ketch

name	ketch
description	Research skill for ketch — a fast stateless CLI for web search, OSS code search, curated library docs, page scraping, and site crawling; an optional MCP server exists for operators who want it, but the CLI is the primary interface. Use when a question needs live sources: 'research X', 'what are people saying about Y', 'find docs or real-world examples for Z', 'scrape/crawl this site' — or when installing or configuring ketch backends. Routes search vs code vs docs vs scrape vs crawl, keeps every fetch inside a token budget, turns error prefixes into control flow, and produces cited syntheses. Not for local codebase search, private repos, or pages behind auth.
version	0.1.0

Ketch

Route every live-source question to one of ketch's five research surfaces — search, code, docs, scrape, crawl — over the transport the operator gave you, with a token budget on every fetch and a source URL on every claim. ketch is one stateless binary — call, result, exit — with web search, OSS code grep, curated library docs, and page/site extraction together, so a complete research pipeline needs no other tool and no daemon.

Transport: stateless CLI by default, MCP when the operator wired it

The CLI is ketch's identity: call → result → exit, --json on every call, exit codes as control flow, zero daemon. That is the default transport and the zero-infrastructure path. The MCP server is a supported alternative for operators who want it — never a prerequisite.

Decide once per session, before the first call:

1. which ketch succeeds → the CLI is your transport: --json on every call, exit codes as control flow.
2. Also check for ketch's five MCP tools in your tool list — search, code, docs, scrape, crawl from a server named ketch (in Claude Code: mcp__ketch__search, …). Present → the operator wired them up on purpose, and using them for research calls is correct and good: structured output, per-URL errors, no shell round-trip. Do not shell out around tools the operator set up.
3. Both live → either transport serves research calls, but know the tradeoff: a running MCP server holds the single-process page-cache lock, so concurrent CLI scrapes silently run cache-disabled.
4. Neither CLI nor MCP tools → ketch is not installed. Offer brew install 1broseidon/tap/ketch or go install github.com/1broseidon/ketch@latest — an operator action: propose, wait for confirmation.

The rule: use the transport the operator gave you — when both are live, either is fine for research calls, and operator actions are always CLI.

Config discovery is CLI regardless of transport: ketch config prints effective settings and available backends as JSON; there is no config tool over MCP. Operator actions — config set, cache, browser install, crawl --background/status/stop, doctor — are deliberately not in MCP. They are always CLI.

Glossary

Use only these terms in ketch output.

Term	Meaning
surface	One of the five research operations: search, code, docs, scrape, crawl
transport	How a surface is called: the CLI binary (default) or the optional MCP tools
backend	The provider behind a surface: brave/ddg/searxng/exa/firecrawl/keenable (search), grepapp/sourcegraph/github (code), context7 (docs)
operator action	A system-managing or diagnostic command — config set, cache, browser install, background crawls, doctor — CLI-only by design
error prefix	The stable class on every ketch error: CLI exit codes 2–6, mirrored as the bracketed prefix opening every MCP tool error — [validation], [not_found], [upstream], [precondition], [cancelled]
fan-out	How many queries are searched and URLs scraped under one plan
token budget	The per-call output bound: max_chars/trim on scrapes, tokens on docs, limit/--minimal on lists
probe	One cheap read-only call that tests whether a surface is configured and reachable

How to use this skill

• Default: answer one question with one or two routed calls. Use the surface routing table, token budgets, and error control flow below.
• ketch research <question>: deep multi-source research — search fan-out → scrape top hits → optional code/docs corroboration → synthesized, cited answer. Read references/verbs/research.md.
• ketch setup: configure backends with the operator — probe current state, propose exact commands, mutate only on confirmation. Read references/verbs/setup.md. Enter this verb whenever any call returns [precondition] / exit 5.

One question = one plan. Escalate a default run into ketch research when the first search shows the answer is contested, multi-part, or needs corroboration.

Non-negotiable disciplines

1. Use the transport the operator gave you. The CLI is the default; MCP tools in your list mean the operator opted in — use them for research rather than shelling out around them. When both are live, either serves research calls (a running MCP server holds the page-cache lock, so concurrent CLI scrapes run uncached); operator actions — config, cache, browser, background crawls, doctor — are always CLI.
2. Bound every fetch. max_chars 4000–8000 plus trim on any scrape of a page you have not seen — an unguarded page can cost ~25k tokens. Skipping the cap requires a stated one-line reason ("known ~200-word page").
3. Cite every claim. A research synthesis without source URLs is not a deliverable.
4. Error prefixes are control flow. Classify before reacting. Never retry [validation] or [not_found] unchanged.
5. Propose, then mutate. config set, browser install, docker runs, installs — only after the operator confirms the exact command. Never touch a value that is already configured and working.
6. The binary outranks this file. ketch config and --help are ground truth; where they disagree with a table here, trust the binary and flag the skill as drifted.

Gold decision trace

Request: "ketch research — do people actually use Go's iter.Seq in real projects, and what are the gotchas?"

Transport: operator wired mcp__ketch__* into this session → honor it; research calls go over MCP.
Plan: 2 queries · scrape top 3 · max_chars 6000 + trim · ≤8 calls

search {query: "Go iter.Seq real-world experience gotchas", limit: 5}
  → "[upstream] ddg rate limited" → rotate to next entry in available_backends,
    retry once: search {query: ..., backend: "brave", limit: 5} → ok
search {query: "Go range-over-func adoption production", backend: "brave", limit: 5}
  → 10 results, 8 unique hosts → picked 3: official blog post, one experience
    report, one issue thread (primary sources over aggregators)

scrape {urls: [u1, u2, u3], max_chars: 6000, trim: true}
  → isError=false; checked results[] one by one: u1, u2 ok;
    u3.error = "[upstream] … 503" → dropped, will be named in synthesis

code {query: "iter.Seq", lang: "go", limit: 3}      # corroborate real usage
  → 3 repos with file/line URLs

Synthesis: five claims, each cited to its URL; u3 listed as unretrieved;
one conflict between u1 and u2 stated and attributed, not averaged.
Budget: 5 of 8 calls (the rate-limited attempt counts).

Surface routing

First match wins:

The question needs	Surface	Not
Current web pages, opinions, news, comparisons	search	docs — that is curated library docs only
How real projects call an API	code	search — blogs talk about code; code greps public OSS repos via grep.app
A library's own documentation, version-aware	docs	scrape of the docs site — docs is already extracted and token-budgeted
The content of a URL you already hold	scrape	search — never re-find a known URL
Many pages from one site	crawl	looped scrape — crawl dedupes, bounds, and streams

In reverse: search finds URLs; scrape reads them; crawl reads a site; code reads public source; docs reads library docs. search with scrape: true fuses the first two when you will want full content from every hit — budget it like a scrape.

Token budgets

Call	Bound with	Measured cost
search, limit 5	limit	~1.4 KB
code, limit 3	limit	~0.7 KB
docs, default budget	tokens (default 4000)	~3.3 KB
scrape, unknown page	max_chars 4000–8000 + trim	unguarded: up to ~100 KB (~25k tokens)
crawl (MCP)	max_pages + per-page max_chars	30 pages default, 100 cap, 3-min wall clock
Any CLI list	--minimal	roughly halves output

Error control flow

Exit (CLI)	Prefix (MCP)	Meaning	Do
2	[validation]	Bad input	Fix the call; retrying unchanged can never succeed
3	[not_found]	Nothing matched	Change the query or selector; not an outage
4	[upstream]	Backend or network failure	Rotate backend (available_backends in ketch config) or retry once
5	[precondition]	Operator config missing	Stop researching; enter ketch setup
6	[cancelled]	Cancelled or timed out	Rerun with smaller scope

Situations → class: unknown backend, regexp on github → [validation]. Selector matched nothing → [not_found]. ddg rate limit (it rate-limits readily under fan-out), DNS failure, grepapp's intermittent 504 → [upstream], rotate or retry once. Missing API key, docs backend local (planned, unimplemented), force_browser with no browser configured → [precondition]. One asymmetry: a CLI crawl interrupted by SIGINT exits 0 with partial results, by design.

Gotchas

Detail for each lives in references/surfaces.md.

• Scraping a bare domain auto-probes /llms.txt and may silently return that instead of the homepage — the title field reveals the swap; no_llms_txt opts out.
• docs is a two-step: resolve the name → vet the matches → fetch by library ID. Resolve never returns empty — garbage in gets confident fuzzy matches out, so check the name, not just the trust score.
• Batch scrape reports per-URL failures inside a successful call: isError=false with results[].error set. Check every entry.
• regexp works on grepapp and sourcegraph only; github rejects it with a pointer to those backends.
• Background crawls (--background, status, stop) are CLI-only; the MCP crawl is synchronous and capped.
• The page cache (bbolt, 72h default TTL) is single-process: a long-running MCP server holds the lock, so concurrent CLI scrapes silently run cache-disabled — ketch doctor reports the cache as locked by another process. Running the server degrades the CLI; prefer CLI-only when both would run long-term.

BAD/GOOD contrasts

BAD: scrape {url: "https://docs.example.com"} — no bound; you get llms.txt or ~25k tokens, whichever is worse. GOOD: scrape {url: "https://docs.example.com/quickstart", max_chars: 6000, trim: true} — plus no_llms_txt: true when you want the page itself, not the site's llms.txt.

BAD: Telling a user they must run an MCP server to use ketch with agents — the CLI plus a prompt block is the zero-infrastructure path, and a long-running server holds the page-cache lock against every CLI call. GOOD: CLI by default; MCP when the operator wired it — and when mcp__ketch__* tools are in your list, use them for research instead of shelling out around the operator's setup.

BAD: [upstream] ddg rate limited → retry the identical call three times. GOOD: Rotate — backend: "brave" (or the next entry in available_backends) — retry once, and note the swap.

BAD: Fetch docs from resolve's first match because its trust score is high, even though its name is not the library you asked about. GOOD: Vet name + snippet count + trust; if no match names the intended library, say so instead of fetching junk docs.

Reference loading

• ketch research … → read references/verbs/research.md before starting.
• ketch setup, any [precondition]/exit 5, or an install → read references/verbs/setup.md.
• Full flag/param tables, CLI↔MCP name mapping, backend/key matrix, or a surface behaving oddly → read references/surfaces.md.

Scope

In scope: the five research surfaces over both transports, the research and setup verbs, token budgets, error-prefix control flow, backend configuration. Out of scope: local or private codebase search (use repo tools), pages behind auth or paywalls, bulk archival crawling beyond the caps, browser automation beyond ketch's headless-rendering fallback.

Bound every fetch; cite every claim.