deep-research

# Deep Research Use Sandwalk as the durable system of record. Sandwalk never supplies reasoning or prose: perform those tasks yourself, then submit bounded artifacts through its commands. Use the default compact hint mode (or set `SANDWALK_HINT_MODE=compact` explicitly). ## Required startup Choose exactly one writable directory prefix: - If `SANDWALK_DIRECTORY_PREFIX` is non-empty, use it and never pass a conflicting `--directory-prefix`. - Otherwise choose one prefix and pass the same `--directory-prefix` to every workspace command. If the user names an existing workspace, start with the continuation loop below. Do not initialize it and do not reconstruct its state from chat. For a new workspace, read [references/commands.md](references/commands.md) completely before the first Sandwalk or network command. Treat its command forms as canonical; do not guess flags or subcommands. After `sandwalk init --slug <slug>` succeeds, never initialize that workspace again. ## Resume existing research First ensure the previous worker has stopped. Never run two sessions concurrently against the same claim. Then repeat this loop: 1. Run `sandwalk continue --slug <slug>` with the established directory prefix. If it reports phase `completed`, stop and report the exported result. 2. Read the returned `artifacts/work/current.json` and every artifact it explicitly asks you to inspect. For review evidence with `kind: "visual"`, open every `image_path` with the available vision tool. Treat `description` only as an agent observation, never as source text, and put every inspected visual reference into the review's `reviewed_visuals` list exactly once. 3. Edit only fields inside `editable`. Never change `fixed`, `workspace`, `action`, identifiers, or `integrity_md5`. The integrity hash excludes `editable`, so it remains valid after the intended edit and must never be removed or recomputed. Make the requested semantic decisions from the exact artifacts, not from memory or search snippets. 4. Run the exact `sandwalk apply --file ...` command returned by `continue`. 5. Run the `continue` command returned by `apply` and repeat until the reported phase is `completed`. The current packet and durable state override chat history, controller checklists, old errors, and any lost command response. After compaction, uncertainty, or a partially applied action, discard the session-local procedure and run `continue` again. It migrates old state and derives a valid action from the state that actually committed. If `apply` reports `INVALID_WORK_PACKET`, run `continue` once to regenerate the packet, then edit only `editable` and retry the exact returned `apply` command. Do not recompute the integrity hash, use `next`, or switch to the manual command workflow while a current packet exists. The packet presents one deterministic valid path; other legal research actions may exist. If inspecting its fixed artifact shows that candidate is unsuitable, set its editable decision to `reject`, give a concise reason, and apply the packet. Sandwalk will durably skip it and derive another action. Never create an excerpt or finding from a bot challenge, access-denied page, irrelevant result, or content that does not address the packet's research objective and step. Search packets contain an editable query and initially empty `editable.source_root` and `editable.source_index` fields. Confirm that the query names the research subject and current step goal. For a one-off lexical local search, put the authorized directory in `source_root`. For a previously built Sandwalk semantic index, put its directory in `source_index`. Never set both. ## Exclusive retrieval While this skill is active, Sandwalk is the only permitted path for every network, local-document, or GNU Info search and fetch, including reconnaissance. Do not load or invoke another web-search, browsing, or fetch skill. Do not call `ddgr`, `curl`, `wget`, `yt-dlp`, `ug`, `ugrep`, `mq`, Xberg, Docling, a browser, or an HTTP client directly. These programs may run only behind a Sandwalk adapter or for bounded inspection of already-returned Sandwalk artifact paths. Use `sandwalk search` to discover persisted hits, `sandwalk fetch` to create immutable snapshots, and read only the returned `document_path`. If either command fails, diagnose and retry its adapter; never substitute another retrieval path. The fetch result also declares `document_media_type`. Navigate `text/markdown` through `mq` headings. For `text/plain`, locate relevant terms with `rg -n`, then read only bounded line ranges around matches. Do not run `mq` against plain text or read an entire long transcript into context. For installed GNU Info or active Emacs Info manuals, run `sandwalk search` with `--adapter sandwalk-search-texiq`. Use `--source-root <info-directory>` only when an explicit Info directory is required; the adapter otherwise searches the active Emacs `Info-directory-list` followed by the ordinary Info path. Returned `info://texiq/` hits select the matching fetch adapter automatically and become exact-node `text/plain` snapshots. Read the returned `document_path` with the same bounded `rg -n` workflow as any other plain-text snapshot; do not invoke `texiq` directly while this skill is active. For user-authorized local documents, run `sandwalk search` with exactly one `--source-root <directory>`. The root must already be readable inside the agent's filesystem sandbox. Sandwalk records it with every hit, defaults to the `ugrep+` connector, and later selects the local-file fetch dispatcher for `file://` hits. Ordinary text and source files return `text/plain`; rich documents are delegated to the structured Docling connector. Remote PDF hits use the same structured normalization. For arXiv hits, Sandwalk prefers the article HTML for `document.md` and retains the matching versioned `source.pdf` for the user; an unusable HTML representation falls back to Docling automatically. Inspect the returned primary document according to its declared media type; never treat the search snippet as document content. When the source meaning depends on a diagram, page layout, handwriting, or another visual feature that normalized text does not preserve, use `sandwalk visual create` on the retained rich-document snapshot page. It accepts only a supported paginated artifact named by the immutable snapshot manifest, renders one full page with a bounded fixed profile, and never invokes a model. PDF is direct; Office, OpenDocument, RTF, EPUB/FB2, Visio, and Publisher inputs use a temporary LibreOffice-to-PDF conversion before Poppler. Treat the returned PNG—not page numbering from another native viewer—as authoritative visual evidence, because font availability and renderer differences can affect non-PDF pagination. Inspect the PNG with vision before attaching it. The required description is your short observation for navigation; it is not a quotation and cannot substitute for inspecting the image. Prefer exact excerpts for ordinary textual claims. Visual capture is a permitted bounded detour from a current excerpt/finding packet: create and attach the visual with the current claim, do not apply the now-stale packet, then run `sandwalk continue` to derive a fresh packet from durable state. The explicit `sandwalk-fetch-xberg` alternative targets the Xberg v1 CLI and is intended for simple local documents where its lower operational cost is useful. Its bundled profile disables result caching, layout-to-Markdown, layout-dependent reading-order rewriting, VLM, and remote enrichment. It retains Xberg's structured JSON plus quality metrics and rejects structured tables that disappear from Markdown. Keep Docling as the default for complex PDF or rich-document hierarchy unless a broader reviewed corpus establishes a new default. For repeated or semantic retrieval over local documents or Info manuals, first build the discovery cache outside the research workflow with exactly one of: ```console sandwalk index build --source-root <authorized-directory> \ --index-directory <index-directory> sandwalk index build --info-manual <manual-or-path> [--emacs] \ --index-directory <index-directory> ``` Then use `sandwalk search ... --source-index <index-directory>`. This runs one typed QMD `vec:` query with `--no-rerank`, so neither query expansion nor a reranking model is loaded. QMD projections and snippets are never evidence; `sandwalk fetch` verifies the selected entry and unchanged original input, then publishes the exact normalized document under its original `file://` or `info://texiq/` provenance. Rebuild the index when fetch reports that it is stale. Use `qmd doctor` to verify automatically selected acceleration. Set `QMD_FORCE_CPU=1` only when Metal/CUDA/Vulkan is unavailable or prohibited by the agent sandbox; expect slower queries. Do not invoke QMD search or retrieval directly while this skill is active. For YouTube hits, the default constructor uses source chapters when they exist. If it returns `text/plain`, the video had no usable source chapter structure: search the timestamped transcript with `rg -n` and read bounded ranges. Never invent headings, summarize unread transcript regions, or fetch the video, audio, or captions outside Sandwalk. ## Workflow 1. Create one workspace with a short lowercase slug using the required startup rules above. 2. Perform bounded reconnaissance whenever the topic name is unfamiliar or ambiguous. Resolve its identity with Sandwalk search and a fetched primary document, never from memory or search snippets, before recording the objective or sealing the plan. 3. Record the objective, an append-only step plan, and dependency edges; validate and seal the plan. Then enter the continuation loop. 4. Claim one eligible step. When parallel workers are available, give each worker a different claim. Otherwise process steps sequentially. 5. Search, select relevant hit identifiers from the JSON response, fetch those hits, and read only the necessary portions of the returned immutable primary documents. 6. Create exact excerpts and, only when material visual content requires it, bounded rich-document page visuals. Write narrow findings, attach evidence with typed relations, seal each finding revision, and review it against every attached artifact. Prefer an independent validation worker; with one worker, perform the review as a separate evidence-only pass. 7. Complete the claim only after its current findings pass review. Repeat until all required steps complete. 8. Prepare the writer pack, draft using only its typed citation tokens, submit the report, review every hashed block, and finalize. When the user requests a PDF, run `sandwalk export pdf --slug <slug>` after completion. Report blocks are separated by blank lines. Every non-heading block needs a current citation token, including short lead-in prose before a list. When Sandwalk reports `REPORT_BLOCK_UNCITED`, use the returned block preview to edit that exact block inside `editable.report_markdown`; do not repair a separate draft file and do not guess the ordinal. On interruption, use the continuation loop. Use `sandwalk resume` only when you need its richer crash diagnostics or bounded recovery report. Do not edit the SQLite database, projections, snapshots, excerpts, or audit log. ## Guardrails - Keep claims narrower than their evidence. - Preserve source disagreement using `contradicts` or `qualifies`; never erase it to force consensus. - Cite exact excerpts or inspected immutable visual pages, not search snippets, uninspected descriptions, or remembered page content. - Never query or edit the Sandwalk SQLite database directly. Candidate selection and recovery must go through the current packet. - Never fabricate or manually alter `hit_`, `snap_`, `excerpt_`, `visual_`, or `claim_` identifiers. - Never rewrite citation numbering. Use `[cite:step-key/finding-key]`; Sandwalk renders final citations. - If a manual command fails, execute its `next` command when present. Otherwise use `sandwalk explain CODE`, repair the stated invariant, and retry. If `apply` fails after a child mutation committed, run `continue`; do not replay the entire packet blindly. - Keep at most one active claim per worker. Claims do not expire; recover the existing claim through `continue` after interruption, and checkpoint before handing work off. - Relation `context` is supplemental. Every finding needs at least one `supports`, `contradicts`, or `qualifies` text or visual artifact before sealing.