distill

name

distill

description

Compress an expository note into readable prose backed by a certified Glossary via a standalone CLI — abstractive idea-compression, not extractive trimming. Re-expresses the note as flowing prose (carrying the thesis and the relations among terms) above an optional `## Workflow` checklist of the note's actionable directives and a definitions-only `## Glossary` table, collapsing restatements to one entry each and keeping only operational tokens verbatim; a different model fidelity-grades the glossary definitions and the workflow steps against the source and surfaces anything that did not translate. Replaces /cut. Use on /distill, "distill this", "compress this note", "make a glossary of this note", "tighten this into its ideas", "summarize as a glossary", "compress this guide/procedure into steps", "this note is too long/verbose", «дистиллируй», «сократи в глоссарий», «выжимка», «сделай глоссарий», «сократи гайд в шаги»; route whole-repo glossary maintenance to /glossary.

distill

A standalone headless CLI that re-expresses an expository note as readable prose backed by a certified glossary. Unlike cut (extractive, retired), distill is abstractive: it does not keep a verbatim subset of the input — it rebuilds the note around a canonical form. The default output is a readable note: flowing connective prose that carries the thesis and the relations among terms, then an optional ## Workflow checklist of the note's actionable directives, then a ## Glossary table of definitions only (no duplication — definitions live in the table, relations in the prose). Restatements of one idea collapse structurally to a single glossary entry.

Two certified channels share the pipeline: the glossary (declarative — concepts to know) and the workflow (procedural — directives to do). The workflow channel is what lets distill keep a note's actionable payload — a practices list, a procedure — that the glossary alone would dissolve; it is optional and appears only when the note prescribes actions. Word count goes down; input and output do not match line for line. Run it on a finished expository or how-to note (a vault note, a track section, a concept explainer, a practices list), not on a pure command runbook or code.

How to use

Distillation is a two-step emit → review → apply flow: distill-text writes an interactive intermediary beside the note, you resolve its residue by checking boxes, and distill-text apply writes the finished note back to source. The source note is never touched until apply.

1. Emit. Give distill-text the note — a positional file path (distill-text input.md) or piped on stdin with --out <dest.md> (required for stdin, since the destination can't be inferred) — with FIREWORKS_API_KEY in env (the .local/bin wrapper fills it from the macOS Keychain; doppler run --project claude-code --config std -- is another way). For a vault entry named rather than pathed, resolve the path first with vault-query get: distill-text "$(vault-query get "Target distance")". On success it writes <note>.tmp.md beside the note and prints the intermediary path as stdout's only line; the footer, ending · review: N items + gate (or · review: gate when nothing needs triage), goes to stderr. Flags: --glossary drops the prose and emits just the glossary; --lang ru overrides autodetect; --max-retries N caps gate recovery (default 2); --no-gate / --no-revise skip stage 5 / stage 4. Exit codes: 0 intermediary written; 2 usage error; 3 passthrough — no intermediary, the stdout path points at a temp copy of the unmodified original (truncation/transient failsafe, expand-guard growth, nothing to distill), prefer the source (empty input exits 3 with nothing on stdout); 4 a prior <note>.tmp.md is still pending — apply it or delete it, then re-emit. On exit 3 the stdout path line is present except for empty input, where stdout is empty, so a $status-branching caller must not assume a path on 3.

2. Review the intermediary. Open <note>.tmp.md. Its frontmatter carries epistemic_status: in-review, its body is the distilled note, and above the gate it may hold a <!-- interact: pick-any id=residue --> block — one item per entry that failed or dodged a gate, each carrying the verbatim source in a fenced payload. Checkbox is the whole interface; never hand-edit content or the indentation inside a block. Per item, by its reason prefix:

   reason prefix (on the item)	verb	check the box to	leave unchecked to
   inverted: / entails-missing: (a glossary def)	recover	re-render that def from its fenced source	drop the def from the note
   workflow:N (a step-group)	recover	splice the source's directive back into the steps	drop those steps
   gate-inconclusive: (judge returned no verdict)	keep	ship the entry as it stands	drop it
   any other prefix (dropped:, prose/payload)	—	(nothing — apply cannot place it; leave it)	keep it dropped; hand-add later

   The last row is content the pipeline dropped (a collided wikilink, a compacted table row, a prose flake) — apply has no position to splice it back, so checking its recover box is refused (exit 2), never silently lost. Leave it unchecked; if you want it, copy the fenced payload into the note by hand after apply. For the actionable rows, unchecked always removes the entry — it is never "skip silently." Then check the single reviewed: box in the <!-- interact: confirm-all id=triage-final … --> gate last — it is the "I looked" bit, and nothing applies until it is checked.

3. Apply. Run distill-text apply <note>.tmp.md. It verifies the gate and the stamp (the note must be unchanged since emit), fires the checked verbs, writes the finished note back to <note>.md (all scaffold stripped, epistemic_status flipped to distilled), and deletes the intermediary. Stdout is the note path alone; — applied: N recovered · M kept · K removed (V verbatim) · re-projected|re-projection skipped goes to stderr. Exit 0 applied; 1 the key is missing and a checked recover needed the model (nothing written); 2 refused, nothing written — the gate is unchecked (… gate 'triage-final' is not satisfied), the note changed since emit (stamp mismatch — re-run distill), a malformed block, or a bad/already-applied path. Apply never prompts and has no exit 3.

At a terminal, emit also offers to finish in one sitting: it prints the review path and an apply later with: distill-text apply <path> hint on stderr, then waits — check the boxes in Obsidian and press y to apply in place, or anything else (or Ctrl-C) to leave the intermediary for a later apply. Piped and command-substitution callers never see the prompt (the stdout path line is all they get), so the flow above is unchanged for scripts.

What it does (6 stages)

1. Extract a combo (gpt-oss-120b): {description (authored frontmatter description: verbatim when present — the one independent anchor; generated otherwise), thesis (the spine claim), glossary (load-bearing concepts with dense definitions, relations, and source-block pointers), workflow (the note's actionable directives in order, each with source-block pointers — empty when the note prescribes nothing)}. Restatements of one concept collapse to a single entry.
2. Grade each block (gpt-oss-120b): drop (off-thesis or already in the glossary) / distill (re-express densely — the default for prose) / retain (only compact content that rewording would destroy — code, commands, paths, flags, a wikilink-reference list). A block carrying a [[wikilink]] always survives to the output (retained or distilled, never silently dropped). A step whose every source block is retained is dropped from the workflow so the directive is not duplicated.
3. Synthesize, source-grounded: each glossary definition is written from its cited source text — the def states what the concept is, with relations withheld from the def-writer so the edges ride the connective prose rather than bloating the def — and each workflow step is tightened against its source draft. (A --synth regenerate dial that wrote from the extracted idea-graph alone was removed: the 2026-06-25 stability experiment showed source-grounding dominates — equal stability, more compression.) Then write the connective prose head from the definitions + relations. The body is assembled deterministically: the connective prose, then the ## Workflow checklist (numbered, one directive per line), then the ## Glossary table (Term | Definition only — relations live in the prose, not a column), then retained-verbatim blocks. --glossary replaces the prose with the short tie-together and emits the workflow + glossary alone.
4. Revise (gpt-oss-120b, 4 passes): the distilled prose, the steps, and the definitions pass through words → sentences → paragraphs → AI patterns. Code, wikilinks, and inline code are masked and restored verbatim.
5. Fidelity-grade (glm-5p2, the different model), two criteria: glossary definitions by definition-scoped entailment (does each output capture what the source says the concept is, and invent nothing — relations, rationale, and examples ride the prose, so a def that omits them is never "missing"; thesis still recoverable?); workflow steps by directive coverage (does every source directive appear as a step and every step trace to one, judged as a set per shared source block — a step keeps a reason the source states but the gate flags an invented one, while a dropped reason never counts as missing). A failing definition is re-rendered from source; a failing step-group is repaired from the judge's own finding (the rewrite is told the dropped or inverted directive, not just re-compressed) and, if it still fails within --max-retries, floored with the source's verbatim imperative — a substring of source that covers the action and cannot invert, surfaced as · N kept-verbatim rather than residue. Whatever still fails (a definition, or a step-group whose source yields no extractable clause) is surfaced as <residue>, never silently shipped. The judge model occasionally returns no parseable verdict; askJson retries once, and if it still cannot grade an item, that item is marked gate-inconclusive and surfaced unverified — the distillation ships intact rather than the whole run dropping to the passthrough failsafe.
6. Prose QA (glm-5p2 judges, gpt-oss-120b fixes): the connective prose is gated against its own contract — no closing meta-summary, no document self-reference (the note/this concept), no AI vocabulary, thesis-first opening. Defects trigger one fix pass that may delete offending spans (unlike revise, which only rewords). This sits below the fidelity line: prose defects are repaired best-effort, never blocking. Skipped by --no-gate and in --glossary (no prose). The footer reports · N prose fixes when it fires.

Render a prose note from a glossary (separate, on-demand)

distill-text prose <file> runs the inverse: it reconstructs a flowing prose note from an already-distilled glossary. Since the default compress output already pairs prose with the glossary, reach for prose mainly to turn a glossary-only file back into prose — a --glossary output, or a saved glossary note that has no prose head.

• Input: a distilled file — this tool's own output (the <result>…</result> wrapper is stripped; any <residue> is ignored) or a saved glossary note. It reads the frontmatter, the prose/tie head, the ## Glossary table, and any other section. The parse is order-independent: a ## Workflow checklist is recognized wherever it sits and passed through verbatim — prose mode reconstructs prose from the glossary, never from the workflow, so the certified checklist is never dissolved into prose.
• Output: frontmatter verbatim, then flowing prose grounded only in the glossary (no claim, term, or example absent from it), then the preserved sections (the ## Workflow checklist, a wikilink reference list). The ## Glossary table itself is dropped. Same output contract — the temp .md path on stdout, the footer (— rendered prose · 221→281 words · 5 entries) on stderr. A skip (no ## Glossary table, empty prose, error) exits 3 (output = the unmodified original), reason on stderr — the same code compress passthrough uses.
• Passes / flags: reuses the four revise passes (--no-revise skips them) and honors --lang. It is not fidelity-gated — there is no <residue>; the glossary is the certified artifact, so re-ground the prose against it if a claim looks off.
• Limit: prose mode works from the Term | Definition table alone. A glossary-only file has no relation list to draw on (the default keeps relations in its prose, not the table), so a re-rendered note's relations are softer than the original's — they survive only as far as the definitions' own wording carries them. The glossary remains the source of truth.

Install / run

Requires FIREWORKS_API_KEY (e.g. via doppler run --project claude-code --config std --).

distill-text input.md                      # emit input.tmp.md beside the note
distill-text apply input.tmp.md            # write the reviewed result back to input.md
distill-text < input.txt --out out.md      # stdin: --out names the destination (required)
distill-text "$(vault-query get "Entry name")"  # vault entry resolved by name → path
distill-text --glossary input.md          # glossary only (tie + definitions), no prose
distill-text --lang ru input.md            # force the Russian rubric
distill-text --no-gate input.md            # skip the stage-5 fidelity gate
distill-text prose glossary.md             # separate: prose note FROM a glossary-only file (no gate)

The binary is distill-text; it is on PATH via .local/bin/distill-text. Emit's stdout is exactly the data — one line, the intermediary path (nothing on empty input); the footer and every other diagnostic go to stderr. Capture is plain: path=$(distill-text input.md); status=$?. Then branch on $status: 0 = intermediary at $path (review it, then distill-text apply "$path"), 3 = passthrough, $path is a temp copy of the unmodified original (prefer the source), 4 = a pending input.tmp.md already exists (apply or delete it), 2 = usage error, 1 = missing key.

Limits

• Scope: built for expository prose (notes, concept explainers, track sections). On a short or list-heavy note the expand-guard reverts rather than shipping a larger note — exit 3, footer notes the revert; prefer the original.
• Latency: ~30–60 s (extract + grade + synth + tie + 4 revise passes + thinking-judge gate + prose-QA judge). Built for a generous budget, not an interactive hook. --no-gate / --no-revise trade fidelity/polish for speed.
• Abstractive risk: distill writes new text, so a definition or a step can drift from or invent against the source. The fidelity gate catches and surfaces this for glossary entries and workflow steps; a fidelity drift inside the prose still escapes it. The prose-QA judge (stage 6) gates the prose for style defects only (meta-summary, self-reference, AI vocabulary, opening) — not for invention against the source.
• Workflow granularity: steps are gated as a set per shared source block, so a practices list written without blank lines between items (one block) is judged — and on failure repaired — as a whole; a procedure whose steps are separated into distinct blocks is gated step by step. Formatting the source into discrete steps buys finer residue pinpointing.
• Rationale rides on the source, never synthesized: a workflow step keeps the source's why when the source states one ("do X because Y"), and the gate forbids inventing a reason the source omits. So a checklist carries reasoning the note already gives but cannot supply reasoning the note leaves implicit — distill has no argument sink to derive a missing why. A guide whose load-bearing reasoning is never stated outright still loses it; that gap is a missing source claim, not a gate failure — see the design note in distill.ts.
• Failsafe: a parse error or timeout in an earlier stage → passthrough (original text, footer notes the skip, no intermediary written). A fidelity-judge parse failure no longer passes through — it degrades to gate-inconclusive residue so the distillation still ships (see stage 5). A missing API key exits non-zero with a clear message rather than passing through; every compress-mode passthrough exits 3 (empty input additionally prints nothing on stdout), while a prose skip (no glossary table, empty prose, transient error) exits 3 (output = the unmodified original), reason on stderr.
• Pending intermediary: emit refuses (exit 4, nothing on stdout, before any model call) when <note>.tmp.md already exists — an interrupted review is a pending decision, not garbage. Apply it (distill-text apply <note>.tmp.md) or delete it, then re-emit. Apply consumes the intermediary on success; re-applying an already-applied note fails with exit 2 (no intermediary at … — already applied, or re-run distill).
• In-vault indexing: the <note>.tmp.md intermediary is excluded from vault-query search/consult by a *.tmp.md suffix rule in .vaultignore — but that rule only takes effect once the vault-query binary is rebuilt (it is a nix-store build, unlike distill-text, which runs live from the repo). Until then a half-reviewed intermediary can surface in consult results.