| name | evolve-distill |
| description | Use when turning a repeated, successful multi-step workflow from past sessions into a reusable overlay skill — distilling a proven procedure into a class-level skill proposal ("distill a skill from this", "turn this workflow into a skill", "make a skill from how I keep doing X", "extract a skill from my history"). Creates an eval-gated, human-approved proposal — never writes a skill directly. For auditing existing skills use evolve-skills; for session corrections use evolve-review. |
evolve-distill — a proven workflow becomes a skill, safely
This is evolve's answer to autoskill-style skill creation
(docs/evolve-plan.md): distill a successful multi-step workflow
into a reusable skill — but behind every evolve gate. The difference that
matters: autoskill runs an LLM subprocess and auto-writes the skill to
disk. Here you are the extractor (the agent-is-deriver pattern), and the
output is a proposal, not a skill — it lands only after an eval and a human
say so. That prevents the sprawl and poisoning that ungated auto-creation
invites.
Paths: EVOLVE=${CLAUDE_PLUGIN_ROOT}/scripts/evolve; resolve state with
node $EVOLVE/honcho_client.ts status (honors XDG_DATA_HOME).
When NOT to distill
Most sessions produce nothing distillable, and that is the correct outcome.
A workflow earns a skill only when all hold — otherwise stop:
- Reusable class, not a one-off. A procedure for a class of tasks
("set up a new FastAPI service"), never a single task ("fix today's bug").
- ≥2 witnessing sessions (the anti-overfit bar — autoskill's headline
failure is single-trajectory skills that bake in run-specific paths). The
exception is an explicit user ask ("make a skill from this"), which is its
own evidence.
- Non-obvious. It saved real work and isn't common knowledge or a
single command.
- Not already covered by a hub skill or an existing overlay (check
overlay.ts report; near-duplicate → propose a patch via evolve-review,
not a new skill).
Procedure
1. Find candidates. Recurring signal points at what to read:
node $EVOLVE/overlay.ts distill-candidates
Task classes seen in ≥2 distinct sessions, with their session-ids. Thin by
design — it says where to look, not what to create. No candidates? Say
"nothing to distill" and stop (run digest.ts --backfill first if the
journal is empty).
2. Read the evidence — you are the extractor. For a candidate, open the
named sessions' transcripts (under the projects dir from status) and confirm
the same multi-step workflow actually recurred and succeeded. Two sessions
that merely touched the same skill are not a workflow — read them and judge.
Verify success from artifacts (git log, test output) where the transcript is
ambiguous; never trust a self-report of "it worked."
3. Distill class-level, strip the session. Draft the SKILL.md body as
When to use / Core principles / Workflow with concrete tool commands.
Ruthlessly remove anything specific to one run — fixed paths, values, repo
names, calibrations. Whatever only one trajectory needed goes to a
references/ note, not the body. A session-specific name is an automatic
reject; name for the class (setup-fastapi-service-local, never
fix-x-today-local).
4. Write the eval — no eval, no registration. A skill with no way to
prove it works never enters the catalog (the catalog is trusted recall). Draft
a bench scenario that asserts on artifacts: {"id": ..., "prompt": ..., "assert": "<shell that exits 0 iff the skill worked>", "judge": "<optional rubric>"}. This is the gate — overlay.ts propose refuses an
overlay-create without it.
5. Stage the proposal (never write the skill). Pipe the proposal to
propose — it validates the eval and the ≥2-session bar, then stages to
pending/:
echo '{"kind":"overlay-create","name":"<class>-local",
"description":"Use when …","body":"## When to use…\n## Workflow\n1. …",
"summary":"distilled <class> workflow","signal":"recurred in N sessions",
"sessions":["<id1>","<id2>"],
"eval":{"id":"smoke","prompt":"/<class>-local …","assert":"…"}}' \
| node $EVOLVE/overlay.ts propose
6. Record the mining (symmetric mining, Law 9). Bank what worked, not
only failures: honcho_client.ts observe --type technique --target agent --content "[technique] project=… — <the reusable method>" so the next
distill pass sees the reinforcing signal.
7. Hand off. The proposal sits in pending/; /evolve-review presents it
for approval, and approval replays it via overlay.ts apply-pending →
GR4 skill-write scan → overlay scaffolded at confidence 0.0 with its
bench.json bundled. It earns trust only through overlay.ts record outcomes;
never present a fresh distilled skill as proven.
Hard rules
-
De-identification reuse test (R1): after stripping case-specific
entities, ask whether this user would need the same policy again — if what
remains is generic advice, distill nothing. Empty is the default success.
-
Never invent steps. Only workflow steps the sessions actually
demonstrate; single-trajectory details go to references/, not the body.
-
Propose, never write. This skill stages proposals only. Applying is
/evolve-review's human-gated step; scaffolding runs the poisoning scan.
-
≥2 sessions or an explicit ask. One impressive session is the overfit
trap, not evidence.
-
No eval, no registration. Enforced mechanically at propose.
-
Class-level names; hub skills are read-only. A distilled skill is always
a user-scope *-local overlay. A gap in a hub skill routes to the fleet
channel (observe --target skill__<name>), never a local edit.
-
Distill from success; strip the session. The failure mode you are
guarding against is baking one run's specifics into a "reusable" skill.
