imprint-reteach-audit

name	imprint-reteach-audit
version	1.0.0
description	Re-teach one or more imprint sites from their existing recordings and verify them with `imprint audit`, including the concurrency rules, background-monitoring recipe, and the known failure modes.
triggers	["reteach site","re-teach","teach from scratch","audit site","rebuild tools","verify site"]
allowed-tools	["Bash","Read"]

Re-teach + audit imprint sites

Use this to rebuild a site's tools from its existing recording and confirm they work.

Commands

# Re-teach (wipes compiled artifacts, PRESERVES sessions/, recompiles from the
# newest raw recording, tracing on, compile concurrency 2):
IMPRINT_TEACH_TIMEOUT=30m bash scripts/teach-from-scratch.sh <site>

# Audit (spawns a headless claude that calls every compiled tool live and grades
# correct/broken; exit 0 = PASS, default threshold 95%):
bun run src/cli.ts audit <site>

Hard rules

• Compile concurrency is 2 — never raise it, and never run two teaches at once. Each teach already runs 2 compile agents; two parallel teaches = 4 concurrent Opus agents → trips rate limits. Re-teach sites strictly sequentially. Avoid overlapping a teach with an audit for the same reason.
• Heavy search tools need IMPRINT_TEACH_TIMEOUT=30m+ (batchexecute / multi-filter search; the default is too tight once param-fidelity verification runs).
• Teach is long: ~40–75 min per site, dominated by the heavy tools' compile.generate. Run it in the background and monitor.
• To re-teach several sites + audit each as one hands-off job, loop them sequentially in a single background driver script (teach → audit → next).

Monitoring a background teach (the log is spinner-heavy)

# strip ANSI + collapse \r redraws + drop spinner frames
sed -E 's/\x1b\[[0-9;?]*[a-zA-Z]//g' LOG | tr '\r' '\n' \
  | grep -aviE '◐|◓|◑|◒|◇|◆|●|Triaging|Replaying|Compiling •|cycle [0-9]/5|^\s*$|^│|^├'

Watch for: shared modules built + verified vs pruning; per-tool index.ts appearing under ~/.imprint/<site>/<tool>/; final Done! N tools ready. Avoid grepping cycle/Compiling • lines raw — they are giant \r-concatenated blobs.

Known failure modes

• Valid shared module pruned for "must export a transform function" — was a bun stale-.ts-import-cache bug; fixed in src/imprint/prereq-builder.ts (importModuleFresh copies the module to a unique sibling path before re-importing, defeating the cache). If it recurs, the fix is there.
• Audit grader non-determinism on a no-op param. A param that legitimately has no observable effect (e.g. a brand filter where two brands share the same data) can be graded correct one run and tool_broken the next, flipping a site between PASS and ~85%. The tools still return correct data — re-audit once to confirm it's variance before treating it as a defect (and report honest numbers, don't re-roll just to pass).
• A network outage during the replay/capture stage corrupts the session — events time out with 0 requests captured (the captured-count plateaus). Replay normally takes ~3 min; if it's dragging for many minutes with a flat capture count, the network dropped. Re-teach when the connection is stable. Diagnose via the per-event capture trajectory in the teach log (Replaying event N/M (K requests captured)).

Verify + expose for live testing

• PASS = audit exit 0 and score ≥ 95% (graded N of N invocations).
• The MCP registration imprint mcp-server <site> is a command pointer — it re-reads ~/.imprint/<site>/ on every spawn, so it always serves the latest teach. After a re-teach, a running Claude session must reconnect (/mcp) or start fresh to pick up new tools; a new session needs nothing.
• Confirm what's served: claude mcp list (look for imprint-<site> ✓ Connected) and claude mcp get imprint-<site>.