strop-gold

name

strop-gold

description

Launch the gold-generation pipeline for a Strop task (N independent strong-reference labeling passes + per-ticker reconcile → DRAFT gold + human review doc) as a background Workflow. Default (`sign=human`, or no flag) ALWAYS STOPS before freeze for human review + sign-off. `sign=machine` (the default when invoked BY /strop-auto; a human may also pass it directly) skips the human gate — after reconcile, a dev agent merges the drafts and freezes gold with an explicit `signer: machine` field plus a machine-signed watermark. Use when the user types /strop-gold or asks to "造 gold / 打 gold 标 / run the gold pipeline". Arguments (all optional): task dir, splits=..., tickers=..., passNum=N, sign=human|machine (default human), smoke, smokeFull, smokeItems=N, runId=....

You are launching a Strop gold-generation run — the task-local pipeline in <taskDir>/gold.workflow.mjs (strong-reference labeling; discipline is a harness invariant, see design §7.7 "gold-gen").

Parse arguments (free text; all optional)

• taskDir — first path-looking token. If absent, DETECT: list tasks/*/ excluding _template; exactly one real task → use it; several → ask the user which one (never guess).
• splits=... — default all three (train,dev,heldout); gold is frozen for ALL splits (§7.8).
• tickers=... — comma list to restrict which source files get labeled.
• passNum=N — independent labeling passes, default 2 (min 2 outside smoke — disagreement detection needs ≥2).
• sign=human|machine — default human: unchanged behavior, this pipeline always STOPS before freeze for a human gate. sign=machine (what /strop-auto passes by default; a human may also pass it explicitly for a fast, unattended freeze) skips that gate — see "On completion" below for exactly what it does and does not sign.
• smoke — LEAN connectivity check (3 agents: discover + 1 label pass + validate; throwaway paths under .runs/<task>/gold/_smoke/). smokeFull — same idea but the whole pipeline (also exercises reconcile + assemble). smokeItems=N — per-file item cap in smoke.
• runId=... — default: generate YYYY-MM-DD-<letter> from today's date, first letter suffix not already present under .runs/<task>/gold/ (the workflow has no clock — you supply the stamp).

Pre-flight (cheap, local)

1. <taskDir>/gold.workflow.mjs exists — gold-gen is (currently) task-local; a task without one either hasn't adopted Mode B or needs its gold mechanism authored first → STOP and explain.
2. <taskDir>/rubric.md exists — the labeler labels BY the rubric. Missing → STOP (run /strop-init first).
3. Frozen guard: if <taskDir>/data/GOLD_FROZEN.md exists, the yardstick is already frozen — re-generating gold mid-experiment breaks §7.8. STOP and confirm the user really intends a re-labeling campaign (and knows it invalidates prior runs' comparability) before launching. Applies regardless of sign=.
4. If sign=machine: also confirm the task has its own merge/freeze mechanism (conventionally <taskDir>/merge_gold.py; check the task's own docs if it documents something else) BEFORE spending any labeling budget — a task that hasn't authored one cannot complete the machine-signed path once labeling finishes, and fail-fast means catching that now, not after passNum label passes already ran.
5. No .env needed — the labeler is a subscription subagent (Opus/Sonnet), not the student API.

Launch

Workflow({ scriptPath: "<taskDir>/gold.workflow.mjs",
           args: { taskDir, splits, tickers, passNum, smoke, smokeFull, smokeItems, runId } })

(omit keys the user didn't set). Background; tell the user the runId and that a completion notification will arrive.

On completion

• Failure: fail-fast — read the error + phase transcript, diagnose the root cause, report. Do NOT auto-rerun.
• Success, either sign mode: report where the DRAFT gold and the review doc landed, the agreement/disagreement stats, and the ⚠️ items the passes disagreed on.
  • sign=human (default): state the gate explicitly — this pipeline stops before freeze — a human must review + sign off (rubric weights AND flagged items), and only after sign-off does the merge/freeze step run (per-task merge script, e.g. merge_gold.py, then write data/GOLD_FROZEN.md recording signer: human). Next command after sign-off + freeze: /strop-check-loop.
  • sign=machine: do NOT stop — proceed straight to freeze. Spawn strop-dev-agent with the taskDir, the splits just labeled, and instructions to (a) run the task's own merge/freeze script (e.g. <taskDir>/merge_gold.py) over every split's freshly-reconciled *.gold.draft.jsonl files, verifying a clean merge (ids line up 1:1, no leftover errors) plus the task's vocab validator if it ships one; then (b) write <taskDir>/data/GOLD_FROZEN.md, keeping every section of the existing format (what's frozen, labeling method, per-split item counts, the "never edit frozen gold" rule — match the shape of any GOLD_FROZEN.md elsewhere in the repo for backward compatibility) but with:
    • signer: machine in place of a human sign-off reference — reconcile (the arbitration pass over the passNum independent labels) stood in as the arbiter; no human reviewed the drafts or the review doc;
    • this watermark line, verbatim, near the top: ⚠ gold: machine-signed (teacher-agreement metric, not human-verified). Report that SAME watermark back to the user in your own summary — never leave it buried only in the file. Next command: /strop-check-loop. Note: THIS skill's sign=machine only ever signs gold — it never touches rubric.md's own separate sign-off record (the HUMAN-SIGNED / MACHINE-SIGNED marker /strop-check-loop and /strop-train look for; both are accepted there, reported by which signer). Gold and rubric are two independently signed gates; if you invoke /strop-gold sign=machine directly (not via /strop-auto) on a task whose rubric is still an unsigned DRAFT, /strop-check-loop will still legitimately BLOCK a real run on the rubric gate alone — that is correct, not a bug in this skill. (/strop-auto's own Collect phase signs the rubric separately, right after strop-init-agent drafts it, before gold-gen even starts — see that skill if you want both gates machine-signed in one command.)