| name | task-blueprint |
| description | Run at the START of a recurring, high-friction task in this repo — BEFORE opening a worktree or writing code — whenever the work matches one of the proven shapes this repo repeats weekly: (a) run a pre-registered / gated eval or benchmark, (b) land an evidence artifact (a results row, a gate JSON, a failure-ledger entry) so CI stays green, or (c) resolve a ledger / manifest / results merge conflict. Instead of improvising the scaffold each time, stamp the matching blueprint below, fill the [PLACEHOLDERS], and adapt. Fires on "run the eval", "land the result", "add a ledger entry", "the ledger/manifest won't merge", "regenerate the manifest", "start task X", "/task-blueprint".
|
| metadata | {"short-description":"Stamp a proven, parameterized scaffold for the eval / evidence-artifact / merge-conflict tasks this repo repeats"} |
task-blueprint
Parameterized scaffolds for the three task shapes that recur in this repo. Each is a checklist you
COPY, fill the [PLACEHOLDERS], and adapt — not a script to run blind. This skill is orchestration
only: it routes to the deeper skills for the mechanics. Zero capability/AGI claim; canClaimAGI
stays false; every headline stays honest-or-NO-GO.
Depends on git-discipline (git writes), spark-cluster-ops (box + GPU + git-crypt),
ci-artifact-drift (green-before-push), session-handover (continuity). Read those for the
mechanics; this skill sequences them into a per-task recipe.
0. Always first (every blueprint) — situational awareness + isolated worktree
git fetch origin --prune
python .agents/skills/git-operations/scripts/git_situational_awareness.py
git -c filter.git-crypt.smudge=cat -c filter.git-crypt.clean=cat -c filter.git-crypt.required=false \
worktree add --detach [WORKTREE_DIR] origin/main
head -1 [WORKTREE_DIR]/README.md
Then claim it in the live ledger so no peer double-launches you (spark-cluster-ops §1):
append to SESSION-COORDINATION.md a block with your session id + pid + explicit
OWNS / CLAIM / do-not-touch lines. Skim the newest SESSION-HANDOVER-*.md + the
agi-proof/failure-ledger.md open rows so you don't redo finished work.
Blueprint A — Run a pre-registered / gated eval or benchmark
- Pin, don't drift. State up front, in your claim block: task/domain, seeds (multi-seed for
comparability), judge families (≥2 DISTINCT families for a VALIDATED gate; 1 = CANDIDATE only),
temp (temp=0 for headline judging), and the pass/fail bar (e.g. κ≥0.40 with a CI excluding zero).
Freeze the harness across the whole sweep (
rlvr-harness-traps §C).
- Pick the box (
spark-cluster-ops §0,7). Iterate on the free Spark via the bridge poller;
the Mac serves the 2nd (Llama/MLX) judge family; a registered/headline number ONLY on
RunPod x86 behind runpod-paid + --yes. aarch64 Spark numerics are never a registered number.
- Hold the one-GPU-job invariant (
spark-cluster-ops §2). Route heavy Spark work THROUGH the
bridge poller (the cluster-wide lock). If you must launch directly, nvidia-smi headroom-check
FIRST — never start a judge/teacher while a cert/train is live (a stray load silently KILLS it).
- Watch it (memory: report-KTA). Surface elapsed + ETA + progress proactively — use the
trainwatch skill / the cluster dashboard (binds 0.0.0.0:8420, funnel :8455).
- Land the result → Blueprint B. If the gate flips to NO-GO, that is a real result: label it
CANDIDATE/NO-GO and file a ledger row (§C) — NEVER lower a threshold to force a pass.
Blueprint B — Land an evidence artifact (green-first, no-overclaim)
- Edit the SOURCE, never the generated copy. Results flow
published-results.json →
build_results_page.py → RESULTS.md (hand-editing RESULTS.md is a drift-gate fail).
Regenerate exactly what you touched, then run the --check form (ci-artifact-drift §2 table).
- NEVER stage machine-local run-results:
agi-proof/benchmark-results/rlvr.public-report.json
(tracked, rewritten every RLVR run), .venv/, training/rlvr/checkpoints/. git checkout -- them.
- Isolate your diff (
git-discipline §4): git add [EXPLICIT_FILES] — never -A/.; confirm
with git diff --cached --stat. Never git stash (shared across worktrees).
- Pass the measurement contract before pushing:
make claim-check
python tools/validate_failure_ledger.py --check
Run the one-shot pre-push sweep in ci-artifact-drift §3; it must print ARTIFACTS CLEAN.
- PR + merge mechanics per
git-discipline / spark-cluster-ops §6: required checks fast +
ci-complete; required_review_thread_resolution:true is the SILENT blocker (green checks can
still be blocked by an unresolved thread); --squash for linear history.
Blueprint C — Resolve a ledger / manifest / results merge conflict ← this week's dominant friction
The agi-proof/failure-ledger.md Markdown table and the generated *.json manifests/results are
what conflict when several agents land evidence at once. Do not hand-splice conflict markers —
apply the shape-appropriate resolve, then re-run the verifier.
C1 — Append-only tables (agi-proof/failure-ledger.md): UNION-RESOLVE.
Both sides only ADDED rows. The correct merge is the union of all rows, not a pick-one.
git diff --check
python tools/validate_failure_ledger.py --check
Each row is | <Failure ID> | <Status> | <Claim impact> | <Required response> | — a unique id, a
Status classifiable as open/resolved, no overclaim. Preserve every peer's row; losing one loses evidence.
C2 — Generated JSON (manifests / published-results.json / gate JSONs): REGEN, don't merge.
These are produced from a source of truth, so a textual 3-way merge is meaningless. Take EITHER side
whole (or git checkout --theirs/--ours), then rebuild from source and let the --check verifier
be the arbiter:
git checkout --theirs [GENERATED_JSON]
python tools/build_results_page.py
python tools/verify_replication_manifest.py
Then re-run the relevant row of the ci-artifact-drift §2 regenerate/verify table and
make claim-check. Never resolve a conflict by silently deleting a peer's evidence row or flipping a
gate verdict — if two verdicts genuinely disagree, stop and ask the human (no-overclaim contract).
C3 — Shared-branch rebase (spark-cluster-ops §4): stage only your explicit files → commit →
git pull --rebase origin [BRANCH] → push --force-with-lease. If the rebase halts on
"unstaged changes" from rlvr.public-report.json, git checkout -- it and re-rebase.
Close-out (every blueprint)
- Confirmed green locally (
make claim-check + artifact sweep) BEFORE push; CI green first try.
- Any measured-but-not-fully-validated result has a
failure-ledger.md row (single-seed / single-family
/ no third-party = not yet VALIDATED).
- If the session was substantial, write the dated
SESSION-HANDOVER-*.md (session-handover skill).
- Faithful reporting: report the number WITH its seeds, CI, and judge families, or not at all.
This is dev-workflow tooling — it makes NO capability/uplift/AGI claim.