| name | ci-artifact-drift |
| description | Run BEFORE committing or pushing; a handover, ledger, pointer, index, or artifact drift — or a red no-overclaim gate — is the usual reason a PR in this repo goes red, not a real bug. (Trigger note: the skill index derives its keyword list from the FIRST 12 non-stopword tokens of this description — `_triggers_from_text(limit=12)` — so any drift source that must auto-fire the skill has to be named in THAT opening sentence, not further down. That is exactly how the handover drift source stayed invisible to the trigger while being documented in the body.) Concretely, PRs go red on (a) generated artifacts that drifted from their source of truth (RESULTS.md, the RAG index, wiki pages, the local-Sophia dataset, the version stamp, the failure ledger, and the SESSION-HANDOVER -> CURRENT.json -> Agent-Index chain) and (b) the no-overclaim measurement gates (lint_claims, claim_gate, eval_stats, assert_decontam, lint_training_rows). Use to regenerate/verify these locally so the PR lands green the first time. ALSO fires the moment you add or rename ANY `SESSION-HANDOVER-*.md` (a handover doc is a drift source the rest of this trigger omitted, and it is the one that has bitten most): a handover edit silently drifts `data/knowledge_graph/CURRENT.json` (the pointer globs handovers), and `docs/00-Index/Agent-Index.md` is a PROJECTION of that pointer (it never scans handovers itself) — so regenerate the pointer FIRST (`build_current_pointer.py`), the Agent-Index SECOND (`generate_obsidian_views.py`); the reverse order rebuilds the index from a stale pointer and stays red. Handover slugs are lowercase-only in the pointer's glob — an uppercase slug is silently INVISIBLE to the read-first chain, with no error. ALSO fires the moment `build_skill_index.py --check` reports the skill-index STALE, when `validate-core`/`test_skill_meta_tools` fails on `skills/registry/index.json`, or you are about to regenerate that index: NEVER regenerate it on an unlocked checkout (bakes decrypted descriptions in → fails CI's byte-compare AND leaks encrypted-skill IP) — a local unlocked STALE while CI is green is the git-crypt view asymmetry, NOT real drift. See Notes for the keyless-clone regen. ALSO fires the instant you edit `.gitattributes` in ANY way — adding/removing a `filter=git-crypt` path (encrypting a doc/skill), a `!filter !diff` plaintext exception, or an LFS rule: `.cbmignore` is DERIVED from `.gitattributes`, so the `guard` job's `derive_cbmignore.py --check` goes red until you run `python tools/cbm/derive_cbmignore.py` and commit it in the SAME PR. Encrypting a file also means regenerating the skill index (keyless) if it was a skill.
|
| metadata | {"short-description":"Pre-push guard: regenerate generated artifacts + pass the measurement-contract gates"} |
CI artifact-drift guard
See also spark-cluster-ops §4: agi-proof/benchmark-results/rlvr.public-report.json is a
TRACKED run-result that every RLVR run rewrites and is NOT gitignored — never stage it;
git checkout -- it to unblock a shared-branch rebase. (A never-stage tracked file, distinct from
the regenerate-from-source artifacts below.)
In this repo several files are generated from a source of truth and a CI gate fails closed if
the committed copy drifts. The other recurring red is the measurement (no-overclaim) contract.
Both are deterministic and reproducible locally — run them before you push so CI is green first try.
1. The measurement contract (fast-ci runs this on every PR)
make claim-check
This list was WRONG until 2026-07-15 — it named 6 of the 10 steps, silently omitting
lint_recipe, build_tool_disclosure --check, leiden_receipt --check and
task_board.py render --check. Two sessions then debugged a claim-check failure while
holding a summary that said the failing step could not exist, and one of them "confirmed" it
with a truncated grep -A6 that stopped at 6 of 10 — a self-inflicted false negative that
AGREED with this doc, so two independent stale sources corroborated each other. A summary of
a gate is not the gate. When make claim-check fails, read Makefile and the actual failing
line; do not reason from this block, or from a grep -A<n> whose window you did not verify
covers the whole target. Keep this list byte-honest with the Makefile or delete it.
make claim-check-fast is the lighter subset the pre-commit hook runs. Install the hook once with
make hooks (sets core.hooksPath=.githooks). Never lower a gate threshold to force a pass —
if a gate flips to NO-GO, that is a real result: label it candidate and add a failure-ledger entry.
2. Generated-artifact drift gates (ci.yml fails on a stale committed copy)
Regenerate (drop --check/--verify) only what your change actually touched, then re-run the
check form to confirm it's clean. If you changed none of these sources, just run the checks.
| If you changed… | Regenerate | Verify (CI form) |
|---|
agi-proof/benchmark-results/published-results.json | python tools/build_results_page.py | python tools/build_results_page.py --check |
wiki sources / wiki/ | python tools/wiki_sync.py sync | python tools/wiki_sync.py check |
| corpus / RAG inputs | python tools/build_rag_index.py --local | python tools/build_rag_index.py --verify |
| training pack inputs | python tools/build_local_sophia_dataset.py | python tools/build_local_sophia_dataset.py --check |
VERSION / version refs | — | python tools/check_version_consistency.py |
agi-proof/failure-ledger.md | (edit by hand) | python tools/validate_failure_ledger.py --check |
| provenance export demo | python tools/export_prov.py --demo | python tools/export_prov.py --demo --check |
.claude/skills/*/SKILL.md (new/changed agent skill) | python tools/build_skill_index.py (locked clone — see Notes) | python tools/build_skill_index.py --check |
.gitattributes skill/crypt exemptions | python tools/cbm/derive_cbmignore.py | python tools/cbm/derive_cbmignore.py --check |
tests/test_*.py added to .github/workflows/ | — | python tools/lint_script_tests.py |
The deterministic builders below are wired as fast-ci's "Artifact drift gate
(P7)" step (PR #949) — they rebuild the artifact in memory and byte-compare to the
committed copy, so a drifted artifact fails CI even though ci.yml only runs them
without --check. The P7 step runs on EVERY PR (the fast job has no changes
filter / label gate), so these gates also catch drift on artifact-only ledger PRs that
skip validate-core — the exact hole that let CURRENT.json / Agent-Index.md
silently fall behind ledger edits until a code PR ate the red. Same regenerate/verify
shape as the rows above.
| If you changed… | Regenerate | Verify (CI form) |
|---|
agi-proof/benchmark-results/prosoche/focus-frontier-battery.json (Focus-Efficiency-Frontier task battery + decontam receipt) | python tools/build_focus_battery.py | python tools/build_focus_battery.py --check |
docs/math-physics/index.html (math/physics verification dashboard) | python tools/build_math_physics_dashboard.py | python tools/build_math_physics_dashboard.py --check |
training/prosoche/attention_sft.jsonl (goal-anchored attention SFT pack) | python tools/build_prosoche_sft.py | python tools/build_prosoche_sft.py --check |
data/wisdom_provenance_benchmark/heldout_v1.jsonl (wisdom provenance-behavior benchmark) | python tools/build_wisdom_provenance_benchmark.py | python tools/build_wisdom_provenance_benchmark.py --check |
data/knowledge_graph/CURRENT.json (machine-readable state pointer; drifts on any ledger / handover / knowledge-graph edit) | python tools/build_current_pointer.py | python tools/build_current_pointer.py --check |
docs/00-Index/Agent-Index.md (shared agent-facing index; derived from CURRENT.json + intent spine) | python tools/generate_obsidian_views.py | python tools/generate_obsidian_views.py --check |
RESULTS.md is generated — never hand-edit it. To change a result, edit
published-results.json and re-run build_results_page.py.
2a. The recurring evidence-manifest.json merge conflict (root cause + resolution)
Every ledger PR regenerates agi-proof/evidence-manifest.json from the ledger, so two
concurrent ledger PRs always collide on it — and JSON cannot union-merge. This bit
PRs #970/#959/#984 repeatedly. It is a generated file with ~6 runtime consumers
(wiki_coupling_gate, mine_evidence_edges, run_hidden_eval_sophia, bench,
run_long_horizon, build_web_data), so it must stay committed — do not gitignore it.
3. One-shot pre-push sweep
python -m compileall -q agent okf tools agi-proof 2>/dev/null
python tools/lint_script_tests.py
make claim-check
python tools/build_skill_index.py --check
python tools/cbm/derive_cbmignore.py --check
python tools/build_results_page.py --check && \
python tools/wiki_sync.py check && \
python tools/build_rag_index.py --verify && \
python tools/build_local_sophia_dataset.py --check && \
python tools/check_version_consistency.py && \
python tools/validate_failure_ledger.py --check && \
python tools/build_focus_battery.py --check && \
python tools/build_math_physics_dashboard.py --check && \
python tools/build_prosoche_sft.py --check && \
python tools/build_wisdom_provenance_benchmark.py --check && \
python tools/build_current_pointer.py --check && \
python tools/generate_obsidian_views.py --check && echo "ARTIFACTS CLEAN — safe to push"
If any line fails, regenerate that one artifact (table above), re-run, then push.
4. Running these gates in a THROWAWAY tree — two FALSE-RED traps
git archive origin/main | tar -x -C <dir> is a good way to run the gates without a worktree (it
sidesteps the git worktree + git-crypt fatal entirely — see spark-cluster-ops). But several
gates shell out to git, so a tree whose .git is missing or broken reports failures that are
not in your diff. Both of these cost a session real time on 2026-07-26:
git add -A fails SILENTLY without git-lfs on PATH. .gitattributes declares LFS filters,
so staging dies with git-lfs filter-process: git-lfs: command not found / fatal: the remote end hung up unexpectedly — invisible if you redirected stderr. The tree then has zero tracked
files. tools/task_board.py reads git ls-files and fails OPEN when git is ABSENT but not
when git ANSWERS EMPTY, so it drops every row and reports
TASK-BOARD.md is STALE vs tasks/board. That STALE is a lie, and the perverse consequence is
that no .git at all passes while a broken .git fails — "it passed before I ran git init"
is not reassurance.
Diagnostic: git ls-files | wc -l (expect ~11k, not 0). Fix: export PATH="$HOME/bin:$PATH"
(where git-lfs lives on this Mac) before git add.
Never commit a TASK-BOARD.md regenerated in that state — it deletes real rows.actionlint cannot find .github/actionlint.yaml without a .git. It locates the project
root via git, so in a bare extract every self-hosted label false-flags
(label "spark" is unknown, label "aarch64" is unknown). Those labels are declared and
legitimate. Pass the config explicitly: actionlint -config-file .github/actionlint.yaml <file>.
Before believing ANY red from a throwaway tree, re-run that gate on a pristine extract of
origin/main. If main fails it too, it is inherited, not yours — the same discipline as the
cancelled-vs-failure check in git-discipline §3.
Notes
skills/registry/index.json must be regenerated from a LOCKED view. The committed
index carries "(encrypted — unlock to index)" placeholder rows for git-crypt skills;
CI runs locked and rebuilds those placeholders. Regenerating on an UNLOCKED checkout
bakes the decrypted descriptions in — which (a) fails validate-core's
test_skill_meta_tools byte-compare on CI and (b) LEAKS encrypted-skill IP into a
plaintext artifact. When unlocked, regenerate via a keyless clone:
git clone --no-hardlinks . /tmp/locked && (cd /tmp/locked && python tools/build_skill_index.py) && cp /tmp/locked/skills/registry/index.json skills/registry/.
Corollary: a local unlocked build_skill_index.py --check that says STALE while CI is
green is the SAME asymmetry, not real drift.- The required CI checks are
fast (fast-ci.yml) and ci-complete (ci.yml). Both must be
green for main-protection to allow a merge.
- A
test/ci-complete shown as failed may actually be a cancelled run (main moved mid-run).
Confirm the job conclusion before treating it as a real failure — see the git-discipline skill.
