verify-decoder
Audit a predecoder checkpoint against holdout seeds. Runs independent_eval (3 fair-baseline guards) and interprets borderline cases with LLM reasoning. Use when a round produces a promising Δ_LER and the user wants to confirm it is not a reward-hacking artifact.
Source facts
- Repository
- qualit527/qec-ai-decoder
- Last source activity
- April 22, 2026 at 05:58
- Detected SKILL.md language
- English
- Stars
- 3
- Forks
- 0
Install options
The review-first prompt is selected by default. You can switch to a direct command or download a local copy.
Review the source files
Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.
Showing SKILL.md
SKILL.md
Source instructions · Read-only preview- name
- verify-decoder
- description
- Audit a predecoder checkpoint against holdout seeds. Runs independent_eval (3 fair-baseline guards) and interprets borderline cases with LLM reasoning. Use when a round produces a promising Δ_LER and the user wants to confirm it is not a reward-hacking artifact.
# /verify-decoder
## When to use
- An AutoQEC round produced `delta_ler > 0` and the user wants final sign-off.
- User asks to "verify this checkpoint" or "audit this round".
## Inputs
- `round_dir`: path to `runs/<id>/round_N/` (must contain `checkpoint.pt` + `config.yaml`)
- `env_yaml`: env used by the round (defaults to `config.yaml`'s `env_name`)
## Behavior
1. Run `python -m cli.autoqec verify <round_dir> --env <env_yaml>`.
2. Read `verification_report.json` + `training.log`.
3. LLM-reason over:
- If `verdict=VERIFIED`: write one paragraph noting key confidence intervals and whether delta is within ablation threshold. Output "APPROVED".
- If `verdict=SUSPICIOUS`: read training.log, the DSL config, the history of previous rounds; diagnose whether this is (a) genuine small improvement, (b) overfitting, (c) reward-hacking. Recommend next action: accept, re-run with more shots, or reject.
- If `verdict=FAILED`: produce a diagnostic report matching the failure pattern (seed leak, ablation failure, negative delta). Archive into `round_N/failure_diagnosis.md`.
## Tool-use rules
- Read: training.log, config.yaml, verification_report.md, previous round metrics.json files.
- Bash: `python -m cli.autoqec verify ...` only. No other commands.
## Output
- Short decision block: APPROVED / SUSPICIOUS_KEEP / SUSPICIOUS_REJECT / FAILED_REJECT.
- Diagnostic paragraph saved to `round_N/decision.md`.
View on GitHub