메뉴
Audit a predecoder checkpoint against holdout seeds. Runs independent_eval (3 fair-baseline guards) and interprets borderline cases with LLM reasoning. Use when a round produces a promising Δ_LER and the user wants to confirm it is not a reward-hacking artifact.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Use when preparing an AutoQEC demo walkthrough, advisor presentation, hackathon pitch, recorded narration, or evidence-backed explanation of why the demos matter and why their outputs are correct.
Read the artifacts of a completed AutoQEC run and propose improvements to the framework code itself — DSL gaps, weak baselines, miscalibrated safety thresholds, prompt drift, env limitations, orchestration friction. Advisory only; never edits framework files. Use after a run completes when the user asks "what should we change before the next run?" or feels the system is plateauing.
Run the AutoQEC research loop on a given env YAML. Orchestrates the autoqec-ideator / autoqec-coder / autoqec-analyst subagents via the Agent tool, invokes the Runner CLI for training + evaluation, and writes history.jsonl + pareto.json. Use when the user asks "run AutoQEC on <env>", "start a research round", or provides an EnvSpec YAML.
Read Zulip stream/topic history for project context, summarize decisions, and recover requirements or action items. Use when task context may depend on prior Zulip discussion rather than only repository files.
Inspect a stalled or failed run, identify root cause (bad hyperparameter / NaN pattern / OOM / env misconfig), and recommend a fix. Does NOT apply fixes autonomously.
Read an entire runs/<id>/log.md and assess research narrative coherence, identify stuck hypotheses, detect overfitting signs, and write a review markdown. Use after a full run completes or when a loop has been stuck for many rounds.
| name | verify-decoder |
| description | Audit a predecoder checkpoint against holdout seeds. Runs independent_eval (3 fair-baseline guards) and interprets borderline cases with LLM reasoning. Use when a round produces a promising Δ_LER and the user wants to confirm it is not a reward-hacking artifact. |
delta_ler > 0 and the user wants final sign-off.round_dir: path to runs/<id>/round_N/ (must contain checkpoint.pt + config.yaml)env_yaml: env used by the round (defaults to config.yaml's env_name)python -m cli.autoqec verify <round_dir> --env <env_yaml>.verification_report.json + training.log.verdict=VERIFIED: write one paragraph noting key confidence intervals and whether delta is within ablation threshold. Output "APPROVED".verdict=SUSPICIOUS: read training.log, the DSL config, the history of previous rounds; diagnose whether this is (a) genuine small improvement, (b) overfitting, (c) reward-hacking. Recommend next action: accept, re-run with more shots, or reject.verdict=FAILED: produce a diagnostic report matching the failure pattern (seed leak, ablation failure, negative delta). Archive into round_N/failure_diagnosis.md.python -m cli.autoqec verify ... only. No other commands.round_N/decision.md.