Skip to main content

verify-decoder

Audit a predecoder checkpoint against holdout seeds. Runs independent_eval (3 fair-baseline guards) and interprets borderline cases with LLM reasoning. Use when a round produces a promising Δ_LER and the user wants to confirm it is not a reward-hacking artifact.

معلومات المصدر

المستودع
qualit527/qec-ai-decoder
آخر نشاط في المصدر
٢٢ أبريل ٢٠٢٦ في ٠٥:٥٨
لغة SKILL.md المكتشفة
الإنجليزية
النجوم
٣
التفرعات
٠

خيارات التثبيت

يُحدَّد Prompt الذي يراجع المصدر أولًا بشكل افتراضي. يمكنك التبديل إلى أمر مباشر أو تنزيل نسخة محلية.

مراجعة ملفات المصدر

اقرأ SKILL.md وأي ملفات مرافقة يعرضها SkillsMP قبل أن تقرر التثبيت.

عرض SKILL.md

SKILL.md
تعليمات المصدر · معاينة للقراءة فقط
name
verify-decoder
description
Audit a predecoder checkpoint against holdout seeds. Runs independent_eval (3 fair-baseline guards) and interprets borderline cases with LLM reasoning. Use when a round produces a promising Δ_LER and the user wants to confirm it is not a reward-hacking artifact.
# /verify-decoder ## When to use - An AutoQEC round produced `delta_ler > 0` and the user wants final sign-off. - User asks to "verify this checkpoint" or "audit this round". ## Inputs - `round_dir`: path to `runs/<id>/round_N/` (must contain `checkpoint.pt` + `config.yaml`) - `env_yaml`: env used by the round (defaults to `config.yaml`'s `env_name`) ## Behavior 1. Run `python -m cli.autoqec verify <round_dir> --env <env_yaml>`. 2. Read `verification_report.json` + `training.log`. 3. LLM-reason over: - If `verdict=VERIFIED`: write one paragraph noting key confidence intervals and whether delta is within ablation threshold. Output "APPROVED". - If `verdict=SUSPICIOUS`: read training.log, the DSL config, the history of previous rounds; diagnose whether this is (a) genuine small improvement, (b) overfitting, (c) reward-hacking. Recommend next action: accept, re-run with more shots, or reject. - If `verdict=FAILED`: produce a diagnostic report matching the failure pattern (seed leak, ablation failure, negative delta). Archive into `round_N/failure_diagnosis.md`. ## Tool-use rules - Read: training.log, config.yaml, verification_report.md, previous round metrics.json files. - Bash: `python -m cli.autoqec verify ...` only. No other commands. ## Output - Short decision block: APPROVED / SUSPICIOUS_KEEP / SUSPICIOUS_REJECT / FAILED_REJECT. - Diagnostic paragraph saved to `round_N/decision.md`.
عرض على GitHub