메뉴
For an accepted improvement, attribute the gain to the specific component via ablation, and confirm it is real (not noise) with multi-seed re-runs, before trusting or shipping it. Use after ratchet-experiment keeps a change.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Run a disciplined keep/revert experiment loop against a frozen scalar evaluator. Use when iterating on a model, algorithm, or piece of code to find improvements: propose a change, run a bounded experiment, score it, and keep the change only if it improves the metric (otherwise revert). Requires a frozen evaluator that emits a single scalar plus hard-constraint vetoes.
Handle the two situations the greedy ratchet cannot: route safety-critical or high-risk changes to human review, and switch search strategy when the loop plateaus (greedy local search -> grid / tree / evolutionary). Use from inside a ratchet-experiment loop.
Read a paper or product writeup (and its key references) and distill it into a structured, implementable spec: the core mechanism, the claimed metric, a minimal reproduction, and the constraints. Use at the start of adopting a community idea, before reproducing or grafting it.
Integrate a confirmed idea into an existing large codebase behind tests — the step generic paper-to-code tools skip. Use after repro-harness, before validating with ratchet-experiment.
Confirm a distilled idea actually works in isolation, against a frozen evaluator, before grafting it into a real codebase. Guards against fabricated or hardcoded results. Use after distill-paper and before graft-change.
| name | attribute |
| description | For an accepted improvement, attribute the gain to the specific component via ablation, and confirm it is real (not noise) with multi-seed re-runs, before trusting or shipping it. Use after ratchet-experiment keeps a change. |
Status: v0 scaffold — unverified. Behavior not yet validated end-to-end in this repo. See
../../STATUS.md.
A kept change is not yet an understood change. Before trusting a win, find out what
caused it and whether it survives noise (guards against overfitting / lucky runs —
see ../../docs/pitfalls.md).
An attribution report: which component caused the gain, the effect size, and its seed-stability — attached to the experiment in the ledger. If the gain does not survive ablation or seeds, revert it.
