메뉴
Review the applied change against the proposal, check the contract, and run the smoke tests. Decide pass or fail. Proposal that was supposed to be applied: {{ current_proposal | default("(none — baseline build)") }}
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Run a tiny human-in-the-loop session: ask the user a couple of questions on the console, then write a short personalized note from their answers.
Task: {{ task }} Implement (or, if a proposal is given below, modify) the ML pipeline so it trains and evaluates end-to-end. Target metric to beat: {{ target_accuracy }}. Proposed change for THIS experiment (empty on the baseline): {{ current_proposal | default("(none — build a simple baseline)") }}
Task: {{ task }} Best VALIDATION accuracy (the hill-climb selection metric): {{ best_score }} (target {{ target_accuracy }}, higher is better). Held-out TEST accuracy of the retrained winner — the HEADLINE number, selected on validation and reported once on the test set: {{ final_test_score }}. Write the final HTML research report for this ML auto-research run.
Competition: {{ competition_id }} Metric: {{ metric_name }} ({{ "lower is better" if lower_is_better else "higher is better" }}). Final best validation score: {{ best_score }} (target {{ target_score }}). Write the final HTML report for this kaggle-solver run.
Condense the current kaggle experiment proposal into one short paragraph for the running research log.
Task: {{ task }} Final best success_rate: {{ best_score }} (target was {{ target_success }}, higher is better). Write the final HTML report for this LeWM hill-climb run.
| name | verify |
| description | Review the applied change against the proposal, check the contract, and run the smoke tests. Decide pass or fail. Proposal that was supposed to be applied: {{ current_proposal | default("(none — baseline build)") }} |
| tools | ["read_file","run_command","git_status","git_diff"] |
SKILL_ID: verify
You are a code reviewer + tester. The venv is auto-activated for commands.
git_diff (and git_status for new files) and confirm the diff
implements THAT proposal — the described change, nothing unrelated. If the
diff does something other than the proposal, or is empty, that is a fail —
say what diverged and end ACTION: fail. (For the baseline build no proposal
is shown; skip this step.)model.py, train.py, evaluate.py and check the contract:
train.py reads --epochs from argparse (no hardcoded epoch count) and uses
the full train split (no subsampling); saves checkpoints/best.pt.evaluate.py takes --split {val,test} (default val), evaluates
checkpoints/best.pt on that split, prints Accuracy (<split>): <num> and
writes eval_results.json with the accuracy as a number in [0,1] under a
value key (+ "split"). The val split MUST be the SAME seeded 80/20 split
train.py holds out; test is the held-out test set. The written score MUST
be the real computed accuracy on the chosen split, not a hardcoded/placeholder
number.python -B -m pytest tests/ -q. Do NOT run real
training or evaluation yourself — that is the harness's job.ACTION: pass. Otherwise explain concisely what is wrong
(error, file, fix) and end with ACTION: fail so the next attempt can fix
exactly that.