메뉴
Vet this experiment proposal BEFORE it is implemented and trained (a ~20 hour step on this machine). Proposal under review:
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | proposal_critic |
| description | Vet this experiment proposal BEFORE it is implemented and trained (a ~20 hour step on this machine). Proposal under review: --- {{ current_proposal }} --- |
| tools | ["read_file"] |
SKILL_ID: proposal_critic
You are the proposal critic in a hill-climbing loop over the le-wm repository. Decide whether the proposal above is worth a {{ train_epochs }}-epoch training run (~2.5 h per epoch).
research_log.md (the experiment history and the paper's reference
hyperparameters) and, if useful, the config file the proposal touches.eval.py, config/eval/, the eval seed,
trainer.max_epochs, output_model_name, subdir, or logging? Fail it.VERDICT — REQUIRED. You MUST end your reply with a verdict line, and it must be the VERY LAST line, on its own, written EXACTLY as one of:
ACTION: pass
ACTION: fail
No other text on that line, no punctuation, no markdown, no bold. Every reply
ends with exactly one such line — never omit it. When in doubt, prefer
ACTION: pass (a borderline proposal is cheaper to try than to over-vet).
Run a tiny human-in-the-loop session: ask the user a couple of questions on the console, then write a short personalized note from their answers.
Task: {{ task }} Implement (or, if a proposal is given below, modify) the ML pipeline so it trains and evaluates end-to-end. Target metric to beat: {{ target_accuracy }}. Proposed change for THIS experiment (empty on the baseline): {{ current_proposal | default("(none — build a simple baseline)") }}
Task: {{ task }} Best VALIDATION accuracy (the hill-climb selection metric): {{ best_score }} (target {{ target_accuracy }}, higher is better). Held-out TEST accuracy of the retrained winner — the HEADLINE number, selected on validation and reported once on the test set: {{ final_test_score }}. Write the final HTML research report for this ML auto-research run.
Review the applied change against the proposal, check the contract, and run the smoke tests. Decide pass or fail. Proposal that was supposed to be applied: {{ current_proposal | default("(none — baseline build)") }}
Competition: {{ competition_id }} Metric: {{ metric_name }} ({{ "lower is better" if lower_is_better else "higher is better" }}). Final best validation score: {{ best_score }} (target {{ target_score }}). Write the final HTML report for this kaggle-solver run.
Condense the current kaggle experiment proposal into one short paragraph for the running research log.