| name | submission-review |
| description | Use immediately before finishing a benchmark task to verify the final solution is reproducible, clean, committed if required, and does not rely on temporary files, original executables, hidden artifacts, or accidental environment state. |
| tags | submission, review, programbench, final-check, reproducibility |
Submission Review
Use this skill immediately before final answer or benchmark submission.
The goal is to catch zero-score mistakes after the implementation appears correct.
Required Checks
Run a final build check. Do not create or update /tmp/original-executable during final review; if differential testing still needs an oracle, it should already have been saved before implementation. This check assumes ./executable is the candidate generated by the solution.
git status --short
test -f compile.sh && echo compile.sh-present
if test -f compile.sh; then
sh ./compile.sh > /tmp/final-build.out 2> /tmp/final-build.err
build_code=$?
else
echo "compile.sh-missing" > /tmp/final-build.out
: > /tmp/final-build.err
build_code=0
fi
echo "build_exit=$build_code"
tail -80 /tmp/final-build.out
tail -120 /tmp/final-build.err
test -e ./executable && echo executable-present
test -x ./executable && echo executable-executable
Then run one smoke command relevant to the task:
timeout 5 ./executable >/tmp/final-run.out 2>/tmp/final-run.err; run_code=$?
echo "run_exit=$run_code"
tail -40 /tmp/final-run.out
tail -40 /tmp/final-run.err
If no-args intentionally fails, the smoke command should still confirm expected stderr/exit behavior.
Review Checklist
Check:
compile.sh exists when required
compile.sh creates the final artifact from repo files
./executable exists after compile.sh
./executable has execute permission
solution does not call the original executable
solution does not depend on /tmp files
solution does not use absolute local paths
solution does not require network access
temporary probes/fuzz/log files are not included
only intended files are changed
important smoke or regression tests pass
commit exists if the benchmark expects a commit
Search for accidental dependencies:
rg -n "/tmp/original|original-executable|/home/|/workspace/|TODO|FIXME|apt-get|pip install|npm install" . || true
Review generated wrappers/scripts:
sed -n '1,200p' compile.sh
file ./executable 2>/dev/null || true
head -40 ./executable 2>/dev/null || true
Commit Check
If the environment requires committing:
git status --short
git diff --stat
git log --oneline -1
Commit only intended files. Do not include temporary outputs, fuzz inputs, or logs.
Final Report
Return:
Submission review:
- build reproducible: yes/no
- executable present: yes/no
- smoke test: pass/fail and command
- original dependency check: pass/fail
- temp file check: pass/fail
- git status:
- remaining risk:
