Grade a backend corrective (debugging) benchmark submission under submissions/be-corrective/ by running each question's Node check.mjs (binary 10/0). Use when asked to evaluate, score, or grade a be-corrective submission.
Grade a data-analyst constructive (build-from-scratch) benchmark submission under submissions/da-constructive/ against the hidden reference answers and rubrics, producing per-question and overall scores out of 10. Use when asked to evaluate, score, or grade a…
Grade a data-analyst corrective (find-the-mistakes) benchmark submission under submissions/da-corrective/ - run the scripted Q1 HTML checker (10/0), the scripted Q2 verify.py (net counting), and LLM-match the Q3 mistakes.md entries to the MISTAKES keys. Use…
Grade a frontend corrective (debugging) benchmark submission under submissions/fe-corrective/ by running each question's Playwright check.mjs (binary 10/0). Use when asked to evaluate, score, or grade a fe-corrective submission.
Take the backend corrective (debugging) capability benchmark - create a submission folder named <llm>+<harness> under submissions/be-corrective/ and fix all nine broken Node backends (api/db/service × easy/mid/hard). Use when asked to take the be-corrective…
Take the data-analyst constructive (build-from-scratch) capability benchmark - create a submission folder named <llm>+<harness> under submissions/da-constructive/ and produce all three deliverables (offline dashboard, CSV analysis, architecture design) from…
Take the data-analyst corrective (find-the-mistakes) capability benchmark - create a submission folder named <llm>+<harness> under submissions/da-corrective/ and answer all nine corrective questions across three types (fix a broken HTML dashboard, find the…
Take the frontend corrective (debugging) capability benchmark - create a submission folder named <llm>+<harness> under submissions/fe-corrective/ and fix all nine broken frontend apps (web/react/app-shell × easy/mid/hard). Use when asked to take the…