Plan and execute code changes using a spec-driven workflow with structured documents. Use when making non-trivial changes (multi-file, multi-step, or behavior-impacting) that need structured planning before implementation. Not for trivial single-file edits.
Create and manage changesets for versioning with @changesets/cli. Use when making changes that need a changelog entry, preparing a release, or managing package versions.
Diagnose and fix failed benchmark runs. Use when a benchmark run fails, scores are unexpected, judges show errors, or traces are incomplete.
Build the pnpm monorepo correctly respecting inter-package dependencies. Use when building, verifying changes, or troubleshooting build failures.
Write and run tests using Node.js built-in test runner (node:test) with .mjs files. Use when creating new tests, debugging test failures, or understanding test infrastructure.
Understand the vanilla JS web-report app structure, patterns, and contribution guidelines. Use when modifying the UI, adding new views, or fixing frontend bugs.
Add a new AI coding agent adapter so it can participate in benchmarks. Use when integrating any new CLI-based coding tool (Cursor-like, Copilot alternative, open-source agent, etc.) into the adapter system.
Add a new evaluation judge type so task packs can use it to verify agent correctness. Use when creating a new check, custom validator, or specialized assertion.