| name | review-questions |
| description | Adversarially review an Anthropic University question bank for correctness, ambiguity, duplication, and fairness. Use before committing question changes or when auditing an existing content/<exam>/questions.json. |
| argument-hint | <exam-id> [domain-id] |
| allowed-tools | Read, Grep, Glob, Edit |
Review a question bank
Audit the questions in content/$ARGUMENTS/questions.json (optionally scoped to one domain). Your
job is to catch wrong keys, ambiguity, and unfair items before they reach a learner.
Verify each question against the docs
For every question, confirm the keyed answer is actually correct using the claude-code-docs MCP
server and the claude-api skill. Flag any key you cannot substantiate from official documentation.
Common things to double-check: stop_reason values, batch API (50% cost, up to 24h), -p/--print,
--output-format json, .claude/rules/ glob scoping, skill frontmatter (context: fork,
allowed-tools, argument-hint), MCP .mcp.json scopes and ${ENV} expansion, prompt caching, and
plugin/marketplace layout.
Checklist (report findings, most severe first)
- Wrong key — the keyed answer is not the best/only correct one, or another option is also correct.
- Ambiguous — two options are defensibly correct, or the stem is under-specified.
- Tell — the correct answer is obvious from length, specificity, or grammar rather than knowledge.
- Weak distractors — a wrong option is so implausible it gives the answer away.
- Duplication — near-identical stem/scenario to another question.
- Schema/tag issues — bad
domain, missing studyArea, empty explanations.
- Fairness/scope — tests trivia or an undocumented detail rather than a real concept.
Output
List each finding as: id — category — one-line problem — suggested fix. Offer to apply fixes. When
done, run node scripts/build.mjs --check and confirm it passes.
