菜单
Assess test suite quality using Farley's 8 Properties and Tautology Theatre detection. Use when user says review tests, test quality, are my tests good, test assessment, or test design review. Not for writing tests (use language skills) or code review (use gemini-review).
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | test-design-reviewer |
| description | Assess test suite quality using Farley's 8 Properties and Tautology Theatre detection. Use when user says review tests, test quality, are my tests good, test assessment, or test design review. Not for writing tests (use language skills) or code review (use gemini-review). |
Identify all test files in scope. Use language-appropriate patterns:
*_test.gotest_*.py, *_test.py*_spec.rb*.test.ts, *.spec.tsRate each property 0-10 across the test suite. Provide evidence.
| # | Property | Question to ask | Red flags |
|---|---|---|---|
| 1 | Understandable | Can you tell what's being tested in 5 seconds? | Cryptic names, no arrange/act/assert structure, shared state |
| 2 | Maintainable | Will this break when implementation changes? | Testing private methods, brittle selectors, hardcoded values |
| 3 | Repeatable | Same result every run, any order, any machine? | Time-dependent, filesystem-dependent, test ordering, shared DB state |
| 4 | Atomic | One reason to fail? | Multiple assertions testing different behaviors, setup-heavy |
| 5 | Necessary | Does this test earn its keep? | Duplicate coverage, testing framework/language behavior |
| 6 | Granular | Pinpoints the failure location? | Coarse assertions (assert result), catch-all tests |
| 7 | Fast | Runs in milliseconds? | Real HTTP calls, sleep/wait, full DB setup per test |
| 8 | First | Written before production code? | Tests that mirror implementation structure, not behavior |
Scoring methodology:
lib/cli_calculator.py for deterministic math (JSON in, JSON out).final_property_score = 0.60 * static_score + 0.40 * llm_score per propertyPer-property scoring rubrics: anchor each 0-10 score to the full 8-properties x 5-bands matrix in references/bands.md. Rough guide: 9-10 exemplary, 7-8 strong, 5-6 mixed, 3-4 weak, 1-2 harmful.
Aggregation methodology:
Sampling for large suites:
Weighted Farley Index = (U*1.5 + M*1.5 + R*1.25 + A*1.0 + N*1.0 + G*1.0 + F*0.75 + T*1.0) / 9.0
Divisor is 9.0 (sum of weights), not 8 (number of properties). U/M weighted highest (readability, coupling); F weighted lowest (speed is contextual).
| Range | Rating | Interpretation |
|---|---|---|
| 9.0-10.0 | Exemplary | Model suite; tests serve as living documentation |
| 7.5-8.9 | Excellent | High quality with minor improvement opportunities |
| 6.0-7.4 | Good | Solid foundation with clear areas for improvement |
| 4.5-5.9 | Fair | Functional but needs significant attention to test design |
| 3.0-4.4 | Poor | Tests provide limited value; major refactoring needed |
| 0.0-2.9 | Critical | Tests may be harmful; consider rewriting from scratch |
The critical question: "Would this test still pass if all production code were deleted?"
Scan for these 4 patterns:
| Pattern | What it looks like | Example |
|---|---|---|
| Mock tautology | Test verifies that a mock returns what it was told to return | mock.return_value = 42; assert service.get() == 42 (only tests the mock) |
| Mock-only test | Every dependency is mocked, nothing real executes | Test with 5 mocks and zero real objects |
| Trivial tautology | Assertion is always true regardless of code | assert isinstance(result, dict) when function signature guarantees dict |
| Framework test | Tests framework behavior, not application logic | Testing that Rails validations work, that pytest fixtures inject |
Also scan for mock interaction anti-patterns (affect Maintainable score):
| Pattern | What it looks like |
|---|---|
| Over-specified interactions | verify with exact call counts, call ordering, verifyNoMoreInteractions |
| Testing internal details | ArgumentCaptor deep inspection, verify(never()) mirroring branches, high verify-to-assert ratio |
For each tautology or anti-pattern found: report the file, line, pattern type, and why it's problematic.
## Test Design Review
### Farley Index: X.X / 10.0 (Rating)
| Property | Static | LLM | Blended | Weight | Weighted | Key Evidence |
|----------|--------|-----|---------|--------|----------|--------------|
| Understandable | X.X | X.X | X.X | 1.50x | X.XX | ... |
| Maintainable | X.X | X.X | X.X | 1.50x | X.XX | ... |
| Repeatable | X.X | X.X | X.X | 1.25x | X.XX | ... |
| Atomic | X.X | X.X | X.X | 1.00x | X.XX | ... |
| Necessary | X.X | X.X | X.X | 1.00x | X.XX | ... |
| Granular | X.X | X.X | X.X | 1.00x | X.XX | ... |
| Fast | X.X | X.X | X.X | 0.75x | X.XX | ... |
| First (TDD) | X.X | X.X | X.X | 1.00x | X.XX | ... |
### Tautology Theatre Analysis
Each subsection always present; use "None detected." when empty.
#### Mock Tautologies
| Test Method | Line | Mock Setup | Assertion |
#### Mock-Only Tests
| Test Method | Line | Evidence |
#### Trivial Tautologies
| Test Method | Line | Assertion |
#### Framework Tests
| Test Method | Line | Assertion | What It Actually Tests |
**Summary**: {total} instances across {affected}/{total_methods} test methods.
### Top 3 Improvements
1. [Highest-impact fix targeting weakest high-weight property]
2. [Second priority]
3. [Third priority]
### Methodology Notes
- Static/LLM blend: 60/40
- Files analyzed: {count} ({sampling note})
- Language: {lang}, Framework: {framework}
This skill is invoked by the orchestrator when test files are in scope (see the orchestrator skill, review routing step). Can also be invoked directly via /test-design-reviewer.
See references/scoring.md.
See references/scoring.md.
Execution trace capture and token baselining. Use when user says trace, extract trace, count tokens, token baseline, or /harness-trace. Not for harness optimization (use harness-mechanic).
Release prep and tagging with pre-flight verification. Use when user says release, tag, ship it, push to production, create release, or bump version. Not for editing a version string outside a release, a lockfile or a changelog entry by hand, and not for reading what changed in a past release: those are ordinary edits, just do them.
Mine LEARNING.md retrospectives across all repos for recurring failure-modes, then propose harness changes (hooks, rules, skills, checklist items) with falsifiable change-contracts. Also runs the doc-gardening pass over the governance docs (stale rules, dead cross-references, skill-table drift). Use when user says learning loop, mine learnings, recurring failures, what should I fix in my process, cross-repo retrospective, doc gardening, stale docs, or run learning-loop. Runs on human schedule, never autonomously. Not for single-project retrospectives (use learning-docs) or trace/token analysis (use harness-mechanic).
Full contractor-mode loop: sub-protocols (LOCALIZE, BENCH-BASELINE, REPRODUCE, DRIFT), review routing by file pattern, blast radius, UAT, parallelism and effort caps, escalation, just-do-it, goal-backed runs. Load it to RUN that loop, once you have an approved plan and are entering step 1, or when a step needs its detail. It is a protocol, not a way to do the work: it never replaces the language, review or planning skill the task itself calls for, and a task small enough to just edit (SKIP_SET) needs none of it. The always-on spine lives in rules/orchestrator-protocol.md. Not for trace capture (harness-trace) or harness optimization (harness-mechanic).
Ruby on Rails with service-oriented architecture, Dry-validation, Sidekiq/Solid Queue, Hotwire. Use for Rails API, Rails services, Rails forms, RSpec, ActiveRecord, Rails migrations. Not for standalone Ruby gems (use ruby skill), and not for judging whether a migration is safe to run in production (that is the database-reviewer agent).
Get three independent second opinions (isolated Claude + isolated Gemini + isolated DeepSeek) on a problem Claude is analyzing. Use when user says second opinion, another perspective, challenge this approach, or asks gemini/deepseek about a decision, approach, or diagnosis. Also invoke on your own initiative, without being asked: on a complex task before planning, after 2 failed root-cause attempts, on a 🔴 decision (rewrite, core logic, security), or when the review loop is stuck below 80 after 2 fix rounds. Gathers context, writes a focused prompt, calls all reviewers in isolated Docker containers, and synthesizes all viewpoints. Not for reviewing code or changes, even phrased as 'ask gemini to review' (use gemini-review).