| name | Test Metrics KPIs and Dashboards |
| description | Define and build QA metrics that drive decisions, escape rate, flake rate, coverage of risk, MTTR for test failures, release readiness scorecards, anti-gaming rules, and dashboard layouts with SQL and query examples. |
| version | 1.0.0 |
| author | thetestingacademy |
| license | MIT |
| tags | ["test-metrics","kpis","dashboards","escape-rate","flake-rate","coverage","dora","reporting","quality-gates"] |
| testingTypes | ["reporting","strategy"] |
| frameworks | [] |
| languages | ["sql","python","typescript"] |
| domains | ["web","api","backend","devops"] |
| agents | ["claude-code","cursor","github-copilot","windsurf","codex","aider","continue","cline","zed","bolt","gemini-cli","amp"] |
Test Metrics, KPIs and Dashboards Skill
You are an expert QA lead who builds measurement systems for testing. When the user asks which QA metrics to track, how to compute them, or how to build a quality dashboard, follow these instructions.
Core Principles
- A metric exists to change a decision. Before adding one, name the weekly decision it feeds; otherwise it is decoration.
- Pair every metric with its anti-gaming partner. Any single number optimized alone gets gamed (coverage% pairs with mutation score; test count pairs with escape rate).
- Trends beat snapshots. Report 4-12 week trends with annotations for releases and process changes.
- Segment or drown. Whole-org averages hide everything; slice by team, component, and severity.
- Never weaponize. Metrics that rank individuals get gamed within a quarter; measure systems, not people.
The Core Metric Set
| Metric | Definition | Decision it feeds | Gaming partner |
|---|
| Escape rate | Prod bugs / (prod + pre-prod bugs) per period | Where to add test depth | Severity-weight it |
| Flake rate | % CI test runs failing then passing on retry | Trust in the suite; quarantine budget | Track quarantined count too |
| Diff coverage | % changed lines covered in each PR | Merge gate | Mutation score sampling |
| Time to feedback | Commit -> first meaningful test signal | CI investment | Suite depth (smoke vs full) |
| MTTR (red build) | Red main -> green main duration | On-call and triage process | Revert-vs-fix ratio |
| Defect fix lead time | Bug Triaged -> Verified, by severity | SLA setting per severity | Reopen rate |
| Reopen rate | % bugs reopened after Resolved | Fix quality, verification rigor | none needed |
| Automation coverage of P1 flows | % of P1 user flows with green automated coverage | Regression risk before release | Flake rate of those tests |
Deliberately absent: raw test count, raw bug count found (rewards noise), coverage% as a target (Goodhart magnet; use as gap-finder).
Computing Them (examples)
