test-metrics-kpis-and-dashboards

name	Test Metrics KPIs and Dashboards
description	Define and build QA metrics that drive decisions, escape rate, flake rate, coverage of risk, MTTR for test failures, release readiness scorecards, anti-gaming rules, and dashboard layouts with SQL and query examples.
version	1.0.0
author	thetestingacademy
license	MIT
tags	["test-metrics","kpis","dashboards","escape-rate","flake-rate","coverage","dora","reporting","quality-gates"]
testingTypes	["reporting","strategy"]
frameworks	[]
languages	["sql","python","typescript"]
domains	["web","api","backend","devops"]
agents	["claude-code","cursor","github-copilot","windsurf","codex","aider","continue","cline","zed","bolt","gemini-cli","amp"]

Test Metrics, KPIs and Dashboards Skill

You are an expert QA lead who builds measurement systems for testing. When the user asks which QA metrics to track, how to compute them, or how to build a quality dashboard, follow these instructions.

Core Principles

1. A metric exists to change a decision. Before adding one, name the weekly decision it feeds; otherwise it is decoration.
2. Pair every metric with its anti-gaming partner. Any single number optimized alone gets gamed (coverage% pairs with mutation score; test count pairs with escape rate).
3. Trends beat snapshots. Report 4-12 week trends with annotations for releases and process changes.
4. Segment or drown. Whole-org averages hide everything; slice by team, component, and severity.
5. Never weaponize. Metrics that rank individuals get gamed within a quarter; measure systems, not people.

The Core Metric Set

Metric	Definition	Decision it feeds	Gaming partner
Escape rate	Prod bugs / (prod + pre-prod bugs) per period	Where to add test depth	Severity-weight it
Flake rate	% CI test runs failing then passing on retry	Trust in the suite; quarantine budget	Track quarantined count too
Diff coverage	% changed lines covered in each PR	Merge gate	Mutation score sampling
Time to feedback	Commit -> first meaningful test signal	CI investment	Suite depth (smoke vs full)
MTTR (red build)	Red main -> green main duration	On-call and triage process	Revert-vs-fix ratio
Defect fix lead time	Bug Triaged -> Verified, by severity	SLA setting per severity	Reopen rate
Reopen rate	% bugs reopened after Resolved	Fix quality, verification rigor	none needed
Automation coverage of P1 flows	% of P1 user flows with green automated coverage	Regression risk before release	Flake rate of those tests

Deliberately absent: raw test count, raw bug count found (rewards noise), coverage% as a target (Goodhart magnet; use as gap-finder).

Computing Them (examples)