Author LLM/RAG/agent evaluation suites in DeepEval that prove a feature is correct with gated numbers, not vibes. Use when asked to "eval an LLM", "test a prompt", "measure RAG quality", "check for hallucination", "score answer relevancy", "verify tool calls", or gate a release on model output quality. Ships the metric definitions baked in — faithfulness, answer relevancy, contextual precision/recall/relevancy, hallucination, tool correctness, G-Eval, bias, toxicity — with the inputs each needs, the score formula, and which direction passes. Enforces guardrails against unpinned judge models, exact-matching non-deterministic output, contaminated goldens, single-run scores, and metric-picking that ignores the real failure mode. See `reference.md` for runnable DeepEval suites and CI wiring, `tooling.md` for install/versions and alternatives.
Test web UIs for accessibility to WCAG 2.2 AA with axe-core automation plus the manual keyboard, focus, and screen-reader checks automation can't catch. Use when adding a11y assertions to a suite, auditing a page or component, wiring a11y into CI, or reviewing UI for WCAG conformance. Enforces guardrails against aria-label spam, div-soup role hacks, disabling axe rules to go green, and treating an axe pass as full coverage. See `reference.md` for setup code, rule/tag config, contrast thresholds, and a manual checklist.
Design and implement API contract tests so a provider can't break its consumers. Enforces guardrails against exact-value matching, unversioned pacts, contract tests that are secretly E2E, and deploys that ship without a can-i-deploy gate. Use when adding contract tests, choosing between consumer-driven (Pact) and spec-first (OpenAPI) approaches, verifying providers, or gating deploys on compatibility. Covers REST, GraphQL, and event/message contracts. See `reference.md` for runnable tests, broker commands, and CI wiring.
Detect, quantify, classify, and fix flaky tests — compute flake rate and per-test flake scores, reproduce non-determinism, classify the root cause, and either fix at the root or quarantine with an owner + issue + SLA. Use when tests pass-on-retry, fail intermittently in CI, or someone reaches for `retries` to make the pipeline green. Enforces guardrails against masking flake with retries, deleting tests, and ownerless quarantine. See `reference.md` for runnable rerun commands, formulas, and per-cause before/after fixes.
Design and run performance tests that prove a system meets its SLOs under realistic load — backend load/stress/soak/spike with k6 and frontend web-vitals with Lighthouse. Use when asked to "load test", "check performance", "find our capacity", "set a latency budget", or gate a release on speed and scale. Enforces guardrails against reporting averages instead of percentiles, zero-think-time hammering, extrapolating from a localhost run, trusting a single run, and ignoring error rate at load. Defines SLOs first, models a realistic workload, and turns thresholds into pass/fail CI gates. See `reference.md` for a full k6 script, stress/soak variants, Lighthouse CI budgets, and metric formulas.
Author and maintain resilient Playwright end-to-end tests using the Page Object Model, fixtures, and stable, user-facing locators. Enforces guardrails against flakiness, brittle selectors, and shared state. Use when writing, reviewing, or debugging Playwright E2E specs, or when wiring E2E tests into CI. See `reference.md` for config, fixture patterns, and commands.
Produce a tailored, risk-based QA strategy for a team or project. Use when asked to "create a QA strategy", "assess our testing approach", "build a test plan/roadmap", or decide what and how much to automate. Enforces guardrails against vanity coverage targets, big-bang rewrites, and metrics with no gate. First gathers a defined set of inputs (tech stack, team size, release cadence, current maturity, risk/compliance), then writes the strategy against a consistent template. See `reference.md` for the risk-scoring rubric, metric formulas, and quality-gate examples.
Analyze a full-stack application (frontend, backend, middleware) and design a complete test pyramid — deciding exactly which tests belong in the FE, which in the BE, which at middleware/seams, and at what level (unit, integration/component, contract, E2E). Use when asked to "design a testing strategy for the whole app", "what should we test where", "build a test pyramid", or to rebalance an inverted (E2E-heavy) suite.