failure-triage

name	failure-triage
description	Triage and analyze Playwright test failures. Use when: test failures need diagnosis, cross-pattern analysis of shared root causes, categorizing UI vs API failures, reading test-results/ error reports, proposing fixes for broken tests.

Failure Triage

Analyze test failures from test-results/, categorize them, identify shared root causes, and produce a structured analysis report with proposed fixes.

When to Use

• After a test run produces failures (test-results/ contains error folders)
• When diagnosing why Playwright tests are failing
• When proposing fixes for broken test code

Procedure

Step 1: Discover Failed Tests

1. Read all subdirectory names under test-results/
2. Each folder represents one failed test — pattern: {spec}.{describe}-{testname}-{browser}/
3. Extract the spec file name and test name from each folder name
4. Read .last-run.json for execution metadata

Step 2: Read Error Reports

Minimise token usage — use the tiered reading strategy below.

Tier 1 — Digest-first (preferred)

If test-results/failure-digest.json exists (generated by the SessionStart hook), read it instead of opening any error-context.md files. The digest contains all fields needed for steps 3–5:

• testName, location, errorType, errorMessage, folder

Skip to Step 3 immediately after reading the digest.

Tier 2 — Selective file reads (fallback)

Only open individual error-context.md files when the digest is absent or when snapshot inspection is required for a UI TimeoutError/locator failure. Apply the reading rules from .github/instructions/error-context-reading.instructions.md:

Failure type	Sections to read	Sections to skip
API (any)	# Test info, # Error details	# Instructions, # Test source, # Page snapshot
UI — AssertionError / TypeError	# Test info, # Error details	# Instructions, # Test source, # Page snapshot
UI — TimeoutError (locator)	# Test info, # Error details, first 30 lines of # Page snapshot	# Instructions, # Test source

Never read # Test source from error-context.md — open the actual spec file directly if source is needed.

Step 3: Categorize Failures

Split into two groups:

Category	Spec Files	Apply Rules
UI	auth.broken.spec.ts, tasks-ui.broken.spec.ts	Locator, Async/Await, Assertion, Dropdown, UI-specific
API	tasks-api.broken.spec.ts	Assertion, API Diagnostics

Step 4: Cross-Pattern Analysis (Rule 6)

Before proposing individual fixes, sweep ALL failures at once:

1. Scan beforeEach/beforeAll — bugs here cascade to every test in the block; fix first
2. Identify duplicate selectors — search all test files for the same broken selector string; fix all occurrences simultaneously
3. Batch API fixes — if response shape is wrong in one API test, check all other API tests for the same structural assumption
4. Group by root cause — present fixes grouped by root cause, not by test
5. Estimate blast radius — for each root cause, list all affected tests before proposing the fix

Common shared patterns to detect:

• Shared Selector Bug — wrong locator in beforeEach breaks all tests in that describe block
• Shared URL Assumption — multiple tests assert the same wrong redirect URL
• API Response Shape — same body.data.tasks path appears in multiple API tests
• Hardcoded IDs — multiple tests use static IDs (t1, t3) that no longer exist
• HTTP Method Mismatch — patch/post/get/delete vs what endpoints actually accept

Step 5: Propose Fixes

For each failure, reference the detailed analysis rules to diagnose and propose a fix.

Step 6: Generate Analysis Report

Produce a structured report in this format:

## Test Failure Analysis Summary

**Total Tests**: X | **Failed**: Y | **Passed**: Z | **Iteration**: N of MAX_ITERATIONS

---

### Cross-Pattern Analysis (Shared Root Causes)

| Root Cause | Affected Tests | Fix Strategy |
|------------|---------------|--------------|
| ... | ... | ... |

---

### UI Test Failures

#### [TC-UI-XXX] Test Name
- **File**: use-case-2/tests/filename.spec.ts:line
- **Error Type**: TimeoutError | AssertionError | TypeError
- **Root Cause**: Brief explanation
- **Shared With**: [list other tests with same root cause, or "unique"]
- **Impact**: Critical | High | Medium | Low
- **Proposed Fix**: code snippet with before/after
- **Confidence**: High | Medium | Low

---

### API Test Failures

#### [TC-API-XXX] Test Name
- **File**: use-case-2/tests/filename.spec.ts:line
- **Error Type**: HTTP 4XX | TypeError | AssertionError
- **Root Cause**: Brief explanation
- **Shared With**: [list other tests with same root cause, or "unique"]
- **Impact**: Critical | High | Medium | Low
- **Proposed Fix**: code snippet with before/after
- **Confidence**: High | Medium | Low

---

## Recommended Actions
1. [Fix shared root cause X — resolves N tests]
2. [Fix UI-specific issue Y]
3. [Fix API-specific issue Z]

**Ready to apply fixes?** (yes/no)