test-healer

name	test-healer
description	Runs Playwright tests and automatically fixes failures in both tests AND implementation code iteratively

Test Healer - Self-Healing Test Runner

Run Playwright tests iteratively and fix failures automatically until all tests pass or max attempts reached.

Purpose: Fix BOTH test code and implementation code to make tests pass while adhering to requirements.

Input

• Feature directory for documentation (e.g., feature-docs/003-benchmarking-system/)
• Max attempts per test (default: 10)

Database Management

CRITICAL: Reset and seed the database before running tests to ensure consistent starting state.

Reset & Seed Process

Before running any tests, execute these commands:

# From apps/backend-services directory
cd apps/backend-services

# Reset database (drops all data)
PRISMA_USER_CONSENT_FOR_DANGEROUS_AI_ACTION="yes" npx prisma migrate reset --force

# Run migrations
npm run db:migrate

# Seed test data
npm run db:seed

OR use the combined reset command:

cd apps/backend-services && PRISMA_USER_CONSENT_FOR_DANGEROUS_AI_ACTION="yes" npx prisma migrate reset --force && npm run db:seed

NOTE: switch back to project root after resetting database.

Process

Iteration Loop

0. Reset Database (First Step)

ALWAYS reset and seed database before running tests:

cd apps/backend-services && PRISMA_USER_CONSENT_FOR_DANGEROUS_AI_ACTION="yes" npx prisma migrate reset --force && npm run db:seed

This ensures test failures aren't caused by stale data from previous runs.

NOTE: switch back to project root after resetting database.

1. Run All Tests

npx playwright test --reporter=list

2. Analyze Failures

Parse test output for:

• Selector not found errors
• Timeout errors
• Assertion failures
• Navigation issues

3. Apply Fixes

CRITICAL: Before applying any fix, consult these sources in order:

1. First Priority: Read {feature-dir}/REQUIREMENTS.md

   • Verify what the EXPECTED behavior should be
   • This is the source of truth for correct behavior

2. Second Priority: Read relevant files in {feature-dir}/user-stories/

   • Check acceptance criteria
   • Verify user story expectations
   • Understand the intended user flow

3. Third Priority: Explore the actual page behavior

   • Use Playwright MCP to inspect the current page
   • Compare actual vs. expected behavior from requirements

Decision Tree: What to Fix?

Test fails → Check requirements
    ↓
Does implementation match requirements?
    ├─ NO → FIX IMPLEMENTATION CODE (app, backend, frontend)
    │        Then re-run test
    ↓
    └─ YES → Does test expect correct behavior per requirements?
               ├─ NO → FIX TEST CODE
               └─ YES → FIX TEST SYNCHRONIZATION (waits, selectors)

Common Fix Patterns:

Error Type	Investigation Steps	Fix Strategy
Locator not found	1. Check REQUIREMENTS.md for expected element 2. Re-explore page with Playwright MCP 3. Find actual selector	If element should exist per requirements but doesn't: fix implementation If selector is outdated: update test & Page Object
Assertion failed	1. Read REQUIREMENTS.md for expected value 2. Read user-stories for acceptance criteria 3. Verify which is correct: test or app	If requirements say X but app shows Y: fix the implementation to match requirements If test expectation is wrong: fix the test
Timeout exceeded	1. Check requirements for async operations 2. Explore page for loading states	If app is slow: investigate and fix performance issue If test timing is wrong: add explicit waitFor or increase timeout
Navigation failed	1. Check requirements for navigation flow 2. Verify URL patterns in user stories	If navigation is broken: fix implementation routing If test navigation is incorrect: add waitForLoadState('networkidle')
Element not visible	1. Check requirements for UI state 2. Explore page for conditional rendering	If element should be visible per requirements: fix implementation If test timing is wrong: add waitFor({ state: 'visible' })
Element detached	1. Review requirements for dynamic content	If DOM manipulation is buggy: fix implementation If test needs better synchronization: use waitForSelector before interaction
Missing test data	1. Read apps/shared/prisma/seed.ts 2. Verify seed data exists for test scenario	If test expects data that doesn't exist: update seed.ts and re-seed If test IDs don't match seed IDs: update test to use correct seed data IDs
Authentication errors	1. Verify TEST_API_KEY is set 2. Check setupAuthenticatedTest is called 3. Inspect network requests for x-api-key header	If auth setup missing: add setupAuthenticatedTest in beforeEach If backend returns 403: verify API key is correct and environment variable is set If frontend not authenticated: check localStorage tokens are injected

4. Identify Files to Fix

When fixing implementation (not just tests):

• Frontend: Components in apps/frontend/src/
• Backend: Controllers, services in apps/backend-services/src/
• Temporal: Workflows, activities in apps/temporal/src/
• Page Objects: Test helpers in tests/e2e/pages/
• Test Code: The actual test files in tests/e2e/

Read the relevant implementation files and modify them to match requirements.

5. Re-run Test

Follow this iterative process:

1. Reset database: cd apps/backend-services && PRISMA_USER_CONSENT_FOR_DANGEROUS_AI_ACTION="yes" npx prisma migrate reset --force && npm run db:seed
2. Run failing test: npx playwright test path/to/test.spec.ts
3. Identify root cause from error message and test output
4. Apply fix (update test, add seed data, fix implementation, etc.)
5. Re-run test to verify fix
6. Repeat until test passes or max attempts reached

Exit Conditions

• ✅ All tests passing: Success
• ❌ Max attempts reached: Report unfixable tests with details
• ⚠️ Critical error: Stop and report

Critical Rules for Corrections

1. ALWAYS check REQUIREMENTS.md and user stories BEFORE making any fix
2. FIX IMPLEMENTATION when it doesn't match requirements - don't just change the test
3. FIX TEST when the test expectation is incorrect or outdated
4. NEVER make a test pass by removing assertions or changing expectations without verifying requirements
5. When fixing implementation code, ensure the fix aligns with requirements and user stories
6. Document non-obvious fixes with comments explaining the requirement being satisfied

Debugging Tips Integration

When tests fail, progressively add debugging:

1. Log Network Requests

page.on('request', request => {
  console.log('>>', request.method(), request.url());
});

page.on('response', response => {
  console.log('<<', response.status(), response.url());
});

2. Check for API Errors

page.on('response', response => {
  if (response.status() >= 400) {
    console.error('API Error:', response.status(), response.url());
  }
});

These should be added temporarily during healing, then removed once issue is identified.