Testing patterns for Playwright AI-powered agents including the Planner, Generator, and Healer architecture for self-healing test automation, intelligent test generation, and adaptive test execution strategies.
Testing patterns for Playwright AI-powered agents including the Planner, Generator, and Healer architecture for self-healing test automation, intelligent test generation, and adaptive test execution strategies.
You are an expert in Playwright AI-powered agent architecture. When the user asks you to implement Planner, Generator, or Healer agents for test automation, build self-healing test infrastructure, or create adaptive testing pipelines with Playwright, follow these detailed instructions.
Core Principles
Three-agent architecture -- Structure test automation around three specialized agents: Planner (decides what to test), Generator (creates test code), and Healer (fixes broken tests). Each agent has a distinct responsibility and interface.
Selector resilience hierarchy -- Use a prioritized selector strategy: test IDs > ARIA roles > text content > CSS selectors. Self-healing agents should try alternatives in this order when primary selectors fail.
Snapshot-driven healing -- When tests fail, capture accessibility tree snapshots, DOM state, and screenshots. Feed these to the Healer agent to produce corrected selectors and actions.
Incremental generation -- Generate tests incrementally from user stories or natural language descriptions. Validate each generated step against the running application before generating the next.
Execution feedback loops -- Every test run produces structured feedback that informs the next generation or healing cycle. Use JSON-formatted execution reports, not raw console output.
Human-in-the-loop checkpoints -- Allow developers to review and approve generated or healed tests before they become part of the official suite. Never auto-commit generated tests without review.
Cost-aware agent orchestration -- Track LLM token usage for each agent invocation. Set budgets per test generation and healing cycle to prevent runaway API costs.
`You are a test planning agent for Playwright browser automation.
Given a user story, create a detailed test plan with concrete steps.
Each step must map to a Playwright action (navigate, click, fill, select, assert, wait).
Use descriptive selectors based on ARIA roles and test IDs.
Return the plan as a JSON object matching the TestPlan schema.`
const
`User Story: ${userStory}${appContext ? `App Context: ${appContext}` : ''}
Create a test plan. Return ONLY valid JSON matching this schema:
{
"id": "string",
"title": "string",
"userStory": "string",
"priority": "critical|high|medium|low",
"steps": [{"order": number, "action": "string", "target": "string", "value?": "string", "description": "string"}],
"preconditions": ["string"],
"expectedOutcome": "string",
"estimatedComplexity": number
}`
const
await
this
client
messages
create
model
this
model
max_tokens
2048
temperature
0
system
messages
role
'user'
content
const
content
0
type
'text'
content
0
text
''
const
match
/\{[\s\S]*\}/
if
throw
new
Error
'Planner failed to generate valid JSON'
return
JSON
parse
0
as
TestPlan
async
prioritizeTests
plans
TestPlan
riskAreas
string
Promise
TestPlan
const
`Given these test plans and known risk areas, reorder by priority.
Risk areas: ${riskAreas.join(', ')}
Plans: ${JSON.stringify(plans.map((p) => ({ id: p.id, title: p.title, priority: p.priority })))}
Return a JSON array of plan IDs in priority order.`
const
await
this
client
messages
create
model
this
model
max_tokens
512
temperature
0
messages
role
'user'
content
const
content
0
type
'text'
content
0
text
''
const
ids
string
JSON
parse
match
/\[[\s\S]*\]/
0
'[]'
return
map
(id) =>
find
(p) =>
id
filter
Boolean
as
TestPlan
Generator Agent
// tests/agents/generator/generator-agent.tsimportAnthropicfrom'@anthropic-ai/sdk';
import { TestPlan, TestStep } from'../planner/planner-agent';
exportinterfaceGeneratedTest {
planId: string;
code: string;
imports: string[];
fixtures: string[];
pageObjects: string[];
}
exportclassGeneratorAgent {
privateclient: Anthropic;
privatemodel: string;
constructor(model = 'claude-sonnet-4-20250514') {
this.client = newAnthropic();
this.model = model;
}
asyncgenerateTest(plan: TestPlan, existingPageObjects?: string[]): Promise<GeneratedTest> {
const systemPrompt = `You are a Playwright test code generator agent.
Generate TypeScript Playwright tests following these rules:
1. Use page object model pattern
2. Prefer getByRole, getByTestId, getByLabel over CSS selectors
3. Use web-first assertions (expect(locator).toBeVisible())
4. Include proper setup and teardown
5. Add meaningful test descriptions
6. Handle loading states with appropriate waits
7. Generate data-testid selectors when no semantic selector exists`;
const prompt = `Generate a Playwright test for this plan:
${JSON.stringify(plan, null, 2)}${existingPageObjects?.length ? `Available page objects: ${existingPageObjects.join(', ')}` : 'No existing page objects.'}
Return ONLY the complete TypeScript test file content. Use @playwright/test imports.`;
const response = awaitthis.client.messages.create({
model: this.model,
max_tokens: 4096,
temperature: 0,
system: systemPrompt,
messages: [{ role: 'user', content: prompt }],
});
const text = response.content[0].type === 'text' ? response.content[0].text : '';
const codeMatch = text.match(/```typescript\n([\s\S]*?)```/) || text.match(/```ts\n([\s\S]*?)```/);
const code = codeMatch ? codeMatch[1] : text;
return {
planId: plan.id,
code: code.trim(),
imports: this.extractImports(code),
fixtures: this.extractFixtures(code),
pageObjects: this.extractPageObjects(code),
};
}
asyncgeneratePageObject(
pageName: string,
pageUrl: string,
accessibilitySnapshot: string
): Promise<string> {
const prompt = `Generate a Playwright Page Object for "${pageName}" at URL "${pageUrl}".
Accessibility tree snapshot:
${accessibilitySnapshot}
Generate a TypeScript class with:
1. Locator properties for all interactive elements
2. Action methods for common user flows
3. Assertion methods for page state verification
4. Use getByRole and getByTestId selectors
5. Export the class as default
Return ONLY TypeScript code.`;
const response = awaitthis.client.messages.create({
model: this.model,
max_tokens: 4096,
temperature: 0,
messages: [{ role: 'user', content: prompt }],
});
const text = response.content[0].type === 'text' ? response.content[0].text : '';
const codeMatch = text.match(/```typescript\n([\s\S]*?)```/);
return codeMatch ? codeMatch[1].trim() : text.trim();
}
privateextractImports(code: string): string[] {
const importRegex = /import\s+.*from\s+['"](.+?)['"]/g;
constimports: string[] = [];
let match;
while ((match = importRegex.exec(code)) !== null) {
imports.push(match[1]);
}
return imports;
}
privateextractFixtures(code: string): string[] {
const fixtureRegex = /test\.extend<\{([\s\S]*?)\}>/;
const match = code.match(fixtureRegex);
if (!match) return [];
return match[1].split(';').map((f) => f.trim()).filter(Boolean);
}
privateextractPageObjects(code: string): string[] {
const poRegex = /new\s+(\w+Page)\(/g;
constpageObjects: string[] = [];
let match;
while ((match = poRegex.exec(code)) !== null) {
pageObjects.push(match[1]);
}
return [...newSet(pageObjects)];
}
}
Healer Agent
// tests/agents/healer/healer-agent.tsimportAnthropicfrom'@anthropic-ai/sdk';
exportinterfaceHealingContext {
testName: string;
failedStep: string;
errorMessage: string;
failedSelector: string;
accessibilitySnapshot: string;
screenshot?: string;
previousSelectors?: string[];
domDiff?: string;
}
exportinterfaceHealingResult {
healed: boolean;
newSelector: string;
confidence: number;
reasoning: string;
alternativeSelectors: string[];
suggestedAction?: string;
}
exportclassHealerAgent {
privateclient: Anthropic;
privatemodel: string;
privatehealingHistory: Map<string, string[]> = newMap();
constructor(model = 'claude-sonnet-4-20250514') {
this.client = newAnthropic();
this.model = model;
}
asyncheal(context: HealingContext): Promise<HealingResult> {
const systemPrompt = `You are a self-healing test automation agent for Playwright.
When a test selector breaks, analyze the accessibility snapshot and error to find the correct new selector.
Prefer selectors in this order:
1. getByTestId('...') - most stable
2. getByRole('...', { name: '...' }) - semantic and resilient
3. getByLabel('...') - for form elements
4. getByText('...') - for content-based selection
5. CSS selectors - last resort
Provide multiple alternatives ranked by confidence.`;
const prompt = `A Playwright test failed. Help me fix the selector.
Test: ${context.testName}
Failed Step: ${context.failedStep}
Error: ${context.errorMessage}
Failed Selector: ${context.failedSelector}${context.previousSelectors ? `Previous selectors that also failed: ${context.previousSelectors.join(', ')}` : ''}
Current accessibility snapshot:
${context.accessibilitySnapshot}${context.domDiff ? `DOM changes since last success:\n${context.domDiff}` : ''}
Return a JSON object:
{
"healed": boolean,
"newSelector": "string",
"confidence": 0-1,
"reasoning": "string",
"alternativeSelectors": ["string"],
"suggestedAction": "optional string if the action type should change"
}`;
const response = awaitthis.client.messages.create({
model: this.model,
max_tokens: 1024,
temperature: 0,
system: systemPrompt,
messages: [{ role: 'user', content: prompt }],
});
const text = response.content[0].type === 'text' ? response.content[0].text : '';
const jsonMatch = text.match(/\{[\s\S]*\}/);
if (!jsonMatch) {
return {
healed: false,
newSelector: context.failedSelector,
confidence: 0,
reasoning: 'Failed to parse healer response',
alternativeSelectors: [],
};
}
const result = JSON.parse(jsonMatch[0]) asHealingResult;
// Track healing history for this testconst key = `${context.testName}:${context.failedStep}`;
const history = this.healingHistory.get(key) || [];
history.push(result.newSelector);
this.healingHistory.set(key, history);
return result;
}
asyncbatchHeal(contexts: HealingContext[]): Promise<HealingResult[]> {
returnPromise.all(contexts.map((ctx) =>this.heal(ctx)));
}
getHealingHistory(testName: string): Map<string, string[]> {
const filtered = newMap<string, string[]>();
for (const [key, value] ofthis.healingHistory) {
if (key.startsWith(testName)) {
filtered.set(key, value);
}
}
return filtered;
}
}
Start with the Planner, validate with the Generator -- Always create a test plan before generating code. Plans catch missing preconditions and ambiguous requirements before code generation wastes tokens.
Use accessibility snapshots for healing, not screenshots -- Accessibility trees are structured and machine-readable. Screenshots require vision models and are slower and less reliable for selector resolution.
Set confidence thresholds for auto-healing -- Only auto-apply healed selectors when confidence exceeds 0.8. Below that threshold, flag for human review.
Version generated tests separately from handwritten tests -- Keep generated tests in a separate directory with clear naming so developers know which tests are agent-maintained.
Run healing in dry-run mode first -- Before applying healed selectors to the test suite, run the healed version in a sandboxed environment to verify it actually passes.
Limit healing attempts per test -- Set a maximum of 3 healing attempts per failing test. If healing fails after 3 tries, the test needs manual intervention.
Track selector drift metrics -- Monitor how often selectors need healing. High heal rates indicate unstable UI or poor initial selector choices.
Use test IDs as the primary selector strategy -- Invest in adding data-testid attributes to the application. They survive UI refactors and are the most reliable selector type.
Generate page objects alongside tests -- When generating tests for a new page, also generate the page object model. This promotes reuse and reduces duplication.
Review agent-generated code with the same rigor as human code -- Generated tests can contain anti-patterns, hardcoded values, and fragile assertions. Always review before merging.
Anti-Patterns
Auto-committing generated tests without review -- Generated code can contain hardcoded secrets, flaky assertions, or incorrect business logic. Always review before committing.
Using screenshots for healing instead of accessibility trees -- Screenshots are expensive to process, slower, and less accurate than structured accessibility data.
Letting the healer run indefinitely -- Without attempt limits, the healer can enter infinite loops and burn through API budgets.
Generating tests without a plan -- Skipping the planning phase produces unfocused tests that miss critical paths and edge cases.
Healing selectors without understanding why they broke -- A healed selector treats the symptom, not the cause. Track why selectors break to improve initial selector quality.
Using CSS selectors as the primary strategy -- CSS selectors are brittle against UI changes. Prefer ARIA roles and test IDs.
Running all agents sequentially when parallel execution is possible -- Independent healing operations can run in parallel. Sequential execution wastes time.
Not tracking agent costs -- Without cost tracking, a single test generation session can cost more than expected. Always monitor token usage.
Trusting low-confidence healing results -- A healed selector with 0.3 confidence is likely wrong. Set minimum confidence thresholds.
Generating tests for unstable UI during active development -- Wait for UI components to stabilize before generating tests. Generating against rapidly changing UIs wastes resources on constant healing.
