systematic-testing

name	systematic-testing
description	Bottom-up systematic validation for multi-layer architectures. Use when testing domain plugins, proxy routes, API endpoints, browser connections, traffic capture, codegen output, or dashboard UI. Test each layer independently before testing integration.

Systematic Validation

When testing or debugging a multi-layer system, work bottom-up. Never test a higher layer until the layer below it is verified. This approach isolates failures to the specific layer where they occur.

The Four Layers

Layer	Name	Key Files	Validation
L1	Domain Plugin	domains/<name>/src/routes.ts	Routes registered, handler logic correct
L2	API Proxy	apps/api/src/register-domains.ts + packages/browser/src/handler/api-proxy.ts	Endpoint responds, browser dispatches correctly
L3	Browser + Traffic Capture	packages/browser/src/handler/index.ts	Browser connected, navigation works, CDP traffic captured
L4	Dashboard UI	apps/web/src/app/(dashboard)/	Data renders, interactions work, states handled

Dependency chain: L1 → L2 → L3 → L4

---

Per-Layer Validation

L1 — Domain Plugin

Verify the route handler logic is correct before the browser is involved.

# List all registered domains and their routes
curl -s http://localhost:3001/api | jq '.domains'

# Verify a specific route exists
curl -s http://localhost:3001/api | jq '.domains[] | select(.name == "<domain>")'

Common L1 failures:

• handler function not exported from routes.ts
• DomainRoute type mismatch (missing handler or targetUrl)
• Route path typo (e.g., /search vs /searches)

---

L2 — API Proxy

Browser must be connected before proxy routes work. Note: the Access Gap table (browser vs direct HTTP status per endpoint) is generated during discovery Step 2e — this layer uses those results but does not regenerate them.

# Check browser health
curl -s http://localhost:3001/browser/health | jq '.browser'

# Call a domain route directly
curl -s "http://localhost:3001/api/<domain>/<path>?<params>" | jq '.'

Expected browser health when connected:

{ "active": true, "ready": true, "profile": "<profile-name>", "domain": null }

Common L2 failures:

• 503 { "error": "Browser not connected" } → Connect the browser first via WebSocket
• 404 → Route not registered in register-domains.ts or path typo
• 500 → Handler threw an error — check server logs

---

L3 — Browser + Traffic Capture

Connect the browser and verify it can navigate and capture traffic.

# After connecting via WebSocket at ws://localhost:3001/browser/stream?profile=<name>&url=<url>

# Verify connected browser can navigate
curl -s "http://localhost:3001/api/<domain>/search?q=test" | jq '.total'

# Check traffic buffer (for Type B2 domains)
curl -s http://localhost:3001/browser/traffic | jq '[.entries[] | {url: .url[:100], status}]'

# Clear traffic buffer before a Type B2 test
curl -s -X DELETE http://localhost:3001/browser/traffic | jq '.'

Type classification:

• Type A — targetUrl proxy. Browser navigates to target, headers forwarded.
• Type B — handler with page.evaluate(). Browser navigates, DOM parsed server-side.
• Type B2 — handler with traffic capture. Browser navigates, page JS fires XHR/fetch, buffer read.
• Type C — handler with browserFetch(). Single same-origin request, no navigation.

Common L3 failures:

• Browser navigates but page renders empty (stubborn page, CAPTCHA) → check page content with page.evaluate(() => document.body.innerText.slice(0, 500))
• Traffic buffer empty after navigation (Type B2) → XHR/API calls not firing; increase wait time
• RegionalDomain mismatch → filter extracted URLs to correct domain

---

L4 — Dashboard UI

Use the visual-dev skill for screenshot-based validation.

Validation checklist:

• Empty state renders (no data, no errors)
• Loading state shows skeleton/spinner
• Data renders correctly after API call
• Error state shows clear message when API fails
• Interactive elements (search, click) trigger correct API calls and update state

# Quick smoke test — does the page load?
curl -s http://localhost:3000/<page> | grep -o '<title>[^<]*</title>'

---

Gate Sequence

Run this sequence before calling a phase complete:

# Gate 1: Domain routes registered
curl -s http://localhost:3001/api | jq '.domains[].name'

# Gate 2: Browser connected and proxy works
curl -s "http://localhost:3001/api/<domain>/search?q=test" | jq '.total'

# Gate 3: Real data returned (not empty, not error)
curl -s "http://localhost:3001/api/<domain>/search?q=<real-query>" | jq '.events[0].name // .performers[0].name // .[0]'

# Gate 4: Listings/detail data returns
curl -s "http://localhost:3001/api/<domain>/listings/<id>" | jq '.listings[0]'

All four must pass before building UI.

---

Fix Before Ascending

When a test fails at any layer, fix it before moving up. If L3 traffic capture is empty, check L2 proxy (is the right route called?), then L1 handler (is the wait time sufficient?). Don't move to L4 UI until L3 is clean.

When Tests Fail Unexpectedly

Use the debug-logs skill to add targeted DEBUG() calls inside route handlers to observe what the browser page actually contains when the handler runs.