forge-test

name	forge-test
description	QA test Forge issue changes against preview deployments. Use this skill to test like a human QA — hitting the preview backend API and navigating the preview frontend to verify acceptance criteria are met. Triggers on: /forge-test, testing an issue, QA testing, verifying changes on preview, checking if acceptance criteria pass. Also use when the pipeline needs to verify changes at testing status.
user_invocable	true
arguments	documentId

Forge Test

Automated QA agent that tests the issue's actual output against the preview or staging deployment — like a human tester would. Hits the backend API and navigates the frontend to verify acceptance criteria are met.

This is NOT a test runner (vitest/playwright). It's a manual QA replacement that uses live URLs to verify the change works end-to-end.

Usage

/forge-test <documentId>

Tools

• forge_issues — get issue data (acceptance criteria, preview URLs, plan, changeHistory)
• forge_comments — list previous comments (review/fix feedback) + post test report
• forge_config — get project config (staging URLs, test credentials)
• forge_coolify_deploy — check deployment status before testing
• Browser (optional) — if a browser-automation MCP is available, use whatever browser tools the runner exposes (auto-detected; usually surfaced as browser_*: navigate, click, type, snapshot/screenshot). Do not hardcode a provider. If no browser MCP is available, fall back to curl/WebFetch HTML checks.
• HTTP — WebFetch / Bash (curl) for API testing

Read references/test-approach.md for detailed testing patterns (API auth, browser tools, what to verify). Read references/result-format.md for report template and verdict rules. Read references/browser-playbook.md for step-by-step browser interaction guides (login, navigation, form input, code inspection) — follow these exactly to avoid rediscovering UI flows.

Workflow

Pull-model note: On large issues forge_step_start returns a lean manifest (bodyTruncated:true). Fetch acceptanceCriteria/plan as needed via forge_issues.get { documentId, fields: ['acceptanceCriteria'] } rather than assuming full body is present.

Step 0: Local-only mode guard

Call forge_config → get and forge_coolify_deploy → list. If previewDeploy is null/missing AND Coolify list is empty → project is in local-only mode.

In local-only mode there is no preview/staging deployment to QA against. The build + unit tests already ran in forge-code / forge-fix. Post a comment and exit without changing status:

forge_comments → create → {
  data: {
    body: "**QA skipped** — project is in local-only mode (no Coolify, no preview URL). Build and unit tests were executed by forge-code/forge-fix. The pipeline ends at `developed` for human review; the human closes the issue manually. Skip forge-test.",
    issue: "<documentId>",
    author: "Machamp"
  }
}

Stop. Do NOT call forge_issues → update.

Step 0.5: Docs-only deliverable guard

If the issue's change is a no-code deliverable (a docs/proposals/<topic>.md decision/audit/spike artifact) — detect mechanically: the diff for the issue's branch touches only docs/** (no packages/**) — there is no UI/API to QA. Do NOT try to walk acceptance criteria as a browser flow; that would FAIL on a doc that has no runtime surface and bounce the issue back into a loop.

Instead verify the artifact: confirm the planned docs/proposals/<topic>.md exists, is non-trivial (the actual decision/rationale/recommendations, not a stub), and is indexed in docs/proposals/README.md. PASS on presence + substance. Post the report and set status as usual (Step 9). If any packages/** file is in the diff, this is not docs-only — run the normal QA below.

Step 0.6: Decompose-aware guard (epic child vs parent integration)

If the issue has metadata.branchConfig or metadata.useIntegrationBranch, it is part of a decomposed epic — QA behavior differs. Read .claude/skills/forge-plan/references/decompose-execution.md and follow the forge-test section. In short: a child is not deployed individually (no per-child preview), so don't FAIL it for lacking one — note "verified via build+review; e2e deferred to parent" and advance it; its merge target is the integration branch (mark_merged target:'feature'), never base/deploy. The parent (useIntegrationBranch) is where the assembled epic gets its real end-to-end QA, and is the only issue that promotes the integration branch to base. For a non-decompose issue (no such metadata), ignore this step.

Step 1: Fetch Issue + Pipeline Context

forge_issues → get → { documentId: "<id>" }
forge_comments → list → { filters: { issue: "<id>" } }

Read from issue: title, description, acceptanceCriteria, aiAcceptanceCriteria, plan, previewUrl, previewApiUrl, changeHistory.

Read from comments: triage report, review findings, previous QA reports, fix notes.

Step 2: Detect Reopen Cycle

Check changeHistory for previous testing → reopen transitions.

• First test (no prior reopen): test all acceptance criteria normally.
• Reopen cycle (prior QA failure exists): find the most recent QA Test Report comment. Extract the FAIL items — these are mandatory regression tests that must pass this time. Also re-test all other acceptance criteria.

Step 3: Wait for Deployment Readiness

Before testing, ensure the deployed code is up to date. Deployments triggered by upstream skills (forge-code, forge-fix) may still be building when this step starts.

forge_coolify_deploy → status → {}

Check the most recent deployment for each resource:

1. If the most recent deployment was created within the last 5 minutes and status is still in-progress/building → wait 60 seconds, then re-check
2. Repeat up to 3 times (~3 min max wait)
3. If deployment is complete or no recent deployment found → proceed immediately
4. If still building after 3 retries → proceed but add a note to the test report: "Warning: Coolify deployment may still be in progress. Results may reflect stale code."

If no Coolify resources are configured, skip this step.

Step 4: Get Test URLs & Credentials

Fetch project config via forge_config → get. This returns previewDeploy with stagingUrl, stagingApiUrl, and testCredentials.

URL resolution (issue preview takes priority, staging as fallback):

• testUrl = issue previewUrl ?? project previewDeploy.stagingUrl
• testApiUrl = issue previewApiUrl ?? project previewDeploy.stagingApiUrl

If both testUrl and testApiUrl are null → post comment "No preview or staging deployment found, cannot test", stop.

Use previewDeploy.testCredentials (array of {label, username, password}) for authenticated flows. Pick the credential whose label matches the role the scenario needs (match by label — don't assume a fixed role set).

Step 5: Build Test Cases

Test cases come from upstream pipeline steps — forge-test executes, it doesn't invent.

Source 1 — Plan QA Scenarios (primary): Read the plan field and find the ## QA Scenarios section. Each scenario has Setup → Action → Verify → Contrast. Execute these directly — they were written by forge-plan which has codebase context. Tag as Plan source.

Source 2 — Acceptance Criteria (fill gaps): If the plan has no QA Scenarios section, or if some AC items aren't covered by scenarios, derive test cases from acceptance criteria:

• "User can X" → navigate to feature and do X
• "Y should display Z" → navigate to Y and verify Z is visible
• "API returns X when Y" → call API with Y and check response Tag as AC #N source.

Source 3 — Review findings (from comments): If forge-review posted findings, check if any were flagged as concerns. Tag as Review source.

Source 4 — Reopen regression (from comments): If this is a reopen cycle, extract FAIL items from the previous QA report. These are mandatory regression tests that MUST pass. Tag as Regression source.

Step 6: Test Backend API

For each backend-related test case:

• Authenticate via the project's login flow using credentials from previewDeploy.testCredentials (match by label); don't hardcode the auth endpoint — it varies by stack
• Construct requests against testApiUrl
• Use WebFetch or curl to hit endpoints
• Verify: status codes, response shape, data correctness
• Test with multiple roles when the AC involves role-based behavior

Step 7: Test Frontend UI — functional AND quality

Drive testUrl with the project's wired browser MCP. Test in two passes — a feature that works but ships broken UX is still a FAIL.

Pass A — Functional flow (per test case):

• Navigate, login with the appropriate test credential for the scenario, walk the user flow from the acceptance criteria.
• Verify elements are present, interactive, and show correct data (snapshot/read the page, fill forms, click through).
• Watch the console: any error thrown during the flow is a FAIL even if the screen looks fine.

Pass B — UX & accessibility quality bar (every screen the change touches): Judge each affected screen as the user, not as a checklist — adopt the AC's persona, predict what they expect at each step (entry / action / result / error / empty) BEFORE you look, then walk the flow and record each gap as Expectation → Observation → user impact → severity → concrete fix. The full expectation-first procedure lives in references/ui-quality-checklist.md — run it over each affected screen as the lens for where to look. The bar is "would you ship this to a user", not "does the element exist".

Run the Pass-B quality checklist (references/ui-quality-checklist.md) over each affected screen. Cover, and record a verdict for each:

• Responsive — re-check the flow at narrow / medium / wide viewports; layout must not break, overflow, or hide primary actions at the narrow width.
• States — exercise the empty, loading, and error states of any list / form / async surface the change touches — not just the happy path. A missing empty/error state is a FAIL.
• Accessibility — every new interactive element reachable and operable by keyboard alone; icon-only controls and inputs have accessible labels; text legible and not conveyed by colour alone.
• Role correctness (if the project has roles) — controls a role may not use are hidden/disabled, not just unenforced; no other tenant's/user's data leaks onto the screen.
• Design consistency — matches the app's existing components, spacing, and typography; no ad-hoc one-off styling.

If the project has a ux-contract projectFact (a per-project UX standard, injected into your context), hold each affected screen to its Definition of UX-Done — that is the project's own bar; a missing applicable item there is a FAIL.

Calibrate the verdict so the pipeline doesn't loop on polish: only a gap that genuinely breaks or strands the user is a FAIL → reopen (missing empty/error state, a primary action unreachable at a supported width, a control a role must not use is operable, another tenant's data on screen). A "would-be-nicer" or out-of-scope want is recorded in the Verification note, NOT a FAIL row — don't bounce an issue for polish.

Tag quality failures distinctly (UX / A11y / Responsive) so forge-fix can triage severity — they are real FAILs, kept separate from functional FAILs. Scope to the change — audit the screens this issue adds or modifies; don't re-audit the whole app. For backend-only changes with no UI surface, skip Pass B and note "no UI surface".

Emit UX findings (if the project has a ux-contract). When the project has a ux-contract projectFact and the change is user-facing, for each UX / A11y / Responsive gap you record above, also write it via forge_ux_findings → write { stage: "verify-live", kind: "missing-state"|"a11y"|"microcopy"|"responsive"|"design-system"|"other", detail, severity: "must"|"should" }. This is a non-blocking side-channel — separate from the PASS/FAIL verdict (a failed emit never changes the verdict); issue/run are resolved server-side. It feeds the per-project contract's learning loop. Skip for projects without a ux-contract or backend-only changes.

If no browser tools available: use curl/WebFetch to fetch the page HTML and verify key elements are present/absent. Note in the report that testing was HTML-only (no interactive browser) — Pass B cannot be fully verified, say so explicitly rather than claiming PASS.

Step 8: Post Test Report

forge_comments → create → {
  data: {
    body: "<test report>",
    issue: "<documentId>",
    author: "Forge QA"
  }
}

Report format:

**QA Test Report** {cycle indicator if reopen: "(Cycle 2)"}

**Test environment:** {testUrl} (preview|staging)

| # | Test Case | Source | Result | Notes |
|---|-----------|--------|--------|-------|
| 1 | Description | AC #1 | PASS/FAIL | Details |
| 2 | Edge case from plan | Plan | PASS/FAIL | Details |
| 3 | Empty/error state of <surface> | UX | PASS/FAIL | Details |
| 4 | Keyboard reachability of <control> | A11y | PASS/FAIL | Details |
| 5 | Flow at narrow viewport | Responsive | PASS/FAIL | Details |
| 6 | Previous failure regression | Regression | PASS/FAIL | Details |

**Verification:** what I actually walked — which flows, at which viewports, which states/roles exercised, and what I did NOT cover. "Looks right" is not verification; name the evidence. For UI changes, attach a screenshot of the final state.

**Summary:** X/Y passed
**Verdict:** PASS / FAIL

Source ∈ AC #N / Plan / Review / Regression / UX / A11y / Responsive. See references/result-format.md for full template and failure detail format, and references/ui-quality-checklist.md for the Pass-B quality bar.

Step 9: Stamp merged_at (base) on PASS, then Set Status

Status update must be the LAST action. It triggers downstream pipeline steps.

• All pass:
  1. Stamp merged_at (base). forge-code merges every complexity's ISS-* branch into the base branch at the code step, and you just QA'd that base/staging deployment — so a PASS means it is verified-on-base. Stamping merged_at is what releases the blocks-gate for any issue this one blocks (a blocked dependent only dispatches once its blocker's merged_at is set) and any decompose parent waiting on this child. Idempotent (COALESCE keeps the first timestamp): forge_issues → mark_merged → { data: { issueId: "<documentId>", target: "base" } } Skip for a decomposed-epic issue (metadata.branchConfig/useIntegrationBranch) — Step 0.6 stamps those against the integration branch (target:'feature'); don't double-stamp.
  2. Then set status as the LAST action → forge_issues → update → { data: { status: "tested" } }
• Any fail → forge_issues → update → { data: { status: "reopen" } } + detailed failure report with actionable info for forge-fix

Test-specific output reminder

The QA report goes to forge_comments.create, NOT to chat. Don't print it twice. (See pipeline preamble for general output rules.)