dev-qa-workflow

name	dev-qa-workflow
description	Orchestrated dev + QA workflow with mandatory browser testing, codex review gate, and feedback loop. Dev builds, Codex reviews the diff, QA runs browser tests via /qa-test. Failures loop back to dev automatically.
allowed-tools	["Bash","Read","Write","Edit","Glob","Grep","Task","TaskCreate","TaskUpdate","TaskList","TeamCreate","TeamDelete","SendMessage","ToolSearch","AskUserQuestion","Skill","mcp__codex-pair*"]

Dev-QA Workflow

Orchestrated development workflow with three gates: Dev builds, Codex reviews the diff, QA runs functional browser tests. Failures loop back automatically.

Triggers

• "/dev-qa", "dev qa workflow", "build and test", "dev with QA"

Usage

/dev-qa <task-description>
/dev-qa <task-description> --no-parallel   # Force serial QA (override auto-detect)

You will be prompted for:

• App URL (for QA browser testing)
• Criteria file (optional - path to success criteria markdown)
• Project path (defaults to cwd)

Parallel QA is automatic. The coordinator counts criteria during preflight and uses parallel Playwright instances (playwright-1 through playwright-4) when there are 8 or more criteria. Use --no-parallel to force serial mode.

---

Architecture

You (Coordinator)
│
├── Phase 1: CODEX ARCHITECTS
│   Coordinator calls codex-pair MCP
│   Codex analyzes the codebase and proposes an implementation plan
│   Do NOT pre-diagnose - let Codex think from scratch
│
├── Phase 2: DEV AGENT
│   Task-spawned general-purpose agent
│   Implements CODEX'S PLAN (not its own ideas)
│   Commits when done, messages coordinator
│
├── Phase 3: CODEX REVIEW GATE
│   Coordinator calls codex-reply (same session)
│   Reviews the diff against its own plan
│   Blocks QA until codex approves (or flags issues → back to dev)
│
├── Phase 4: QA AGENT
│   Task-spawned general-purpose agent
│   MUST invoke /qa-test skill for browser testing
│   Code review alone is NOT sufficient
│   Reports PASS/FAIL with evidence
│
└── Phase 5: FEEDBACK LOOP (if QA fails)
    QA failures → dev fix → codex re-review → QA re-test
    Loop until pass or max 3 iterations

Why This Shape

• Codex architects first because it catches design issues before any code is written, and the review is stronger when Codex checks against its own plan
• Dev is a spawned agent so it works autonomously and the coordinator isn't blocked
• Codex review runs on the coordinator because it's a single MCP call using the same session as the architect phase
• QA is a spawned agent because /qa-test is a long-running browser automation workflow
• Feedback loop is coordinator-driven so it can make the call to stop after max iterations

---

Best Practice: Define the Eval Upfront

Every dev-qa run is an iterative eval. Before any agent is spawned, the coordinator must define three things clearly. These flow through the entire pipeline - dev knows what to build, Codex knows what to check, QA knows what to test.

1. What is the problem?

State the issue or feature in concrete, observable terms. Not "fix the login" but "the login form submits successfully but the session cookie is not set, so the user is redirected back to /login on the next page load."

Bad: "Improve the dashboard" Good: "The dashboard loads in 4.2s on a cold start. Target is under 2s. The bottleneck appears to be the /api/metrics endpoint which returns 2MB of unfiltered data."

2. How do you test it?

Define the test methodology before writing code. This tells QA exactly what to do and prevents "I looked at the code and it seems fine" non-testing.

• What URL/page to test
• What actions to perform (click X, fill Y, submit Z)
• What to observe (network response, UI change, error message, timing)

3. What are the success criteria?

Each criterion must be binary pass/fail. No subjective judgments. If you can't tell a machine "check if this passed," the criterion is too vague.

Bad: "The page should load faster" Good: "Time to interactive is under 2 seconds as measured by Playwright's waitForLoadState('networkidle')"

Bad: "The form should handle errors well" Good: "Submitting with an empty email field shows 'Email is required' below the input within 500ms, and the form does not submit a network request"

Why This Matters

Without these three things defined upfront, the feedback loop devolves into opinion-based iteration. Dev guesses what "done" means, Codex reviews against vibes, QA tests whatever seems important. With them defined, every agent in the chain shares the same definition of success, and the feedback loop converges faster.

The coordinator should write these into the criteria file (or verify they're present in a user-provided one) during Phase 0 preflight.

---

Phase 0: Preflight

Before spawning any agents, the coordinator must:

Capture Baseline SHA

cd {project_path} && git rev-parse HEAD

Store this as {baseline_sha}. All diffs will be measured from this point.

Verify App is Accessible (if URL provided)

curl -fsS -o /dev/null -w "%{http_code}" {url}

If this fails, stop and tell the user. QA cannot run against a down app.

Generate Criteria File (if none provided)

If the user didn't provide a criteria file, generate one from the task description using the Success Criteria Template below. Apply the three-part eval framework (see "Define the Eval Upfront" above): every criterion must state the problem, how to test it, and a binary pass/fail condition. Save to {project_path}/qa-criteria-generated.md. This MUST happen before QA is spawned.

If the user DID provide a criteria file, review it against the three-part framework. If criteria are vague (e.g., "should work correctly"), refine them into testable statements before proceeding.

Count Criteria and Decide Parallel Mode

Read the criteria file (provided or generated) and count the number of acceptance criteria (headings matching ### AC- or numbered test cases). Store the count as {criteria_count}.

Auto-detect parallel mode:

• If the user passed --no-parallel, set {use_parallel} = false
• Otherwise, if {criteria_count} >= 8, set {use_parallel} = true
• Otherwise, set {use_parallel} = false

This decision is used in Phase 4 when spawning the QA agent.

Verify qa-test Skill Exists (BLOCKING)

The QA agent depends on the qa-test skill. If /qa-test isn't in the available skills list, stop and tell the user. Do not proceed - QA will fail without it. The user needs to install the skill before running this workflow.

---

Phase 1: Codex Architects the Solution

Before any code is written, Codex analyzes the codebase and proposes an implementation plan. Do NOT pre-diagnose the problem - let Codex think from scratch.

Step 1: Create Team

TeamCreate: team_name="dev-qa", description="Dev + QA workflow for: {task}"

Step 2: Load Codex MCP

ToolSearch: query="codex-pair"

Step 3: Send Problem to Codex

mcp__codex-pair__codex:
  prompt: |
    I need to implement the following:
    {task description}

    Project path: {project_path}
    Relevant files: {any file paths the user mentioned, or "explore the codebase"}

    Analyze the codebase, understand the current architecture, and propose a specific implementation plan. Include:
    1. Which files to modify or create
    2. What changes to make in each file (function names, logic)
    3. The order of changes
    4. Any risks or edge cases to watch for

    Do NOT give vague guidance. Give a concrete plan that a developer can follow step by step.
  cwd: "{project_path}"
  sandbox: "read-only"
  developer-instructions: "You are the architect. Analyze thoroughly, then propose a specific implementation plan with file paths, function names, and reasoning. Be concise but complete."

Step 4: Capture conversationId

Save the conversationId from the response. This SAME session will be used for the review in Phase 3 and all subsequent reviews in the feedback loop.

Step 5: Evaluate the Plan

Read Codex's plan. If it looks reasonable, proceed. If it seems off, use codex-reply to ask clarifying questions before proceeding. Present the plan to the user if the task is significant.

---

Phase 2: Spawn Dev Agent

Dev implements Codex's plan - not its own ideas.

Dev Agent Task

TaskCreate:
  subject: "Implement: {task description}"
  description: "Implement according to Codex's plan below."

Dev Agent Prompt

Spawn via Task tool with team_name="dev-qa", name="dev":

You are the dev agent on a dev-qa team. Your job is to implement the plan that Codex (the architect) designed.

## Your Task
{task description}

## Codex's Implementation Plan
{paste Codex's full plan here}

## Project
{project path}

## Rules
1. Follow Codex's plan above. Do NOT deviate or improvise your own approach.
2. If the plan won't work mid-build, STOP and message the coordinator explaining why. Do not silently change the approach.
3. Implement the minimum changes needed - no over-engineering beyond the plan.
4. If a test command is obvious (e.g., `npm test`, `pytest`, visible in package.json scripts), run it. If not obvious, skip.
5. When done, commit your changes with a descriptive message.
6. Message the coordinator with a summary of what you changed and which files.

## When Complete
Use SendMessage to tell the coordinator:
- What you implemented
- Which files changed (exact paths)
- Any concerns or edge cases
- Whether tests passed (or "no obvious test command found")

Then STOP. Do not cycle through the task list looking for more work.
After messaging the coordinator, you are done. Do not call TaskList. Do not look for unblocked tasks.

Validate Dev Output Before Shutdown

When dev messages completion:

1. Read the message
2. Check for a commit: Run git diff {baseline_sha}..HEAD in {project_path}. If the diff is empty, dev didn't commit. Message dev: "You need to commit your changes before I can proceed." Wait for dev to respond with confirmation, then re-check.
3. Once a commit exists: Immediately send SendMessage(type="shutdown_request", recipient="dev")
4. Do NOT wait after shutdown - proceed to Phase 3

---

Phase 3: Codex Review Gate

After dev completes, Codex reviews the diff against its own plan. This uses the SAME Codex session from Phase 1 (via codex-reply), so Codex already has full architectural context.

Step 1: Get the Diff

Use the baseline SHA captured in Phase 0:

cd {project_path} && git diff {baseline_sha}..HEAD

This captures ALL changes since the workflow started, regardless of how many commits dev made.

If dev didn't commit (no diff), message dev to commit first. Do not proceed without a diff.

Step 2: Send Diff to Codex (Same Session)

Use codex-reply with the conversationId saved from Phase 1:

mcp__codex-pair__codex-reply:
  conversationId: "{saved_conversation_id}"
  prompt: |
    Dev has implemented your plan. Here is the diff:
    ```
    {paste the actual output of git diff {baseline_sha}..HEAD here}
    ```

    Changed files: {list from dev's message}
    Re-read each changed file and review against your plan.

    Focus on:
    1. Did dev follow your plan correctly?
    2. Any bugs, logic errors, or off-by-one issues?
    3. Any deviations from the plan that introduce risk?
    4. Could these changes break existing functionality?

    Verdict: APPROVE or REJECT with specific issues.

Step 3: Handle Codex Verdict

• APPROVE - Proceed to Phase 4 (QA)
• REJECT - Create a new dev task with Codex's specific issues, re-spawn dev agent, loop back to Phase 2. Do NOT proceed to QA on rejected code.

---

Phase 4: Spawn QA Agent

This is the critical phase. The QA agent MUST run browser-based functional testing, not just code review.

QA Agent Task

TaskCreate:
  subject: "QA test: {task description}"
  description: "Run functional browser tests against {url}. Use /qa-test skill. See QA agent prompt for mandatory requirements."

QA Agent Prompt

Spawn via Task tool with team_name="dev-qa", name="qa":

You are the QA agent on a dev-qa team. Your job is to verify that the dev changes actually work in the browser.

## MANDATORY: Browser Testing Required

You MUST use the Skill tool to invoke the qa-test skill. This is not optional.
Code review alone will be REJECTED by the coordinator. You must test through the UI.

Your first action should be invoking the skill. Do not read source code first.

## How to Invoke QA Testing

Use the Skill tool exactly like this:

    Skill(skill="qa-test", args="{url} {qa_args} {criteria_file}")

The coordinator has already decided the mode for you. `{qa_args}` will be either:
- `--parallel` (8+ criteria detected, uses dedicated Playwright instances for concurrent testing)
- empty (serial mode for smaller test suites)

If `{criteria_file}` is missing or empty, STOP and message the coordinator. Do not proceed without a criteria file.

## Eval Framing - Read This First

**Criteria file (source of truth): `{criteria_file}`**

Before running tests, read the criteria file. Each acceptance criterion has three parts:
1. **Problem** - what is broken or needs to exist
2. **Test method** - exact steps to perform in the browser
3. **Pass/Fail condition** - binary, measurable outcome

Your job is to execute the test method for each AC and check it against the pass/fail condition. Your report must mirror this structure - not invent your own criteria.

## What Was Changed
{summary from dev agent's completion message}

This is context only. The criteria file is the source of truth for what to test. If the dev summary and criteria conflict, stop and ask the coordinator.

## Testing Focus
- Execute the test method for each AC in the criteria file
- Check each result against its pass/fail condition
- Test at least one edge case per AC
- Verify existing functionality wasn't broken (AC for regressions should be in the criteria file)

## Completion Requirements - ALL of these must be in your message

Your message to the coordinator MUST include ALL of the following. Messages missing any of these will be rejected:

1. **Overall verdict:** PASS or FAIL
2. **QA report file path** (generated by /qa-test, e.g., QA-Test-Report.md or similar)
3. **Screenshot paths** as evidence (generated by /qa-test during browser testing)
4. **Per-criterion results** - for each AC from the criteria file, include:
   - AC ID (e.g., AC-001)
   - Problem (from criteria file)
   - Test method you performed
   - Pass/Fail condition (from criteria file)
   - Result: PASS or FAIL
   - Evidence: screenshot path or observed behavior

If anything FAILED, also include:
- Exact steps to reproduce
- Expected vs actual behavior
- Screenshot evidence

If you cannot run /qa-test (e.g., skill not available, browser won't launch), message the coordinator explaining the blocker. Do NOT fall back to code review.

Then STOP. Do not look for more work. Do not call TaskList. You are done.

Coordinator: Validate QA Output

When QA messages you, perform these checks:

1. Message contains a QA report file path (e.g., QA-Test-Report.md)
2. Verify the report file exists: Use the Read tool to read the report path. If it doesn't exist, QA fabricated the path.
3. Message contains screenshot paths and at least one screenshot file exists (use Glob to check)
4. Report content references Playwright/browser interactions (not just code analysis)
5. Report references AC IDs from the criteria file: Read both the criteria file and the QA report. Every AC in the criteria file must appear in the report with Problem/Test/Pass fields and a PASS/FAIL result. If ACs are missing or the QA agent invented its own criteria, reject.

If ANY of these checks fail, the QA output is invalid. REJECT it and re-spawn QA with this escalated prompt:

"Your previous report was rejected. You MUST use: Skill(skill='qa-test', args='{url} {qa_args} {criteria_file}'). Your FIRST action must be calling the Skill tool. Your report must cover every AC in the criteria file using the Problem/Test method/Pass condition structure. Do NOT invent your own criteria. Do NOT do a code review."

Shutdown QA After Validation

Only shut down QA AFTER validating the output. If validation fails and you need to re-spawn, you haven't wasted the shutdown/respawn cycle.

Once validated: SendMessage(type="shutdown_request", recipient="qa")

---

Phase 5: Feedback Loop

If QA reports failures:

Iteration Tracking

Track the current iteration. Maximum 3 loops before escalating to the user.

On QA Failure

1. Create a new dev task with Codex's plan + QA's failure details:

   TaskCreate:
     subject: "Fix QA failures (iteration {n}): {specific issues}"
     description: |
       Original plan from Codex:
       {Codex's plan from Phase 1}
   
       QA found these failures:
       {paste QA's specific failure details with screenshots/evidence}
   
       Fix these issues following Codex's original architecture. The QA agent will re-test after you're done.
   

2. Re-spawn dev agent (name="dev") with the fix task (include Codex's plan in prompt)

3. Validate commit before shutdown - same as Phase 2: check git diff {baseline_sha}..HEAD has new changes. If dev didn't commit, message them to commit first. Only shutdown after commit is confirmed.

4. Get the updated diff: git diff {baseline_sha}..HEAD (captures all changes including the fix)

5. Codex re-review using mcp__codex-pair__codex-reply with the saved conversationId:

   mcp__codex-pair__codex-reply:
     conversationId: "{saved_conversation_id}"
     prompt: |
       Dev made fixes for the QA failures. Here is the updated diff:
       ```
       {paste the actual output of git diff {baseline_sha}..HEAD here}
       ```
       Changed files: {list from dev's message}
       Re-read each changed file and review the fix. Verdict: APPROVE or REJECT.
   

6. If Codex REJECTS the fix - go back to step 1 with Codex's issues (do NOT spawn QA on rejected code)

7. If Codex APPROVES - re-spawn QA agent (name="qa") to re-test

8. Validate QA output - all 5 checks from Phase 4 (report exists, screenshots exist, browser evidence, AC IDs from criteria file present, per-criterion Problem/Test/Pass structure)

9. Shutdown QA after validation

10. If still failing after 3 full iterations (dev → codex → QA = 1 iteration) - stop and present all findings to the user with a summary of what was tried

On QA Pass

1. Shut down all agents
2. Present the results to the user:
   • What was implemented
   • Codex's review summary
   • QA test results
   • Number of iterations needed
3. Clean up: TeamDelete

---

Coordinator Checklist

Use this as your execution guide:

PREFLIGHT (Phase 0)
[ ] 1. Gather inputs (task, url, criteria file, project path)
[ ] 2. Capture baseline SHA: git rev-parse HEAD → {baseline_sha}
[ ] 3. Verify app accessible: curl -fsS {url} (if URL provided)
[ ] 4. Generate criteria file if none provided → qa-criteria-generated.md
[ ] 5. Count criteria → {criteria_count}. Auto-parallel: >=8 criteria → {use_parallel}=true
[ ] 6. Verify qa-test skill is available (STOP if missing)

CODEX ARCHITECTS (Phase 1)
[ ] 7. Create team: dev-qa
[ ] 8. Load codex-pair: ToolSearch("codex-pair")
[ ] 9. Send problem to Codex (do NOT pre-diagnose) → save {conversation_id}
[ ] 10. Evaluate Codex's plan. If significant task, present to user.

DEV IMPLEMENTS CODEX'S PLAN (Phase 2)
[ ] 11. Create dev task (include Codex's full plan in description)
[ ] 12. Spawn dev agent (name="dev") with Codex's plan in prompt
[ ] 13. Wait for dev completion message
[ ] 14. Verify commit exists: git diff {baseline_sha}..HEAD must be non-empty
[ ] 15. If no commit → message dev to commit, wait, re-check
[ ] 16. Shutdown dev: SendMessage(type="shutdown_request", recipient="dev")

CODEX REVIEWS ITS OWN PLAN'S EXECUTION (Phase 3)
[ ] 17. Get diff: git diff {baseline_sha}..HEAD
[ ] 18. Send diff to Codex via codex-reply with {conversation_id}
[ ] 19. If codex REJECTS → create fix task, go to step 12
[ ] 20. If codex APPROVES → continue

QA BROWSER TESTING (Phase 4)
[ ] 21. Create QA task. Set {qa_args}="--parallel" if {use_parallel}=true, else ""
[ ] 22. Spawn QA agent (name="qa") with {qa_args} baked into the prompt
[ ] 23. Wait for QA completion message
[ ] 24. Validate QA output BEFORE shutdown:
      - Read the QA report file path (must exist)
      - Glob for screenshot files (at least one must exist)
      - Report must reference browser/Playwright interactions
      - Report lists AC IDs from criteria file with Problem/Test/Pass structure
      - No invented criteria - every AC must come from the criteria file
[ ] 25. If validation fails → reject, re-spawn QA with escalated prompt, go to 22
[ ] 26. Shutdown QA: SendMessage(type="shutdown_request", recipient="qa")

FEEDBACK LOOP (Phase 5, max 3 iterations)
[ ] 27. Create fix task with QA failure details
[ ] 28. Re-spawn dev (name="dev") with Codex's plan + QA failures, verify commit, shutdown
[ ] 29. Get updated diff: git diff {baseline_sha}..HEAD
[ ] 30. Codex re-review via codex-reply with SAME {conversation_id} (include diff)
[ ] 31. If codex REJECTS → back to step 27 (don't spawn QA on rejected code)
[ ] 32. If codex APPROVES → re-spawn QA (name="qa") with same {qa_args}, validate output, then shutdown
[ ] 33. If still failing after 3 iterations → stop, present findings to user

CLEANUP
[ ] 34. Present results to user (what was done, codex plan, codex review, QA results, iterations)
[ ] 35. TeamDelete to clean up

---

Success Criteria Template

If the user doesn't provide a criteria file, generate one from the task description. Every criterion must follow the three-part eval framework: problem, test method, pass/fail condition.

# Success Criteria for: {task}

## Problem Statement
{Concrete, observable description of the issue or feature. What is broken, missing, or needed? Include specifics - error messages, measurements, user-visible behavior.}

## Acceptance Criteria

### AC-001: {primary requirement}
- **Priority:** P0
- **Problem:** {What specifically is wrong or needs to exist}
- **Test method:**
  1. Navigate to {relevant page}
  2. {action}
  3. {action}
  4. Observe {what to check}
- **Pass condition:** {Binary, measurable outcome - no subjective language}
- **Fail condition:** {What failure looks like}

### AC-002: {secondary requirement}
- **Priority:** P1
- **Problem:** {What specifically is wrong or needs to exist}
- **Test method:**
  1. {steps}
- **Pass condition:** {Binary, measurable outcome}
- **Fail condition:** {What failure looks like}

### AC-003: No regressions
- **Priority:** P0
- **Problem:** Changes must not break existing functionality
- **Test method:**
  1. Navigate to main pages
  2. Verify existing functionality still works
  3. Check console for new errors
- **Pass condition:** All existing features work as before, no new console errors
- **Fail condition:** Any existing feature breaks or new errors appear in console

Save to {project_path}/qa-criteria-generated.md and pass to QA agent.

---

Edge Cases

Dev agent finishes but didn't commit

Ask dev to commit before proceeding. Codex needs a diff to review.

Codex MCP is unavailable

Fall back to CLI: codex exec --sandbox read-only --full-auto "{review prompt}" If codex CLI is also unavailable, stop and tell the user. Do NOT skip the review gate. Codex review is mandatory per global rules.

QA agent doesn't invoke /qa-test

This should not happen with the mandatory prompt above. But if QA reports results without browser evidence (no screenshots, no Playwright interactions), REJECT the QA report and re-spawn with an even more explicit prompt: "Your previous report was rejected because you did not run browser tests. You MUST use: Skill(skill='qa-test', args='{url} {qa_args} {criteria_file}'). Do NOT proceed without browser testing. Your report must cover every AC in the criteria file using the Problem/Test method/Pass condition structure."

App isn't running

Handled in Phase 0 preflight. If curl -fsS -o /dev/null {url} returns non-zero, stop and tell the user before spawning any agents.

No test URL provided

Ask the user via AskUserQuestion. QA cannot run without a URL to test against. Do not proceed without it.