workflow-test-fix

<purpose> Unified test-fix orchestrator that combines **test planning generation** (Phase 1-4) with **iterative test-cycle execution** (Phase 5) into a single end-to-end pipeline. Creates test sessions with progressive L0-L3 test layers, generates test tasks, then executes them with adaptive fix cycles until pass rate >= 95% or max iterations reached. Triggered via skill name routing for full pipeline or execute-only modes. </purpose> <process> ## 1. Architecture Overview ``` ┌───────────────────────────────────────────────────────────────────────────┐ │ Workflow Test Fix Orchestrator (SKILL.md) │ │ → Pure coordinator: Route entry point, track progress, pass context │ │ → Five phases: Session → Context → Analysis → TaskGen → Execution │ └──────────────────────────────────┬────────────────────────────────────────┘ │ ┌────────────┬────────────┬──────┴──────┬────────────┬────────────┐ ↓ ↓ ↓ ↓ ↓ ┌──────────┐┌──────────┐┌──────────┐┌──────────┐ ┌──────────────┐ │ Phase 1 ││ Phase 2 ││ Phase 3 ││ Phase 4 │ │ Phase 5 │ │ Session ││ Context ││ Analysis ││ Task Gen │ │ Test Cycle │ │ Start ││ Gather ││ Enhanced ││ Generate │ │ Execute │ │ ││ ││ ││ │ │ │ │ Input ││ Coverage ││ Gemini ││ IMPL_PLAN│ │ 1. Discovery│ │ Detect + ││ or Code ││ L0-L3 ││ IMPL-* │ │ 2. Execute │ │ Session ││ Scan ││ AI Issue ││ TODO_LIST│ │ 3. Fix Loop │ │ Create ││ ││ ││ │ │ 4. Complete │ └────┬─────┘└────┬─────┘└────┬─────┘└────┬─────┘ └──────────────┘ │ │ │ │ ↑ │testSessionId │ │ │ └──→────────┘contextPath│ │ │ └──→───────┘AnalysisRes│ │ └──→──────┘ testSessionId │ └──→──(Summary)──→┘ Task Pipeline (generated in Phase 4, executed in Phase 5): ┌──────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │ IMPL-001 │──→│ IMPL-001.3 │──→│ IMPL-001.5 │──→│ IMPL-002 │ │ Test Gen │ │ Code Validate │ │ Quality Gate │ │ Test & Fix │ │ L1-L3 │ │ L0 + AI Issues │ │ Coverage 80%+ │ │ Max N iter │ │@code-developer│ │ @test-fix-agent │ │ @test-fix-agent │ │@test-fix-agent│ └──────────────┘ └─────────────────┘ └─────────────────┘ └──────────────┘ ``` ## 2. Key Design Principles 1. **Unified Pipeline**: Generation and execution are one continuous workflow - no manual handoff 2. **Pure Orchestrator**: SKILL.md coordinates only - delegates all execution detail to phase files 3. **Auto-Continue**: Phase 1→2→3→4→(Summary)→5 automatically 4. **Task Attachment/Collapse**: Sub-tasks attached during phase execution, collapsed after completion 5. **Progressive Phase Loading**: Phase docs read **only** when that phase executes, not upfront 6. **Adaptive Strategy**: Fix loop auto-selects strategy (conservative/aggressive/surgical) based on iteration context 7. **Quality Gate**: Pass rate >= 95% (criticality-aware) terminates the fix loop 8. **Phase File Hygiene**: Phase files reference `workflowPreferences.*` for preferences, no CLI flag parsing ## 3. Usage Full pipeline and execute-only modes are triggered by skill name routing (see Mode Detection). Workflow preferences (auto mode) are collected interactively via AskUserQuestion before dispatching to phases. **Full pipeline** (workflow-test-fix): Task description or session ID as arguments → interactive preference collection → generate + execute pipeline **Execute only** (workflow-test-fix): Auto-discovers active session → interactive preference collection → execution loop ## 4. Interactive Preference Collection Before dispatching to phase execution, collect workflow preferences via AskUserQuestion: ```javascript // ★ 统一 auto mode 检测：-y/--yes 从 $ARGUMENTS 或 ccw 传播 const autoYes = /\b(-y|--yes)\b/.test($ARGUMENTS) if (autoYes) { // 自动模式：跳过所有询问，使用默认值 workflowPreferences = { autoYes: true } } else { const prefResponse = AskUserQuestion({ questions: [ { question: "是否跳过所有确认步骤（自动模式）？", header: "Auto Mode", multiSelect: false, options: [ { label: "Interactive (Recommended)", description: "交互模式，包含确认步骤" }, { label: "Auto", description: "跳过所有确认，自动执行" } ] } ] }) workflowPreferences = { autoYes: prefResponse.autoMode === 'Auto' } } ``` **workflowPreferences** is passed to phase execution as context variable, referenced as `workflowPreferences.autoYes` within phases. ## 5. Compact Recovery (Phase Persistence) Multi-phase test-fix pipeline (Phase 1-5) spans long conversations, especially Phase 5 fix loops. Uses **双重保险**: TodoWrite 跟踪 active phase 保护其不被压缩，sentinel 作为兜底。 > **⚠️ COMPACT DIRECTIVE**: Context compression MUST check TodoWrite phase status. > The phase currently marked `in_progress` is the active execution phase — preserve its FULL content. > Only compress phases marked `completed` or `pending`. ## 6. Execution Flow ``` Entry Point Detection: ├─ /workflow-test-fix → Full Pipeline (Phase 1→2→3→4→Summary→5) └─ /workflow-test-fix → Execution Only (Phase 5) Phase 1: Session Start (session-start) └─ Read("phases/01-session-start.md") ├─ Step 1.0: Detect input mode (session | prompt) ├─ Step 1.1: Create test session → testSessionId └─ Output: testSessionId, MODE Phase 2: Test Context Gather (test-context-gather) └─ Read("phases/02-test-context-gather.md") ├─ Step 1.2: Gather test context → contextPath └─ Output: contextPath Phase 3: Test Concept Enhanced (test-concept-enhanced) └─ Read("phases/03-test-concept-enhanced.md") ├─ Step 1.3: Test analysis (Gemini) → TEST_ANALYSIS_RESULTS.md └─ Output: TEST_ANALYSIS_RESULTS.md Phase 4: Test Task Generate (test-task-generate) └─ Read("phases/04-test-task-generate.md") ├─ Step 1.4: Generate test tasks → IMPL_PLAN.md, IMPL-*.json, TODO_LIST.md └─ Output: testSessionId, 4+ task JSONs Summary Output (inline after Phase 4): └─ Display summary, auto-continue to Phase 5 Phase 5: Test Cycle Execution (test-cycle-execute) └─ Read("phases/05-test-cycle-execute.md") ├─ Step 2.1: Discovery (load session, tasks, iteration state) ├─ Step 2.2: Execute initial tasks (IMPL-001 → 001.3 → 001.5 → 002) ├─ Step 2.3: Fix loop (if pass_rate < 95%) │ ├─ Select strategy: conservative/aggressive/surgical │ ├─ Generate fix task via @cli-planning-agent │ ├─ Execute fix via @test-fix-agent │ └─ Re-test → loop or exit └─ Step 2.4: Completion (summary, session archive) └─ Output: final pass_rate, summary ``` **Phase Reference Documents** (read on-demand when phase executes): | Phase | Document | Purpose | Compact | |-------|----------|---------|---------| | 1 | phases/01-session-start.md | Detect input mode, create test session | TodoWrite 驱动 | | 2 | phases/02-test-context-gather.md | Gather test context (coverage/codebase) | TodoWrite 驱动 | | 3 | phases/03-test-concept-enhanced.md | Gemini analysis, L0-L3 test requirements | TodoWrite 驱动 | | 4 | phases/04-test-task-generate.md | Generate task JSONs and IMPL_PLAN.md | TodoWrite 驱动 | | 5 | phases/05-test-cycle-execute.md | Execute tasks, iterative fix cycles, completion | TodoWrite 驱动 + 🔄 sentinel | **Compact Rules**: 1. **TodoWrite `in_progress`** → 保留完整内容，禁止压缩 2. **TodoWrite `completed`** → 可压缩为摘要 3. **🔄 sentinel fallback** → Phase 5 包含 compact sentinel；若 compact 后仅存 sentinel 而无完整 Step 协议，必须立即 `Read("phases/05-test-cycle-execute.md")` 恢复 ## 7. Core Rules 1. **Start Immediately**: First action is TaskCreate initialization, second action is Phase 1 (or Phase 5 for execute-only entry) 2. **No Preliminary Analysis**: Do not read files or gather context before starting the phase 3. **Parse Every Output**: Extract required data from each step output for next step 4. **Auto-Continue**: Phase 1→2→3→4→(Summary)→5 automatically (for full pipeline entry) 5. **Track Progress**: Update TaskCreate/TaskUpdate dynamically with task attachment/collapse pattern 6. **Task Attachment Model**: Sub-tasks **attached** during phase, **collapsed** after completion 7. **DO NOT STOP**: Continuous workflow until quality gate met or max iterations reached 8. **Progressive Loading**: Read phase doc ONLY when that phase is about to execute 9. **Entry Point Routing**: `workflow-test-fix` skill → Phase 1-5; `workflow-test-fix` skill → Phase 5 only ## 8. Input Processing ### test-fix-gen Entry (Full Pipeline) ``` User input → Detect type: ├─ Starts with "WFS-" → MODE=session, sourceSessionId=input ├─ Ends with ".md" → MODE=prompt, description=Read(input) └─ Otherwise → MODE=prompt, description=input ``` ### test-cycle-execute Entry (Phase 5 Only) ``` Arguments → Parse flags: ├─ --resume-session="WFS-xxx" → sessionId=WFS-xxx ├─ --max-iterations=N → maxIterations=N (default: 10) └─ (no args) → auto-discover active test session ``` ## 9. Data Flow ``` User Input (session ID | description | file path) ↓ [Detect Mode: session | prompt] ↓ Phase 1: Session Start ───────────────────────────────────────── ↓ 1.0+1.1: session:start → testSessionId, MODE ↓ Phase 2: Test Context Gather ──────────────────────────────────── ↓ 1.2: test-context-gather/context-gather → contextPath ↓ Phase 3: Test Concept Enhanced ────────────────────────────────── ↓ 1.3: test-concept-enhanced → TEST_ANALYSIS_RESULTS.md ↓ Phase 4: Test Task Generate ───────────────────────────────────── ↓ 1.4: test-task-generate → IMPL_PLAN.md, IMPL-*.json, TODO_LIST.md ↓ Summary Output (inline) ───────────────────────────────────────── ↓ Display summary with next step ↓ Phase 5: Test Cycle Execution ─────────────────────────────────── ↓ 2.1: Load session + tasks + iteration state ↓ 2.2: Execute IMPL-001 → 001.3 → 001.5 → 002 ↓ 2.3: Fix loop (analyze → fix → retest) until pass_rate >= 95% ↓ 2.4: Completion → summary → session archive ``` ## 10. Summary Output (after Phase 4) After Phase 4 completes, display the following summary before auto-continuing to Phase 5: ``` Test-fix workflow created successfully! Input: [original input] Mode: [Session|Prompt] Test Session: [testSessionId] Tasks Created: - IMPL-001: Test Understanding & Generation (@code-developer) - IMPL-001.3: Code Validation Gate - AI Error Detection (@test-fix-agent) - IMPL-001.5: Test Quality Gate - Static Analysis & Coverage (@test-fix-agent) - IMPL-002: Test Execution & Fix Cycle (@test-fix-agent) Quality Thresholds: - Code Validation: Zero CRITICAL issues, zero compilation errors - Minimum Coverage: 80% line, 70% branch - Static Analysis: Zero critical anti-patterns - Max Fix Iterations: 5 Review artifacts: - Test plan: .workflow/[testSessionId]/IMPL_PLAN.md - Task list: .workflow/[testSessionId]/TODO_LIST.md - Analysis: .workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md ``` **CRITICAL - Next Step**: Auto-continue to Phase 5: Test Cycle Execution. Pass `testSessionId` to Phase 5 for test execution pipeline. Do NOT wait for user confirmation — the unified pipeline continues automatically. ## 11. Test Strategy Overview Progressive Test Layers (L0-L3): | Layer | Name | Focus | |-------|------|-------| | **L0** | Static Analysis | Compilation, imports, types, AI code issues | | **L1** | Unit Tests | Function/class behavior (happy/negative/edge cases) | | **L2** | Integration Tests | Component interactions, API contracts, failure modes | | **L3** | E2E Tests | User journeys, critical paths (optional) | **Quality Thresholds**: - Code Validation (IMPL-001.3): Zero CRITICAL issues, zero compilation errors - Minimum Coverage: 80% line, 70% branch - Static Analysis (IMPL-001.5): Zero critical anti-patterns - Pass Rate Gate: >= 95% (criticality-aware) or 100% - Max Fix Iterations: 10 (default, adjustable) ## 12. Strategy Engine (Phase 5) | Strategy | Trigger | Behavior | |----------|---------|----------| | **Conservative** | Iteration 1-2 (default) | Single targeted fix, full validation | | **Aggressive** | Pass rate >80% + similar failures | Batch fix related issues | | **Surgical** | Regression detected (pass rate drops >10%) | Minimal changes, rollback focus | Selection logic and CLI fallback chain (Gemini → Qwen → Codex) are detailed in Phase 5. ## 13. Agent Roles | Agent | Used In | Responsibility | |-------|---------|---------------| | **Orchestrator** | All phases | Route entry, track progress, pass context | | **@code-developer** | Phase 5 (IMPL-001) | Test generation (L1-L3) | | **@test-fix-agent** | Phase 5 | Test execution, code fixes, criticality assignment | | **@cli-planning-agent** | Phase 5 (fix loop) | CLI analysis, root cause extraction, fix task generation | ## 14. TodoWrite Pattern **Core Concept**: Dynamic task tracking with attachment/collapse for real-time visibility. > **Implementation Note**: Phase files use `TodoWrite` syntax to describe the conceptual tracking pattern. At runtime, these are implemented via `TaskCreate/TaskUpdate/TaskList` tools from the allowed-tools list. Map `TodoWrite` examples as follows: > - Initial list creation → `TaskCreate` for each item > - Status changes → `TaskUpdate({ taskId, status })` > - Sub-task attachment → `TaskCreate` + `TaskUpdate({ addBlockedBy })` > - Sub-task collapse → `TaskUpdate({ status: "completed" })` + `TaskUpdate({ status: "deleted" })` for collapsed sub-items ### Full Pipeline (Phase 1-5) ```json [ {"content": "Phase 1: Session Start", "status": "in_progress"}, {"content": "Phase 2: Test Context Gather", "status": "pending"}, {"content": "Phase 3: Test Analysis (Gemini)", "status": "pending"}, {"content": "Phase 4: Test Task Generate", "status": "pending"}, {"content": "Phase 5: Test Cycle Execution", "status": "pending"} ] ``` ### Phase 1-4 Collapsed → Phase 5 Active ```json [ {"content": "Phase 1: Session Start", "status": "completed"}, {"content": "Phase 2: Test Context Gather", "status": "completed"}, {"content": "Phase 3: Test Analysis (Gemini)", "status": "completed"}, {"content": "Phase 4: Test Task Generate", "status": "completed"}, {"content": "Phase 5: Test Cycle Execution", "status": "in_progress"}, {"content": " → Execute IMPL-001 [code-developer]", "status": "in_progress"}, {"content": " → Execute IMPL-001.3 [test-fix-agent]", "status": "pending"}, {"content": " → Execute IMPL-001.5 [test-fix-agent]", "status": "pending"}, {"content": " → Execute IMPL-002 [test-fix-agent]", "status": "pending"}, {"content": " → Fix Loop", "status": "pending"} ] ``` ### Fix Loop Iterations ```json [