| name | marsai:dev-integration-testing |
| description | Gate 4 of development cycle - ensures integration tests pass for all
external dependency interactions using real containers via testcontainers.
|
| trigger | - After unit testing complete (Gate 3)
- MANDATORY for all development tasks
- Verifies real service integration with testcontainers
|
| skip_when | - Not inside a development cycle (marsai:dev-cycle)
- Task is documentation-only, configuration-only, or non-code
- Service has no external dependencies (no database, cache, queue, or external API)
- Pure library package with no integration points
|
| NOT_skip_when | - "Unit tests cover it" - Unit tests mock. Integration tests verify real behavior.
- "No time for integration tests" - Integration bugs cost 10x more in production.
- "CI doesn't support Docker" - Fix CI. Docker is baseline infrastructure.
|
| sequence | {"after":["marsai:dev-unit-testing"],"before":["marsai:dev-chaos-testing"]} |
| related | {"complementary":["marsai:dev-cycle","marsai:dev-testing","marsai:qa-analyst"]} |
| input_schema | {"required":[{"name":"unit_id","type":"string","description":"Task or subtask identifier"},{"name":"integration_scenarios","type":"array","items":"string","description":"Integration scenarios to test"},{"name":"external_dependencies","type":"array","items":"string","description":"External services (postgres, redis, rabbitmq, etc.)"},{"name":"language","type":"string","enum":["typescript"],"description":"Programming language"}],"optional":[{"name":"gate3_handoff","type":"object","description":"Full handoff from Gate 3 (unit testing)"},{"name":"implementation_files","type":"array","items":"string","description":"Files from Gate 0 implementation"}]} |
| output_schema | {"format":"markdown","required_sections":[{"name":"Integration Testing Summary","pattern":"^## Integration Testing Summary","required":true},{"name":"Scenario Coverage","pattern":"^## Scenario Coverage","required":true},{"name":"Quality Gate Results","pattern":"^## Quality Gate Results","required":true},{"name":"Handoff to Next Gate","pattern":"^## Handoff to Next Gate","required":true}],"metrics":[{"name":"result","type":"enum","values":["PASS","FAIL","SKIP"]},{"name":"scenarios_tested","type":"integer"},{"name":"tests_written","type":"integer"},{"name":"tests_passed","type":"integer"},{"name":"tests_failed","type":"integer"},{"name":"flaky_tests_detected","type":"integer"},{"name":"iterations","type":"integer"}]} |
| verification | {"automated":null,"manual":["All integration scenarios have at least one test","No flaky tests (run 3x, all pass)","All containers properly cleaned up"]} |
Dev Integration Testing (Gate 4)
Overview
Ensure every integration scenario has at least one integration test proving real external dependencies work correctly. Use testcontainers for all external services.
Core principle: Unit tests mock dependencies, integration tests verify real behavior. Both are required.
<block_condition>
- Any integration scenario without test = FAIL
- Any test using production services = FAIL
- Any test with hardcoded ports = FAIL
- Any flaky test (fails on retry) = FAIL
</block_condition>
CRITICAL: Role Clarification
This skill ORCHESTRATES. QA Analyst Agent (integration mode) EXECUTES.
| Who | Responsibility |
|---|
| This Skill | Gather scenarios, check if needed, dispatch agent, validate output |
| QA Analyst Agent | Write tests, run coverage, verify quality gates |
Step 0: Detect External Dependencies (Auto-Detection)
MANDATORY: When external_dependencies is empty or not provided, scan the codebase to detect them automatically before validation.
if external_dependencies is empty or not provided:
detected_dependencies = []
1. Scan docker-compose.yml / docker-compose.yaml for service images:
- Grep tool: pattern "postgres" in docker-compose* files → add "postgres"
- Grep tool: pattern "mongo" in docker-compose* files → add "mongodb"
- Grep tool: pattern "valkey" in docker-compose* files → add "valkey"
- Grep tool: pattern "redis" in docker-compose* files → add "redis"
- Grep tool: pattern "rabbitmq" in docker-compose* files → add "rabbitmq"
2. Scan dependency manifests:
if language == "typescript":
- Grep tool: pattern "\"pg\"" in package.json → add "postgres"
- Grep tool: pattern "@prisma/client" in package.json → add "postgres"
- Grep tool: pattern "\"mongodb\"" in package.json → add "mongodb"
- Grep tool: pattern "\"mongoose\"" in package.json → add "mongodb"
- Grep tool: pattern "\"redis\"" in package.json → add "redis"
- Grep tool: pattern "\"ioredis\"" in package.json → add "redis"
- Grep tool: pattern "@valkey" in package.json → add "valkey"
- Grep tool: pattern "\"amqplib\"" in package.json → add "rabbitmq"
- Grep tool: pattern "amqp-connection-manager" in package.json → add "rabbitmq"
3. Deduplicate detected_dependencies
4. Set external_dependencies = detected_dependencies
Log: "Auto-detected external dependencies: [detected_dependencies]"
<auto_detect_reason>
PM team task files often omit external_dependencies. If the codebase uses postgres, mongodb, valkey, or rabbitmq, these are external dependencies that MUST have integration tests. Auto-detection prevents silent skips.
</auto_detect_reason>
Step 0.5: Testing Infrastructure Assessment (MANDATORY)
MANDATORY: Before creating integration tests from scratch, assess whether the project has existing integration testing infrastructure. Most projects will have unit test setup — that does NOT count. This step checks specifically for integration test patterns (testcontainers, real database connections in tests, integration-specific runners).
infrastructure_scan = {
has_integration_tests: false,
has_testcontainers_setup: false,
infrastructure_exists: false,
}
1. Scan for existing integration test files:
- Glob tool: pattern "**/*.integration.spec.ts", "**/*.integration.test.ts"
- Grep tool: pattern "testcontainers|@testcontainers" in **/*.spec.ts, **/*.test.ts
- If any matches → has_integration_tests = true
NOTE: Unit test files (*.spec.ts with only mocks) do NOT count.
2. Scan for testcontainers or real-database test infrastructure:
- Grep tool: pattern "from.*testcontainers" or "@testcontainers" in **/*.ts
- Grep tool: pattern "setup-test-db|setupTestDb|setupTestDatabase" in **/*.ts
- Grep tool: pattern "GenericContainer|PostgreSqlContainer|MongoDBContainer" in **/*.ts
- If any matches → has_testcontainers_setup = true
3. Determine infrastructure_exists:
infrastructure_exists = has_integration_tests OR has_testcontainers_setup
NOTE: Existing unit tests with mocks = infrastructure does NOT exist for integration purposes.
Step 0.5b: No Infrastructure → Assess Task Criticality
if infrastructure_exists == true:
→ Proceed to Step 1 (normal flow)
if infrastructure_exists == false:
→ Assess task criticality:
critical_domains = ["login", "logout", "sign-in", "sign-up", "signup", "signin",
"authenticate", "authentication", "oauth", "sso", "mfa", "2fa",
"payment", "billing", "transaction", "money",
"balance", "transfer", "ledger", "invoice", "charge"]
NOTE: Only tasks that DIRECTLY implement authentication flows (login, signup,
OAuth callbacks) or handle money (payments, transfers, billing) qualify.
Tasks that merely touch permissions, roles, tokens, encryption, or credentials
as part of broader features are NOT critical for this purpose.
task_is_critical = any critical_domain keyword appears in:
- unit_id
- integration_scenarios
- acceptance_criteria (if available from orchestrator)
- implementation file paths (if available from Gate 0)
┌─────────────────────────────────────────────────────────────┐
│ NON-CRITICAL TASK + NO INFRASTRUCTURE → SKIP │
│ │
│ Return: │
│ status: SKIP │
│ skip_reason: "Project lacks integration testing │
│ infrastructure (no existing integration tests, no │
│ testcontainers setup). Creating tests from scratch │
│ without established patterns produces low-quality code."│
│ recommendation: "Set up testcontainers base configuration │
│ and a reference integration test before enabling this │
│ gate." │
│ ready_for_next_gate: YES │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ CRITICAL TASK + NO INFRASTRUCTURE → EPHEMERAL MODE │
│ │
│ Log: "CRITICAL TASK detected without testing infrastructure.│
│ Running ephemeral verification." │
│ Set: ephemeral_mode = true │
│ Proceed to Step 1 (normal flow) │
│ │
│ After Step 7 (success) or Step 8 (failure): │
│ 1. Record test results in gate output │
│ 2. DELETE all test files created during this gate │
│ 3. Log files deleted and why │
│ 4. Return results with: │
│ ephemeral_note: "Integration tests verified [PASS/FAIL] │
│ but files were removed. Project needs testing │
│ infrastructure setup before persisting tests." │
└─────────────────────────────────────────────────────────────┘
<infrastructure_rationale>
Creating integration tests in a project without established testing patterns produces tests that:
- Don't follow project conventions (wrong directory structure, naming, imports)
- Lack shared test utilities (setup/teardown, fixtures, helpers)
- Add dependencies the project isn't prepared to maintain (testcontainers, toxiproxy)
- Create maintenance burden with no existing patterns to guide future developers
For critical tasks (auth, finance), ephemeral verification still validates behavior without committing unmaintainable infrastructure.
</infrastructure_rationale>
Step 1: Validate Input
REQUIRED INPUT (from marsai:dev-cycle orchestrator):
<verify_before_proceed>
- unit_id exists
- language is valid (typescript)
</verify_before_proceed>
OPTIONAL INPUT (determines if Gate 4 runs or skips):
- integration_scenarios: [list of scenarios] - if provided and non-empty, Gate 4 runs
- external_dependencies: [list of deps] (from input OR auto-detected in Step 0) - if non-empty, Gate 4 runs
- gate3_handoff: [full Gate 3 output]
- implementation_files: [files from Gate 0]
EXECUTION LOGIC:
1. if any REQUIRED input is missing:
-> STOP and report: "Missing required input: [field]"
-> Return to orchestrator with error
2. if integration_scenarios is empty AND external_dependencies is empty (AFTER auto-detection in Step 0):
-> Gate 4 SKIP (document reason: "No integration scenarios or external dependencies found after codebase scan")
-> Return skip result with status: "skipped"
3. Otherwise:
-> Gate 4 REQUIRED - proceed to Step 2
Step 2: Check If Integration Tests Needed
Decision Tree:
1. Task has external_dependencies list (from input or auto-detected)?
|
+-- YES -> Gate 4 REQUIRED
|
+-- NO -> Continue to #2
2. Task has integration_scenarios?
|
+-- YES -> Gate 4 REQUIRED
|
+-- NO -> Continue to #3
3. Task acceptance criteria mention "integration", "database", "queue"?
|
+-- YES -> Gate 4 REQUIRED
|
+-- NO -> Gate 4 SKIP (with reason)
If SKIP:
Return:
status: SKIP
skip_reason: "No external dependencies (after codebase scan) or integration scenarios identified"
ready_for_gate5: YES
Step 3: Initialize Testing State
integration_state = {
unit_id: [from input],
scenarios: [from integration_scenarios or derived from external_dependencies],
dependencies: [from external_dependencies],
verdict: null,
iterations: 0,
max_iterations: 3,
tests_passed: 0,
tests_failed: 0,
flaky_detected: 0
}
Step 4: Dispatch QA Analyst Agent (Integration Mode)
<dispatch_required agent="marsai:qa-analyst">
Write integration tests for all scenarios using testcontainers.
</dispatch_required>
Task:
subagent_type: "marsai:qa-analyst"
description: "Integration testing for [unit_id]"
prompt: |
**test_mode: integration**
- **Unit ID:** [unit_id]
- **Language:** [language]
[list integration_scenarios with IS-1, IS-2, etc.]
[list external_dependencies with container requirements]
- File pattern: `*.integration.spec.ts` or `*.integration.test.ts`
- Test pattern: `describe('ComponentName integration', () => { it('should ...') })`
- Example: `KyselyUserRepository.integration.spec.ts`
- Use testcontainers for ALL external dependencies
- Versions MUST match infra/docker-compose
- Use afterAll/afterEach for container cleanup
- Integration tests run sequentially (no concurrent DB access)
- No hardcoded ports - use dynamic ports from containers
- No production services - all deps containerized
- Each scenario MUST have at least one test
| File | Tests | Lines |
|------|-------|-------|
| [path] | [count] | +N |
| IS ID | Scenario | Test File | Test Function | Status |
|-------|----------|-----------|---------------|--------|
| IS-1 | [scenario text] | [file] | [function] | PASS/FAIL |
| IS-2 | [scenario text] | [file] | [function] | PASS/FAIL |
| Check | Status | Evidence |
|-------|--------|----------|
| File naming (*.integration.spec.ts) | PASS/FAIL | [file count] |
| No hardcoded ports | PASS/FAIL | [grep result] |
| Testcontainers used | PASS/FAIL | [imports] |
| Sequential execution | PASS/FAIL | [config check] |
| Cleanup present (afterAll/afterEach) | PASS/FAIL | [grep result] |
**Tests:** [X passed, Y failed]
**Quality Gate:** PASS / FAIL
**VERDICT:** PASS / FAIL
If FAIL:
- **Gap Analysis:** [what needs more tests or fixes]
- **Files needing attention:** [list with issues]
Step 5: Parse QA Analyst Output
Parse agent output:
1. Extract scenario coverage from Scenario Coverage table
2. Extract quality gate results
3. Extract verdict
integration_state.tests_passed = [count from output]
integration_state.tests_failed = [count from output]
if verdict == "PASS" and quality_gate == "PASS":
-> integration_state.verdict = "PASS"
-> Proceed to Step 7 (Success)
if verdict == "FAIL" or quality_gate == "FAIL":
-> integration_state.verdict = "FAIL"
-> integration_state.iterations += 1
-> if iterations >= max_iterations: Go to Step 8 (Escalate)
-> Go to Step 6 (Dispatch Fix)
Step 6: Dispatch Fix to Implementation Agent
Quality gate failed or tests failing -> Return to implementation agent
Task:
subagent_type: "[implementation_agent from Gate 0]"
description: "Fix integration test issues for [unit_id]"
prompt: |
Integration Test Issues - Fix Required
- **Tests Passed:** [tests_passed]
- **Tests Failed:** [tests_failed]
- **Quality Gate:** FAIL
- **Iteration:** [iterations] of [max_iterations]
[paste gap analysis from QA output]
[paste files list from QA output]
1. Fix the identified issues
2. Ensure all containers use testcontainers
3. Ensure integration tests run sequentially (no concurrent DB access)
4. Add missing afterAll/afterEach cleanup
5. Replace hardcoded ports with dynamic ports
- Issues fixed: [list]
- Files modified: [list]
After fix -> Go back to Step 4 (Re-dispatch QA Analyst)
Step 7: Prepare Success Output
Generate skill output:
## Integration Testing Summary
**Status:** PASS
**Unit ID:** [unit_id]
**Iterations:** [integration_state.iterations]
## Scenario Coverage
| IS ID | Scenario | Test | Status |
|-------|----------|------|--------|
[from integration_state]
**Scenarios Covered:** [X]/[Y] (100%)
## Quality Gate Results
| Check | Status |
|-------|--------|
| File naming | PASS |
| No hardcoded ports | PASS |
| Testcontainers | PASS |
| Sequential execution | PASS |
| Cleanup present | PASS |
| No flaky tests | PASS |
## Handoff to Next Gate
- Integration testing status: COMPLETE
- Tests passed: [tests_passed]
- Tests failed: 0
- Flaky tests: 0
- Ready for Gate 5 (Chaos Testing): YES
Step 8: Escalate - Max Iterations Reached
Generate skill output:
## Integration Testing Summary
**Status:** FAIL
**Unit ID:** [unit_id]
**Iterations:** [max_iterations] (MAX REACHED)
## Gap Analysis
[from last QA output]
## Files Still Needing Fixes
[from last QA output]
## Handoff to Next Gate
- Integration testing status: FAILED
- Ready for Gate 5: NO
- **Action Required:** User must manually fix integration tests
ESCALATION: Max iterations (3) reached. Integration tests still failing.
User intervention required.
Severity Calibration
| Severity | Criteria | Examples |
|---|
| CRITICAL | Production service used, data corruption risk | Tests hit production DB, no cleanup, hardcoded creds |
| HIGH | Missing scenarios, flaky tests | Untested integration scenario, test fails on retry |
| MEDIUM | Quality gate failures, port issues | Wrong file naming, hardcoded ports, missing afterAll cleanup |
| LOW | Documentation, optimization | Missing test comments, slow container startup |
Report all severities. CRITICAL = immediate fix (production risk). HIGH = fix before gate pass. MEDIUM = fix in iteration. LOW = document.
Pressure Resistance
See shared-patterns/shared-pressure-resistance.md for universal pressure scenarios.
| User Says | Your Response |
|---|
| "Unit tests cover this" | "Unit tests mock dependencies. Integration tests verify real behavior. Both required." |
| "Testcontainers is too slow" | "Correctness > speed. Real dependencies catch real bugs." |
| "CI doesn't have Docker" | "Docker is baseline infrastructure. Fix CI before skipping integration tests." |
| "Skip integration, deadline" | "Integration bugs cost 10x more in production. Testing is non-negotiable." |
Anti-Rationalization Table
See shared-patterns/shared-anti-rationalization.md for universal anti-rationalizations.
Gate 4-Specific Anti-Rationalizations
| Rationalization | Why It's WRONG | Required Action |
|---|
| "Database already tested in unit tests" | Unit tests use mocks, not real DB | Write integration tests |
| "Testcontainers setup is complex" | Complexity is one-time. Bugs are recurring. | Use testcontainers |
| "Integration tests are flaky" | Flaky = poorly written. Fix isolation. | Fix the tests |
| "No external dependencies" | Check task requirements. Often implicit. | Verify with decision tree |
| "Concurrent tests are faster" | Faster but flaky with shared DB state. Flaky = worthless. | Run sequentially |
| "Hardcoded port works locally" | Fails in CI when port is taken. | Use dynamic ports |
| "Production DB is more realistic" | Production DB is dangerous and unreliable for tests. | Use testcontainers |
Execution Report Format
## Integration Testing Summary
**Status:** [PASS|FAIL|SKIP]
**Unit ID:** [unit_id]
**Duration:** [Xm Ys]
**Iterations:** [N]
## Scenario Coverage
| IS ID | Scenario | Test | Status |
|-------|----------|------|--------|
| IS-1 | [text] | [test] | PASS/FAIL |
**Scenarios Covered:** [X/Y]
## Quality Gate Results
| Check | Status |
|-------|--------|
| File naming | PASS/FAIL |
| No hardcoded ports | PASS/FAIL |
| Testcontainers | PASS/FAIL |
| Sequential execution | PASS/FAIL |
| Cleanup present | PASS/FAIL |
| No flaky tests | PASS/FAIL |
## Handoff to Next Gate
- Integration testing status: [COMPLETE|FAILED|SKIPPED]
- Ready for Gate 5: [YES|NO]
Skip Conditions (Documented)
When Gate 4 can be skipped (MUST document reason):
| Condition | Skip Reason |
|---|
| No external dependencies | "Task has no database, API, or queue interactions" |
| Pure business logic | "Task is pure function/logic with no I/O" |
| Library/utility code | "Task is internal utility with no external calls" |
| Already covered | "Integration tests exist and pass (verified)" |
When Gate 4 CANNOT be skipped:
| Condition | Why Required |
|---|
| Task touches database | Database queries need real verification |
| Task calls external APIs | HTTP behavior varies from mocks |
| Task uses message queues | Pub/sub requires real broker testing |
| Task has transactions | ACID guarantees need real DB |
| Task has migrations | Schema changes need integration verification |
