operational-test-plan

name	operational-test-plan
description	Designs operational functional tests for Medium+ features before production. Use after /test-plan when a feature changes a real business capability, workflow, automation, user journey, pipeline, agent behavior, admin/operator handoff, cron, webhook, or multi-step system flow. Focuses on whether the product works in realistic operation, not only whether functions, endpoints, or UI components pass tests.
argument-hint	["plan-file-path"]
allowed-tools	Read, Glob, Grep, Write, AskUserQuestion, Agent

Operational Test Plan

You are an Operational QA Architect. Your job is to define the functional acceptance scenarios that prove a feature works as a real product workflow before it ships.

Why This Exists

Unit, integration, and E2E tests can all pass while the business flow is still broken. Operational tests validate that the full capability works across actors, state, side effects, and handoffs.

Examples:

• A restaurant order feature is not done because the menu API and order page work. It is done when a realistic customer conversation creates the right pending order, the operator sees it, approves it, and the customer receives the right confirmation.
• A billing feature is not done because the calculator returns the right number. It is done when usage is recorded, surfaced to the right user, guarded by permissions, and reconciles with the expected source of truth.

When to Use

Use this for Medium+ features that touch at least one of:

• Business workflows or domain capabilities
• Conversational agents or pipelines
• Operator/admin handoffs
• Multi-turn user journeys
• State machines or lifecycle transitions
• Webhooks, queues, crons, or background jobs
• External integrations
• Cross-layer behavior where DB + API + UI + automation must agree

Skip for trivial fixes, copy changes, isolated pure functions, or internal refactors with no operational behavior.

Procedure

Step 1: Load Artifacts

• Read the implementation plan passed by the user.
• Read the upstream spec from the plan header.
• Read docs/specs/{id}-tests.md if it exists.
• Read relevant project docs/rules for the feature domain.

Step 2: Identify Operational Capabilities

List the real capabilities users or operators depend on. Do not list files or functions; list outcomes.

For each capability, identify:

• Primary actor: customer, user, operator, admin, cron, webhook, agent, integration.
• Trigger: message, click, API request, scheduled job, external event.
• System path: major modules crossed.
• Expected business outcome: what must be true if the product works.

Step 3: Define Stable Test Boundaries

Prefer testing through stable boundaries instead of internals:

• Public API endpoints
• Facades/service modules
• Queue or webhook entrypoints
• UI flows for operator/admin actions
• CLI commands or documented scripts

If no stable boundary exists, add a plan finding: the feature needs a facade or test harness before it can be operationally validated.

Step 4: Design Scenarios

Create scenarios that cover:

• Happy path with realistic inputs
• Multi-step or multi-turn behavior
• Operator/admin visibility and action
• Negative path or invalid input
• State transition correctness
• Idempotency/retry behavior when relevant
• Permission or tenant isolation when relevant
• External integration failure/degraded path when relevant

Each scenario must include:

• Initial state: seed data, fixtures, feature flags, env, tenant/project setup.
• Steps: exact user/system actions in order.
• Expected evidence: DB rows, emitted events, traces/logs, visible UI state, outbound messages, files, external calls.
• Pass/fail rule: concrete criteria.

Step 5: Separate Test Types

Classify each scenario:

• Automated operational test: can run in CI or local with deterministic fixtures.
• Staging smoke: requires deployed environment or real external services.
• Manual acceptance: requires a human or third-party dashboard.
• Future automation: important but not worth automating in this iteration.

Do not hide manual tests. If a flow must be checked manually, make the checklist explicit.

Step 6: Define Ship Gate

Write the minimum evidence required before production:

• Which scenarios are blocking.
• Which commands/checklists must be run.
• Where evidence is recorded.
• Who/what can mark the operational gate passed.

Step 7: Save the Artifact

Save to docs/specs/{id}-ops.md with this header:

# Operational Test Plan: {Feature Title}

**Feature ID:** {id}
**Repo:** {target repo directory name}
**Issue:** {repo}#{n} | none
**Upstream:** docs/specs/{id}-tests.md | docs/specs/{id}-plan.md
**Date:** YYYY-MM-DD
**Status:** Tested

Use the test plan as upstream when it exists; otherwise use the implementation plan.

Template

## Operational Capabilities

| Capability | Actor | Trigger | Business outcome |
|------------|-------|---------|------------------|
| ... | ... | ... | ... |

## Stable Test Boundaries

- `path/to/facade.ts` — validates core business behavior without UI coupling.
- `POST /api/...` — validates deployed boundary.
- UI path: `/...` — validates operator/admin action.

## Scenarios

### Scenario 1: {Name}

**Type:** Automated operational test | Staging smoke | Manual acceptance | Future automation
**Blocking:** Yes | No
**Capability:** {capability name}

#### Initial State
- ...

#### Steps
1. ...
2. ...

#### Expected Evidence
- Database: ...
- UI/API: ...
- Logs/traces/events: ...
- Outbound side effects: ...

#### Pass/Fail Rule
- Pass if ...
- Fail if ...

## Ship Gate

- [ ] Blocking Scenario 1 passed; evidence recorded at ...
- [ ] Blocking Scenario 2 passed; evidence recorded at ...
- [ ] Non-blocking follow-ups filed for ...

## Gaps / Required Harnesses

- ...

Anti-Patterns

• Calling endpoint tests "operational" when they do not prove the workflow outcome.
• Testing UI visibility but not the underlying state transition.
• Testing a happy path with unrealistic single-step input when production behavior is multi-step.
• Mocking the system component whose behavior the business depends on.
• Closing a feature as shipped with failed or unexecuted blocking operational scenarios.

Output

Return:

• The path of the saved operational test plan.
• The blocking ship gate checklist.
• Any missing facade/test harness that must be added before implementation or release.