13-deploy-smoke

name	13-deploy-smoke
description	Executes the deployment, runs API smokes (H1–H3) plus browser connectivity gates (H4–H5: CORS + frontend bundle wiring), health checks, changelog, and monitoring baseline. Final pipeline stage. Blocking — user must approve deployment results.

13 — Deploy & Smoke Check

Deploy the application and verify it works with smoke tests and health checks.

Preamble: pipeline-preamble.md — shared conventions for stages 00–19. Sessions: sessions-reference.md — requires active_session unless waived; reports under docs/sessions/{id}/reports/. Cross-cutting: considerations.md, connectivity-gates.md. State agent: project .cursor/agents/workflow-state-manager.md if present; else edit workflow-state.yaml per workflow-state-reference.md — mandatory read/update.

Connectivity (stage 13)

Mandatory sequence: H0c (pre) → deploy → H1–H3 → verify_connectivity.sh (H4–H5). Do not mark deployed without H4–H5 pass or user-waived checklist entry. See connectivity-gates §Stage 13.

Single-env / live role

Inherit env_role from 12 (or re-resolve). If staging_as_live / sole stack:

• Smokes and promotes target production traffic — say so in every AskQuestion and report.
• Do not describe the cutover as “staging-only” when no distinct prod stack exists.
• Prefer labeling results as live even when hostnames still contain staging.

TAC-to-IWXXM (ADR-034 / EV-044): Smoke staging cluster at api|app.staging.tac-to-iwxxm.com (or CI Staging smoke, Host-header via STAGING_LB_IP=143.244.202.13 until DNS/TLS Ready) before promoting via PR stage→main only; then smoke prod at api|app.tac-to-iwxxm.com. Never call staging URLs production; never promote while Staging smoke/gate is red.

CI/CD tip gate

Before deploy, promote, or cutover CLI: tip SHA required workflows must be green. Red/cancelled → hard stop unless the user explicitly waives via AskQuestion.

Health ≠ primary-journey-ready

Liveness//health (and dependency probes) can pass while the primary user journey fails.

• H3 (primary journey smoke): timeout, hang, or hard error = FAIL even if H1 health is green.
• Do not mark health_tiers.h3: pass on health-only evidence.
• On H3 P0 failure during an evolve cycle: recommend pause 16-evolve → 14-hotfix.

Prerequisites

1. 12-verify-deploy must be completed — deploy checklist approved
2. Required:
   • `{artifacts_dir}/reports/deploy-checklist.md`` — all items checked
   • Deployment plan document — deploy commands and configuration
   • docs/test-plan.md — smoke test definitions
3. Deployment platform CLI/tools must be installed and authenticated

Session management

{artifacts_dir} = active_session.artifacts_dir from workflow-state-manager.

Per sessions-reference.md §10 and workflow-state-agent-protocol.md.

1. Agent read_context must return active_session (or blocking deviation).
2. Current stage must appear in active_session.routing_plan unless user amends plan.
3. Write stage reports to active_session.artifacts_dir/reports/ when this stage produces a report.
4. On completion: update routing-plan entry status; mirror project.stages.{key} via agent update.
5. 00-context exempt from active_session requirement (session opener). Report: reports/deploy-smoke.md.

State management

Agent protocol: workflow-state-agent-protocol.md. Stage key: stages.13-deploy-smoke.

Invoke workflow-state-manager read_context before any other action; update after each substep. Do not edit workflow-state.yaml directly.

Detail: docs/deploy-state.md — sync URL/status with YAML on each deploy step.

Deployment record (workflow-state.yaml §deployment)

This stage writes the deployment section that downstream skills (11, 15) consume:

On event	Update
Deploy succeeds	deployment.staging.status: deployed, commit_deployed, URLs
H1–H3 pass	deployment.staging.health_tiers.h1/h2/h3: pass
H4–H5 pass	deployment.staging.health_tiers.h4/h5: pass
H4–H5 not run	deployment.staging.health_tiers.h4/h5: pending + note
Local T0 verified	deployment.local_build.status: green, t0_result, journey count
URL discovered	deployment.staging.urls.<app>: <url>
Commit drift	deployment.staging.drift: true when commit_deployed != commit_head

After each smoke phase, update workflow-state.yaml §deployment immediately so that 15-service-health and 11-verify-impl can read current tier status without re-running checks. Use deployment.url_discovery.method to refresh URLs when needed.

On invocation — check state

1. Read both state sources (stages.13-deploy-smoke and deployment).
2. If deployed: Ask: "Re-deploy, validate existing, or rollback?"
3. If in_progress or failed: Report what happened. Resume or restart.
4. If pending: Start fresh.

Commit-as-you-go

Commit artifacts to an appropriate branch before transitioning to the next stage or asking the user a blocking question. Branch type per workflow-state-reference.md §Git history. Record every commit in workflow-state.yaml §git_history.commits with stage: "13-deploy-smoke".

State file: docs/deploy-state.md

# Deploy State

> Last updated: [date]
> Status: pending | in_progress | deployed | failed | rolled_back

## Deployment Log

| # | Step | Status | Started | Completed | Notes |
|---|------|--------|---------|-----------|-------|
| 1 | Deploy | pending | — | — | — |
| 2 | Smoke tests | pending | — | — | — |
| 3 | Health check | pending | — | — | — |
| 4 | Changelog | pending | — | — | — |
| 5 | Monitoring baseline | pending | — | — | — |

## Current Deployment

| Field | Value |
|-------|-------|
| App name | — |
| Deploy URL | — |
| Deploy mode | — |
| Commit | — |
| Branch | — |

Delta / feature-addition mode

• Redeploy only services affected by new Fn; run H1–H5 for changed browser/API paths.
• Update deployment block via workflow-state-manager after smokes.

Workflow

Phase 1.5 — Pre-deploy integration (T1)

Before production / live deploy (env_role: prod or staging_as_live):

1. T0 green: pytest tests/e2e/ -m "e2e and not live" (or project equivalent)
2. H0c green: CORS unit gate when the project uses connectivity-gates
3. Migrations apply on target DB (the live stack when single-env)
4. Optional: local smoke against TestClient
5. Connectivity readiness (12): deploy-checklist includes H0c + frontend/API origin rows
6. CI/CD tip green: project CI watch for tip SHA — hard stop if red

Deploy gate: T1 fail → 14-hotfix or fix-in-place; do not deploy.

Phase 1 — Deploy

Execute deploy per docs/deploy.md §Integration (Render, Docker, etc.). Run migrations on target before serving traffic.

Capture full stdout/stderr.

If deploy succeeds:

• Record app URL and deployment details in deploy-state.md
• Proceed to Phase 2

If deploy fails:

• Parse the error
• Present to user via AskQuestion:
  • "Approve fix — apply recommended fix and retry"
  • "Abort deployment"
  • "Modify — I'll fix differently"
  • "Let me explain / provide more context"
• Retry up to 3 times after fixes
• If still failing after 3 attempts, abort and report

Phase 2 — Smoke Tests (Parallel Agents)

Run backend smokes first, then browser connectivity (H4–H5). Backend-only pass is not sufficient for deployed service hybrid deploys — see connectivity-gates.md.

Operator env (live stack — names may still say “staging”):

# Resolve URLs via project deploy helpers; export API/frontend bases for smokes
export {{ENV_PREFIX}}_STAGING_ADMIN_API_URL=https://{{STAGING_URL_HOST}}

When env_role is staging_as_live / sole stack, state in the report: these URLs are production.

Agent 1 — API connectivity (H1):

• Project staging/live smoke script or health tests (-m live when used)
• Verify TLS and liveness//health on public APIs; dependency probes ok when applicable
• Return: pass/fail, response times — H1 green alone does not pass the stage

Agent 2 — Functional smoke (H2–H3):

• H2: DB / migration head (or project equivalent)
• H3: primary user-journey request (warm cold dependencies first if needed)
• H3 timeout / hang / error = FAIL even if H1 health is green
• Negative API paths per test-plan / config-spec where applicable
• Return: pass/fail, response details

Agent 3 — Browser connectivity (H4–H5) — blocking:

• Run bash scripts/deploy/verify_connectivity.sh (H0c + live H4/H5 when URLs set)
• H4: CORS OPTIONS from each frontend origin → API returns Access-Control-Allow-Origin
• H5: Live JS bundle contains expected API hosts (not localhost)
• Return: pass/fail per tier; cite failing origin/path

Agent 4 — Resource verification:

• DB pool healthy; worker backlog below threshold
• No container crash loops on DO/Modal
• Return: resource status report

If any smoke test fails, present to user:

• "Investigate and fix — the service is not working correctly"
• "Accept — the failure is expected / non-critical"
• "Rollback — stop the deployment"

Phase 3 — Health Check

Verify service health:

• All functions/endpoints respond
• Error rates are acceptable (check logs)
• No container restarts
• Memory/CPU within expected ranges

Record monitoring baseline.

Phase 4 — Generate Changelog + release tags

For stage → main promotes, release prep belongs on stage before the promote PR (semver + CHANGELOG); tags land on main after merge. See docs/deploy.md §Release checklist and doks-promote-from-stage.mdc §Release on promote.

Aggregate commits and PRs into a structured changelog:

1. git log --oneline [last-deploy-tag]..HEAD
2. Group by phase and milestone using [T{id}] and [M{id}] prefixes
3. Prefer updating docs/CHANGELOG.md (dated section); keep root CHANGELOG.md only if the project still uses it:

# Changelog

## [version] — [date]

### Phase 1: [Phase Name]
- **M1**: [milestone] ([PR link])
  - [T1.1] test: [description]
  - [T1.2] feat: [description]
...

4. Tag on main tip after promote merge: git tag vYYYY.MM.DD-deploy (push tag)
5. If publishable packages shipped: confirm semver was bumped on stage; run pypi-release-checklist, then push per-package tags (tac2iwxxm-v*, …)

Phase 5 — Generate Deploy Report

Write docs/deploy-report.md:

# Deploy Report

> Date: [date]
> Status: deployed
> URL: [deployment URL]

## Pre-Deploy
- Checklist: all items passed (`{artifacts_dir}/reports/deploy-checklist.md`)

## Deployment
- Command: [deploy command]
- Duration: [time]
- Result: SUCCESS

## Smoke Tests
| Test | Status | Response Time |
|------|--------|---------------|
| H1 API connectivity | PASS | [ms] |
| H2 DB | PASS | — |
| H3 RAG ask | PASS | [ms] |
| H4 CORS (browser) | PASS | — |
| H5 Frontend bundle | PASS | — |
| Resources | PASS | — |

## Health Check
- Error rate: [%]
- Avg response time: [ms]
- Container restarts: [N]

## Monitoring Baseline
- [metrics recorded]

## Rollback
- Command: [rollback command]
- Last known good: [commit]

## Changelog
- See CHANGELOG.md

Phase 6 — Summary

Deploy & Smoke Check Complete.

  Deployment:     SUCCESS — [URL]
  Smoke tests:    [N] / [N] passed
  Health:         OK
  Changelog:      CHANGELOG.md (v[version]-deploy)
  Monitoring:     baseline recorded

  Report: docs/deploy-report.md
  State:  docs/deploy-state.md (status: deployed)

Pipeline Complete ✓

  Phase A (Product Planning):       completed
  Phase B (Technical Planning):     completed
  Phase C (Build):                  completed
  Phase D (Verification & Deploy):  completed

  Total stages: 14 (00-13)
  Total issues surfaced: [N]
  Total artifacts: [N] documents

Update workflow-state.yaml overall status to completed. Update docs/deploy-state.md status to deployed.

Rollback

If the user requests a rollback:

1. Execute the rollback command from the deployment plan
2. Update deploy-state.md status to rolled_back
3. Report the rollback and last known good state

Output Rules

1. Pre-deploy checks done: Never deploy without 12-verify-deploy passing.
2. Failures require user choice: Every failure gets AskQuestion with options.
3. Max 3 deploy retries: Prevent infinite loops.
4. Smoke tests are minimal: Quick validation, not full test suite — but must include H4–H5 for multi-origin UI deploys.
5. Never mark deployed if H4/H5 fail without documented user waiver in deploy-checklist.
6. Rollback documented: Every deployment has a rollback command.
7. State persists: Deploy state survives session boundaries.
8. Changelog: Aggregate commits into structured changelog at deploy time.