| name | heater-appb-testing-ci |
| description | Use when writing or fixing tests for HEATER's api/ or web/, when CI is red (ruff format skew, shard failures, coverage floor, openapi-ts-sync, bandit), when a test passes alone but fails in the full suite or on CI, or when a test needs "now", the DB, or the network. |
App B Testing & CI
Writing api tests
- Naming:
tests/api/test_api_*.py = endpoint tests via dependency_overrides (76 files); plain tests/api/test_*.py = unit tests. Follow the convention — reviewers and guards assume it.
- The endpoint-test pattern (copy
tests/api/test_api_chat.py): build a bare FastAPI(), include_router(<router>), then app.dependency_overrides[get_<x>_service] = lambda: _FakeService() and override identity deps (require_app_user, get_auth_verifier, stores → InMemory). Assert status + contract shape, including the 401/402/409 paths. Tenancy example: test_api_tenancy_resolver.py:92 (InMemory user/league/membership stores + fake ClerkVerifier → assigned team beats query param).
- DB-free is mandatory — git worktrees and CI have a fresh EMPTY
draft_tool.db (only the main checkout has the real 26MB one). A test that "needs data" must monkeypatch the engine loaders at source (e.g. test_api_lineup_daily_inputs.py:39 patches src.database.load_park_factors etc.) or use fakes. Never assert row counts from the ambient DB.
- Monkeypatch the BOUND name: if the consumer does
from src.x import y, patch consumer.y, not src.x.y — a source-patch silently misses and the test leans on shared state (passes alone, fails in suite). tests/conftest.py:278-314 demonstrates: it patches load_league_standings in every importing module.
- Time: patch
src.et_today._utcnow (the :19 patch point), never datetime.now — engines are ET-canonical; wrong-clock patches flake after ~8 PM ET and in CI (test_api_matchup_periods.py:56 is the model).
- Network: conftest installs a socket guard — any non-loopback connect raises
NetworkBlockedError (Windows xdist workers wedge on blocking socket.recv; pytest-timeout can detect but not interrupt it). Opt out per-test with @pytest.mark.allow_network or globally HEATER_ALLOW_NETWORK=1. A blocked-network error in a test means MOCK THE FETCH, not allowlist it.
- xdist isolation: each worker gets its own DB via
HEATER_DB_PATH=data/_xdist/draft_tool_<worker>.db (conftest.py:30-42) — don't fight it.
CI shape (.github/workflows/ci.yml)
| Job | Key facts |
|---|
| Lint & Style | ruff==0.14.10 (ci.yml:41) — MUST equal requirements.txt's pin; a mismatched local ruff "formats clean" then CI fails (the 2026-06-23 skew — always python -m ruff format . with the pinned version) |
| Test shards | 4 groups × -n 2 --dist loadfile, duration-balanced by .test_durations (pytest-split) |
| Coverage Floor | full run, --cov-fail-under=60 |
| openapi-ts-sync | pnpm 10 (ci.yml:227 — pnpm 9 misreads pnpm-workspace.yaml's ignoredBuiltDependencies as a packages-less manifest; this exact bug broke CI until #148) → pnpm gen:api → git diff --exit-code web/src/lib/api/generated.ts |
| Security Scan | bandit==1.9.4, api/ only, fails on HIGH |
| Structural guards | run inside shards — test_no_logic_in_routers (AST: no src. imports, no arithmetic assignments in routers), test_openapi_contract (snapshot; regen with python scripts/export_openapi.py), test_api_pro_gating (introspects ROUTER OBJECTS because FastAPI 0.137's lazy include_router hides routes from app.routes — don't "simplify" it to walk app.routes) |
Failure stories
- A wall-clock perf test for the leaders lens needed the live 26MB DB → 0 rows + flaky in worktree/CI; replaced with a count-based DB-free guard (2026-06-20). Perf assertions on ambient data don't survive CI.
- Green fake-service tests hid two real engine-shape bugs (optimizer daily mode: matchup always "", urgency always {}). Fakes prove routing/contracts, NOT seam correctness — pair them with one real-engine smoke or review against engine source.
- Known pre-existing rednesses (don't chase): shard-1 xdist flake
test_draft_service_recommend_engine_failure_logs (passes in isolation; fix deferred by owner) and date/env-sensitive test_ai_keys/test_usage_analytics failures noted 2026-06-27.
When NOT to apply
Engine-layer test conventions (frozen-reference equivalence, TDD on new src modules) → heater-appb-engine-composition. web/ has NO test suite — pnpm build + tsc --noEmit is the frontend gate.
