verify-spec

# Spec Verification Verify that every test in entity surface suites reflects actual protocol behavior as defined by the authoritative sources. Adds traceability links and flags discrepancies before we commit to changing production code. ## Args ``` /verify-spec <entity-name-1> [entity-name-2] ... ``` Entity names (space-separated). Each must already have a test suite from `/surface`. If the suite doesn't exist, the formula STOPS at pre-check. ## Why This Step Exists The pipeline from spec to test has multiple translation layers: ``` adcp spec repo (JSON schemas, OpenAPI) ↓ interpreted into adcp-client-python (Pydantic models) ↓ extracted into docs/test-obligations/ (requirements artifacts) ↓ mapped into tests/unit/test_{entity}.py (surface tests + stubs) ``` Each arrow can introduce errors. Before `/remediate` changes production code to make tests pass, we must confirm the tests expect the RIGHT behavior. ## Sources of Truth (Priority Order) 1. **`adcontextprotocol/adcp`** — The protocol spec. JSON schemas at `dist/schemas/3.0.0-beta.3/core/`, OpenAPI at `static/openapi/`. Local clone: `../adcp` (commit-pinned). GitHub: `https://github.com/adcontextprotocol/adcp` 2. **`adcontextprotocol/adcp-client-python`** — The Python implementation. Pydantic models at `adcp/types/generated_poc/`. Local clone: `../adcp-client-python` (upstream remote). GitHub: `https://github.com/adcontextprotocol/adcp-client-python` 3. **`adcp-req`** (derivative, NOT authoritative) — Requirements artifacts at `../adcp-req/docs/requirements/`. Useful as an INDEX with source links back to (1) and (2). Follow those links to confirm faster than researching from scratch. Never treat adcp-req as the source of truth itself. ## Entity to Spec File Mapping | Entity | JSON Schemas (adcp repo) | Python Types (adcp-client-python) | adcp-req Index | |--------|-------------------------|----------------------------------|----------------| | creative | creative-manifest.json, creative-asset.json, creative-variant.json, creative-filters.json, creative-assignment.json, creative-policy.json, format.json, format-id.json | media_buy/sync_creatives_*, media_buy/list_creatives_* | BR-UC-006/ | | media-buy | media-buy.json, package.json, pricing-option.json, targeting.json, brand-ref.json | media_buy/create_media_buy_*, media_buy/update_media_buy_* | UC-002/, UC-003/ | | delivery | delivery-metrics.json, reporting-webhook.json, reporting-capabilities.json | media_buy/get_media_buy_delivery_* | UC-004/ | ## What You Get For each entity, the test file is updated with: 1. **Spec permalink in each test docstring**: ```python def test_create_media_buy_requires_brand(self): """UC-002-S01: CreateMediaBuyRequest requires brand (BrandReference). Spec: https://github.com/adcontextprotocol/adcp/blob/{COMMIT}/dist/schemas/3.0.0-beta.3/core/media-buy.json#L42 Library: https://github.com/adcontextprotocol/adcp-client-python/blob/{COMMIT}/adcp/types/generated_poc/media_buy/create_media_buy_request.py#L15 """ ``` 2. **Discrepancy report** — tests flagged where expectation doesn't match spec: - `CONFIRMED` — spec supports this test expectation - `UNSPECIFIED` — spec doesn't explicitly address this behavior (implementation-defined) - `CONTRADICTS` — spec says something different than what the test expects - `SPEC_AMBIGUOUS` — spec is unclear, needs interpretation 3. **Verification summary** at the top of the test file: ```python """Entity test suite: creative Spec verification: 2026-02-26 adcp spec commit: 975402d5 adcp-client-python commit: a08805d Verified: 89/95 CONFIRMED, 4 UNSPECIFIED, 2 CONTRADICTS """ ``` ## Protocol ### Step 1: Cook the molecule ```bash python3 .claude/scripts/cook_formula.py \ --formula .claude/formulas/verify-spec.yaml \ --var "ENTITY_NAMES={all_args}" \ --epic-title "Verify spec: {all_args}" ``` **Dry run first** (recommended): ```bash python3 .claude/scripts/cook_formula.py \ --formula .claude/formulas/verify-spec.yaml \ --var "ENTITY_NAMES={all_args}" \ --epic-title "Verify spec: {all_args}" \ --dry-run ``` ### Step 2: Walk the molecule ``` bd ready -> bd show <atom-id> -> execute -> bd close <atom-id> -> repeat ``` Each entity goes through: pre-check -> pin-commits -> extract-expectations -> verify-against-spec -> annotate -> review -> commit. ### Step 3: Done when all atoms closed All entity suites annotated with spec links and verified. Discrepancy report generated. ## Verification Strategy For each test, the verification atom should: 1. **Read the test docstring** to understand what behavior it asserts 2. **Check adcp-req first** (as an index) — follow the source link it provides 3. **Confirm at the source** — read the actual JSON schema or Python type 4. **For stubs**: verify the EXPECTED behavior described in the skip reason 5. **For real tests**: verify the assertions match what the spec defines ### Shortcut via adcp-req adcp-req artifacts list their source in each requirement: ```markdown **Source**: adcp/dist/schemas/3.0.0-beta.3/core/media-buy.json, property `brand` ``` Follow this link directly — it's faster than searching the spec from scratch. But always CONFIRM at the actual source file, don't just trust the adcp-req interpretation. ## Batching Each entity is one batch (unlike /remediate which batches within an entity). For an entity with 130+ tests, the verify atom will be the longest — but it's read-only analysis, not code changes, so it's safe to do in one pass. ## Pipeline Position ``` /surface -> /verify-spec -> /remediate (map) (confirm) (implement) ``` ## See Also - `/surface` — Create the entity test suite (prerequisite) - `/remediate` — Fill verified stubs (runs after this) - `/guard` — Structural guards (orthogonal)