verify-spec

name	verify-spec
description	Verify every test expectation in entity test suites against the authoritative adcp spec and library sources. Adds spec permalinks to each test for traceability. Flags discrepancies where our tests assume behavior the spec doesn't define. Run this after /surface and before /remediate.
args	<entity-name-1> [entity-name-2] ...

Spec Verification

Verify that every test in entity surface suites reflects actual protocol behavior as defined by the authoritative sources. Adds traceability links and flags discrepancies before we commit to changing production code.

Args

/verify-spec <entity-name-1> [entity-name-2] ...

Entity names (space-separated). Each must already have a test suite from /surface. If the suite doesn't exist, the formula STOPS at pre-check.

Why This Step Exists

The pipeline from spec to test has multiple translation layers:

adcp spec repo (JSON schemas, OpenAPI)
    ↓ interpreted into
adcp-client-python (Pydantic models)
    ↓ extracted into
docs/test-obligations/ (requirements artifacts)
    ↓ mapped into
tests/unit/test_{entity}.py (surface tests + stubs)

Each arrow can introduce errors. Before /remediate changes production code to make tests pass, we must confirm the tests expect the RIGHT behavior.

Sources of Truth (Priority Order)

1. adcontextprotocol/adcp — The protocol spec. JSON schemas at dist/schemas/3.0.0-beta.3/core/, OpenAPI at static/openapi/. Local clone: /Users/konst/projects/adcp (commit-pinned). GitHub: https://github.com/adcontextprotocol/adcp

2. adcontextprotocol/adcp-client-python — The Python implementation. Pydantic models at adcp/types/generated_poc/. Local clone: /Users/konst/projects/adcp-client-python (upstream remote). GitHub: https://github.com/adcontextprotocol/adcp-client-python

3. adcp-req (derivative, NOT authoritative) — Requirements artifacts at /Users/konst/projects/adcp-req/docs/requirements/. Useful as an INDEX with source links back to (1) and (2). Follow those links to confirm faster than researching from scratch. Never treat adcp-req as the source of truth itself.

Entity to Spec File Mapping

Entity	JSON Schemas (adcp repo)	Python Types (adcp-client-python)	adcp-req Index
creative	creative-manifest.json, creative-asset.json, creative-variant.json, creative-filters.json, creative-assignment.json, creative-policy.json, format.json, format-id.json	media_buy/sync_creatives_, media_buy/list_creatives_	BR-UC-006/
media-buy	media-buy.json, package.json, pricing-option.json, targeting.json, brand-ref.json	media_buy/create_media_buy_, media_buy/update_media_buy_	UC-002/, UC-003/
delivery	delivery-metrics.json, reporting-webhook.json, reporting-capabilities.json	media_buy/get_media_buy_delivery_*	UC-004/

What You Get

For each entity, the test file is updated with:

1. Spec permalink in each test docstring:

   def test_create_media_buy_requires_brand(self):
       """UC-002-S01: CreateMediaBuyRequest requires brand (BrandReference).
   
       Spec: https://github.com/adcontextprotocol/adcp/blob/{COMMIT}/dist/schemas/3.0.0-beta.3/core/media-buy.json#L42
       Library: https://github.com/adcontextprotocol/adcp-client-python/blob/{COMMIT}/adcp/types/generated_poc/media_buy/create_media_buy_request.py#L15
       """
   

2. Discrepancy report — tests flagged where expectation doesn't match spec:

   • CONFIRMED — spec supports this test expectation
   • UNSPECIFIED — spec doesn't explicitly address this behavior (implementation-defined)
   • CONTRADICTS — spec says something different than what the test expects
   • SPEC_AMBIGUOUS — spec is unclear, needs interpretation

3. Verification summary at the top of the test file:

   """Entity test suite: creative
   
   Spec verification: 2026-02-26
   adcp spec commit: 975402d5
   adcp-client-python commit: a08805d
   Verified: 89/95 CONFIRMED, 4 UNSPECIFIED, 2 CONTRADICTS
   """
   

Protocol

Step 1: Cook the molecule

python3 .claude/scripts/cook_formula.py \
  --formula .claude/formulas/verify-spec.yaml \
  --var "ENTITY_NAMES={all_args}" \
  --epic-title "Verify spec: {all_args}"

Dry run first (recommended):

python3 .claude/scripts/cook_formula.py \
  --formula .claude/formulas/verify-spec.yaml \
  --var "ENTITY_NAMES={all_args}" \
  --epic-title "Verify spec: {all_args}" \
  --dry-run

Step 2: Walk the molecule

bd ready -> bd show <atom-id> -> execute -> bd close <atom-id> -> repeat

Each entity goes through: pre-check -> pin-commits -> extract-expectations -> verify-against-spec -> annotate -> review -> commit.

Step 3: Done when all atoms closed

All entity suites annotated with spec links and verified. Discrepancy report generated.

Verification Strategy

For each test, the verification atom should:

1. Read the test docstring to understand what behavior it asserts
2. Check adcp-req first (as an index) — follow the source link it provides
3. Confirm at the source — read the actual JSON schema or Python type
4. For stubs: verify the EXPECTED behavior described in the skip reason
5. For real tests: verify the assertions match what the spec defines

Shortcut via adcp-req

adcp-req artifacts list their source in each requirement:

**Source**: adcp/dist/schemas/3.0.0-beta.3/core/media-buy.json, property `brand`

Follow this link directly — it's faster than searching the spec from scratch. But always CONFIRM at the actual source file, don't just trust the adcp-req interpretation.

Batching

Each entity is one batch (unlike /remediate which batches within an entity). For an entity with 130+ tests, the verify atom will be the longest — but it's read-only analysis, not code changes, so it's safe to do in one pass.

Pipeline Position

/surface  ->  /verify-spec  ->  /remediate
  (map)        (confirm)        (implement)

See Also

• /surface — Create the entity test suite (prerequisite)
• /remediate — Fill verified stubs (runs after this)
• /guard — Structural guards (orthogonal)