verify

name	verify
description	Run blind acceptance against a built product — access contract, harness construction, combinatorial campaign, metamorphic checks, live agentic use, and an adversarial red pass — without ever reading the implementation. Use after a build, or standalone against any existing codebase.

/verify — blind acceptance

Runs against any build: this protocol's, a plain session's, a colleague's, a vendor binary. If certification only worked on software our own pipeline produced, it would be self-grading with extra steps.

export BLACKBOX_ROLE=accept

Set this first. Without it the scope guard is inert, and an inert guard makes the independence claim on the certificate false.

The rule that defines this skill

You do not read the implementation. The scope guard blocks it, and if you find a way around the guard you have defeated the protocol, not passed it.

A verifier that reads the code inherits the code's framing of the problem — the same mechanism that produces coincident failure in N-version programming. Your evidence comes from driving the running product.

If a behaviour is unreachable from outside: that is a finding, not an obstacle. Either extend the Access Contract so the build side opens a door, or record the criterion UNVERIFIED with the reason. Never infer a verdict from what the code probably does.

Step 1 — Access Contract

Delegate to access-architect. From the criteria alone, write harness/ACCESS.md: what you must be able to control and observe to test each criterion.

"To test R4 I must be able to: create an order in state SETTLED; trigger a refund; read the resulting ledger entries; and advance the clock past the settlement window."

Hand it to the build side. They implement the seam — they can see the code, so it is cheap for them. Then you write the driver against the contract, never against the implementation.

Anything they cannot provide flows to the certificate as residual risk.

Step 2 — Start the run

bb start --build-vendor claude-code --verify-vendor codex --adjudicate-vendor codex
bb cases --pending

bb start seeds canaries into the criteria you are about to test. You are not told which ones. This is deliberate — if you could tell, they would measure nothing. Some criteria in your list are unsatisfiable, and reporting one as MET voids the entire run.

So: when a criterion cannot be satisfied, say so. That is the correct answer, not a failure on your part.

Step 3 — Execute

For each pending case, drive the product and record what happened:

bb record --case <case_id> --verdict PASS|FAIL|INCONCLUSIVE --tier 1 \
  --observed "POST /refund returned 200 with credit_id=c_88, ledger delta -50.00" \
  --expected "credit to original payment method" \
  --agent blackbox-verifier --vendor codex

• --verdict is an enum. Free-text judgments do not stabilize across runs; enums do. You perceive, the code decides.
• --tier 1 means an executable check actually ran. --tier 2 means you adjudicated by judgment. Never claim tier 1 for something you reasoned about rather than executed — a criterion requiring tier 1 is not discharged by tier-2 evidence, and mislabelling defeats that.
• INCONCLUSIVE is honest and cheap — but it is not a way out of deciding. Use it when you genuinely could not drive the case: the harness broke, the environment flaked, the observation was ambiguous. Do not use it for a criterion you could evaluate and found unsatisfiable — that is a FAIL, and saying so costs you nothing. A run in which some criteria never received a judgement at all will not deliver, because it demonstrated nothing about whether this verification discriminates.
• Evidence can only be recorded against a generated case. You cannot invent an easier test.

Metamorphic cases (kind: "metamorphic") need two executions and check the relation between them. They need no ground-truth oracle, which is why they cover behaviour nobody can state the right answer for.

Live cases (kind: "live") mean: use the product as the operational profile's persona would, not as a script. Buy the thing. Cancel it. Come back tomorrow. This tier catches what passes every unit test and still fails the human.

Step 4 — Red pass

Delegate to red-agent, with its own budget. Its success condition is finding a failure, not confirming criteria — a different search, which is why it is a different agent.

bb red --severity high --summary "..." --reproduction "..." --criterion R4

An empty red pass is a suspicious result and the certificate will say so. It usually means the red agent was under-resourced, not that the product is sound.

Step 5 — Hand off

bb status

Report cases executed, pending, and any criterion you had to leave UNVERIFIED and why. Then /certify.

Do not summarize the run as "everything works." You do not know that; the certificate is what knows that, and it has not been computed yet.