test-everything

name	test-everything
description	Run the full cupertino post-promote verification pipeline. Clean → build → test → 3 CI guards → mock-ai-agent → 27-command CLI smoke → MCP tool probe → DB integrity. Optional --deep adds the canonical-type wrong-winner audit, BUG 5 scope check, and search→read round-trip on the top-50 symbols. Single PASS/FAIL verdict + per-step table written to ~/Downloads/cupertino-test-everything-<date>.md.

/cupertino-test-everything

The post-promote verification pipeline. Run this after every FF push from develop → main (or after any non-trivial code change on develop) to confirm nothing regressed.

Saves the ~10-step manual dance: clean build → test → 3 guards → mock → CLI sweep → MCP probe → DB integrity → optional deep audits. Output is a single PASS/FAIL verdict plus a per-step results table.

Invocation

/cupertino-test-everything                  # full pipeline
/cupertino-test-everything --no-clean       # skip swift package clean (faster re-run)
/cupertino-test-everything --deep           # add the optional deep audits (#610 wrong-winner + BUG 5 scope + round-trip)
/cupertino-test-everything --no-clean --deep

Preflight

1. cd to the cupertino repo root (the one with Packages/Package.swift + CHANGELOG.md).
2. Confirm the cupertino binary is built or buildable: xcrun swift build -c release --package-path Packages will produce / refresh Packages/.build/arm64-apple-macosx/release/cupertino.
3. Confirm the shipped DBs are in place: ls ~/.cupertino/{search,packages,samples}.db. If any are missing, run cupertino setup first. Step 9 (DB integrity) needs them.
4. Set the report path: REPORT=~/Downloads/cupertino-test-everything-$(date +%Y-%m-%d-%H%M).md.
5. Schema-gap detection (#639). Run this once up front and export SCHEMA_GAP_REASON so Steps 7, 8, 11 can short-circuit search.db-dependent probes when the binary's expected schema doesn't match the bundle's PRAGMA user_version:

BIN="Packages/.build/arm64-apple-macosx/release/cupertino"
SQLITE=/usr/bin/sqlite3
export SCHEMA_GAP_REASON=""
if [ -f ~/.cupertino/search.db ]; then
    ON_DISK_VER=$("$SQLITE" ~/.cupertino/search.db "PRAGMA user_version;" 2>/dev/null)
    # Probe binary's expected version by opening a scratch DB.
    SCRATCH=/tmp/test-everything-schema-probe-$$.db
    touch "$SCRATCH"
    "$BIN" inheritance _probe_ --search-db "$SCRATCH" >/dev/null 2>&1
    BIN_VER=$("$SQLITE" "$SCRATCH" "PRAGMA user_version;" 2>/dev/null)
    rm -f "$SCRATCH"
    if [ -n "$ON_DISK_VER" ] && [ -n "$BIN_VER" ] && [ "$ON_DISK_VER" != "$BIN_VER" ]; then
        export SCHEMA_GAP_REASON="bundle schema $ON_DISK_VER ≠ binary schema $BIN_VER: re-run after \`cupertino save --docs\` or v1.2.0 publish"
        echo "⚠ Schema gap detected: $SCHEMA_GAP_REASON"
        echo "   Steps 7b / 8 / 11 search.db-dependent probes will SKIP, not FAIL."
    fi
fi

Between schema bumps (binary schemaVersion gets bumped for new FTS columns / tables, bundle ships at the previous schema), develop's binary expects a higher version than the shipped bundle. The schema-stamp guard from #635 correctly refuses to open the mismatched DB, which means every search.db-dependent probe gets rc=1. Pre-#639 those collected as fail, polluting the verdict with what is actually an expected-state gap. Post-#639 they emit assert_skip with the gap reason instead.

6. Open the report with the header:

cat > "$REPORT" <<EOF
# cupertino /test-everything report

- Date: $(date -u +%Y-%m-%dT%H:%M:%SZ)
- Branch: $(git rev-parse --abbrev-ref HEAD) @ $(git rev-parse --short HEAD)
- Args: ${1:-(none)}

| # | Step | Result | Duration | Notes |
|---|------|--------|----------|-------|
EOF

Each step appends one row to $REPORT plus prints to stdout for live progress.

Step 1: clean

Unless --no-clean is passed:

START=$(date +%s)
xcrun swift package --package-path Packages clean
DURATION=$(( $(date +%s) - START ))
echo "| 1 | swift package clean | ✅ | ${DURATION}s | (skipped if --no-clean) |" >> "$REPORT"

Failure criterion: non-zero exit. Skip with --no-clean to keep the incremental build cache.

Step 2: release build

START=$(date +%s)
BUILD_LOG=$(mktemp)
xcrun swift build -c release --package-path Packages 2>&1 | tee "$BUILD_LOG"
EXIT=${PIPESTATUS[0]}
DURATION=$(( $(date +%s) - START ))
WARNINGS=$(grep -c "warning:" "$BUILD_LOG")
[ "$EXIT" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 2 | swift build -c release | $VERDICT | ${DURATION}s | ${WARNINGS} warnings, exit ${EXIT} |" >> "$REPORT"
rm "$BUILD_LOG"

Failure criterion: non-zero exit, OR warnings count > 0 on a clean build (warning escalation is project policy per #586).

Step 3: swift test

START=$(date +%s)
TEST_LOG=$(mktemp)
xcrun swift test --package-path Packages 2>&1 | tee "$TEST_LOG"
EXIT=${PIPESTATUS[0]}
DURATION=$(( $(date +%s) - START ))
# Extract counts from the final summary line: "Test run with N tests in M suites passed/failed"
SUMMARY=$(grep -E "Test run with " "$TEST_LOG" | tail -1)
PASS=$(echo "$SUMMARY" | grep -oE "[0-9]+ tests" | head -1 | grep -oE "[0-9]+")
[ "$EXIT" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 3 | swift test | $VERDICT | ${DURATION}s | ${PASS}/${PASS} ${VERDICT}, exit ${EXIT} |" >> "$REPORT"
rm "$TEST_LOG"

Failure criterion: non-zero exit, OR any failing test in the output.

Step 3a: CLITests + CLICommandTests (focused, separately reported)

swift test (Step 3) already runs these as part of the full suite, but failures inside a 1808-test bundle are hard to localise. Re-running just the CLI test targets with their counts in their own row makes regression triage faster.

START=$(date +%s)
CLI_LOG=$(mktemp)
xcrun swift test --package-path Packages --filter "CLITests|CLICommandTests" 2>&1 | tee "$CLI_LOG"
EXIT=${PIPESTATUS[0]}
DURATION=$(( $(date +%s) - START ))
PASS=$(grep -oE "Test run with [0-9]+ tests" "$CLI_LOG" | tail -1 | grep -oE "[0-9]+")
[ "$EXIT" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 3a | CLI unit tests | $VERDICT | ${DURATION}s | ${PASS} cases, exit ${EXIT} |" >> "$REPORT"
rm "$CLI_LOG"

Covers Packages/Tests/CLITests/ (CLI surface tests: Cupertino.swift, top-level flags, help text) and Packages/Tests/CLICommandTests/ (per-subcommand suites: DoctorTests/, SaveTests/, ServeTests/, FetchTests/).

Step 3b: MockAIAgentTests (focused, separately reported)

Mirror of Step 3a for the MockAIAgent target's unit tests. Distinct from Step 6, which runs the agent end-to-end against a live server: Step 3b runs the in-process unit tests for the agent's probe sequence + assertion helpers.

START=$(date +%s)
MOCK_TEST_LOG=$(mktemp)
xcrun swift test --package-path Packages --filter "MockAIAgentTests" 2>&1 | tee "$MOCK_TEST_LOG"
EXIT=${PIPESTATUS[0]}
DURATION=$(( $(date +%s) - START ))
PASS=$(grep -oE "Test run with [0-9]+ tests" "$MOCK_TEST_LOG" | tail -1 | grep -oE "[0-9]+")
[ "$EXIT" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 3b | MockAIAgent unit tests | $VERDICT | ${DURATION}s | ${PASS} cases, exit ${EXIT} |" >> "$REPORT"
rm "$MOCK_TEST_LOG"

Covers Packages/Tests/MockAIAgentTests/, the in-process unit tests for the agent's phase sequencing, assertion helpers, and JSON-RPC frame parsing. Distinct from Step 6's end-to-end run against a live cupertino serve.

Step 4: 3 CI guards

Run all three in sequence; any failure flips this step to ❌.

START=$(date +%s)
GUARD_STATUS="✅"
GUARD_DETAIL=""
for guard in check-package-purity check-target-foundation-only check-docs-commands-drift; do
    if scripts/${guard}.sh > /tmp/${guard}.log 2>&1; then
        GUARD_DETAIL="${GUARD_DETAIL}${guard} ✓, "
    else
        GUARD_STATUS="❌"
        GUARD_DETAIL="${GUARD_DETAIL}${guard} ✗, "
    fi
done
DURATION=$(( $(date +%s) - START ))
echo "| 4 | 3 CI guards | $GUARD_STATUS | ${DURATION}s | ${GUARD_DETAIL%, } |" >> "$REPORT"

Failure criterion: any guard returns non-zero.

Step 5: build mock-ai-agent

START=$(date +%s)
xcrun swift build --product MockAIAgent --package-path Packages -c release 2>&1 > /tmp/mock-build.log
EXIT=$?
DURATION=$(( $(date +%s) - START ))
[ "$EXIT" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 5 | build mock-ai-agent | $VERDICT | ${DURATION}s | exit ${EXIT} |" >> "$REPORT"

Step 6: run mock-ai-agent end-to-end

START=$(date +%s)
MOCK_LOG=$(mktemp)
Packages/.build/arm64-apple-macosx/release/MockAIAgent 2>&1 | tee "$MOCK_LOG"
EXIT=${PIPESTATUS[0]}
DURATION=$(( $(date +%s) - START ))
# Expected: 6 phases (init / tools/list / tools/call search / resources/list / resources/read / shutdown), all green, exit 0, <30s.
PHASES_OK=$(grep -cE "✓|✅" "$MOCK_LOG" || true)
if [ "$EXIT" = "0" ] && [ "$DURATION" -lt 30 ] && [ "$PHASES_OK" -ge 6 ]; then
    VERDICT="✅"
else
    VERDICT="❌"
fi
echo "| 6 | mock-ai-agent E2E | $VERDICT | ${DURATION}s | phases=${PHASES_OK}/6, exit ${EXIT} |" >> "$REPORT"
rm "$MOCK_LOG"

Failure criterion: non-zero exit, OR duration ≥ 30s, OR fewer than 6 phase ticks in output.

Step 7: 27-command CLI smoke

Each command runs against the release binary with a perl-alarm timeout (macOS lacks timeout(1)). All must exit 0 unless explicitly expected to fail. Output is captured to per-command logs; assertions check exit codes + non-empty bodies for the content commands.

BIN="Packages/.build/arm64-apple-macosx/release/cupertino"
PROJID=$("$BIN" list-samples --limit 1 2>&1 | grep "ID:" | head -1 | awk '{print $2}')
declare -a COMMANDS=(
    # doctor
    "doctor"
    "doctor --save"
    # search × 8 sources (apple-docs, apple-archive, hig, swift-evolution, swift-org, swift-book, packages, samples)
    "search 'View' --source apple-docs --limit 1"
    "search 'application architecture' --source apple-archive --limit 1"
    "search 'navigation' --source hig --limit 1"
    "search 'actors' --source swift-evolution --limit 1"
    "search 'frameworks' --source swift-org --limit 1"
    "search 'protocols' --source swift-book --limit 1"
    "search 'logging' --source packages --limit 1"
    "search 'AVCam' --source samples --limit 1"
    # read four forms
    "read apple-docs://swiftui/view"
    "read https://developer.apple.com/documentation/swiftui/view"
    "read apple-docs://swiftui/view --format markdown"
    "read apple-docs://swiftui/view --format json"
    # list-frameworks + list-samples
    "list-frameworks --limit 5"
    "list-samples --limit 5"
    # samples: read-sample + read-sample-file (post-#594 path)
    "read-sample $PROJID"
    "read-sample-file $PROJID AVCam/AVCamApp.swift"
    # package-search
    "search-packages alamofire --limit 3"
    # --help for the headline subcommands
    "--help"
    "search --help"
    "read --help"
    "serve --help"
    "save --help"
    "fetch --help"
    "doctor --help"
)
START=$(date +%s)
FAILS=0
PASSES=0
for cmd in "${COMMANDS[@]}"; do
    OUT=$(perl -e 'alarm 30; exec @ARGV' "$BIN" $cmd 2>&1)
    if [ $? -eq 0 ] && [ -n "$OUT" ]; then
        PASSES=$((PASSES + 1))
    else
        FAILS=$((FAILS + 1))
        echo "FAIL: $cmd" >> /tmp/cli-smoke-fails.log
    fi
done
DURATION=$(( $(date +%s) - START ))
TOTAL=${#COMMANDS[@]}
# #639: when the schema-gap is detected at preflight, treat search/read
# command failures as expected (skip-equivalent) and downgrade the verdict
# accordingly. The verdict line carries a `(schema-gap)` note so the report
# reader knows the failures aren't regressions.
if [ "$FAILS" = "0" ]; then
    VERDICT="✅"
    NOTE="${PASSES}/${TOTAL} ok, ${FAILS} fail"
elif [ -n "$SCHEMA_GAP_REASON" ]; then
    VERDICT="⚪"
    NOTE="${PASSES}/${TOTAL} ok, ${FAILS} skip (schema-gap: $SCHEMA_GAP_REASON)"
else
    VERDICT="❌"
    NOTE="${PASSES}/${TOTAL} ok, ${FAILS} fail"
fi
echo "| 7 | CLI smoke (${TOTAL}-cmd sweep) | $VERDICT | ${DURATION}s | $NOTE |" >> "$REPORT"

Failure criterion: any command exits non-zero OR returns an empty body. Failing command names go to /tmp/cli-smoke-fails.log. Search/read commands needing search.db will fail when SCHEMA_GAP_REASON is set; the verdict downgrades to ⚪ (skip) instead of ❌ (fail) in that case so the report stays honest about the state.

Note: the command list is 27. The count comes from doctor (2) + search-by-source (8) + read four forms (4) + list-* (2) + samples (2) + package-search (1) + --help (7) + a buffer for adjustments. Adjust the array if cupertino's surface grows.

Step 8: MCP tool probe

Drive the server through 10 tools across happy + edge paths. Each probe sends initialize + the tool call + asserts a non-error response.

BIN="Packages/.build/arm64-apple-macosx/release/cupertino"
START=$(date +%s)
declare -a PROBES=(
    # Happy paths: 6 tools
    '{"method":"tools/list"}'
    '{"method":"tools/call","params":{"name":"search_docs","arguments":{"query":"View","sources":["apple-docs"]}}}'
    '{"method":"tools/call","params":{"name":"read_document","arguments":{"uri":"apple-docs://swiftui/view"}}}'
    '{"method":"tools/call","params":{"name":"list_frameworks","arguments":{}}}'
    '{"method":"tools/call","params":{"name":"search_symbols","arguments":{"name":"View","kinds":["protocol"]}}}'
    '{"method":"tools/call","params":{"name":"search_conformances","arguments":{"protocol":"View"}}}'
    # Edge cases: 4
    '{"method":"resources/list","params":{"cursor":"INVALID_CURSOR_xyz"}}'   # malformed cursor → JSON-RPC -32602 (#601)
    '{"method":"tools/call","params":{"name":"search_docs","arguments":{"query":""}}}'   # empty query → -32602 (#602)
    '{"method":"unknown/method","params":{}}'                                # unknown method → -32601 (#611)
    '{"method":"notifications/initialized"}'                                  # notification → ZERO response (#613 item 3)
)
PASSES=0
FAILS=0
for probe in "${PROBES[@]}"; do
    # Wrap into a JSON-RPC frame with id=2 (or no id for notifications)
    HAS_METHOD=$(echo "$probe" | python3 -c "import json,sys; print(json.loads(sys.stdin.read()).get('method',''))")
    if [[ "$HAS_METHOD" == "notifications/"* ]]; then
        FRAME=$(echo "$probe" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); d['jsonrpc']='2.0'; print(json.dumps(d))")
        EXPECT_FRAMES=1   # only the init response
    else
        FRAME=$(echo "$probe" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); d['jsonrpc']='2.0'; d['id']=2; print(json.dumps(d))")
        EXPECT_FRAMES=2   # init + the probe's response/error
    fi
    INIT='{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}'
    OUT=$(printf '%s\n%s\n' "$INIT" "$FRAME" | perl -e 'alarm 10; exec @ARGV' "$BIN" serve --no-reap 2>/dev/null)
    GOT=$(echo "$OUT" | grep -c "^{")
    if [ "$GOT" = "$EXPECT_FRAMES" ]; then
        PASSES=$((PASSES + 1))
    else
        FAILS=$((FAILS + 1))
        echo "FAIL: expected ${EXPECT_FRAMES} frames, got ${GOT} for: ${probe}" >> /tmp/mcp-probe-fails.log
    fi
done
DURATION=$(( $(date +%s) - START ))
TOTAL=${#PROBES[@]}
[ "$FAILS" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 8 | MCP probe (${TOTAL} tools/edges) | $VERDICT | ${DURATION}s | ${PASSES}/${TOTAL} ok, ${FAILS} fail |" >> "$REPORT"

Concurrent-serve sibling check: spawn 2 parallel cupertino serve instances against the same ~/.cupertino/*.db. Both must complete cleanly without DB corruption. Run after the 10 probes:

# concurrent test: 2 sequential serves + 2 parallel serves
for i in 1 2; do
    printf '%s\n%s\n' "$INIT" '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
        | perl -e 'alarm 10; exec @ARGV' "$BIN" serve --no-reap > /tmp/concurrent-${i}.out 2>/dev/null &
done
wait
PARALLEL_OK=0
for i in 1 2; do
    grep -q '"id":2' /tmp/concurrent-${i}.out && PARALLEL_OK=$((PARALLEL_OK + 1))
done
[ "$PARALLEL_OK" = "2" ] && CONC_VERDICT="✅" || CONC_VERDICT="❌"
echo "| 8b | concurrent serves (2 parallel) | $CONC_VERDICT | - | ${PARALLEL_OK}/2 ok |" >> "$REPORT"

Step 8c: CLI ↔ MCP equivalence probe (many questions, both paths)

Cupertino exposes the same data via two transports: the CLI (cupertino search, cupertino read) and MCP-over-stdio (tools/call search_docs, tools/call read_document). They should return equivalent results. This step issues a wide spread of queries across both paths and asserts the top result URI matches between transports for every query.

Catches divergences like:

• A ranker change that fires on the CLI but not the MCP path (or vice versa)
• A source filter that's parsed differently between transports
• A new MCP tool that doesn't have a CLI counterpart (intentional or accidental)

BIN="Packages/.build/arm64-apple-macosx/release/cupertino"
START=$(date +%s)

# ~30 varied queries: canonical types, common APIs, multi-word topics,
# special chars, source-filtered, framework-filtered, etc. Mix of search
# + read shapes.
declare -a SEARCH_QUERIES=(
    # Canonical Swift types (post-#610 Class A fixes)
    "Task" "View" "String" "Array" "Hashable" "Equatable" "Codable" "Identifiable" "Sendable"
    # Common UIKit / AppKit
    "URLSession" "JSONDecoder" "DateFormatter" "FileManager"
    # SwiftUI surfaces
    "List" "NavigationStack" "ScrollView" "VStack"
    # Multi-word queries
    "structured concurrency" "async sequence" "view modifier"
    # Edge tokens
    "@available" "Task()" "?.flatMap"
    # Less canonical, longer-tail
    "actor isolation" "main actor" "task local"
    # Cross-source (search.db has 6 sources)
    "DocC" "WWDC" "Swift Evolution proposal"
)
declare -a READ_URIS=(
    "apple-docs://swiftui/view"
    "apple-docs://foundation/url"
    "apple-docs://swift/task"
    "swift-evolution://SE-0304"
    "hig://technologies/technologies-appledeveloperdocumentation"
    "apple-archive://10000047i/RevisionHistory"
)

PASSES=0
FAILS=0

# Search queries: compare top-1 URI from CLI vs MCP for each
for q in "${SEARCH_QUERIES[@]}"; do
    CLI_URI=$("$BIN" search "$q" --limit 1 --source apple-docs 2>/dev/null \
              | grep -oE 'apple-docs://[a-z0-9_/().-]+' | head -1)
    MCP_OUT=$(printf '%s\n%s\n' \
        '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}' \
        "$(jq -nc --arg q "$q" '{jsonrpc:"2.0",id:2,method:"tools/call",params:{name:"search_docs",arguments:{query:$q,sources:["apple-docs"],limit:1}}}')" \
        | perl -e 'alarm 10; exec @ARGV' "$BIN" serve --no-reap 2>/dev/null)
    MCP_URI=$(echo "$MCP_OUT" | grep -oE 'apple-docs://[a-z0-9_/().-]+' | head -1)
    if [ -n "$CLI_URI" ] && [ "$CLI_URI" = "$MCP_URI" ]; then
        PASSES=$((PASSES + 1))
    else
        FAILS=$((FAILS + 1))
        echo "DIVERGE: '$q' → CLI=$CLI_URI MCP=$MCP_URI" >> /tmp/cli-mcp-divergence.log
    fi
done

# Read URIs: compare body length (proxy for content equivalence)
for uri in "${READ_URIS[@]}"; do
    CLI_BYTES=$("$BIN" read "$uri" 2>/dev/null | wc -c | tr -d ' ')
    MCP_OUT=$(printf '%s\n%s\n' \
        '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}' \
        "$(jq -nc --arg u "$uri" '{jsonrpc:"2.0",id:2,method:"tools/call",params:{name:"read_document",arguments:{uri:$u}}}')" \
        | perl -e 'alarm 10; exec @ARGV' "$BIN" serve --no-reap 2>/dev/null)
    MCP_BYTES=$(echo "$MCP_OUT" | wc -c | tr -d ' ')
    # Acceptable variance: CLI emits a text/json wrapper, MCP wraps in a JSON-RPC frame;
    # asserting both are > 200 bytes is the bare correctness check (a hung path returns
    # the ~130-byte empty wrapper).
    if [ "$CLI_BYTES" -gt 200 ] && [ "$MCP_BYTES" -gt 200 ]; then
        PASSES=$((PASSES + 1))
    else
        FAILS=$((FAILS + 1))
        echo "DIVERGE: read '$uri' → CLI=${CLI_BYTES}b MCP=${MCP_BYTES}b" >> /tmp/cli-mcp-divergence.log
    fi
done

DURATION=$(( $(date +%s) - START ))
TOTAL=$((${#SEARCH_QUERIES[@]} + ${#READ_URIS[@]}))
# #639: same downgrade pattern as Step 7. With the schema gap active,
# every query and read hits the schema-rejection path; divergences are
# "expected empty on both sides" not real regressions.
if [ "$FAILS" = "0" ]; then
    VERDICT="✅"
    NOTE="${PASSES}/${TOTAL} match, ${FAILS} diverge"
elif [ -n "$SCHEMA_GAP_REASON" ]; then
    VERDICT="⚪"
    NOTE="${PASSES}/${TOTAL} match, ${FAILS} skip (schema-gap: $SCHEMA_GAP_REASON)"
else
    VERDICT="❌"
    NOTE="${PASSES}/${TOTAL} match, ${FAILS} diverge"
fi
echo "| 8c | CLI ↔ MCP equivalence probe (${TOTAL} queries) | $VERDICT | ${DURATION}s | $NOTE |" >> "$REPORT"

Failure criterion: any divergence between CLI top-1 URI and MCP top-1 URI for the same query, OR either transport returns <200 bytes on a read that should produce real content. Divergence list goes to /tmp/cli-mcp-divergence.log. With SCHEMA_GAP_REASON set, divergences downgrade to ⚪ (skip) instead of ❌ (fail): both sides return empty because the DB rejects the binary's open call, not because of a CLI↔MCP behavioural drift.

Tuning the query list: when a new source / framework / canonical-type query class lands, append the new shape to SEARCH_QUERIES and the relevant URI to READ_URIS. The list is intentionally generous (~30 queries); cupertino search is fast (~50 ms per call) so the whole probe runs in ~30s.

Step 9: DB integrity

START=$(date +%s)
ALL_OK=1
for db in search.db packages.db samples.db; do
    if [ -f ~/.cupertino/$db ]; then
        INTEGRITY=$(sqlite3 ~/.cupertino/$db "PRAGMA integrity_check;" 2>&1)
        FK=$(sqlite3 ~/.cupertino/$db "PRAGMA foreign_key_check;" 2>&1)
        if [ "$INTEGRITY" = "ok" ] && [ -z "$FK" ]; then
            echo "  $db: integrity=ok, foreign_keys=ok"
        else
            ALL_OK=0
            echo "  $db: integrity=$INTEGRITY, foreign_keys=$FK"
        fi
    else
        echo "  $db: not found (skip)"
    fi
done
DURATION=$(( $(date +%s) - START ))
[ "$ALL_OK" = "1" ] && VERDICT="✅" || VERDICT="❌"
echo "| 9 | DB integrity (3 DBs) | $VERDICT | ${DURATION}s | PRAGMA integrity_check + foreign_key_check |" >> "$REPORT"

Failure criterion: PRAGMA integrity_check returns anything other than ok on any DB; OR PRAGMA foreign_key_check returns any rows.

Step 11: per-bug live regression smoke

For every bug shipped in the session that produced PRs #604 through #621, run one live-binary assertion that would catch a deployment-shape regression even when the unit tests stay green. Unit tests cover most of these, but the #618 case (all 1808 unit tests passed and the release binary still hung forever) proved that unit-test green doesn't imply deployment-shape correctness. Belt-and-braces.

Each probe must complete in <5s; the whole step runs in ~30s.

BIN="Packages/.build/arm64-apple-macosx/release/cupertino"
SQLITE=/usr/bin/sqlite3
START=$(date +%s)
FAILS=0; PASSES=0; SKIPS=0
LOG=/tmp/test-everything-step11.log
: > "$LOG"

assert_pass() { echo "  ✅ $1" | tee -a "$LOG"; PASSES=$((PASSES+1)); }
assert_fail() { echo "  ❌ $1: $2" | tee -a "$LOG"; FAILS=$((FAILS+1)); }
assert_skip() { echo "  ⚪ $1: $2" | tee -a "$LOG"; SKIPS=$((SKIPS+1)); }

# Helper used by probes that need ~/.cupertino/search.db to be openable.
# Reads the `SCHEMA_GAP_REASON` exported by the Preflight check (#639).
# Emits `assert_skip` with the gap reason rather than `assert_fail` when
# the binary and bundle versions don't line up, falsely flagging an
# expected-state mismatch as failure pollutes the verdict.
assert_skip_if_schema_gap() {
    local probe_name="$1"
    if [ -n "$SCHEMA_GAP_REASON" ]; then
        assert_skip "$probe_name" "$SCHEMA_GAP_REASON"
        return 0
    fi
    return 1
}

# 11.1 #618: cupertino serve exits on stdin EOF (<2s, exit 0)
t0=$(date +%s)
echo "" | perl -e 'alarm 5; exec @ARGV' "$BIN" serve --no-reap > /dev/null 2>&1
rc=$?; t1=$(date +%s); d=$((t1-t0))
if [ "$rc" = "0" ] && [ "$d" -lt 2 ]; then assert_pass "#618 serve EOF exit (${d}s, rc=$rc)"; else assert_fail "#618 serve EOF" "${d}s, rc=$rc"; fi
pkill -f "cupertino serve" 2>/dev/null

# 11.2 #611: MCP methodNotFound message is "Method not found: …" (NO 'MCP.Core.Protocols.' prefix)
INIT='{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"p","version":"0"}}}'
OUT=$(printf '%s\n{"jsonrpc":"2.0","id":2,"method":"unknown/method","params":{}}\n' "$INIT" \
      | perl -e 'alarm 5; exec @ARGV' "$BIN" serve --no-reap 2>/dev/null)
if echo "$OUT" | grep -q '"message":"Method not found:' && ! echo "$OUT" | grep -q 'MCP.Core.Protocols.Method'; then
    assert_pass "#611 MCP error message clean (no namespace leak)"
else
    assert_fail "#611 MCP error message" "got: $(echo "$OUT" | grep -oE '"message":"[^"]*"' | head -1)"
fi

# 11.3 #614: 'cupertino search String --source apple-docs' returns swift/string (uses post-fix kind for ranking)
if ! assert_skip_if_schema_gap "#614 String ranking"; then
    URI=$("$BIN" search "String" --limit 1 --source apple-docs 2>/dev/null | grep -oE 'apple-docs://[a-z/-]+' | head -1)
    [ "$URI" = "apple-docs://swift/string" ] && assert_pass "#614 kind-extraction surfaces canonical String (#610 Class A)" || assert_fail "#614 String ranking" "got: $URI"
fi

# 11.4 #597: cupertino save --base-dir <X> does not write to ~/.cupertino
ISO=/tmp/test-everything-base-dir-iso-$$
mkdir -p "$ISO"
PROD_SAMPLES_MTIME_BEFORE=$(stat -f "%m" ~/.cupertino/samples.db 2>/dev/null || echo "0")
# Run save --help only; full --dry-run save is too slow for this step. The --help check
# proves the flag exists + the unit tests in SaveBaseDirIsolationTests pin the actual isolation.
if "$BIN" save --help 2>&1 | grep -q "\\-\\-base-dir"; then
    assert_pass "#597 save --base-dir flag exists"
else
    assert_fail "#597 save --base-dir" "flag missing from save --help"
fi
rmdir "$ISO" 2>/dev/null

# 11.5 #598: cupertino-tui headless exit (<2s, exit 0, stderr message)
TUI_BIN="Packages/.build/arm64-apple-macosx/release/cupertino-tui"
if [ -f "$TUI_BIN" ]; then
    t0=$(date +%s)
    STDERR=$(printf 'q\n' | perl -e 'alarm 5; exec @ARGV' "$TUI_BIN" 2>&1 > /dev/null)
    rc=$?; t1=$(date +%s); d=$((t1-t0))
    if [ "$rc" = "0" ] && [ "$d" -lt 2 ] && echo "$STDERR" | grep -qi "TTY"; then
        assert_pass "#598 cupertino-tui headless guard (${d}s, rc=0, TTY msg)"
    else
        assert_fail "#598 cupertino-tui headless" "${d}s, rc=$rc, stderr=$(echo "$STDERR" | head -1)"
    fi
else
    assert_skip "#598 cupertino-tui headless" "binary not built"
fi

# 11.6 #68: cupertino doctor (no flag) does NOT include 'Raw corpus directories'
OUT=$("$BIN" doctor 2>&1)
if ! echo "$OUT" | grep -q "Raw corpus directories"; then
    assert_pass "#68 doctor default hides corpus dirs"
else
    assert_fail "#68 doctor default" "still shows Raw corpus directories"
fi

# 11.7 #68 inverse: cupertino doctor --save DOES include 'Raw corpus directories'
OUT=$("$BIN" doctor --save 2>&1)
if echo "$OUT" | grep -q "Raw corpus directories"; then
    assert_pass "#68 doctor --save shows corpus dirs"
else
    assert_fail "#68 doctor --save" "did NOT include Raw corpus directories"
fi

# 11.8 #607: read swift-evolution / hig / apple-archive URIs return >500 bytes (post indexer + read-side fix)
if ! assert_skip_if_schema_gap "#607 read multi-source"; then
    for u in "swift-evolution://SE-0304" "hig://technologies/technologies-appledeveloperdocumentation" "apple-archive://10000047i/RevisionHistory"; do
        BYTES=$("$BIN" read "$u" 2>/dev/null | wc -c | tr -d ' ')
        if [ "$BYTES" -gt 500 ]; then
            assert_pass "#607 read $u → ${BYTES}b (>500)"
        else
            assert_fail "#607 read $u" "only ${BYTES}b"
        fi
    done
fi

# 11.9 #594: samples.db has zero rows with path starting '/private' (post-#594 reindex)
if [ -f ~/.cupertino/samples.db ]; then
    N=$("$SQLITE" ~/.cupertino/samples.db "SELECT COUNT(*) FROM files WHERE path LIKE '/private%';" 2>/dev/null)
    if [ "$N" = "0" ]; then assert_pass "#594 samples.db has zero /private-prefixed paths"
    else assert_fail "#594 samples.db /private" "${N} rows still corrupted"; fi
else
    assert_skip "#594 samples.db" "DB not built; cupertino setup first"
fi

# 11.10 #595: MCP resources/list with malformed cursor → JSON-RPC -32602
INIT='{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"p","version":"0"}}}'
OUT=$(printf '%s\n{"jsonrpc":"2.0","id":2,"method":"resources/list","params":{"cursor":"INVALID_xyz"}}\n' "$INIT" \
      | perl -e 'alarm 5; exec @ARGV' "$BIN" serve --no-reap 2>/dev/null)
if echo "$OUT" | grep -q '"code":-32602'; then assert_pass "#595 strict cursor returns -32602"
else assert_fail "#595 strict cursor" "no -32602 in response"; fi

# 11.11 #596: empty search query is rejected
if "$BIN" search '' --limit 1 2>&1 | grep -q "Query cannot be empty"; then
    assert_pass "#596 empty query rejected"
else
    assert_fail "#596 empty query" "no rejection message"
fi

# 11.12 #588: cupertino save --dry-run flag exists
if "$BIN" save --help 2>&1 | grep -q "\\-\\-dry-run"; then
    assert_pass "#588 save --dry-run flag exists"
else
    assert_fail "#588 save --dry-run" "flag missing from save --help"
fi

# 11.13 #621 / #620: read-sample-file with invalid project id → 'Project not found'
OUT=$("$BIN" read-sample-file "no-such-project-xyz" "anything.swift" 2>&1)
if echo "$OUT" | grep -q "Project not found:"; then
    assert_pass "#620 invalid project id distinguished from invalid file path"
else
    assert_fail "#620 read-sample-file" "missing 'Project not found:' shape"
fi

# 11.14a #629 / #628: --framework banana on unified path errors out with the
# 'Unknown framework' message (was silently zeroing results pre-#629).
if ! assert_skip_if_schema_gap "#628 unified --framework banana"; then
    OUT=$("$BIN" search "View" --framework banana 2>&1)
    if echo "$OUT" | grep -q "Unknown framework: 'banana'"; then
        assert_pass "#628 unified --framework banana rejected with clear error"
    else
        assert_fail "#628 --framework banana" "missing 'Unknown framework' rejection"
    fi
fi

# 11.14b #629 / #628: source-scoped --framework banana ALSO errors out (the
# source-scoped path goes through Search.Index.search's strict validator).
if ! assert_skip_if_schema_gap "#628 source-scoped --framework banana"; then
    OUT=$("$BIN" search "View" --source apple-docs --framework banana 2>&1)
    if echo "$OUT" | grep -q "Unknown framework: 'banana'"; then
        assert_pass "#628 source-scoped --framework banana rejected"
    else
        assert_fail "#628 --framework banana (apple-docs)" "missing rejection"
    fi
fi

# 11.14c #629 / #628: --framework foundation scopes apple-docs results.
if ! assert_skip_if_schema_gap "#628 --framework foundation scoping"; then
    OUT=$("$BIN" search "View" --source apple-docs --framework foundation --limit 5 --format json 2>/dev/null)
    if [ -n "$OUT" ] && echo "$OUT" | grep -q '"framework"'; then
        if echo "$OUT" | grep -oE '"framework"[[:space:]]*:[[:space:]]*"[^"]+"' | grep -v '"foundation"' | grep -q .; then
            assert_fail "#628 --framework foundation scoping" "non-foundation rows leaked"
        else
            assert_pass "#628 --framework foundation scopes results to foundation"
        fi
    else
        assert_skip "#628 --framework foundation scoping" "search.db not built or no rows"
    fi
fi

# 11.14d #630: URLSession unified search returns foundation/urlsession (real
# protocol page, not swiftui/urlsession property). Confirms the canonical-
# prepend kind-filter is live in the binary.
if ! assert_skip_if_schema_gap "#630 URLSession ranking"; then
    URI=$("$BIN" search "URLSession" --limit 1 --format json 2>/dev/null | grep -oE 'apple-docs://[a-z/-]+' | head -1)
    if [ "$URI" = "apple-docs://foundation/urlsession" ]; then
        assert_pass "#630 URLSession → foundation/urlsession (canonical protocol page)"
    else
        assert_fail "#630 URLSession ranking" "expected foundation/urlsession, got: $URI"
    fi
fi

# 11.14e #630: URL unified search does NOT force-prepend swift/url
# (the KeyboardType.url enum case). This is a "negative" probe: we don't
# care which row wins (URL+Data are #610 Class B corpus casualties pending
# bundle re-publish), only that the worst force-prepend is removed.
if ! assert_skip_if_schema_gap "#630 URL canonical-prepend"; then
    URI=$("$BIN" search "URL" --limit 1 --format json 2>/dev/null | grep -oE 'apple-docs://[a-z/-]+' | head -1)
    if [ "$URI" != "apple-docs://swift/url" ]; then
        assert_pass "#630 URL no longer force-prepends swift/url (KeyboardType.url enum case)"
    else
        assert_fail "#630 URL canonical-prepend" "swift/url KeyboardType.url still at top"
    fi
fi

# 11.14 #429: search.db has zero 403/502 poison titles indexed AFTER the fix (excludes #290's 68
# pre-existing rows which are tracked separately; this asserts no NEW poison since post-fix).
# Tolerant assertion: count must be ≤ 68 (the #290 backlog). Past 68 means a regression.
if [ -f ~/.cupertino/search.db ]; then
    N=$("$SQLITE" ~/.cupertino/search.db "SELECT COUNT(*) FROM docs_metadata WHERE title IN ('403 Forbidden', '502 Bad Gateway');" 2>/dev/null)
    if [ "$N" -le 68 ]; then
        assert_pass "#429 poison titles ≤ 68 (#290 backlog, no new regressions; live count: $N)"
    else
        assert_fail "#429 poison titles" "count $N exceeds the #290 backlog of 68; new poison crept in"
    fi
else
    assert_skip "#429 poison titles" "search.db not built"
fi

# 11.15 #610 Class A: 9 canonical queries return swift/* (collapsed view of step 10a, in case --deep is off)
if ! assert_skip_if_schema_gap "#610 Class A canonical queries"; then
    DECLARE_A_FAILS=0
    for q in "Task" "View" "String" "Array" "Hashable" "Equatable" "Codable" "Identifiable" "Sendable"; do
        URI=$("$BIN" search "$q" --limit 1 --source apple-docs 2>/dev/null | grep -oE 'apple-docs://[a-z/-]+' | head -1)
        EXP_FRAMEWORK="swift"
        [ "$q" = "View" ] && EXP_FRAMEWORK="swiftui"
        [[ "$URI" != "apple-docs://${EXP_FRAMEWORK}/$(echo $q | tr '[:upper:]' '[:lower:]')" ]] && DECLARE_A_FAILS=$((DECLARE_A_FAILS+1))
    done
    if [ "$DECLARE_A_FAILS" = "0" ]; then
        assert_pass "#610 Class A: 9/9 canonical queries surface the right page"
    else
        assert_fail "#610 Class A" "${DECLARE_A_FAILS}/9 queries still wrong-winners"
    fi
fi

DURATION=$(( $(date +%s) - START ))
TOTAL=$((PASSES + FAILS + SKIPS))
[ "$FAILS" = "0" ] && VERDICT="✅" || VERDICT="❌"
echo "| 11 | per-bug live regression smoke (${TOTAL} probes) | $VERDICT | ${DURATION}s | ${PASSES} pass, ${FAILS} fail, ${SKIPS} skip, detail: $LOG |" >> "$REPORT"

Failure criterion: any probe fails. Skips are allowed when a DB isn't built or a binary isn't compiled (so a clean-only run doesn't false-fail).

What's covered (20 probes against the live binary):

Probe	Bug	What it pins
11.1	#618	cupertino serve exits on stdin EOF in <2s
11.2	#611	MCP methodNotFound message is Method not found: … with no MCP.Core.Protocols. namespace leak
11.3	#614	Canonical String query returns swift/string (uses post-fix kind=struct)
11.4	#597	save --base-dir flag exists
11.5	#598	cupertino-tui exits cleanly when stdin is not a TTY
11.6	#68	cupertino doctor (default) does NOT show corpus directories
11.7	#68	cupertino doctor --save DOES show corpus directories
11.8	#607	read swift-evolution://..., hig://..., apple-archive://... each return >500 bytes
11.9	#594	samples.db has zero rows with path LIKE '/private%'
11.10	#595	MCP resources/list with malformed cursor returns JSON-RPC -32602
11.11	#596	cupertino search '' is rejected with "Query cannot be empty"
11.12	#588	cupertino save --dry-run flag exists
11.13	#620	read-sample-file <bad-id> X distinguishes "Project not found" from "File not found"
11.14a	#628	unified search <q> --framework banana errors with Unknown framework:
11.14b	#628	source-scoped search <q> --source apple-docs --framework banana errors
11.14c	#628

Step 9b: minimal-corpus reindex smoke (#643)

Validates the indexer-side fixes end-to-end on a tiny committed fixture corpus, in seconds, avoiding a 12h full reindex just to find a bug at hour 11. Runs the full cupertino save --docs pipeline against Packages/Tests/Fixtures/SmokeCorpus/ (7 pages: UIButton chain + Foundation.URLSession + SwiftUI.LazyVGrid + NSObject) and asserts:

• Schema user_version == current expected (v15 per #637)
• All 7 fixture pages indexed
• inheritance table populated with the expected ≥5 edges (#274)
• Specific edge UIControl → UIButton exists
• Specific edge NSObject → UIResponder exists
• symbol_components column (#77) populated
• kind=unknown count is 0 (fixtures all carry kind explicitly)
• kind=class rows ≥ 5
• docs_fts row count matches docs_metadata
• PRAGMA integrity_check = ok

START=$(date +%s)
if ./scripts/smoke-reindex.sh > /tmp/smoke-reindex.log 2>&1; then
    DURATION=$(( $(date +%s) - START ))
    PASSES=$(grep -c "^  ✅" /tmp/smoke-reindex.log)
    echo "| 9b | smoke reindex (#643) | ✅ | ${DURATION}s | ${PASSES}/10 checks pass |" >> "$REPORT"
else
    DURATION=$(( $(date +%s) - START ))
    PASSES=$(grep -c "^  ✅" /tmp/smoke-reindex.log)
    FAILS=$(grep -c "^  ❌" /tmp/smoke-reindex.log)
    echo "| 9b | smoke reindex (#643) | ❌ | ${DURATION}s | ${PASSES} pass, ${FAILS} fail, see /tmp/smoke-reindex.log |" >> "$REPORT"
fi

Failure criterion: any of the 10 validation checks fail, OR save fails. A schema bump (e.g. v15 → v16) must update both Search.Index.schemaVersion AND scripts/smoke-reindex.sh's check "schema v15" line in the same PR; otherwise this step catches the drift.

When to add a check: every new indexer-side feature (new column in docs_fts / docs_metadata, new table, new index, new derived field) gets a 2-line validation in the script. The fixtures may need a new content shape; add a JSON file under Packages/Tests/Fixtures/SmokeCorpus/<framework>/ and bump the page-count expectation.

Out of scope: this is NOT a full-corpus reindex. The 12h cupertino save --docs against ~/.cupertino/docs/ is the release ceremony, not a routine check. Smoke catches mechanical breakage in the indexer pipeline; the full reindex is for content quality on the real corpus.

Step 10 (--deep only): canonical-type audit + BUG 5 scope + round-trip

Skip unless --deep is passed. These are heavier and aren't run on every promote.

10a: canonical-type wrong-winner audit (#610)

Audit the 45 hand-picked canonical types (URL, Color, Font, List, View, String, Array, Data, Hashable, Equatable, Codable, Identifiable, Sendable, Task, plus the rest from main's audit chat). Each cupertino search <X> --source apple-docs --limit 1 must return the canonical page as result #1.

10b: BUG 5 multi-URI content-drop scope (#607)

For each of the 3 affected sources (swift-evolution, hig, apple-archive), randomly sample 10 URIs from resources/list and assert read_document returns non-empty content for each. Post-#607-read-side-fix, this should be 30/30 ok.

10c: search → read round-trip on top-50 symbols

Run cupertino search <symbol> --limit 1 --source apple-docs for the 50 most-traffic symbols (collected from production query logs if available, else from a static list). For each top-1 result, follow up with cupertino read <uri>. Assert: search returns at least one result, read returns non-empty content, the read content mentions the search term in its body.

START=$(date +%s)
# implementation per-step above; aggregated into one step row.
DURATION=$(( $(date +%s) - START ))
# [verdict logic based on 10a/10b/10c sub-results]
echo "| 10 | --deep audits | $VERDICT | ${DURATION}s | 10a=${WRONG_WINNERS}, 10b=${BUG5_SCOPE}, 10c=${ROUNDTRIP} |" >> "$REPORT"

Final verdict

After all steps:

TOTAL_FAILS=$(grep -c "❌" "$REPORT")
if [ "$TOTAL_FAILS" = "0" ]; then
    echo "" >> "$REPORT"
    echo "## Verdict: ✅ PASS" >> "$REPORT"
    EXIT_CODE=0
else
    echo "" >> "$REPORT"
    echo "## Verdict: ❌ FAIL (${TOTAL_FAILS} step(s) failed)" >> "$REPORT"
    EXIT_CODE=1
fi

echo ""
echo "Report written to: $REPORT"
exit "$EXIT_CODE"

Exit code mirrors the verdict so CI / heartbeat callers can branch on it.

When to run

• After every git push origin develop:main FF promote.
• After any non-trivial commit on develop (the build + test + 3 guards subset gives a fast sanity check; --no-clean skips the long clean).
• Before issuing a new release tag (--deep is mandatory here).
• After a bundle rebuild (cupertino save) to confirm the new DB doesn't have regressions.

Sample output

| # | Step | Result | Duration | Notes |
|---|------|--------|----------|-------|
| 1 | swift package clean | ✅ | 4s | |
| 2 | swift build -c release | ✅ | 162s | 0 warnings, exit 0 |
| 3 | swift test | ✅ | 43s | 1808/1808, exit 0 |
| 3a | CLI unit tests | ✅ | 8s | 152 cases, exit 0 |
| 3b | MockAIAgent unit tests | ✅ | 3s | 18 cases, exit 0 |
| 4 | 3 CI guards | ✅ | 3s | check-package-purity ✓, check-target-foundation-only ✓, check-docs-commands-drift ✓ |
| 5 | build mock-ai-agent | ✅ | 11s | exit 0 |
| 6 | mock-ai-agent E2E | ✅ | 9s | phases=6/6, exit 0 |
| 7 | CLI smoke (27-cmd sweep) | ✅ | 38s | 27/27 ok, 0 fail |
| 8 | MCP probe (10 tools/edges) | ✅ | 22s | 10/10 ok, 0 fail |
| 8b | concurrent serves (2 parallel) | ✅ | - | 2/2 ok |
| 8c | CLI ↔ MCP equivalence (33 queries) | ✅ | 28s | 33/33 match, 0 diverge |
| 9 | DB integrity (3 DBs) | ✅ | 2s | PRAGMA integrity_check + foreign_key_check |
| 11 | per-bug live regression smoke (15 probes) | ✅ | 24s | 15 pass, 0 fail, 0 skip, detail: /tmp/test-everything-step11.log |

## Verdict: ✅ PASS

Failure investigation rules

• Step 2 warnings > 0: a new warning has slipped in; locate via grep warning: $BUILD_LOG and fix at the source. Project policy is zero warnings (#586).
• Step 3 test fails: identify the failing test from $TEST_LOG. If it's flaky (DB races, temp-dir collisions), serialise the suite. If it's a real regression, find the introducing PR and either fix forward or revert.
• Step 4 guard fails: a producer target has gained a forbidden import. Check the contract doc docs/package-import-contract.md for the target's allowed list. The fix is usually moving the type / dropping the import.
• Step 6 mock-ai-agent fails: an MCP protocol handler regressed. Check MCP.Core.Server.routeRequest for the affected method.
• Step 7 CLI command fails: look at /tmp/cli-smoke-fails.log. Common causes: a subcommand was renamed (CHANGELOG + docs should track it), a flag was removed, the shipped DB is missing a row the smoke depends on (set up a fresh cupertino setup).
• Step 8 MCP probe miscount: compare the expected vs actual frame count in /tmp/mcp-probe-fails.log. Notification responses or missing responses both register here.
• Step 9 integrity fails: rare but serious. sqlite3 ~/.cupertino/<db> "VACUUM;" may recover. If integrity_check reports row corruption, drop the DB and cupertino setup again.

Cross-references

• Canonical pair-workflow rule: the gof-di-rules rule in the rules-swift repo (github.com/mihaelamj/rules-swift), the rules every passing run must honour.
• ./README.md: catalog index.
• The DB integrity step is sourced from the cupertino doctor schema-version + journal-mode probes (#234, #236).