| name | saptak-verify |
| version | 0.1.0 |
| description | Run the Saptak verification suite. Picks the right depth based on
what's asked: `make verify` (95s) against the running stack for
routine checks; `make verify-clean-rebuild` (10 min) for the
reviewer-grade from-empty proof; or just `pack_contract_completeness.py`
(3s, no Docker) for fast static checks. Surfaces failures with
the exact file/cell so the user can fix + re-run. Use when: "verify",
"run verification", "check the demo", "make sure it still works",
"is X still passing?".
|
| allowed-tools | ["Bash","Read"] |
Saptak — verification
The project has a 5-suite acceptance framework. Each suite has different cost + coverage. Pick the right one for the question.
Decision tree
| Question | Use | Cost | What it proves |
|---|
| "Is everything still passing?" (default) | make verify | 95s | 149/149 against running stack |
| "Does the contract hold?" (fast static) | python3 tests/pack_contract_completeness.py | 3s, no Docker | 55/55 contract completeness |
| "Does this reproduce from a clean clone?" | make verify-clean-rebuild | 10 min, destructive | From-empty reproduction proof |
| "Pack-author work specifically" | python3 tests/pack_author_acceptance.py | 5s | 43/43 every pack satisfies HOWTO |
| "Just ABAC behavior" | python3 tests/abac_acceptance.py | 45s | 53/53 ABAC + RLS + agent control |
| "Just the use-case paths" | python3 tests/usecase_acceptance.py | 47s | 25/25 each pack × killer-query |
| "Just layer health" | python3 tests/seven_layer_health.py | <1s | 19/19 L1-L7 alive |
Routine flow
make verify
That runs all 4 acceptance suites + the static lint, renders tests/VERIFICATION_REPORT.md. If anything fails, the report's ## Failures section names the specific cell + the value that was wrong.
Clean-rebuild flow (reviewer-grade)
Requires backup first. Take one:
BACKUP_DIR="backups/$(date -u +%Y%m%dT%H%M%SZ)"
mkdir -p "$BACKUP_DIR"
docker compose -f compose/docker-compose.yml exec -T -e PGPASSWORD=saptak postgres \
pg_dumpall -U saptak --clean --if-exists > "$BACKUP_DIR/postgres-dumpall.sql"
docker run --rm -v saptak_saptak_postgres_data:/source:ro -v "$(pwd)/$BACKUP_DIR":/backup \
alpine sh -c "tar -czf /backup/postgres-volume.tar.gz -C /source ."
Then:
make verify-clean-rebuild
This destroys the running DB state, rebuilds from versioned init SQL, runs verify.sh. Takes 10 min including Ollama model re-pull. If anything fails on the clean rebuild but worked on make verify, that's a "runtime ALTERs not versioned" finding — there's something in the live DB that isn't in init SQL or seeders.
When make verify fails
The report (tests/VERIFICATION_REPORT.md) ends with a ## Failures table:
| Suite | Section | Cell | Detail |
|---|---|---|---|
| usecase_acceptance | 1 | F1 graph traversal... | code=500 body={} |
Common failure patterns + their fixes:
| Failure pattern | Likely cause | Fix |
|---|
code=500 on graph endpoints | Missing schema USAGE grant for saptak_app | Check init/03-tenancy.sql; or graph-populator hasn't run its GRANT |
pack=None tenant=None | Endpoint missing pack-uniform response envelope | Add pack + tenant_id to endpoint's return dict |
count=0 when data expected | Seeder skipped (already_seeded=True) or RLS gate fired | Check pack_status; check app.subject_clearance |
Missing *_PACK_* entry | Pack-contract violation | See docs/PACK_CONTRACT.md; run pack_contract_completeness.py for the specific missing key |
When the boot validator fires
Container starts then exits with Pack-contract violation. The stderr names the missing key. Fix it in layer_details.py and rebuild:
cd compose && docker compose build saptak-demo && \
docker compose up -d --no-deps saptak-demo
Related
