| name | integration-test |
| description | Run the full cold-start end-to-end deploy test in an isolated git worktree with always-safe teardown. Use when: (1) verifying the one-shot `make deploy` works on a FRESH workspace/catalog (cold-start), (2) the user says 'integration test', 'cold-start test', 'test the deploy end to end', or 'does deploy still work on a new workspace', (3) validating a change to deploy.sh / databricks.yml / the seed before opening a PR. PROJECT-AUTHORED (not a vendored skill). |
Cold-start integration test (git-worktree E2E)
Drives scripts/integration_test.sh: deploy the whole stack
to a brand-new Lakebase project + UC schema in an isolated worktree, verify it, then tear it all
down. This is the real proof that make deploy is genuinely one-shot on a fresh workspace — in
particular that the genie_spaces table-validation ordering fix holds (empty operational tables are
created before bundle deploy).
When to use it
- After changing
scripts/deploy.sh, databricks.yml, or the seed — before opening a PR.
- To regression-test the cold-start path (genie table-ordering, BYO-warehouse, pgvector) — see the
cold-start E2E section in
docs/test/integration-testing.md.
- To validate a fresh-workspace deploy without hand-clicking through the UI.
This is not a fast check — it provisions a real Lakebase project + app (~10 min) and then deletes
them. Confirm with the user before running.
Safety model (why teardown is always safe here)
The harness only ever creates throwaway resources, each tagged with the marker itest:
- A throwaway Lakebase project (
scp-itest-<epoch>). The app service principal owns its
memory/write-back schemas, and bundle destroy (app delete) orphans those schemas —
unrecoverable in a shared project (see
docs/lakebase-apps-permissions.md). Because the
project is throwaway, teardown just deletes the whole project, orphans and all.
- A throwaway UC schema (
scp_itest_<epoch>) inside an existing catalog — so the run actually
exercises the cold-start (tables don't pre-exist), and teardown drops it CASCADE.
itest_teardown.py refuses to delete any project/schema whose name lacks the itest marker
(unless --force), so the harness can't nuke a real project like mfg-supply-chain-copilot or the
planner schema. Teardown runs from an EXIT trap, so it fires even if the deploy fails partway.
Never point --lakebase-project / --uc-schema at a real (shared) project or schema — let the
harness auto-generate the throwaway names. The catalog (--uc-catalog) is reused, never deleted;
pass an existing writable one (e.g. main).
Inputs to gather from the user
Required: a CLI profile (--profile) and an existing, writable UC catalog (--uc-catalog).
Ask which workspace/profile, and confirm the catalog. Then check:
- Committed? The worktree tests committed code (
HEAD by default). If there are uncommitted
changes to deploy.sh / databricks.yml / the seed, tell the user to commit them first (or note
they won't be tested). Pass a different ref with --ref <branch|sha>.
- Can the profile create a SQL warehouse? If not (restricted workspace), run with
--target byo --sql-warehouse-id <existing-id>.
- Seed? Default seeds demo data.
--no-seed is faster and still proves the deploy + cold-start.
Prereq the user must already have: serverless_compute_id=auto in the profile (the cold-start table
DDL and the schema-drop teardown use Databricks Connect). Same other prereqs as make deploy.
Run it
scripts/integration_test.sh --profile <p> --uc-catalog main
scripts/integration_test.sh --profile <p> --uc-catalog main --target byo --sql-warehouse-id <id>
scripts/integration_test.sh --profile <p> --uc-catalog main --keep
Interpreting the result
- Core assertion:
make deploy exits 0. A green deploy on a fresh project + schema is the
cold-start fix working (it ran the pre-deploy table DDL, then created the Genie space without the
"schema does not exist" 403). If this step dies, the cold-start path is broken — read the deploy
output; the harness still tears down.
- Health checks (reported, non-core): app compute
ACTIVE, Genie space present via
list_spaces, and verify_deploy.py (Lakebase permission contract). A failed health check exits
non-zero but still tears down — investigate the ⚠ lines.
- The script prints PASS or the failing checks, then runs teardown (unless
--keep).
If something goes wrong
- Stuck / interrupted before teardown: re-run the teardown by hand —
uv run python scripts/itest_teardown.py --project scp-itest-<ts> --catalog <cat> --schema scp_itest_<ts>
then git worktree list / git worktree remove --force <path>.
- Don't retry blindly more than once or twice — if the deploy keeps failing the same way, surface
the deploy output to the user; it's the same code path as
make deploy, so the fix belongs in
deploy.sh / databricks.yml, not the harness.
References
