lakebase-scm-workflows

name	lakebase-scm-workflows
description	Opinionated git-Lakebase branch-pairing workflows. Use when scaffolding a Lakebase-paired project, creating/deleting Lakebase branches in lockstep with git branches, diffing parent-aware schemas, opening or merging PRs that touch Lakebase, or running the same operations the lakebase-scm-extension exposes in VS Code.
compatibility	Requires databricks CLI (>= v0.294.0), git (>= 2.30), Node.js (>= 20), and @databricks-solutions/consort
metadata	{"version":"0.1.0"}
parent	databricks-lakebase

lakebase-scm-workflows – agent contract

Agent-facing contract: operating rules (.env, git hooks, credential single-seam), concrete code patterns for each substrate primitive, and reference pointers.

For the human-facing overview (prerequisites, installation, prompts, journey, CLI cheat sheet) see README.md.

FIRST: load the parent databricks-lakebase skill for Lakebase Postgres CLI basics (project / branch / endpoint shapes, name formats, "never delete the production branch" rule). This skill composes on top of it.

Project state – when .env matters

The substrate API takes explicit args (instance, branch, etc.) on every public function – agents can drive every operation without a project .env at all. But when an agent is acting AS the developer in a checked-out paired project, the project's .env is the source of truth for "which Lakebase branch is this workspace currently paired with."

Agent context	.env contract
In a checked-out paired project (Claude Code / Cursor / Genie Code on a developer's machine)	Respect it. Read LAKEBASE_PROJECT_ID to derive instance. After git checkout, call syncEnvToCurrentBranch({ cwd }) so .env matches the new branch – otherwise the bundled CI scripts (refresh-token.sh, flyway-migrate.sh) and the git hooks operate on stale credentials.
Sandbox / no workspace (Claude Desktop, OpenAI Agent Builder, exploratory)	Ignore it. Pass instance and branch explicitly per call. Substrate never requires .env.
Bootstrapping a new project	N/A. createProject creates the .env for you as step 7. No .env exists before that.

Connection-block keys (managed by syncEnvToCurrentBranch / updateEnvConnection / post-checkout.sh):

LAKEBASE_PROJECT_ID=proj-abc
LAKEBASE_BRANCH_ID=feature-x
LAKEBASE_HOST=ep-....database.cloud.databricks.com
LAKEBASE_ENDPOINT=primary
DB_USERNAME=[e-mail]

This is connection METADATA only , no DB token is persisted. The app + its migrations (alembic / knex / flyway) mint a fresh short-lived Lakebase credential at runtime from this metadata (via the lakebase-get-connection seam), so a token can never go stale on disk. Set DATABASE_URL explicitly to override with a fixed credential (CI secret / Docker). This block is the rewritten set on every branch switch; anything else in .env is preserved verbatim.

Project-level keys (written once by writeEnvFile during project bootstrap, never rewritten):

DATABRICKS_HOST=https://workspace.cloud.databricks.com
LAKEBASE_PROJECT_ID=my-app

If you're an agent dropping into a project mid-session, read these first to know what instance to pass to every subsequent operation.

Workflow state surface – .lakebase/workflow-state.json

The SCM workflow is a five-state machine: scaffold-complete → feature-claimed → pr-ready → ci-green → merged. Its gate surface is a single JSON file at the project root, validated against scm-workflow-state.schema.json. The current state plus its invariants (feature id, branch, parent branch, Lakebase UID, PR URL, CI URL) are persisted there.

Inspect it via:

lakebase-scm-state                          # human-readable, with gate ladder
lakebase-scm-state --json --pretty          # machine-readable report
lakebase-scm-state --project-dir ~/repos/x  # different project root

Read / write programmatically:

import {
  readWorkflowState,
  writeWorkflowState,
  initWorkflowState,
  describeGates,
} from "@databricks-solutions/consort";

const s = readWorkflowState(projectDir);    // ScmWorkflowState | null
writeWorkflowState(projectDir, {            // validates first; atomic write
  ...initWorkflowState({ projectId: "demo", tierTopology: 2 }),
});
const gates = describeGates(s!);            // gate ladder for tooling

Phase A (advisory data layer): lakebase-create-project seeds the scaffold-complete row at end-of-scaffold and lakebase-scm-state reads it. A seed failure surfaces as a project warning, not a scaffold abort.

Phase B (first blocking transition): lakebase-scm-claim-feature-branch is the canonical "start a new feature" verb. It enforces its precondition in code (state must be scaffold-complete or merged), calls the substrate primitive createFeaturePairedBranch (Lakebase branch + git branch + .env sync, 30-day TTL), and advances the state file to feature-claimed. /design's pre-hook invokes it; the substrate-only-path invariant is enforced through this bin.

lakebase-scm-claim-feature-branch initial-domain          # claim
lakebase-scm-claim-feature-branch initial-domain --json   # machine-readable
lakebase-scm-claim-feature-branch hotfix-x --parent main  # override tier-default parent
lakebase-scm-claim-feature-branch initial-domain          # idempotent re-run = no-op

Exit codes: 0 success (incl. idempotent no-op), 1 no state file, 2 precondition refused, 3 substrate failure.

Programmatic equivalent:

import { claimFeatureBranch } from "@databricks-solutions/consort";

const { state, paired, alreadyClaimed } = await claimFeatureBranch({
  projectDir,
  featureId: "initial-domain",
});

Full SCM CLI surface (phase C)

The workflow is driven entirely by CLI bins, one per transition. Each enforces its precondition in code, calls the underlying substrate primitives, and writes the new state row.

Bin	Transition	Substrate it wraps
lakebase-scm-state	inspect (read-only)	(reads .lakebase/workflow-state.json)
lakebase-scm-doctor	diagnose (read-only)	cross-checks state + git + Lakebase + .env
lakebase-scm-adopt-state	seed for existing projects	listBranches + getBranchByName + getCurrentBranch
lakebase-scm-recover-orphans [--claim]	retroactively pair orphan git branches	createFeaturePairedBranch per orphan
lakebase-scm-claim-feature-branch <id>	scaffold-complete | merged -> feature-claimed	createFeaturePairedBranch
lakebase-scm-abandon-feature	feature-claimed -> scaffold-complete	deletePairedBranch + git checkout
lakebase-scm-prepare-pr	feature-claimed -> pr-ready	git push + createPullRequest
lakebase-scm-wait-ci [--timeout-sec N]	pr-ready -> ci-green	getPullRequest poll loop
lakebase-scm-merge [--method squash|merge|rebase]	ci-green -> merged	mergePairedPullRequest + git cleanup
lakebase-reconcile-tier --branch <tier>	reconcile a shared tier whose DB is ahead of code (destructive; refuses when not db-ahead)	reconcileTierBranch (drop named orphan tables + stamp the tier to the code head)

All bins support --project-dir <dir> (default cwd), --json, --pretty, and --help.

End-to-end usage:

# Scaffold a fresh project (Step 8c seeds scaffold-complete).
lakebase-create-project --tiers 2 ...

# Existing projects opt in once.
lakebase-scm-adopt-state
# Or, if they have pre-phase-C orphan branches:
lakebase-scm-recover-orphans          # detect-only
lakebase-scm-recover-orphans --claim  # pair every orphan via the substrate

# Per-feature cycle.
lakebase-scm-claim-feature-branch initial-domain
# ... write code, run tests ...
lakebase-scm-prepare-pr
lakebase-scm-wait-ci
lakebase-scm-merge
# state is now merged; the next claim returns from merged to feature-claimed.

# Inspect / diagnose at any point.
lakebase-scm-state --json --pretty
lakebase-scm-doctor

Exit-code conventions across all bins: 0 success / idempotent no-op / clean doctor report, 1 no state file (or doctor warnings only), 2 precondition refused (or doctor failures), 3 substrate failure. lakebase-scm-wait-ci adds 3 for CI failure (state unchanged) and 4 for timeout (state unchanged). lakebase-scm-merge also returns 4 when the downstream migrate fails or times out (the git + PR state IS already merged).

Phase C: substrate-only-path is now enforced

The post-checkout git hook (templates/project/common/scripts/post-checkout.sh) used to silently create Lakebase branches as a fallback for orphan git branches. Phase C retires that fallback: a git checkout -b feature/foo with no prior substrate call leaves .env untouched and the hook prints a clear error pointing the user at lakebase-scm-claim-feature-branch or lakebase-scm-recover-orphans. The substrate is the only path; the SCM workflow is how that path is enforced in code.

lakebase-scm-merge waits on the downstream migrate by default: after merging it blocks until the parent-branch migrate workflow applies the migrations, so git and the Lakebase schema do not diverge. --no-wait-migrate opts out (returns as soon as the PR merges + local syncs), and a migrate that fails or times out exits 4 with the git + PR state already merged. For targeted remediation, lakebase-scm-doctor --fix <finding-id> applies the fix for one finding (supported ids come from FIXABLE_FINDING_IDS) and attaches a post-fix report.

Sync without an IDE – the git hooks

The construct that keeps a Lakebase branch and a git branch in sync in a plain terminal session (no extension, no explicit substrate call) is the bundled git hooks that scaffoldAll / installHooks drops into .git/hooks/ during project bootstrap. They are the default-on automatic sync mechanism. Agents driving raw git commands inherit them for free.

Hook	Fires on	What it does
post-checkout	git checkout <branch>	Reads new current branch → finds matching Lakebase branch → mints fresh credential → rewrites .env connection block. The primary sync mechanism.
post-merge	git merge	Runs Flyway migrations against the now-current Lakebase branch so schema catches up.
pre-push	git push	Schema-diff guard – surfaces unmigrated changes before remote sees them.
prepare-commit-msg	git commit	Embeds Lakebase branch context in commit messages so the schema-diff CI workflow can find them.

Practical implications for an agent:

1. Don't fight the hooks. If you run git checkout feature-x in a paired project, .env auto-updates. Don't also call syncEnvToCurrentBranch defensively – let the hook own that side of the workflow.
2. Hooks don't create branches. They sync state after-the-fact. To CREATE a Lakebase branch (which has no git equivalent), use the substrate's createPairedBranch – it creates the Lakebase side first, then git checkout -b triggers the hook to populate credentials.
3. If hooks aren't installed, re-arm them. Some workflows clone a paired project without scaffolding (e.g. cloning someone else's checkout). The substrate's installHooks(projectDir) is the recovery – copies scripts/post-checkout.sh and siblings into .git/hooks/ with the right permissions.
4. For pure-API sessions (no checkout) the hooks are irrelevant. A Claude Desktop sandbox or OpenAI Agent Builder session that just calls getConnection({ instance, branch }) doesn't have a .git/ to install hooks into – and doesn't need them. The hooks only matter when an agent (or human) is driving a working tree with git commands.

The bundled hook scripts live in templates/project/common/scripts/.

Credential handoff – two helpers, one pattern

Two narrow auth seams – one for Lakebase, one for GitHub. Both follow the same shape: single module, dynamic-runtime fallback chain, CI grep guard preventing any other file from resolving credentials directly.

GitHub

lakebase-github-token                 # print token to stdout
lakebase-github-token --diagnose      # which sources are configured

import { resolveGitHubToken } from "@databricks-solutions/consort";
const token = await resolveGitHubToken();

Fallback: GITHUB_TOKEN env → VS Code getSession (extension host only, via dynamic import('vscode')) → gh auth token → clear error. Scopes: ['repo', 'workflow', 'delete_repo']. Full docs: references/github-auth.md.

Lakebase

Every workflow op that touches Lakebase resolves credentials through a single seam:

lakebase-get-connection --output dsn --instance <id> --branch <name>
# -> libpq URL string (use for Flyway, Alembic, psql)

import { getConnection } from "@databricks-solutions/consort";
const pool = await getConnection({ output: "pool", instance, branch });
// -> @databricks/lakebase pg.Pool with refresh-on-connect

DSN and Pool resolve to the same database via the same OAuth substrate. Never call databricks postgres generate-database-credential from anywhere else in your code – a CI grep guard fails the build if you do. Full docs: references/get-connection.md.

Operations

Concrete invocations per primitive, in user-journey order. The agent reads these to know what to call; humans see the conversational equivalent in README.md's "How to use".

1. Create-project

End-to-end project bootstrap.

lakebase-create-project \
  --project-name proj-checkout \
  --parent-dir ~/code \
  --databricks-host https://workspace.cloud.databricks.com \
  --github-owner my-org \
  --language java \
  --runner self-hosted
# -> JSON on stdout: { projectDir, githubRepoUrl, lakebaseProjectId, lakebaseDefaultBranch, warnings }

# Local-only (no GitHub side effects):
lakebase-create-project --project-name proj-checkout --parent-dir ~/code \
  --databricks-host https://workspace.cloud.databricks.com --no-github

# Wire Playwright into the project so [E2E]-tagged AC rows have a runner.
# Default-on for --language nodejs; opt in elsewhere with --enable-e2e or
# turn off with --no-e2e:
lakebase-create-project ... --language nodejs --enable-e2e

# Skip the .claude/commands/{design,build}.md scaffold (for projects
# that already have their own slash commands or non-Claude-Code consumers):
lakebase-create-project ... --skip-commands

import { createProject } from "@databricks-solutions/consort";
const result = await createProject({
  projectName: "proj-checkout",
  parentDir: process.env.HOME + "/code",
  databricksHost: "https://workspace.cloud.databricks.com",
  githubOwner: "my-org",
  language: "java",
  runnerType: "self-hosted",
  enableTdd: true,                // default: true – lays down .sftdd/ scaffold
  enableE2e: undefined,           // default: true for nodejs, false otherwise.
                                  // Explicit boolean overrides the language default.
  skipCommands: false,            // default: false – writes .claude/commands/{design,build}.md
});

Eleven-step orchestration. Non-fatal failures (CI secrets sync, runner setup, hook/workflow verification) land in result.warnings[]. Hard-fatal errors (input validation, GitHub repo creation, Lakebase project creation, git operations, push rejection on workflow scope) throw.

2. Branch lifecycle

Lakebase branch CRUD plus paired git+Lakebase+.env ops. The lakebase-branch bin is the shell-friendly entrypoint; same operations import directly from the package for JS/TS hosts.

# Lakebase-only branch lifecycle
lakebase-branch list --instance proj-checkout
lakebase-branch show --instance proj-checkout --branch feature-add-orders
lakebase-branch create --instance proj-checkout --branch feature-add-orders --parent staging
lakebase-branch delete --instance proj-checkout --branch feature-add-orders