k8s-deploy

name	k8s-deploy
description	Drive the helm-charts/bin/ install/uninstall/build/health scripts for both dev and prod profiles — configure, install, reinstall, uninstall, health-check, and rebuild-the-API / rebuild-the-frontend. The dev profile installs umbrella chart + peripherals (nginx-ingress, DataHub, Langfuse, dummy data, dev-lock) and auto-seeds peripheral connection config via the admin API. The prod profile installs the umbrella chart only; operator wires peripherals via /api/v1/admin/peripherals/*.
argument-hint	[configure|install|reinstall|uninstall|health-check|run-api] [--profile dev|prod] [--components <csv>] [other options...]
allowed-tools	Bash(*), Read, Edit, Write, Glob, Grep, Skill(k8s-work), AskUserQuestion

Routing

Parse $ARGUMENTS and the user's request to determine the action. If ambiguous or no arguments given, ask the user which action they want:

Action	Trigger keywords
configure	configure, config, setup, env
install	install, up, create
health-check	health-check, health, check, status
run-api	run-api, run, start, deploy, test-mode, iterate
reinstall	reinstall, reset
uninstall	uninstall, teardown, down, remove, destroy

Profile

Every action requires a profile. Parse --profile dev or --profile prod from $ARGUMENTS. If absent, ask the user — do not default. The dev profile installs peripherals and seeds; the prod profile is umbrella-chart only.

Component names

When the user specifies components, match against these names (the --components flag accepts a comma-separated subset). If no components are specified, operate on all-for-profile.

Component	Profiles	Aliases
nginx-ingress	dev	ingress
datahub	dev	—
langfuse	dev	lf, observability
dataspoke-infra	dev, prod	infra, infrastructure, chart, umbrella
api	dev, prod	(rebuild + helm-upgrade the API only — iteration path)
frontend	dev, prod	ui, web (rebuild + helm-upgrade the Next.js UI only — iteration path)
dummy-data	dev	example, dummy
dev-lock	dev	lock
seed	dev	(post-install admin-API seeding only)

---

Action: configure

Prod splits configuration across three planes. Put each input in the plane that owns it and never mix them:

Plane	Holds	Source
env file (helm-charts/.env.prod)	deployment shape only — DATASPOKE_KUBE_*	.env.prod.example; --env-file <path> overrides the default
dataspoke-secrets K8s Secret	the 12 credential keys	operator pre-creates before install — helm-charts/README.md §2
operator overlay (--values)	ingress hosts, TLS secret names, CORS origins, OAuth redirect + client ID, secrets.existingSecret	copy of values-prod.example.yaml — README.md §3

Steps 1–3 and 5 below apply to both profiles; step 4 is dev-only.

1. Read helm-charts/.env.dev (dev profile) or helm-charts/.env.prod (prod profile). If it does not exist, create it from helm-charts/.env.dev.example (dev) or helm-charts/.env.prod.example (prod).
2. If it already exists, verify the canonical variables are present per spec spec/feature/HELM_CHART.md §Configuration — Four-Tier Env Vars:
   • Kube deployment (both profiles): DATASPOKE_KUBE_CLUSTER, DATASPOKE_KUBE_DATASPOKE_NAMESPACE, DATASPOKE_KUBE_IMAGE_REGISTRY, DATASPOKE_KUBE_CLOUD_VENDOR (GCP → Cloud Build; AWS → ECR via DATASPOKE_AWS_PROFILE, optional DATASPOKE_DOCKER_SUDO; empty → local Docker), DATASPOKE_KUBE_INGRESS_MODE (managed default — install & own nginx-ingress; shared — reuse a pre-existing controller), DATASPOKE_KUBE_INGRESS_CLASS (shared mode), DATASPOKE_KUBE_INGRESS_IP (managed: auto-populated in dev; shared: blank), DATASPOKE_KUBE_INGRESS_DOMAIN (managed: auto-populated <IP>.nip.io; shared: operator pre-set), DATASPOKE_KUBE_INGRESS_SCHEME (http default / https — scheme for every ingress-domain URL the dev install builds; set https when a shared controller terminates TLS), DATASPOKE_KUBE_INGRESS_TLS_SECRET (optional, default empty — when set, emits per-Ingress tls: blocks referencing this K8s TLS Secret).
   • App runtime (DATASPOKE_*): not in .env — injected into pods from the dataspoke-secrets K8s Secret via envFrom. In dev, install.sh auto-generates the Secret; in prod, the operator pre-creates it. DATASPOKE_CORS_ORIGINS is rendered from chart values (config.corsOrigins).
   • Dev only (dev profile): DATASPOKE_DEV_KUBE_{DATAHUB,LANGFUSE,DUMMY_DATA}_NAMESPACE, DATASPOKE_DEV_KUBE_DATAHUB_{,PREREQUISITES_}CHART_VERSION, DATASPOKE_DEV_DATAHUB_MYSQL_{ROOT_,}PASSWORD, DATASPOKE_DEV_DUMMY_DATA_{KAFKA_INSTANCE,POSTGRES_USER,POSTGRES_PASSWORD,POSTGRES_DB}, DATASPOKE_DEV_LLM_{PROVIDER,API_KEY,MODEL}. The Langfuse internals and peripheral connection outputs are auto-populated by the peripheral install scripts.
   • Test access (DATASPOKE_TEST_*): auto-populated by install.sh post-install via _sync_env_from_secret; never manually edited. Read by tests/integration/ for laptop-side cluster access.
3. Do NOT add stub-mode toggles to .env.dev. The four dependency factories are toggled via the runtime_config DB row (stub_redis_client, stub_llm_client, stub_pgvector_manager, stub_notification_service) — flippable via PATCH /api/v1/admin/conf, not in the env file.
4. Dev only — generate secure passwords (16+ chars, mixed case, at least one special character) for any missing password variables. Never generate credentials into a prod env file: prod credentials live only in the pre-created K8s Secret.
5. Show the final env file content to the user and ask for confirmation before writing. Do not proceed until the user approves. (Skip confirmation if the env file already has all required variables.)

---

Action: install

Run configure first if the profile env file (helm-charts/.env.dev or helm-charts/.env.prod) does not exist or is missing required variables.

Pre-flight checks

1. Verify kubectl and helm are installed.

2. Verify the Kubernetes cluster specified in DATASPOKE_KUBE_CLUSTER is reachable (kubectl cluster-info).

3. Report cluster node resources (kubectl get nodes) so the user can confirm the cluster meets the budget for the profile: spec/feature/HELM_CHART.md §Resource Sizing (8+ CPU / 24 GB RAM for the full dev profile), or for prod the operator-sized budget in helm-charts/README.md §1 (2 API + 2 frontend replicas, Postgres 1–2 CPU / 2–6Gi, Airflow's five components, Redis primary + replica).

4. Prod only — install.sh fails fast on all of the following before creating any resource. Check them here and stop with the remedy rather than letting the script abort mid-run (authoritative table: README.md §2):

   • --image-tag <tag> is passed explicitly — prod refuses the mutable :dev tag.
   • The DATASPOKE_KUBE_INGRESS_CLASS IngressClass (default nginx) exists: kubectl get ingressclass <class>.
   • The credential Secret exists in the target namespace — dataspoke-secrets, or whatever the overlay's secrets.existingSecret names.
   • All 12 keys are present in it: DATASPOKE_POSTGRES_{USER,PASSWORD,DB}, DATASPOKE_REDIS_PASSWORD, DATASPOKE_AIRFLOW_{USER,PASSWORD}, DATASPOKE_AIRFLOW_{WEBSERVER_SECRET_KEY,JWT_SECRET}, DATASPOKE_INTERNAL_TOKEN, DATASPOKE_JWT_SECRET_KEY, DATASPOKE_OAUTH_STATE_SECRET, DATASPOKE_GOOGLE_OAUTH_CLIENT_SECRET.
   • None holds a known-bad literal: DATASPOKE_JWT_SECRET_KEY equal to the dev default, DATASPOKE_AIRFLOW_USER equal to admin, DATASPOKE_AIRFLOW_PASSWORD empty or admin, DATASPOKE_GOOGLE_OAUTH_CLIENT_SECRET starting with placeholder-.

   If the Secret is missing, point the user at README.md §2 — deliver the keys via ExternalSecrets, Vault, or SealedSecrets; the kubectl create secret --from-env-file form is the one-off bootstrap floor, never --from-literal (leaks into shell history and ps auxww). Do not create the Secret for the user, and do not put its values in the env file.

5. If any check fails, report clearly and stop.

Full install (all components for the profile)

1. Execute the top-level install script in the background:
   • dev: ./helm-charts/bin/install.sh --profile dev
   • prod: ./helm-charts/bin/install.sh --profile prod --values <operator-overlay> --image-tag <tag> — ask the user for both the overlay path and the tag; never dev. --env-file <path> is optional (defaults to helm-charts/.env.prod); README.md §"Prod: committing an operator env file" describes the credential-free committable variant a deployment fork may adopt.
   • Frontend (--frontend none|local|cluster, default none in dev / cluster in prod): pass --frontend local for the host-pnpm dev workflow (writes src/frontend/.env.local), --frontend cluster to deploy the containerised UI in-cluster, or --frontend none for API-only. local is dev-only. The install summary prints the resulting Web UI URL + default [e-mail] / dataspoke login. If the user hasn't said, ask which frontend mode they want (or default per profile).
   • Note the background task ID and output file path.
2. While the script runs, alternate between two monitoring sources every ~30 seconds: a. Script output: read the background task output file (e.g., tail -20 <output-file>) to report install progress messages. b. Cluster state: invoke the /k8s-work skill to get live pod/Helm status across all namespaces.
   • After each round, summarize what changed since the last check.
   • If a pod enters CrashLoopBackOff, OOMKilled, or Error, report it immediately and show recent logs.
3. Continue until the background script exits (exit code 0 = success, non-zero = failure) and all expected pods are Running/Ready.

Partial install (specific components)

1. Resuming an interrupted full install (starting component plus every component after it in dependency order): prefer ./helm-charts/bin/install.sh --profile dev --from-component <name> — it inherits the orchestrator's step markers, error handling, and final summary.
2. Installing one or a few specific components: ./helm-charts/bin/install.sh --profile dev --components <csv>. Honors phase ordering automatically.
3. Rebuild and redeploy the API only (code-iteration path): ./helm-charts/bin/install.sh --profile dev --components api. This rebuilds the API image, runs helm upgrade, and rolls the deployment.
4. Rebuild and redeploy the frontend only (code-iteration path): ./helm-charts/bin/install.sh --profile dev --components frontend. The dev umbrella keeps frontend.enabled=false (developers run host pnpm dev at src/frontend); this fast path builds the frontend image and helm-upgrades with frontend.enabled=true to deploy the containerised UI in-cluster (verification / prod-parity). Stop it with kubectl scale deployment/dataspoke-frontend --replicas=0 -n "${DATASPOKE_KUBE_DATASPOKE_NAMESPACE}".
5. Monitor with /k8s-work after each component completes.

Post-install (dev)

1. Confirm all expected components are running.
2. Seed dummy data and register datasets in DataHub:

   uv run python -m tests.integration.util --reset-seed
   

3. Show access information (ingress endpoints table is in helm-charts/README.md §Ingress Endpoints; substitute DATASPOKE_KUBE_INGRESS_IP / DATASPOKE_KUBE_INGRESS_DOMAIN from helm-charts/.env.dev). In shared ingress mode INGRESS_IP is blank — HTTP services ride the operator-set DATASPOKE_KUBE_INGRESS_DOMAIN, and TCP services (Postgres/Redis/Kafka/lock) are reached on 127.0.0.1 by running ./helm-charts/bin/port-forward.sh in a separate shell held open for the session.
4. Inform the user that helm-charts/.env.dev has been populated with runtime variables (hosts, URLs, ports — including DATASPOKE_TEST_LOCK_URL and DATASPOKE_TEST_DUMMY_DATA_POSTGRES_HOST_PORT) by install.sh (managed mode also derives ingress IP/domain in dev-peripherals/nginx-ingress.sh), and that they should run source helm-charts/.env.dev to load them into their shell.
5. The post-install seeding step (dev profile) has already wired DataHub + Langfuse + LLM provider/model into the API's runtime config via /internal/admin/peripherals/* and /internal/admin/conf — no manual admin-API calls needed unless --skip-seed was set.

Post-install (prod)

None of the dev steps above apply — prod installs no peripherals, seeds no dummy data, and auto-wires no runtime config. Authoritative sequence: helm-charts/README.md §5–§7.

1. Rotate the auto-seeded admin — REQUIRED, and it is already live. Unless --skip-seed was passed, the install seeded [e-mail] / dataspoke (SKIP_SEED defaults to false, so this runs on every prod install). That credential is published in this repository and authenticates against the API the moment install returns, for anyone who can reach it — how reachable depends on the operator's ingress controller and network posture, since the chart applies no source-range or auth gating on the API ingress. Walk the user through the PATCH /api/v1/auth/me recipe in README.md §5, which keeps the new password out of shell history and argv (read -s plus a heredoc into -d @-). Treat it as urgent if the controller is shared or internet-reachable. Do not report the install as complete until this is done or the user explicitly declines.
2. If the install used --skip-seed, seed manually when ready: ENV_FILE=helm-charts/.env.prod bash helm-charts/bin/post-install/seed-admin-user.sh. The ENV_FILE= prefix is required (the script defaults to .env.dev), and the overlay's API ingress host must equal api.<DATASPOKE_KUBE_INGRESS_DOMAIN> (README.md §6). Then rotate per step 1.
3. Register peripherals and runtime config manually — PATCH /api/v1/admin/peripherals/{datahub,langfuse,smtp} and PATCH /api/v1/admin/conf (README.md §7; request/response contracts in spec/API.md).
4. Confirm all expected components are running, and report access URLs from the overlay's ingress hosts (app., api., airflow.).

---

Action: uninstall

Show current state

1. Show what is currently deployed:
   • helm list --all-namespaces
   • kubectl get pods -n <namespace> for each namespace from .env
   • kubectl get pvc -n <namespace>
2. Ask the user to confirm they want to remove resources before proceeding.

Full uninstall

1. Ask the user whether to also delete PVCs and namespaces (both default to preserved).
2. Execute the uninstall script with flags:
   • Always pass --no-question (user already confirmed).
   • dev — full wipe (PVCs + namespaces): ./helm-charts/bin/uninstall.sh --profile dev --no-question --delete-all
   • dev — release-only (preserve PVCs and namespaces): ./helm-charts/bin/uninstall.sh --profile dev --no-question
   • dev — mix-and-match with --delete-pvcs and/or --delete-namespaces for partial wipes.
   • prod — --delete-pvcs is dev-only and does not apply. Namespace deletion is prod's only full wipe: ./helm-charts/bin/uninstall.sh --profile prod --no-question --delete-namespaces The flagless form uninstalls the release and chart-derived Secrets only. The operator-owned credential Secret is never deleted by the script. Before removing anything by hand, read README.md §"Prod: what survives an uninstall" — deleting dataspoke-secrets destroys the only copy of the 12 credentials and strands a retained Postgres PVC, and dataspoke-airflow-fernet-key must be kept or deleted together with that PVC or Airflow breaks on the next install.
3. Clean up any orphaned PersistentVolumes in Released state.

Partial uninstall (specific components)

uninstall.sh --components frontend is the one supported targeted teardown: it runs helm upgrade --reuse-values --set frontend.enabled=false on the dataspoke release (removes the UI Deployment/Service/Ingress, leaves everything else). --components api is rejected — the api subchart is the core service; to stop it temporarily use kubectl scale deployment/dataspoke-api --replicas=0. For any other single component, delete its Helm release (or manifests) directly with helm uninstall <release> -n <ns> or kubectl delete -f ..., then reinstall via install.sh --components <csv>. Do NOT delete namespaces during partial uninstall.

Post-uninstall

1. Confirm cleanup with /k8s-work.
2. Report the clean state.

---

Action: reinstall

There is no dedicated reinstall.sh, and uninstall.sh does not support --components. Reinstall by deleting the target component's Helm release / manifests directly, then re-running install.sh --components <name> — install.sh is idempotent and helm upgrade --install handles re-creation.

Steps

1. Parse $ARGUMENTS to identify the target component (see the Component names table above). If no component is specified, ask the user which to reinstall.
2. Delete the component's existing release (look up the release/manifest set first — e.g. helm list -A or check the peripheral script for the manifest source), then re-run install:

   # Example: reinstall the dataspoke umbrella chart
   helm uninstall dataspoke -n "${DATASPOKE_KUBE_DATASPOKE_NAMESPACE}"
   ./helm-charts/bin/install.sh --profile dev --components dataspoke-infra
   

   For peripherals managed by plain manifests (dev-lock, dummy-data), kubectl delete -f helm-charts/dev-peripherals/<name>/manifests/ before reinstalling.
3. Monitor output for errors. If teardown or rollout fails, report the error and suggest remediation.
4. On success, confirm the component is running and report access URLs.

---

Action: health-check

1. Run ./helm-charts/bin/health-check.sh and report the results.
2. Supported flags:
   • --quick — TCP-only checks (skip deep application-layer probes)
   • --keep-lock — don't touch an existing dev-env lock
   • --force-release — release a held lock without prompting
3. If any service is unhealthy, show the reinstall command: ./helm-charts/bin/install.sh --profile dev --components <name> (component names per the table above).
4. For deeper cluster-level diagnostics, invoke the /k8s-work skill.

---

Action: run-api

Rebuild the DataSpoke API Docker image, redeploy it via helm upgrade, and roll the API deployment. This is the code-iteration path for the backend. (For the frontend equivalent, use --components frontend — see Partial install above; most frontend iteration happens via host pnpm dev and only needs an in-cluster deploy for prod-parity verification.) The API is accessible via nginx-ingress — no port-forward needed. Airflow callbacks reach it via cluster DNS (http://dataspoke-api:8002). Stub-mode toggles are in runtime_config (stub_redis_client, stub_llm_client, stub_pgvector_manager, stub_notification_service); the dev-profile install seeds them all to true post-install, flippable via PATCH /api/v1/admin/conf.

Pre-flight

1. Verify helm-charts/.env.dev (or .env.prod for prod) exists. If not, run configure first.
2. If the user requests it (or --health-check flag), run ./helm-charts/bin/health-check.sh --quick to confirm infrastructure is reachable. If it fails, suggest /k8s-deploy health-check or /k8s-deploy install and stop.
3. Run uv sync to ensure Python dependencies are up to date.

Option parsing

Parse $ARGUMENTS and the user's request for these options:

Option	CLI flag	Default	Description
skip-build	--skip-build	off	Skip Docker build, deploy existing image only
image-tag	--image-tag <tag>	dev	Override the image tag (CI-built images)
health-check	(runs ./helm-charts/bin/health-check.sh --quick first)	off	Pre-flight infrastructure check
stop	(no flag — see Stop section)	off	Scale down the API deployment

Deploy

1. Run ./helm-charts/bin/install.sh --profile dev --components api with parsed flags in the foreground. The script builds the API image (via helm-charts/bin/build-image.sh api), runs helm upgrade --install, and rolls the API deployment to pick up the new :dev image. First run can take 5–10 minutes due to image builds; --skip-build rebuilds are 1–2 minutes.
2. Monitor the output for errors. If the build or rollout fails, report the error and suggest remediation.
3. On success, report the running state to the user:
   • API URL: ${DATASPOKE_KUBE_INGRESS_SCHEME:-http}://api.${DATASPOKE_KUBE_INGRESS_DOMAIN}/api/v1/
   • ReDoc UI: ${DATASPOKE_KUBE_INGRESS_SCHEME:-http}://api.${DATASPOKE_KUBE_INGRESS_DOMAIN}/redoc
   • Health: ${DATASPOKE_KUBE_INGRESS_SCHEME:-http}://api.${DATASPOKE_KUBE_INGRESS_DOMAIN}/health
   • How to run tests: set -a && source helm-charts/.env.dev && set +a && uv run pytest tests/integration/spot/ (spot) or … tests/integration/api_wired/ (UC user stories) — run in separate invocations. The conftest runtime_conf fixture verifies the API has stub_redis_client / stub_pgvector_manager / stub_notification_service all true before the suite runs.
   • How to stop: kubectl scale deployment/dataspoke-api --replicas=0 -n "${DATASPOKE_KUBE_DATASPOKE_NAMESPACE}"

Stop

If the user asks to stop:

1. Run kubectl scale deployment/dataspoke-api --replicas=0 -n "${DATASPOKE_KUBE_DATASPOKE_NAMESPACE}".
2. Confirm the deployment has been scaled down.