infrastructure-as-code

name	infrastructure-as-code
description	Use this skill when a task involves detecting config drift between live cluster state and code, capturing manual kubectl/psql/gcloud fixes into Helm values or Terraform, preparing populated values files, or verifying a fresh deploy would succeed from code alone. Don't use it for Helm template authoring (use helm-chart-expert) or migration safety checks (use migration-check).
version	1.0.0
owner	swarmery-infra
docs	{"status":"reviewed","source_sha":"0d9d2e7c1934","updated":"2026-08-06T00:00:00.000Z"}

Purpose

Enforce the invariant that a fresh server deploy of the project must succeed from code alone, without any manual kubectl patch, kubectl exec, or helm --set overrides that are not persisted. Covers config drift detection, the populated values pattern, post-incident IaC capture, and the "Where Changes Go" decision map.

When to use this skill (triggers)

• Deploying the platform to a new server or environment and verifying all config is in code
• Auditing config drift after manual cluster fixes (kubectl patch, psql exec, gcloud console changes)
• Preparing or regenerating .populated.yaml values files via the infrastructure repo's mapEnvValuesFromEnv.sh (or equivalent env-mapping script)
• Verifying that all manual cluster changes are captured in the correct source file (Helm values, Terraform, migration script)
• Diagnosing literal $VARIABLE_NAME strings appearing in K8s Secrets

When NOT to use this skill (anti-triggers)

• Writing or debugging Helm chart templates -- use helm-chart-expert
• Checking migration safety or schema alignment -- use migration-check
• Deploying the edge service (project.json → device) to edge devices or managing k3s -- use kubernetes-deployment
• Managing Keycloak realm/client configuration -- use keycloak

Required environment (Runtime: .claude/skills/infrastructure-as-code/SKILL.md)

• Tools/libraries: helm (v3.10+), kubectl, terraform, diff, bash
• Scripts: the infrastructure repo's files/mapEnvValuesFromEnv.sh and files/runDatabaseMigrations.sh (or the project's equivalents)
• Repos: the infrastructure, chart, and Terraform repos (project.json → repos)

Terraform environment layout (canonical reference)

Canonical doc: the Terraform repo's README §Environment layout. Read it before adding any new resource; it is the single source of truth for current vs legacy env dirs.

Common patterns to watch for:

Element	Pattern
Active env dirs	the currently maintained environments/<org>/{shared,dev,...} tree
Legacy (DEAD) env dirs	superseded environments/... trees whose backend state buckets no longer exist -- do not add resources there
GCS state bucket	often a single bucket, with per-env discrimination via prefix = "iac/env-<name>"
Terraform runner GCP project	may differ from the cluster's GCP project -- trust the repo README, not stale variable defaults
Environment naming	the staging alias (project.json → cloud.envAlias) may appear as dev in Terraform paths -- same environment
Cutover history	look for cutover notes committed alongside the env dirs

Rule of thumb: if terraform init complains about a missing GCS bucket, you are probably in a legacy env dir. Switch to the corresponding active env dir; don't try to create the missing bucket.

Inputs

• environment: string -- target environment name (e.g., localdev, <envAlias>, prod)
• drift_source: string (optional) -- what changed: helm-override, kubectl-patch, psql-fix, gcp-console, terraform-drift
• values_file: string (optional) -- path to the template values file (e.g., values.<envAlias>.yaml)

Outputs

Format: A drift report listing what diverged and where to persist the fix, OR a verified "clean deploy would succeed" confirmation.

Length budget: Drift report max 40 lines. Post-incident capture checklist max 20 lines. Verification command output summarized as pass/fail, not echoed in full.

Output template:

## Drift Report -- {environment}

| # | Resource | Live Value | Code Value | Source of Truth File | Action |
|---|----------|------------|------------|---------------------|--------|
| {n} | {resource} | {live} | {code} | {file}:{key} | {persist/escalate} |

### Verification
helm template: {PASS|FAIL}
helm upgrade --dry-run: {PASS|FAIL}
terraform plan: {PASS|FAIL|N/A}

### Confidence: {HIGH|MEDIUM|LOW} -- {rationale}

For post-incident capture: a checklist of files to update with specific paths and keys.

Procedure (Checkpoint: after each step)

1. Identify what changed -- Determine the category of drift: Helm override, kubectl patch, SQL fix, GCP console change, or Terraform resource. Checkpoint: category identified.

2. Detect drift -- Run the appropriate diff commands (see detection commands below). Helm diff and Terraform plan are independent and may run in parallel Bash calls. Checkpoint: drift evidence captured.

3. Map to source of truth -- Use the "Where Changes Go" table to identify which file to update. For ambiguous cases (e.g., a change spanning Helm values and Terraform), list both targets and ask the user which applies. Checkpoint: target file identified.

4. Persist the fix -- Update the correct source file (values template, Terraform .tf, migration script). Checkpoint: source file modified.

5. Verify clean deploy -- Render templates and run dry-run to confirm a fresh deploy would succeed. Checkpoint: helm template and helm upgrade --dry-run both exit 0, or terraform plan shows no diff.

Drift detection commands

# Compare live Helm values vs values file
helm get values <release> -n "$NAMESPACE" > "$(mktemp)"
diff "$(mktemp)" values.$ENV.populated.yaml

# Compare live manifests vs rendered templates
helm get manifest <release> -n "$NAMESPACE" > "$(mktemp)"
helm template <release> . -f values.$ENV.populated.yaml > "$(mktemp)"
diff <live> <rendered>

# Terraform drift
cd <terraform-repo>/environments/$ENV
terraform plan  # Any non-empty plan = drift

Where Changes Go -- Decision Table

Category	Source of Truth	Tool	Drift Owner
K8s workloads, services, ingress	Helm chart templates + values files	helm upgrade	this skill
Secrets (passwords, API keys)	Env vars + mapEnvValuesFromEnv.sh	populated values	this skill
K8s Secret missing entirely (not wrong value)	Bootstrap-secret pattern	the repo's bootstrap-secret script	kubernetes-deployment skill
Database schema	Migration scripts	runDatabaseMigrations.sh	migration-check skill
GCP VMs, networks, firewalls	Terraform .tf files	terraform apply	this skill
GCP IAM, service accounts	Terraform .tf files	terraform apply	this skill
TLS certificates	cert-manager CRDs in Helm templates	helm upgrade	this skill

Self-check before returning (anti-hallucination, confidence labels, format match)

• I never suggested editing a .populated.yaml file -- only the template values.$ENV.yaml
• I confirmed the env var is set before instructing the user to re-run mapEnvValuesFromEnv.sh
• I used $NAMESPACE or <namespace> placeholder, not a hardcoded namespace name
• I used $ENV placeholder for environment-specific paths, not a hardcoded environment name
• I cited which file and key produced the drift evidence, using file:key format
• I cross-referenced the bootstrap-secret pattern (from kubernetes-deployment skill) if the drift involves a missing secret -- distinguished from wrong-valued secrets (which stay in this skill)
• Output matches the drift report template format
• Confidence label (HIGH / MEDIUM / LOW) is attached -- label LOW when diff may produce false positives due to YAML key ordering differences

Common mistakes to avoid (DO NOT patterns)

• DO NOT edit .populated.yaml files -- they are generated outputs, not sources of truth; edit values.$ENV.yaml instead
• DO NOT leave a manual kubectl patch or helm --set override uncaptured -- persist it immediately to the values file
• DO NOT hardcode the infra namespace in commands -- use the $NAMESPACE placeholder
• DO NOT write diff output to /tmp -- use $(mktemp) or the working directory
• DO NOT assume mapEnvValuesFromEnv.sh will succeed if env vars are unset -- check first
• DO NOT confuse a missing secret (bootstrap-secret pattern, owned by kubernetes-deployment) with a wrong-valued secret (populated values pattern, owned by this skill)

Escalation (stop-and-ask conditions)

• Stop and ask when: the env var source is unknown (where does $REDIS_PASSWORD come from?)
• Stop and ask when: the populated values file contains a literal $VARIABLE_NAME string and the env var cannot be located
• Stop and ask when: the drift involves a resource not covered by the "Where Changes Go" table
• Stop and ask when: the drift spans both Helm values and Terraform (ambiguous single-source-of-truth)
• Stop and ask when: mapEnvValuesFromEnv.sh has not been updated to include a newly added env var (stale script)

Examples

## Diagnosing literal $VARIABLE_NAME in a K8s Secret

Symptom: Redis connections fail. Pod env vars look correct, but the K8s Secret <infra-release>-redis contains the literal string $REDIS_PASSWORD.

# 1. Check that the env var is set
echo $REDIS_PASSWORD  # Should output the actual password

# 2. Re-generate the populated values file
./files/mapEnvValuesFromEnv.sh -en <envAlias>

# 3. Re-deploy with the regenerated file (dry-run first)
helm upgrade <infra-release> . \
  -f values.<envAlias>.populated.yaml \
  -n "$NAMESPACE" \
  --dry-run

# 4. If dry-run looks correct, apply
helm upgrade <infra-release> . \
  -f values.<envAlias>.populated.yaml \
  -n "$NAMESPACE" \
  --wait --timeout 8m

Output:

## Drift Report -- <envAlias>

| # | Resource | Live Value | Code Value | Source of Truth File | Action |
|---|----------|------------|------------|---------------------|--------|
| 1 | Secret/<infra-release>-redis | $REDIS_PASSWORD (literal) | <actual password> | values.<envAlias>.yaml:redis.password | Re-run mapEnvValuesFromEnv.sh |

### Verification
helm template: PASS
helm upgrade --dry-run: PASS

### Confidence: HIGH -- env var confirmed set, populated file regenerated

## Post-incident IaC capture checklist

After manually patching a Keycloak deployment's memory limit via kubectl edit:

What Changed	Where to Persist
Keycloak memory limit	values.$ENV.yaml -> keycloak.resources.limits.memory

# values.<envAlias>.yaml (template file, committed to Git)
keycloak:
  resources:
    limits:
      memory: "1Gi"  # was 512Mi, increased during incident

Then verify:

helm template <infra-release> . -f values.<envAlias>.populated.yaml | grep -A5 "memory"
helm upgrade <infra-release> . -f values.<envAlias>.populated.yaml -n "$NAMESPACE" --dry-run

Failure modes (symptom -> detection -> action)

• Literal $VARIABLE in Secret: symptom: service can't authenticate to Redis/PostgreSQL -> detect: kubectl get secret <name> -o jsonpath='{.data.password}' | base64 -d shows $REDIS_PASSWORD literally -> fix: set the env var, re-run mapEnvValuesFromEnv.sh, re-deploy
• Helm values not applied after manual --set: symptom: helm get values shows overrides not in the values file -> detect: diff <(helm get values <release>) values.$ENV.populated.yaml -> fix: add the override to values.$ENV.yaml, regenerate populated file
• Terraform drift undetected: symptom: GCP Console change works but terraform apply would revert it -> detect: terraform plan shows a change -> fix: update the corresponding .tf file, run terraform apply
• Stale mapEnvValuesFromEnv.sh: symptom: new service config key shows $NEW_VAR literally after deployment -> detect: check if mapEnvValuesFromEnv.sh references the new variable -> fix: add the variable mapping to the script, re-run, re-deploy

Related skills (compose vs defer)

• helm-chart-expert -- defer to it for Helm template authoring and chart structure; compose with it when drift involves a template bug
• kubernetes-deployment -- defer to it for k8s cluster operations and the bootstrap-secret pattern (missing secrets); compose when drift involves a secret that does not exist at all (vs wrong value)
• migration-check -- defer to it for migration safety; compose when a manual SQL fix needs to be captured as a migration script
• keycloak -- defer to it for Keycloak realm/client configuration; compose when Keycloak Helm values drift is detected

How to use

What it does

This skill keeps your infrastructure honest: everything a fresh deploy needs must live in code, not in someone's shell history. It compares live cluster and cloud state against what your Helm values, Terraform files, and migration scripts actually say, then tells you exactly which file and key to update so the manual fix survives the next deploy.

When to use it

• Someone ran kubectl patch, helm --set, or a console change during an incident, and you need it captured in code before it gets reverted.
• You are bringing up a new environment and want proof a clean deploy would succeed from the repository alone.
• A Kubernetes Secret holds a literal $VARIABLE_NAME string instead of the real value, and services cannot authenticate.
• You need to regenerate a populated values file and confirm the render and dry-run both pass.

When not to use it

• Writing or debugging Helm chart templates — use helm-chart-expert.
• Checking migration safety or schema alignment — use migration-check.
• Cluster operations on edge devices, or a Secret that is missing entirely rather than wrong — use kubernetes-deployment.
• Realm and client configuration for your identity provider — use keycloak.

How to invoke

Skill(skill: "infra-pack:infrastructure-as-code")

Invoke it directly, or just describe the drift ("we patched the memory limit by hand yesterday") and it activates on its own.

Inputs

• environment — the target environment name, such as localdev, <envAlias>, or prod — required.
• drift_source — what changed: helm-override, kubectl-patch, psql-fix, gcp-console, or terraform-drift — optional.
• values_file — path to the template values file, e.g. values.<envAlias>.yaml — optional.

What you get back

A drift report table (max 40 lines) with one row per divergence: resource, live value, code value, the file:key to change, and the action. Below it, a verification block showing pass or fail for helm template, helm upgrade --dry-run, and terraform plan, plus a HIGH/MEDIUM/LOW confidence label with its rationale. For post-incident work you get a short checklist of files and keys to update instead.

Worked example

Skill(skill: "infra-pack:infrastructure-as-code")

> Redis auth is failing in <envAlias>. The pod env looks fine but the
> secret seems wrong.

The skill checks the env var is set, has you regenerate the populated values file, and runs helm upgrade --dry-run before applying. You end up with a one-row drift report pointing at values.<envAlias>.yaml:redis.password, both verification steps passing, and confidence HIGH.

Related

• helm-chart-expert — prefer it when the root cause is a template bug rather than a value that never made it into code.
• kubernetes-deployment — prefer it for the bootstrap-secret pattern when a Secret does not exist at all.
• migration-check — prefer it when a manual SQL fix needs to become a reviewed migration script.