loom-terraform

# Terraform / OpenTofu ## Overview Terraform/OpenTofu infrastructure: module design, state/backends, multi-environment strategy, provider config, refactoring/import. **Prefer `tofu` over `terraform`** — OpenTofu is the Linux Foundation fork, config-compatible, and has capabilities OSS Terraform lacks (client-side state encryption). Command mapping table is at the end. The single highest-value section is **Expert Practices** below — the anti-patterns/gotchas there are where naive-but-working configs go wrong. ## Module & Configuration Design Canonical layout — shared `modules/`, one root module per environment, lock file committed: ```text terraform/ ├── modules/{vpc,eks,rds}/ # each: main.tf variables.tf outputs.tf versions.tf ├── environments/{dev,staging,prod}/ # each: main.tf backend.tf terraform.tfvars .terraform.lock.hcl └── .terraform.lock.hcl ``` Principles: reusable modules with typed, validated, described inputs; providers pinned in `required_providers`; remote state with locking; `locals` for computed/DRY values; documented outputs; `lifecycle` rules where deletion/replacement risk exists. **Variable validation** — validate at the boundary; `contains()` for enums, length/regex for shape: ```hcl variable "environment" { type = string validation { condition = contains(["dev", "staging", "prod"], var.environment) error_message = "Environment must be dev, staging, or prod." } } ``` **Module composition** — wire flat modules via input/output, never nest deep (see Anti-Patterns). Version external modules with `~>`; pin internal git modules by tag `ref=`: ```hcl module "vpc" { source = "terraform-aws-modules/vpc/aws", version = "~> 5.0" } module "internal" { source = "git::https://github.com/org/tf-modules.git//vpc?ref=v1.2.3" } ``` ## State & Backends **S3 with native locking** (no DynamoDB): ```hcl terraform { backend "s3" { bucket = "myorg-terraform-state" key = "prod/eks/terraform.tfstate" region = "us-west-2" encrypt = true kms_key_id = "arn:aws:kms:us-west-2:123456789012:key/..." use_lockfile = true # native S3 conditional-write lock; no DynamoDB table/IAM } } ``` - `use_lockfile = true` locks via an S3 conditional `PutObject` of `<key>.tflock`. In **Terraform**, `dynamodb_table` locking is deprecated (future removal); migrate by keeping BOTH until all CI/operators cut over, then drop `dynamodb_table`. **OpenTofu has NOT deprecated DynamoDB locking** — this is Terraform-only. - **`backend` blocks cannot reference `var.*`/`local.*`/`data.*`** — evaluated before variables exist; any reference errors at `init`. Use *partial configuration*: omit dynamic args, supply via `-backend-config=KEY=VALUE` or a `.tfbackend` file. - **NEVER pass credentials via `-backend-config`** (persisted into `.terraform/` and plan files) or hardcode them — supply backend creds only through env vars (`AWS_ACCESS_KEY_ID`, etc.). - `lifecycle` is a resource meta-argument — **invalid inside a `backend` block**. Protect the state bucket via `prevent_destroy` on its own `aws_s3_bucket` resource + versioning/object-lock. ```bash tofu init -backend-config="bucket=$TF_STATE_BUCKET" -backend-config="key=prod/app.tfstate" ``` **State operations** — prefer the version-controlled block form (`moved`/`removed`/`import`, see Idioms) over these imperative commands: | Command | Purpose | Prefer | | --- | --- | --- | | `tofu state list` / `show <addr>` | inspect | — | | `tofu state mv A B` | rename/move | `moved` block | | `tofu state rm <addr>` | drop from state (no destroy) | `removed { lifecycle { destroy = false } }` | | `tofu import <addr> <id>` | adopt existing resource | `import` block (+ `-generate-config-out`) | | `tofu state pull > s.json` / `push s.json` | raw read/write (push is DANGEROUS) | — | | `tofu state replace-provider OLD NEW` | after provider migration | — | | `tofu force-unlock <LOCK_ID>` | clear stuck lock (verify nothing running first) | — | **State migration** — configure new backend, then `tofu init -migrate-state` (add `-backend-config=...` for target). Always encrypt remote state; never commit state files. ## Multi-Environment **Strategy 1 (RECOMMENDED): directory-per-env.** Each env is a separate root module with its OWN backend bucket and IAM role — switching envs requires a deliberate `cd`, so a dev shell physically cannot target prod. Right choice for dev/staging/prod. **Strategy 2: workspaces — NOT environment isolation.** Official docs: workspaces "are not appropriate for system decomposition or deployments requiring separate credentials and access controls." All workspaces share ONE backend, ONE auth context, ONE provider config — no prod/dev boundary, and a mistyped `tofu workspace select prod && tofu apply` hits prod with no config guard. Use ONLY for short-lived variants within a single access boundary (PR-preview/ephemeral test envs). **Strategy 3: tfvars files** — one root module, per-env var files. Flexible but shares backend/creds like workspaces unless combined with Strategy 1: ```bash tofu apply -var-file="environments/prod.tfvars" ``` ## Security Compact, non-obvious flags (the surrounding IAM/SG HCL is standard): ```hcl # KMS: rotation + generous deletion window resource "aws_kms_key" "data" { enable_key_rotation = true deletion_window_in_days = 30 } # S3 SSE-KMS with bucket keys (cuts KMS API cost ~99% on high-volume buckets) resource "aws_s3_bucket_server_side_encryption_configuration" "data" { bucket = aws_s3_bucket.data.id rule { apply_server_side_encryption_by_default { kms_master_key_id = aws_kms_key.data.arn, sse_algorithm = "aws:kms" } bucket_key_enabled = true } } # RDS encryption is create-time only — cannot be toggled on an existing instance resource "aws_db_instance" "main" { storage_encrypted = true, kms_key_id = aws_kms_key.data.arn } ``` - **Define SG rules as separate `aws_security_group_rule` (or `aws_vpc_security_group_ingress_rule`) resources**, not inline `ingress`/`egress` blocks, when two SGs reference each other — inline blocks force a dependency cycle; separate rule resources break it. - Store credentials in Secrets Manager/SSM and pass only ARNs; mark any secret-bearing output `sensitive = true` (but see Security below — that does NOT keep it out of state). - IAM: scope `Action`/`Resource` to exact ARNs; avoid `"*"` outside the KMS root key policy. ## Providers ```hcl terraform { # Floor = primitives this config uses. 1.5 import/check blocks; 1.9 cross-object validation. # Raise: >=1.10 ephemeral values; >=1.11 write-only args + native S3 locking. # OpenTofu: >=1.7 state encryption; >=1.10 S3 locking. required_version = ">= 1.9.0" required_providers { aws = { source = "hashicorp/aws", version = "~> 5.0" } # 5.x, not 6.0 kubernetes = { source = "hashicorp/kubernetes", version = "~> 2.23" } } } provider "aws" { region = var.region default_tags { tags = { Environment = var.environment, ManagedBy = "terraform" } } } # Aliases for multi-region/account — resources/modules select via `provider =` / `providers = {}` provider "aws" { alias = "replica", region = "us-east-1" } resource "aws_s3_bucket" "replica" { provider = aws.replica, bucket = "${var.name}-replica" } ``` `default_tags` auto-applies to every taggable resource — no per-resource `merge()` needed for org-wide tags. ## Core Patterns **Dynamic blocks** — generate repeatable nested blocks from a collection: ```hcl dynamic "ingress" { for_each = var.ingress_rules content { from_port = ingress.value.from_port to_port = ingress.value.to_port protocol = ingress.value.protocol cidr_blocks = ingress.value.cidr_blocks } } ``` **`for` expressions** — build maps/lists; filter with `if`: ```hcl subnet_by_az = { for s in aws_subnet.private : s.availability_zone => s.id } prod_instances = { for k, v in var.instances : k => v if v.environment == "production" } ``` `count` vs `for_each`, `lifecycle` (`create_before_destroy`/`prevent_destroy`/`ignore_changes`), and `depends_on` are covered in depth under **Expert Practices** — read the Gotchas there before using any of them. ## Troubleshooting | Symptom | Fix | | --- | --- | | State lock timeout | Verify nothing running, then `tofu force-unlock <LOCK_ID>` | | Corrupt provider cache | `rm -rf .terraform/ && tofu init` — do NOT delete `.terraform.lock.hcl` | | Resource exists but not in state | `import` block (or `tofu import`), then reconcile HCL | | Circular dependency | Extract the coupling into a standalone resource (e.g. `aws_security_group_rule`) or make deps unidirectional | | Force replacement | `tofu apply -replace="aws_x.y"` — `taint` is DEPRECATED (mutates state with no plan preview) | | Drift from console change | `tofu plan -refresh-only` to detect; `tofu apply` to restore code definition | | Module source/version changed | `tofu init -upgrade` | | Long-running create/update/delete | resource `timeouts { create = "60m" ... }` block | **Regenerate (never just delete) a corrupt lock file** across all target platforms — deleting it discards validated checksum pins and can pull a different version next `init`: ```bash tofu providers lock -platform=linux_amd64 -platform=darwin_arm64 -platform=windows_amd64 ``` **Sensitive data in state:** `sensitive = true` only redacts UI output — the value is still plaintext in state. Real fixes: write-only args (TF ≥1.11) + ephemeral resources (≥1.10) never persist; OpenTofu ≥1.7 `encryption` block; or external store passing only ARNs. See Expert Practices → Security. ## Testing & Validation ```bash tofu fmt -recursive # format (run before every commit) tofu validate # config validity (no cloud calls) tfsec . # or trivy/checkov — static security scan infracost breakdown --path . # cost estimate ``` **Plan review checklist:** all changes expected · no unintended `-/+` (destroy/replace) · sensitive values not in outputs · encryption enabled · IAM least-privilege · SGs restrictive · tags present. ## Expert Practices: Idioms, Anti-Patterns & Gotchas High-signal guidance from the official docs. The *why* matters as much as the rule. ### Anti-Patterns **Never put a `provider` block in a reusable child module.** Docs: "A module intended to be called by one or more other modules must not contain any provider blocks," and "a module with its own provider configurations is not compatible with for_each, count, or depends_on" on the call. Mechanism: a provider config and the resources it manages must be destroyed together, but the graph cannot guarantee that ordering once the call is multiplied/ordered. Declare `provider` blocks only in root modules; pass aliased providers explicitly via `providers`. **Aliased providers are NEVER inherited** — forgetting to pass one silently falls back to the default (wrong region/account). Child modules declare `required_providers` for version constraints only — no provider block, no `backend` block. ```hcl provider "aws" { alias = "usw2", region = "us-west-2" } module "app" { source = "./modules/app" providers = { aws = aws.usw2 } for_each = var.environments # works only because the module has no provider block } ``` **Keep module composition flat (single level of child modules).** Modules take inputs, emit outputs, and know nothing about where state lives. Compose flat at the root and wire via input/output rather than nesting (root → A → B → C), which obscures dependency flow, complicates provider passing, and makes refactoring hazardous. ### Idioms **Don't repeat the resource type in the name label.** `aws_security_group.app` already carries the type; `aws_security_group.app_security_group` says the noun twice. Short role-descriptive nouns, underscores not dashes (`aws_instance.web`, not `aws_instance.ec2-web-server`). The `this`/`main` singleton convention is a *community* idiom, NOT in the official style guide. **Use `moved`/`removed`/`import` blocks, not the imperative CLI.** `state mv`/`import`/`taint` are imperative: every operator must run them, they aren't version-controlled or reproducible. Block forms are validated, reviewable in PRs, and run in CI with no manual steps. - `moved` (≥1.1) encodes renames. **Never delete a published `moved` block** — it makes configs referencing the old address plan a *delete* instead of a move. Chain renames across successive moves; retain all historical blocks. - `removed` (≥1.7) with `lifecycle { destroy = false }` drops a resource from management WITHOUT destroying real infra. - `import` (≥1.5) blocks make import plannable; pair with `-generate-config-out=FILE` to auto-generate HCL for brownfield adoption. OpenTofu 1.7+ adds `for_each` for bulk imports. Unlike `moved`, import blocks may be deleted after they succeed. ```hcl moved { from = aws_instance.server, to = aws_instance.app_server } # chain, never prune removed { from = aws_db_instance.legacy, lifecycle { destroy = false } } # keep real infra import { to = aws_instance.app, id = "i-1234567890abcdef0" } # then -generate-config-out ``` **Commit `.terraform.lock.hcl`, pre-populate multi-platform checksums.** `init` records checksums only for the platform it ran on, so a teammate/CI on another OS/arch hits "no matching checksum". The lock pins PROVIDERS only — module pinning lives in the `version` constraint. A CI `init -upgrade` re-resolves from constraints and overwrites lock selections, silently unpinning. ```bash tofu providers lock -platform=linux_amd64 -platform=darwin_amd64 \ -platform=darwin_arm64 -platform=windows_amd64 ``` ### Gotchas **`count` index shift silently recreates the WRONG resources — prefer `for_each`.** Removing a non-tail element of a `count` list reindexes everything after it: removing `[1]` of three makes old-`[2]` become `[1]`, so Terraform destroys & recreates the resource you never touched — a data-loss footgun for RDS/EBS/subnets. Key `for_each` over a map/set by STABLE identity. Caveats: `toset()` on a list silently dedupes duplicates (dropping instances), and a `moved` block migrates existing count-indexed state to for_each keys without recreation. ```hcl # BAD — count: removing var.names[1] reindexes [2]→[1], recreating a survivor resource "aws_instance" "app" { count = length(var.names), ami = var.ami } # GOOD — for_each: address keyed by stable name, deletes only the removed key resource "aws_instance" "app" { for_each = toset(var.names), ami = var.ami, tags = { Name = each.value } } ```