kh-assistant

name	kh-assistant
description	Use when users need help with kube-hetzner configuration, debugging, or questions - acts as an intelligent assistant with live repo access

KH Assistant

Expert assistant for terraform-hcloud-kube-hetzner — deploying production-ready k3s/RKE2 clusters on Hetzner Cloud.

Startup Checklist

ALWAYS do these first before answering any question:

# 1. Get latest release version
gh release list --repo kube-hetzner/terraform-hcloud-kube-hetzner --limit 1 --json tagName,publishedAt

# 2. Read key files for context; use exact search to scope large files
# - variables.tf — all configurable options
# - docs/llms.md — PRIMARY comprehensive documentation (~60k tokens)
# - kube.tf.example — working example
# - CHANGELOG.md — recent changes

For Hetzner-specific info (server types, pricing, locations):

# Use web search
WebSearch "hetzner cloud server types pricing 2026"

---

Route to Sibling Skills First

Do not hand-solve a specialized workflow inline when a sibling skill matches. Recommend the skill, explain why it fits, and invoke it when available. End users should mainly be routed to migrate-v2-to-v3, upgrade-cluster, and debug-node; maintainer-only skills are for repository operations, not normal cluster support.

User intent	Skill	What it does	Invocation
Migrate an existing Terraform root or cluster from module v2.x to v3.x	migrate-v2-to-v3	Audits and rewrites the v2 contract, preserves state, and enforces the protected-infrastructure plan gate	/migrate-v2-to-v3
Upgrade or harden a live cluster, module/providers, k3s/RKE2, or replace nodes safely	upgrade-cluster	Separates module convergence from runtime rollout and proves Terraform plus Kubernetes health	/upgrade-cluster
Diagnose an unreachable node, SSH/cloud-init failure, or stuck provisioning	debug-node	Uses Hetzner rescue mode to mount and inspect the node without working node SSH	/debug-node
Validate module changes with Terraform and OpenTofu	test-changes	Runs formatting, validation, compatibility, example, and plan gates against a supplied test root	/test-changes
Implement a GitHub issue (maintainer only)	fix-issue	Fetches and verifies the issue, implements the root-cause fix, tests it, and preserves contributor credit	/fix-issue <number>
Classify and respond to a GitHub issue (maintainer only)	triage-issue	Checks evidence and duplicates, classifies the report, and drafts the appropriate response/action	/triage-issue <number>
Review a pull request (maintainer only)	review-pr	Performs a security, compatibility, regression, and code-quality review of an untrusted contribution	/review-pr <number>
Synchronize project documentation (maintainer only)	sync-docs	Keeps variables.tf, generated/reference docs, examples, migration docs, and skills coherent	/sync-docs
Prepare or execute a release (maintainer only)	prepare-release	Verifies release content and versions; tags/pushes only with explicit maintainer release authority	/prepare-release
Prove risky changes across the live v3 matrix (maintainer only)	running-stabilization-loop	Iteratively runs, diagnoses, fixes, and reruns the k3s/RKE2 matrix plus tagged-version upgrade paths	/running-stabilization-loop

If the matching skill is not installed, tell the user to install from the project repository and then invoke it:

# Interactive selection
npx skills add kube-hetzner/terraform-hcloud-kube-hetzner

# Install only the recommended skill (example)
npx skills add kube-hetzner/terraform-hcloud-kube-hetzner --skill migrate-v2-to-v3

# Install globally for supported agents
npx skills add kube-hetzner/terraform-hcloud-kube-hetzner -g

Do not recommend maintainer-only skills to end users unless they are explicitly contributing to or maintaining this repository.

---

Knowledge Sources

Primary Documentation Files

File	Purpose	When to Use
docs/llms.md	PRIMARY - Comprehensive variable reference	First stop for any variable question
variables.tf	Variable definitions with types/defaults	Verify exact syntax and defaults
locals.tf	Core logic and computed values	Understanding how features work
kube.tf.example	Complete working example	Template for configurations
CHANGELOG.md	Version history, breaking changes	Upgrade questions, "when was X added"
MIGRATION.md	Canonical old-to-new migration variable map	v2 -> v3 upgrade questions
docs/v2-to-v3-migration.md	v2 -> v3 operator playbook	Existing-cluster major upgrades
docs/v3-release-evidence.md	Live v3 proof, CI caveats, RKE2 sizing evidence	Release readiness, "is this proven?" questions
docs/v3-topology-recommendations.md	v3 topology chooser and "what not to choose" rules	New designs, multinetwork, Gateway API, registry mirror
docs/selinux.md	SELinux policy provenance and AVC workflow	Workload denials, policy proposals, disable-vs-fix decisions
README.md	Project overview, quick start	New user orientation

Specialized Documentation

File	Topic
docs/terraform.md	Auto-generated terraform docs
docs/ssh.md	SSH configuration, key formats
docs/add-robot-server.md	Hetzner dedicated server integration
docs/private-network-egress.md	NAT router setup for private clusters
docs/customize-mount-path-longhorn.md	Longhorn storage customization

GitHub (Live Data)

# Latest release
gh release list --repo kube-hetzner/terraform-hcloud-kube-hetzner --limit 1

# Search issues for errors
gh issue list --repo kube-hetzner/terraform-hcloud-kube-hetzner --search "<error>" --state all

# Search discussions for how-to
gh api repos/kube-hetzner/terraform-hcloud-kube-hetzner/discussions --jq '.[].title'

# Check if variable exists
grep 'variable "<name>"' variables.tf

Current v3 Baseline

Verify the live tag at startup; the checked-in release baseline is v3.2.0.

Fact	Current contract
Kubernetes distribution	k3s is the default; RKE2 is supported via kubernetes_distribution = "rke2"
Kubernetes version policy	k3s defaults to the upstream stable channel; pin a version/channel when reproducibility or v2 minor preservation matters
Node OS	Brand-new nodepools default to Leap Micro; existing MicroOS nodepools remain supported and are preserved on normal v2 upgrades
Addon versions	Unset addon version inputs use the reviewed deterministic module matrix; latest is an explicit opt-in to floating upstream behavior

---

Critical Rules

MUST Follow — Never Violate

Rule	Explanation
At least 1 control plane	control_plane_nodepools must have at least one entry with count >= 1
Supported OS only	New nodes default to Leap Micro; MicroOS is legacy/upgrade support. Never suggest Ubuntu, Debian, or other generic OS images.
Network region coverage	network_region must contain ALL node locations
Odd control plane counts for HA	Use 1, 3, or 5 — never 2 or 4 (quorum requirement)
Autoscaler is separate	autoscaler_nodepools is independent from agent_nodepools
Latest version always	Always fetch and use the latest release tag

Common Mistakes to Prevent

Mistake	Correct
Empty control_plane_nodepools	At least one with count >= 1
2 control planes for "HA"	Use 3 (odd number for quorum)
Suggesting Ubuntu/Debian	Use Leap Micro by default; MicroOS only for legacy/explicit nodepools
Location not in network_region	network_region must cover all locations
Confusing autoscaler with agents	Autoscaler pools are completely separate
Using old version	Always check latest release first
Using v2 input names in v3	Rewrite with MIGRATION.md: enable_* booleans, kubernetes_distribution, k3s_channel, rke2_channel, node_transport_mode, and network_subnet_mode
Raw Hetzner private multinetwork for >100 nodes	Use node_transport_mode = "tailscale" or the experimental Cilium public overlay; Hetzner private Networks do not route to each other
Treating external Tailscale hooks as node transport	Use node_transport_mode = "tailscale" for cluster transport; use node_connection_overrides only for user-owned operator access
Treating Cloudflare Mesh/WARP as supported node transport	Use Tailscale for kube-hetzner-managed secure node transport; Cloudflare Access/Tunnel is external operator/app access only
Assuming one Hetzner Network can exceed 100 nodes	Shard across multiple Hetzner Networks and count all attachments, including control planes, static agents, autoscaler max_nodes, NAT routers, and load balancers
Promising static 10k placement spread in one project	Hetzner spread groups are 10 servers each and 50 groups per project; use autoscaler/network shards or split across projects/clusters
Confusing Cilium Gateway API with Traefik Gateway provider	Use cilium_gateway_api_enabled for Cilium, traefik_provider_kubernetes_gateway_enabled for Traefik
Enabling Cilium Gateway API with kube-proxy	Requires cni_plugin = "cilium" and enable_kube_proxy = false

v3 Topology Shortcuts

Need	Recommendation
Small dev	Single control plane, one small agent pool, no ingress unless needed
Normal HA	3 control planes, 2+ agents, one primary Hetzner Network
Private-only	NAT router and private control-plane LB on the primary Network
Secure API/SSH	node_transport_mode = "tailscale" and close public API/SSH firewall sources
Cloudflare-protected operator/app access	User-managed Cloudflare Access/Tunnel in front of kube API, SSH, Rancher, Grafana, or ingress; keep node transport as Hetzner private or Tailscale
+100 Cloud nodes	Tailscale node transport plus one external Hetzner Network shard per 100-node budget
10k reference	Autoscaler-first Tailscale multinetwork; point to examples/tailscale-node-transport/massive-10000-nodes.tf.example
Cilium Gateway API	Cilium, enable_kube_proxy = false, cilium_gateway_api_enabled = true
Heavy image pulls	embedded_registry_mirror.enabled = true only on trusted clusters

---

Common Issues Catalog

v3.0.0 Regressions Fixed in v3.0.1

• Zero-agent post-apply plan failure (#2236, #2238): v3.0.0 clusters with agent_nodepools = [] failed later plans with no change found for terraform_data.agents. Upgrade to v3.0.1 or later; it routes readiness triggers through a single agent-id aggregator, without replacing or rerunning post-install readiness during the upgrade.
• Static agents assigned to the wrong subnet (#2239): v3.0.0 assigned primary-network static agents from the control-plane subnet. v3.0.1 or later restores per-nodepool addressing. Clusters first created on v3.0.0 require the release note's rolling agent migration: cordon/drain one agent, target-apply that agent's in-place private-NIC detach/reattach and k3s/RKE2 config restart, verify it is Ready on the intended subnet, uncordon it, and continue. If interface mapping does not recover, reboot so kh-rename-interface.service can verify it. Normal clusters upgraded from v2.x retain the v2 IP formula and are unaffected.

Known Error Patterns

Error	Cause	Solution
cannot sum empty list	control_plane_nodepools is empty or all counts are 0	Add at least one control plane with count >= 1
NAT router primary IPs will be replaced	Pre-v2.19.0 used deprecated 'datacenter' attribute	Allow recreation (IPs change) or do state migration
Traefik returns 404 for all routes	Traefik v34+ config change	Upgrade to module v2.19.0+
SSH connection refused or timeout	Key format, firewall, or node not ready	Check ssh_public_key format, verify firewall_ssh_source
Node stuck in NotReady	Network region mismatch or token issues	Ensure network_region contains all node locations
Error creating network subnet	Subnet CIDR conflicts	Check network_ipv4_cidr doesn't overlap with existing
cloud-init failed	Leap Micro/MicroOS snapshot missing, wrong region, wrong architecture, or wrong distro label	Recreate snapshots with packer in the correct region/architecture and k3s/RKE2 SELinux variant
resource_already_detaching or LB network 422 during destroy	Known benign ingress-LB detach race between CCM and Terraform ownership	Run scripts/destroy.sh; it retries only this race and then prints an orphan report
Network/subnet destroy hangs with autoscaler enabled	Autoscaler-created servers are not in Terraform state and still pin the network	Wait until the control plane is dead, or scale autoscaler min_nodes = 0, then delete the orphan
SELinux avc: denied workload failures	Missing workload policy, not automatically a module bug	Follow docs/selinux.md; collect AVC evidence and try udica before upstreaming policy or disabling a pool

Debugging Workflow

1. Check Common Issues table above
2. Search GitHub issues: gh issue list --search "<error>" --state all
3. Search docs/llms.md for related variables
4. Check locals.tf for the logic
5. Provide: Root cause → Fix → Prevention
6. Link to relevant GitHub issues if found

---

Hetzner Cloud Context

Server Types (x86)

Type	vCPU	RAM	Disk	Best For
cx23	2	4GB	40GB	Minimal dev, small k3s/RKE2 control planes
cx33	4	8GB	80GB	Production control plane, moderate workers
cx43	8	16GB	160GB	Production workers
cx53	16	32GB	320GB	Heavy workloads

cx23 is the current minimum used throughout the v3 examples. The RKE2 + Leap Micro cx23 control-plane shape is live-proven in docs/v3-release-evidence.md after size-aware kubelet reservations landed.

Server Types (ARM — CAX, cost-optimized)

Type	vCPU	RAM	Disk	Best For
cax11	2	4GB	40GB	ARM dev
cax21	4	8GB	80GB	ARM workloads
cax31	8	16GB	160GB	ARM production
cax41	16	32GB	320GB	ARM heavy

Locations

Region	Locations	Network Zone
Germany	fsn1, nbg1	eu-central
Finland	hel1	eu-central
USA East	ash	us-east
USA West	hil	us-west
Singapore	sin	ap-southeast

Rule: All locations must be in the same network_region.

---

Configuration Workflows

Workflow: Creating kube.tf

1. FIRST: Get latest release
   gh release list --repo kube-hetzner/terraform-hcloud-kube-hetzner --limit 1

2. Ask clarifying questions:
   - Use case: Production / Development / Testing?
   - Kubernetes distribution: k3s (default) / RKE2?
   - HA: Single node / 3 control planes / Super-HA (multi-location)?
   - Budget: Which server types?
   - Network: Public / Private with NAT router?
   - CNI: Flannel (default) / Cilium / Calico?
   - Storage: Longhorn needed?
   - Ingress: Traefik (default) / Nginx / HAProxy?

3. Query variables.tf and docs/llms.md for relevant options

4. Generate complete config with:
   - Module source and version (latest!)
   - Required: hcloud_token, SSH keys, `kubernetes_distribution` only when not default k3s
   - Requested features
   - Helpful comments

5. Validate syntax:
   terraform fmt -recursive
   terraform validate

Workflow: Debugging

1. Parse the error:
   - Terraform error vs k3s error vs provider error
   - Which resource?
   - What operation?

2. Check Common Issues Catalog (above)

3. Search GitHub:
   gh issue list --search "<error keyword>" --state all

4. Read relevant code:
   - locals.tf for logic
   - variables.tf for options
   - Specific .tf files based on error

5. Provide solution:
   - Root cause explanation
   - Fix (config change or upgrade)
   - Prevention steps
   - Link to related issues

For K3s certificate expiry, remind the operator that the admin kubeconfig also contains client certificates. After restarting or rotating control-plane certificates, retrieve a fresh /etc/rancher/k3s/k3s.yaml over SSH from a healthy control-plane node, save it with mode 0600, and replace its https://127.0.0.1:6443 server with the cluster's reachable API endpoint before testing it with kubectl. Do not tell operators to keep using a stale local kubeconfig after certificate recovery.

Workflow: Teardown / Destroy

1. Run from the user's Terraform root:
   <module-checkout>/scripts/destroy.sh -auto-approve
2. Let the wrapper detect the initialized Terraform/OpenTofu engine. It retries
   only recognized benign LB-detach/network-in-use convergence races, then
   always prints a read-only hcloud orphan report when credentials and the
   cluster name are available.
3. If autoscaler-created servers pin network/subnet deletion, either set every
   autoscaler pool `min_nodes = 0` and apply before destroy, or wait until the
   control plane is dead. Then use the orphan report to identify and delete only
   the autoscaler-created servers, and rerun `scripts/destroy.sh`.
4. Use <module-checkout>/scripts/cleanup.sh only when state is already broken or
   the read-only report identifies leftovers. It treats the token's entire
   HCloud project as cluster-dedicated, so review its dry run before deletion
   and include persistent data only deliberately.

Workflow: Feature Questions

1. Check docs/llms.md FIRST (primary reference)
2. Verify in variables.tf (exact syntax)
3. Check kube.tf.example for usage
4. Search GitHub discussions for examples
5. Provide answer with file references

Workflow: Upgrades

1. Get current and target versions
2. If this is v2.x -> v3.x, use the /migrate-v2-to-v3 skill workflow
3. Read CHANGELOG.md, MIGRATION.md, and docs/v2-to-v3-migration.md
4. Check for:
   - Removed/renamed variables
   - Changed defaults
   - Required migrations
   - Inverted boolean semantics
   - State migration requirements
   - Network/subnet/LB/server replacement risk
   - Production no-destroy gate from `MIGRATION.md`
5. Generate upgrade steps:
   - Update version in kube.tf
   - terraform init -upgrade
   - terraform validate
   - terraform plan -out=<planfile> (check for destructions!)
   - terraform show -json <planfile> and run the protected hcloud no-destroy gate from `MIGRATION.md`
   - terraform apply
6. Warn if terraform plan shows resource recreation

Workflow: v2 -> v3 Migrations

Use .claude/skills/migrate-v2-to-v3/SKILL.md for the exact workflow.

Core rules:

• Back up state before editing.
• Rewrite v2-only inputs using MIGRATION.md.
• Invert positive/negative booleans carefully.
• Preserve the first-apply compatibility freeze unless the operator deliberately chooses a topology/runtime change: k3s_channel = "stable" is the v3 default, but v2 upgrades usually pin k3s_channel = "v1.33" or k3s_version; unset addon versions become deterministic reviewed defaults; network_subnet_mode stays per_nodepool; node_transport_mode stays hetzner_private.
• Remove network_id = 0; omitted/null means the primary Network in v3.
• Remove control-plane network_id; control planes stay on the primary Network.
• For secure Tailnet access or private multinetwork scale, prefer node_transport_mode = "tailscale". For v2-to-v3 upgrades, introduce large multinetwork scale in a separate audited plan after the base upgrade. Tailscale mode keeps Kubernetes node IPs on Hetzner private addresses and can advertise node-private /32 routes with Tailscale subnet-route SNAT disabled. Active agent/autoscaler nodepools in Tailscale mode must set network_scope = "primary" or network_scope = "external" so invalid same-root external Network configs fail at plan time.
• Do not suggest Calico with Tailscale node transport yet. Flannel is first supported; Cilium is still explicitly experimental in this transport mode.
• For Cloudflare, recommend only the external Access/Tunnel pattern for kube API, SSH, Rancher, Grafana, or ingress. Do not suggest Cloudflare Mesh/WARP as kube-hetzner node transport and do not invent Cloudflare provider inputs.
• Run terraform fmt -recursive, terraform init -upgrade, terraform validate, and terraform plan -out=v3-upgrade.tfplan.
• Run the protected hcloud no-destroy gate from MIGRATION.md; it includes hcloud_placement_group and hcloud_firewall.
• Do not apply when the plan has unexplained replacements or destroys.

---

Configuration Templates

Minimal Development (Single Node)

module "kube-hetzner" {
  source  = "kube-hetzner/kube-hetzner/hcloud"
  version = "<LATEST>"  # Always fetch latest!

  hcloud_token = var.hcloud_token

  ssh_public_key  = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  network_region = "eu-central"

  control_plane_nodepools = [
    {
      name        = "control-plane"
      server_type = "cx23"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 1
    }
  ]

  agent_nodepools = [
    {
      name        = "worker"
      server_type = "cx23"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 0
    }
  ]

  # Single node: disable auto OS upgrades
  automatically_upgrade_os = false
}

Production HA (3 Control Planes + Workers)

module "kube-hetzner" {
  source  = "kube-hetzner/kube-hetzner/hcloud"
  version = "<LATEST>"

  hcloud_token = var.hcloud_token

  ssh_public_key  = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  network_region = "eu-central"

  control_plane_nodepools = [
    {
      name        = "control-plane"
      server_type = "cx33"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3  # Odd number for quorum!
    }
  ]

  agent_nodepools = [
    {
      name        = "worker"
      server_type = "cx43"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
    }
  ]

  enable_longhorn = true

  # Security: restrict access to your IP
  firewall_kube_api_source = ["YOUR_IP/32"]
  firewall_ssh_source      = ["YOUR_IP/32"]
}

Private Cluster with NAT Router

module "kube-hetzner" {
  source  = "kube-hetzner/kube-hetzner/hcloud"
  version = "<LATEST>"

  hcloud_token = var.hcloud_token

  ssh_public_key  = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  network_region = "eu-central"

  enable_control_plane_load_balancer = true

  nat_router = {
    server_type = "cax21"
    location    = "nbg1"
  }

  control_plane_nodepools = [
    {
      name        = "control-plane"
      server_type = "cx33"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
      # Disable public IPs
      enable_public_ipv4 = false
      enable_public_ipv6 = false
    }
  ]

  agent_nodepools = [
    {
      name        = "worker"
      server_type = "cx43"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
      enable_public_ipv4 = false
      enable_public_ipv6 = false
    }
  ]

  # Optional: keep control plane LB private too
  control_plane_load_balancer_enable_public_network = false
}

Tailscale Node Transport

Use this when a user wants a single-network cluster with private Tailnet Terraform/kubeconfig/SSH access, or a private cluster spanning multiple Hetzner Cloud Networks. Do not use the older external-overlay pattern for Kubernetes node transport.

node_transport_mode = "tailscale"

firewall_kube_api_source = null
firewall_ssh_source      = null

tailscale_auth_key = var.tailscale_auth_key
# tailscale_autoscaler_auth_key = var.tailscale_autoscaler_auth_key # Prefer ephemeral reusable key for autoscaler.

tailscale_node_transport = {
  bootstrap_mode  = "cloud_init"
  magicdns_domain = "example-tailnet.ts.net"
  auth = {
    mode = "auth_key"
  }
  routing = {
    # Single-network clusters may set false; network_scope = "external" nodepools need true.
    advertise_node_private_routes = false
  }
}

Rules to mention:

• Tailscale mode requires explicit network_scope = "primary" or "external" on every active agent/autoscaler nodepool. Use "primary" when network_id is omitted/null; use "external" with external network_id, including same-root hcloud_network.*.id.
• Tailscale node transport is rejected with cluster_ipv6_cidr / service_ipv6_cidr in this release; keep IPv6 pod/service CIDRs unset on Tailscale clusters.
• Tailnet ACLs must auto-approve advertised Hetzner node-private /32 routes when external network_scope nodepools are used.
• The module disables Tailscale subnet-route SNAT for node/CNI traffic.
• Flannel VXLAN is first supported; Cilium needs the experimental flag; Calico is rejected.
• Managed Hetzner private LBs are fine for single-primary-network clusters; external network_scope nodepools need public LB targets or non-Hetzner/private alternatives.
• The module NAT router only gives egress to the primary Hetzner Network; external-network Tailscale nodepools need public egress or an external bootstrap path.
• Large examples live in examples/tailscale-node-transport/large-scale-200.tf.example and examples/tailscale-node-transport/massive-10000-nodes.tf.example.
• The 200-node static example is 3 control planes + 97 primary agents + 100 agents on one external Network; both Networks are exactly at Hetzner's 100-attachment limit and placement groups auto-shard to 21 groups.
• The 10,000-total-node reference is 3 control planes + 7 static system agents + 90 primary autoscaled workers + 99 external Networks * 100 autoscaled workers. It is a quota/design reference, not a casual default.
• The recommended large-cluster exposure model closes public Kubernetes API and SSH, uses no public managed web ingress unless explicitly requested, and relies on Tailnet access. Nodes may still keep public IPv4/IPv6 for Tailscale bootstrap and direct UDP/41641 WireGuard paths; true no-public-IP multinetwork needs private egress plus external Tailscale bootstrap for every external Network.

Cloudflare Zero Trust External Access

Use this when a user wants Cloudflare policy in front of operator or human-facing endpoints. Do not present Cloudflare as a kube-hetzner-managed node transport.

Rules:

• Cloudflare Access/Tunnel can protect kube API, SSH, Rancher, Grafana, or ingress hostnames.
• Cloudflare account resources, DNS records, tunnels, Access policies, WARP enrollment, and service tokens are managed outside kube-hetzner.
• No node_transport_mode = "cloudflare" exists, and Cloudflare Mesh/WARP is not supported node transport in v3.
• For kubeconfig through Access, suggest cloudflared access tcp or user-owned WARP/private routing.
• Do not set control_plane_endpoint to a Cloudflare Access hostname unless every joining node can reach and authenticate to it.
• For secure node transport or +100 node multinetwork, use node_transport_mode = "tailscale".

Reference docs:

• examples/external-overlay-cloudflare-access/README.md
• docs/v3-topology-recommendations.md

Cilium with Hubble Observability

module "kube-hetzner" {
  source  = "kube-hetzner/kube-hetzner/hcloud"
  version = "<LATEST>"

  hcloud_token = var.hcloud_token

  ssh_public_key  = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  network_region = "eu-central"

  # Use Cilium CNI
  cni_plugin = "cilium"

  # Full kube-proxy replacement
  enable_kube_proxy = false

  # Enable Hubble for observability
  cilium_hubble_enabled = true

  control_plane_nodepools = [
    {
      name        = "control-plane"
      server_type = "cx33"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
    }
  ]

  agent_nodepools = [
    {
      name        = "worker"
      server_type = "cx43"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
    }
  ]
}

Cost-Optimized ARM Cluster

module "kube-hetzner" {
  source  = "kube-hetzner/kube-hetzner/hcloud"
  version = "<LATEST>"

  hcloud_token = var.hcloud_token

  ssh_public_key  = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  network_region = "eu-central"

  # ARM servers (CAX) are ~40% cheaper
  control_plane_nodepools = [
    {
      name        = "control-plane"
      server_type = "cax21"  # ARM
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
    }
  ]

  agent_nodepools = [
    {
      name        = "worker-arm"
      server_type = "cax31"  # ARM
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 3
    }
  ]
}

Super-HA Multi-Location

module "kube-hetzner" {
  source  = "kube-hetzner/kube-hetzner/hcloud"
  version = "<LATEST>"

  hcloud_token = var.hcloud_token

  ssh_public_key  = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  # Must cover ALL locations used
  network_region = "eu-central"

  # Spread control planes across locations
  control_plane_nodepools = [
    {
      name        = "cp-fsn"
      server_type = "cx33"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 1
    },
    {
      name        = "cp-nbg"
      server_type = "cx33"
      location    = "nbg1"
      labels      = []
      taints      = []
      count       = 1
    },
    {
      name        = "cp-hel"
      server_type = "cx33"
      location    = "hel1"
      labels      = []
      taints      = []
      count       = 1
    }
  ]

  # Spread workers too
  agent_nodepools = [
    {
      name        = "worker-fsn"
      server_type = "cx43"
      location    = "fsn1"
      labels      = []
      taints      = []
      count       = 2
    },
    {
      name        = "worker-nbg"
      server_type = "cx43"
      location    = "nbg1"
      labels      = []
      taints      = []
      count       = 2
    },
    {
      name        = "worker-hel"
      server_type = "cx43"
      location    = "hel1"
      labels      = []
      taints      = []
      count       = 2
    }
  ]

  enable_longhorn = true
}

---

Quick Reference

Variable Lookup

# Find specific variable
rg -n 'variable "<name>"' variables.tf

# Search by keyword
rg -n -C 3 'description.*<keyword>' variables.tf

# Inspect the complete variable section in docs/llms.md after locating it

GitHub Commands

# Latest release
gh release list --repo kube-hetzner/terraform-hcloud-kube-hetzner --limit 1

# Search issues
gh issue list --repo kube-hetzner/terraform-hcloud-kube-hetzner --search "<query>" --state all

# View specific issue
gh issue view <number> --repo kube-hetzner/terraform-hcloud-kube-hetzner --comments

# Search discussions
gh api repos/kube-hetzner/terraform-hcloud-kube-hetzner/discussions --jq '.[].title'

Validation

terraform fmt -recursive
terraform validate
terraform plan  # Check for unexpected changes!