| name | fix-flux |
| description | Diagnose and repair a stalled Flux reconciliation — a HelmRelease stuck or not progressing, a Kustomization not reconciling, an OCIRepository not resolving, or an ExternalSecret not syncing. Use for "flux is broken", "HelmRelease stuck", "kustomization not reconciling", "ExternalSecret not syncing", or when flux get shows a resource not ready. Scoped to the Artemis cluster. |
Skill: Fix Flux
Diagnose and fix Flux reconciliation issues in Artemis-Cluster.
Read .agents/references/flux-patterns.md before diagnosing — it covers stuck HelmRelease fixes, cross-namespace gotchas, and the CRD timing race.
Step 1 — Identify What's Broken
flux get all -A --status-selector ready=false
kubectl get helmrelease -A | grep -v "True"
kubectl get kustomization -A | grep -v "True"
Step 2 — Diagnose by Symptom
Stuck HelmRelease (install/upgrade retrying)
flux describe helmrelease <app> -n <namespace>
kubectl get events -n <namespace> --sort-by='.lastTimestamp' | tail -20
Fix:
flux suspend helmrelease <app> -n <namespace>
kubectl delete secret -n <namespace> -l name=<app>,owner=helm
flux resume helmrelease <app> -n <namespace>
Kustomization Not Reconciling
Look up the exact kustomization name first — it is the metadata.name in ks.yaml, not always <namespace>-<app>:
grep "^ name:" kubernetes/apps/<namespace>/<app>/ks.yaml
flux describe kustomization <ks-name> -n flux-system
Common causes:
dependsOn target not Ready — check dependency chain- Schema validation error — run
just kube render-local-ks <namespace> <ks-name> to catch it locally
- Git source not synced —
flux reconcile source git flux-system
ExternalSecret Not Syncing / Empty Secret
kubectl describe externalsecret <app> -n <namespace>
just kube sync es
kubectl get secret <app> -n <namespace> -o yaml
Common causes:
- 1Password field name mismatch — exact field name must match template
{{ .FIELD_NAME }}
- onepassword-connect pod unhealthy —
kubectl get pods -n external-secrets
Pod Stuck in ContainerCreating (RBD CSI)
kubectl describe pod <pod> -n <namespace> | grep -A5 Events
If you see rpc error or volume attachment:
kubectl delete pod -n rook-ceph -l app=csi-rbdplugin --field-selector spec.nodeName=<node>
kubectl get volumeattachment | grep <node>
kubectl delete volumeattachment <name>
talosctl reboot -n <node-ip> --wait
CRD Timing Race (Kustomization fails immediately after new CRDs deployed)
Symptom: kustomization reconciles but resources fail with "no matches for kind X" right after a new CRD-bearing app was added.
Fix — manually re-trigger after CRDs are established:
flux reconcile kustomization <ks-name> -n flux-system
If it's a dependency ordering issue, check dependsOn in ks.yaml — the CRD provider must be listed.
Image Pull Errors
kubectl describe pod <pod> -n <namespace> | grep -A3 "Failed"
Verify image tag exists in registry. If rate limited: wait or check imagePullSecret.
HelmRelease Values Not Applying After Git Change
flux reconcile helmrelease <app> -n <namespace> --with-source
Step 3 — Force Full Reconciliation
Look up the kustomization name before running:
grep "^ name:" kubernetes/apps/<namespace>/<app>/ks.yaml
just kube sync ocirepo
flux reconcile kustomization <ks-name> -n flux-system --with-source
flux reconcile helmrelease <app> -n <namespace>
Step 4 — Verify
flux get helmrelease <app> -n <namespace>
kubectl get pods -n <namespace> -l app.kubernetes.io/name=<app>
kubectl logs -n <namespace> deployment/<app> --tail=20
Nuclear Option — Full Namespace Reconcile
Only if multiple apps in a namespace are broken:
flux reconcile kustomization <namespace> -n flux-system --with-source
