argo-knowledge

# Argo Knowledge Base ## Rules 1. **Always use correct apiVersion and kind.** Every Argo CRD uses `apiVersion: argoproj.io/v1alpha1`. Never invent CRDs or apiVersions that do not exist. 2. **Load references on-demand.** Only read reference files when the user's question requires detailed field-level knowledge. Do not preload all references. 3. **Max 1-2 reference files per question.** Keep context focused. If a question spans more than two reference areas, answer the primary question first and offer to elaborate. 4. **Validate YAML before presenting.** Every YAML example must be syntactically valid and use real field names from the Argo CRD schemas. 5. **Prefer canonical patterns.** Use the numbered patterns below as starting points. Adapt to the user's specifics rather than inventing from scratch. 6. **State trade-offs.** When recommending an approach, briefly note what you give up. ## What is Argo The Argo project is a set of Kubernetes-native tools for running and managing jobs and applications on Kubernetes. - **Argo CD** — Declarative GitOps continuous delivery. Watches Git repos and syncs Kubernetes resources to match the desired state. Supports Helm, Kustomize, plain YAML, Jsonnet, and plugin-based config management tools. Provides a UI, CLI, and API for managing applications across multiple clusters. - **Argo Rollouts** — Progressive delivery controller. Extends Kubernetes Deployments with canary releases, blue-green deployments, experimentation, and automated analysis-driven promotion/rollback. Integrates with service meshes and ingress controllers for traffic management. - **Argo Workflows** — Container-native workflow engine for Kubernetes. Runs DAG and step-based workflows where each step is a container. Used for CI/CD pipelines, data processing, ML pipelines, and infrastructure automation. - **Argo Events** — Event-driven workflow automation. Connects external event sources (webhooks, message queues, cloud events, cron schedules) to triggers that create Argo Workflows or any Kubernetes resource. **How they relate:** Argo CD deploys your applications via GitOps. Argo Rollouts handles the progressive delivery strategy during those deployments. Argo Workflows orchestrates complex multi-step jobs. Argo Events wires external events to trigger Workflows or other actions. Together they form a complete GitOps + progressive delivery + automation platform. ## CRD Table | Kind | apiVersion | Project | Purpose | |------|-----------|---------|---------| | Application | argoproj.io/v1alpha1 | Argo CD | Defines a single application to sync from Git to a cluster | | AppProject | argoproj.io/v1alpha1 | Argo CD | RBAC boundary: restricts sources, destinations, and resources | | ApplicationSet | argoproj.io/v1alpha1 | Argo CD | Generates multiple Applications from templates + generators | | Rollout | argoproj.io/v1alpha1 | Rollouts | Progressive delivery replacement for Deployment | | AnalysisTemplate | argoproj.io/v1alpha1 | Rollouts | Defines metric queries for automated rollout analysis | | ClusterAnalysisTemplate | argoproj.io/v1alpha1 | Rollouts | Cluster-scoped AnalysisTemplate | | AnalysisRun | argoproj.io/v1alpha1 | Rollouts | Instance of an AnalysisTemplate execution (auto-created) | | Experiment | argoproj.io/v1alpha1 | Rollouts | Runs multiple ReplicaSet versions simultaneously for comparison | | Workflow | argoproj.io/v1alpha1 | Workflows | A single workflow execution | | WorkflowTemplate | argoproj.io/v1alpha1 | Workflows | Reusable workflow definition (namespace-scoped) | | ClusterWorkflowTemplate | argoproj.io/v1alpha1 | Workflows | Reusable workflow definition (cluster-scoped) | | CronWorkflow | argoproj.io/v1alpha1 | Workflows | Scheduled workflow execution | | EventSource | argoproj.io/v1alpha1 | Events | Defines external event sources to consume | | EventBus | argoproj.io/v1alpha1 | Events | Message transport layer between EventSources and Sensors | | Sensor | argoproj.io/v1alpha1 | Events | Listens to EventBus, applies filters, fires triggers | ## How Argo CD Works ### Sync Loop 1. **Desired state:** Argo CD reads manifests from a Git repo (source). The source can be Helm charts, Kustomize overlays, plain YAML directories, Jsonnet, or custom config management plugins. 2. **Live state:** Argo CD queries the target Kubernetes cluster (destination) for the current state of resources it manages. 3. **Diff:** Argo CD compares desired vs live state. Resources that differ are marked `OutOfSync`. 4. **Sync:** When triggered (manually or via automated sync policy), Argo CD applies the desired state to the cluster using `kubectl apply`, server-side apply, or `kubectl create/replace` depending on sync options. 5. **Health assessment:** After sync, Argo CD evaluates resource health using built-in health checks (Deployments, StatefulSets, Services, Ingresses, etc.) and custom health checks (Lua scripts in `argocd-cm`). Resources are marked Healthy, Progressing, Degraded, Suspended, or Missing. ### Resource Tracking Argo CD tracks which resources belong to an Application using one of three methods: - **`label`** — Adds `app.kubernetes.io/instance: <app-name>` label. Simple but can conflict with other tools using this label. - **`annotation`** — Adds `argocd.argoproj.io/tracking-id` annotation. Avoids label conflicts. Default in modern Argo CD. - **`annotation+label`** — Uses both. Maximum compatibility. Configured via `resource.trackingMethod` in `argocd-cm` ConfigMap. ### Multi-Cluster Argo CD manages applications across multiple clusters from a single control plane: - The cluster running Argo CD is the **in-cluster** target (referenced as `https://kubernetes.default.svc`). - External clusters are added via `argocd cluster add <context-name>`, which creates a ServiceAccount + ClusterRoleBinding on the target cluster and stores credentials as a Secret in the Argo CD namespace. - Cluster Secrets have label `argocd.argoproj.io/secret-type: cluster` and contain `server`, `name`, `config` (with `bearerToken`, `tlsClientConfig`). ## Decision Trees ### Application vs ApplicationSet Use **Application** when: - You have a single application or a small, fixed number of applications - Each application has unique configuration that doesn't follow a pattern - You want direct, explicit control over each application Use **ApplicationSet** when: - You need to generate many Applications from a pattern (e.g., per-directory, per-cluster, per-team) - Applications share a common template with parameterized differences - You want Applications auto-created/deleted when directories or clusters appear/disappear - You need progressive rollout across environments (rollingSync) ### Rollout vs Deployment Use **Deployment** when: - You want simple rolling updates - You don't need traffic shaping, analysis, or manual promotion gates - The application is non-critical or internal-only Use **Rollout** when: - You need canary or blue-green deployment strategies - You want automated analysis (metrics-based promotion/rollback) - You need traffic percentage control via service mesh or ingress - You need manual approval gates between rollout steps - You want experiment-based A/B testing ### Which ApplicationSet Generator - **Git directory** — One Application per directory in a monorepo. Best for: environments or services organized as directories. - **Git file** — One Application per JSON/YAML config file in a repo. Best for: externalized app configs with arbitrary fields. - **List** — Explicitly enumerated Applications. Best for: small, fixed sets with no dynamic discovery. - **Cluster** — One Application per registered Argo CD cluster. Best for: deploying the same app to all clusters matching a selector. - **Matrix** — Cartesian product of two generators. Best for: deploy N apps to M clusters. - **Merge** — Combine generator outputs, overriding fields from a secondary generator. Best for: cluster-specific overrides on top of a base config. - **Pull request** — One Application per open PR. Best for: PR preview environments. - **SCM provider** — One Application per repo in a GitHub org or GitLab group. Best for: auto-onboarding repos. ## Canonical YAML Patterns ### Pattern 1: Application with Helm Source ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd finalizers: - resources-finalizer.argocd.argoproj.io spec: project: default source: repoURL: https://charts.example.com chart: my-app targetRevision: 1.2.3 helm: releaseName: my-app valuesObject: replicaCount: 3 image: repository: registry.example.com/my-app tag: latest ingress: enabled: true hosts: - my-app.example.com parameters: - name: service.type value: ClusterIP destination: server: https://kubernetes.default.svc namespace: my-app syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - ServerSideApply=true retry: limit: 5 backoff: duration: 5s factor: 2 maxDuration: 3m ``` ### Pattern 2: Application with Kustomize Source ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-prod namespace: argocd spec: project: default source: repoURL: https://github.com/org/k8s-configs.git targetRevision: main path: apps/my-app/overlays/production kustomize: namePrefix: prod- commonLabels: environment: production images: - registry.example.com/my-app:v2.1.0 destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true ``` ### Pattern 3: ApplicationSet with Git Directory Generator ```yaml apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: cluster-addons namespace: argocd spec: goTemplate: true goTemplateOptions: - missingkey=error generators: - git: repoURL: https://github.com/org/cluster-addons.git revision: main directories: - path: addons/* - path: addons/experimental-* exclude: true preserveResourcesOnDeletion: true template: metadata: name: 'addon-{{.path.basename}}' labels: envLabel: staging spec: project: cluster-addons source: repoURL: https://github.com/org/cluster-addons.git targetRevision: main path: '{{.path.path}}' destination: server: https://kubernetes.default.svc namespace: '{{.path.basename}}' syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true strategy: type: RollingSync rollingSync: steps: - matchExpressions: - key: envLabel operator: In values: - staging maxUpdate: 100% - matchExpressions: - key: envLabel operator: In values: - production maxUpdate: 25% ``` ### Pattern 4: ApplicationSet with Cluster Generator + Matrix ```yaml apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: platform-services namespace: argocd spec: goTemplate: true goTemplateOptions: - missingkey=error preserveResourcesOnDeletion: true generators: - matrix: generators: - clusters: selector: matchLabels: tier: production values: revision: main - list: elements: - app: ingress-nginx namespace: ingress-nginx path: platform/ingress-nginx - app: cert-manager namespace: cert-manager path: platform/cert-manager - app: monitoring namespace: monitoring path: platform/monitoring template: metadata: name: '{{.name}}-{{.app}}' spec: project: platform source: repoURL: https://github.com/org/platform-config.git targetRevision: '{{.values.revision}}' path: '{{.path}}' destination: server: '{{.server}}' namespace: '{{.namespace}}' syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - ServerSideApply=true ``` ### Pattern 5: Rollout with Canary Strategy + AnalysisTemplate ```yaml apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: my-app namespace: my-app spec: replicas: 5 revisionHistoryLimit: 3 selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: my-app image: registry.example.com/my-app:v2.0.0 ports: - containerPort: 8080 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 512Mi strategy: canary: maxSurge: 1 maxUnavailable: 0 steps: - setWeight: 10 - pause: { duration: 2m } - setWeight: 30 - analysis: templates: - templateName: success-rate args: - name: service-name value: my-app - setWeight: 60 - pause: { duration: 5m } - setWeight: 100 canaryService: my-app-canary stableService: my-app-stable trafficRouting: istio: virtualServices: - name: my-app-vsvc routes: - primary --- apiVersion: argoproj.io/v1alpha1 kind: AnalysisTemplate metadata: name: success-rate namespace: my-app spec: args: - name: service-name metrics: - name: success-rate interval: 60s count: 5 successCondition: result[0] >= 0.95 failureLimit: 2 provider: prometheus: address: http://prometheus.monitoring:9090 query: | sum(rate(http_requests_total{service="{{args.service-name}}", status=~"2.."}[5m])) / sum(rate(http_requests_total{service="{{args.service-name}}"}[5m])) ``` ### Pattern 6: Rollout with Blue-Green Strategy ```yaml apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: my-app namespace: my-app spec: replicas: 3 revisionHistoryLimit: 3 selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: my-app image: registry.example.com/my-app:v2.0.0 ports: - containerPort: 8080 strategy: blueGreen: activeService: my-app-active previewService: my-app-preview autoPromotionEnabled: false scaleDownDelaySeconds: 30 prePromotionAnalysis: templates: - templateName: smoke-test args: - name: preview-url value: http://my-app-preview.my-app.svc.cluster.local postPromotionAnalysis: templates: - templateName: success-rate args: - name: service-name value: my-app ``` ### Pattern 7: Workflow DAG Pattern ```yaml apiVersion: argoproj.io/v1alpha1 kind: Workflow metadata: generateName: ci-pipeline- namespace: argo spec: entrypoint: ci-pipeline serviceAccountName: argo-workflow arguments: parameters: - name: repo-url value: https://github.com/org/my-app.git - name: revision value: main - name: image value: registry.example.com/my-app templates: - name: ci-pipeline dag: tasks: - name: checkout template: git-clone arguments: parameters: - name: repo-url value: '{{workflow.parameters.repo-url}}' - name: revision value: '{{workflow.parameters.revision}}' - name: unit-test template: run-tests dependencies: - checkout - name: lint template: run-lint dependencies: - checkout - name: build-image template: build-push dependencies: - unit-test - lint arguments: parameters: - name: image value: '{{workflow.parameters.image}}' - name: git-clone inputs: parameters: - name: repo-url - name: revision container: image: alpine/git:latest command: [sh, -c] args: - git clone --branch {{inputs.parameters.revision}} {{inputs.parameters.repo-url}} /work volumeMounts: - name: work mountPath: /work - name: run-tests container: image: golang:1.22