arc-runner-manager

name	arc-runner-manager
description	Manage and troubleshoot GitHub Actions Runner Controller (ARC) runners on Kubernetes. Use when deploying runners, checking status, viewing logs, or diagnosing issues. Supports any Kubernetes cluster (microk8s, GKE, EKS, AKS). Includes safety checks for production environments.
version	1.0.0
allowed-tools	Bash(kubectl:*,helm:*,microk8s:*),Read,Edit,Write

ARC Runner Manager

Manage GitHub Actions Runner Controller (ARC) runners on any Kubernetes cluster. Covers diagnostics, deployment, configuration, and troubleshooting for production environments.

Quick Start

Check runner health:

kubectl get autoscalingrunnersets -n arc-runners -o wide
kubectl get pods -n arc-systems -l app.kubernetes.io/component=runner-scale-set-listener

View listener logs:

kubectl logs -n arc-systems deployment/arc-gha-rs-controller --tail=50

Troubleshoot connection issues: See "Troubleshooting Workflow" below.

Deploy a new runner: See "Deployment Workflow" below.

Workflows

1. Troubleshooting Workflow

Use when runners are unhealthy, failing to connect, or showing errors.

Step 1: Check runner status

kubectl get autoscalingrunnersets -n arc-runners -o wide
kubectl get pods -n arc-systems --field-selector=status.phase!=Running

Step 2: Review logs for errors

• Controller logs: kubectl logs -n arc-systems deployment/arc-gha-rs-controller --tail=100
• Listener pod logs: kubectl logs -n arc-systems <listener-pod-name> --tail=50
• Look for: auth errors, connection timeouts, version mismatches, credential issues

Step 3: Identify root cause (see references/troubleshooting-guide.md)

• Connection failures → GitHub App credentials issue
• Version mismatch → Controller/runner chart versions don't match
• Pod crashes → Resource constraints or missing secrets
• Listener not running → Helm release may not have applied resources

Step 4: Apply fix (see troubleshooting guide for remediation steps)

2. Deployment Workflow

Use when deploying a new ARC runner or reinstalling an existing one.

Prerequisites:

• GitHub App credentials (App ID, Installation ID, Private Key)
• kubectl/helm access to target cluster
• arc-systems and arc-runners namespaces created (controller helm release running)

Step 1: Verify prerequisites

kubectl get namespace arc-systems arc-runners
kubectl get deployment arc-gha-rs-controller -n arc-systems

Step 2: Create/update runner secret (if needed)

kubectl create secret generic <secret-name> -n arc-runners \
  --from-literal=github_app_id=<app-id> \
  --from-literal=github_app_installation_id=<install-id> \
  --from-literal=github_app_private_key=@<path-to-key> \
  --dry-run=client -o yaml | kubectl apply -f -

Step 3: Deploy/update runner using Helm

helm install <runner-name> \
  --namespace arc-runners \
  --set githubConfigUrl=https://github.com/<org> \
  --set githubConfigSecret=<secret-name> \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

Step 4: Verify deployment

kubectl get autoscalingrunnersets -n arc-runners
kubectl get pods -n arc-systems | grep listener

See references/deployment-guide.md for detailed steps and safe upgrade procedures.

3. Configuration Workflow

Use when adjusting runner settings, scaling, or Helm values.

View current configuration:

# Use --export to avoid exposing secret references and sensitive data
kubectl get autoscalingrunnersets <runner-name> -n arc-runners -o json | jq '{apiVersion, kind, metadata: {name, namespace}, spec: {minRunners, maxRunners, labels}}'

# For Helm values, only display non-sensitive settings
helm get values <helm-release> -n arc-runners | grep -E "minRunners|maxRunners|replicas|labels"

⚠️ WARNING: Do NOT use kubectl get ... -o yaml or helm get values without filtering—they expose secret names and references. Always extract only the fields you need.

Common configurations:

• Min/max runners: Edit spec.minRunners and spec.maxRunners
• Resource limits: Edit spec.containerMode.kubernetesModeServiceAccountName
• Runner labels: Edit spec.labels
• Machine type: Edit spec.machineGroupRepositoryNames

See references/configuration-reference.md for all options.

Apply changes:

helm upgrade <release-name> \
  --namespace arc-runners \
  --set key=value \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

Key Notes for Production

Safety First:

• Always test changes on non-production runners first
• Backup Helm values before upgrades: helm get values <release> -n arc-runners > backup.yaml
• Version mismatches between controller and runner charts cause silent failures — verify alignment before changes
• Credential leaks: Never log or expose GitHub App private keys; store in secrets only

Validation Checklist: Before deploying or modifying runners:

1. ✅ Helm release and controller versions match
2. ✅ GitHub App credentials are valid and secret is correctly mounted
3. ✅ Namespace and RBAC resources exist
4. ✅ Resource quotas allow pod creation
5. ✅ Network policies don't block listener → GitHub communication

See references/production-validation.md for detailed safety checks and validation scripts.

Common Issues:

• Pod stuck in ContainerCreating: Check resource requests/limits and node capacity
• Listener auth errors: Verify secret data format and GitHub App permissions
• Runner invisible in GitHub: Check listener logs for registration errors
• Version mismatch deletion: Ensure controller and runner chart versions align

Tool Scoping

This skill uses:

• Bash: kubectl/helm operations (readonly by default; write via explicit user request)
• Read: View manifests, Helm values, logs
• Edit/Write: Modify configurations (requires user confirmation for production changes)

All destructive operations (delete, force upgrade) require explicit user approval.

Security: Protecting Sensitive Data

When debugging or troubleshooting:

• Never request full YAML output: Commands like kubectl get ... -o yaml expose secret references
• Never request full Helm values: Use helm get values | grep to extract only safe fields
• Never request pod specs: Pod specifications may contain secret names or environment variables
• Filter all logs before displaying: Logs may contain credentials, tokens, or sensitive configuration
• Only show necessary fields: Use jq or grep to extract only the diagnostic information needed

Example safe diagnostics:

# ✓ SAFE: Shows only status
kubectl get autoscalingrunnersets -n arc-runners -o wide

# ✓ SAFE: Shows only keys, not values
kubectl get secret <name> -n arc-runners -o jsonpath='{.data}' | jq 'keys'

# ✗ UNSAFE: Exposes secret references and configuration
kubectl get autoscalingrunnersets -n arc-runners -o yaml

# ✗ UNSAFE: Exposes all environment variables and credentials
helm get values <release> -n arc-runners