| name | agent-deployment-pipeline |
| description | Implement CI/CD pipelines for AI agent deployment with evaluation gates. Use for GitHub Actions workflows, GitOps with ArgoCD, container image building, and automated testing. Triggers on "CI/CD", "pipeline", "GitHub Actions", "GitOps", "ArgoCD", "deployment automation", "continuous deployment", or when implementing safe agent release workflows. |
Agent Deployment Pipeline
Overview
Implement CI/CD pipelines for AgentStack agents with evaluation-based safety gates, GitOps workflows, and progressive rollouts.
Pipeline Architecture
┌─────────────────────────────────────────────────────────────────┐
│ Deployment Pipeline │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Push │──▶│ Build │──▶│ Test │──▶│ Eval │──▶ │
│ │ │ │ Image │ │ Unit │ │ MLflow │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ Safety Gate │ │
│ │ Pass? │ │
│ └─────────────┘ │
│ │ │ │
│ Yes ───┘ └─── No │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Deploy │ │ Block │ │
│ │ Staging │ │ + Alert │ │
│ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ Canary │ │
│ │ Rollout │ │
│ └─────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ Production │ │
│ │ 100% │ │
│ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
GitHub Actions Workflow
Main Pipeline
name: Agent Deployment Pipeline
on:
push:
branches: [main]
paths:
- 'agents/**'
pull_request:
branches: [main]
paths:
- 'agents/**'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
detect-changes:
runs-on: ubuntu-latest
outputs:
agents: ${{ steps.changes.outputs.agents }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: changes
with:
filters: |
agents:
- 'agents/**'
list-files: json
build:
needs: detect-changes
if: needs.detect-changes.outputs.agents !=
[, ]
[]
[]
[]
GitOps with ArgoCD
Application Manifest
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: agentstack-agents
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: agentstack
source:
repoURL: https://github.com/org/agentstack-config.git
targetRevision: HEAD
path: environments/production/agents
destination:
server: https://kubernetes.default.svc
namespace: agentstack
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ApplyOutOfSyncOnly=true
retry:
limit: 5
backoff:
duration: 5s
maxDuration: 3m
factor: 2
Progressive Rollout with Argo Rollouts
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: customer-support-agent
namespace: agentstack
spec:
replicas: 5
revisionHistoryLimit: 3
selector:
matchLabels:
app: customer-support-agent
template:
metadata:
labels:
app: customer-support-agent
spec:
containers:
- name: agent
image: ghcr.io/org/agents/customer-support:latest
ports:
- containerPort: 8080
strategy:
canary:
steps:
- setWeight: 10
- pause: {duration: 5m}
- analysis:
templates:
- templateName: agent-success-rate
{ }
{ }
Analysis Template
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: agent-success-rate
namespace: agentstack
spec:
args:
- name: agent-name
metrics:
- name: success-rate
interval: 1m
count: 5
successCondition: result[0] >= 0.95
failureLimit: 2
provider:
prometheus:
address: http://prometheus:9090
query: |
sum(rate(http_requests_total{
app="{{args.agent-name}}",
status=~"2.."
}[5m]))
/
sum(rate(http_requests_total{
app="{{args.agent-name}}"
}[5m]))
- name: error-rate
interval: 1m
count: 5
successCondition: result[0] <= 0.01
failureLimit: 2
provider:
prometheus:
Evaluation Script
import argparse
import json
import subprocess
import mlflow
from typing import Dict, Any
def run_evaluation(
agent_image: str,
dataset_path: str,
output_file: str
) -> Dict[str, Any]:
"""Run MLflow evaluation against agent container."""
container_id = subprocess.check_output([
"docker", "run", "-d", "-p", "8080:8080",
agent_image
]).decode().strip()
try:
wait_for_health("http://localhost:8080/health")
dataset = load_dataset(dataset_path)
with mlflow.start_run():
results = mlflow.evaluate(
model=agent_model_wrapper,
data=dataset,
model_type="databricks-agent",
evaluators="default",
evaluator_config={
"databricks-agent": {
"metrics": [
"safety",
"groundedness",
"relevance",
"chunk_relevance"
]
}
}
)
scores = {
: results.metrics.get(, ),
: calculate_quality_score(results),
: results.metrics.get(, ),
}
mlflow.log_metrics(scores)
output = {
: scores,
: results.to_dict(),
: ([
scores[] >= ,
scores[] >=
])
}
(output_file, ) f:
json.dump(output, f, indent=)
output
:
subprocess.run([, , container_id])
subprocess.run([, , container_id])
__name__ == :
parser = argparse.ArgumentParser()
parser.add_argument(, required=)
parser.add_argument(, required=)
parser.add_argument(, default=)
args = parser.parse_args()
run_evaluation(args.agent_image, args.dataset, args.output_file)
Notification Integration
name: Deployment Notifications
on:
workflow_run:
workflows: ["Agent Deployment Pipeline"]
types: [completed]
jobs:
notify:
runs-on: ubuntu-latest
steps:
- name: Slack notification
uses: slackapi/slack-github-action@v1
with:
channel-id: 'deployments'
payload: |
{
"text": "${{ github.workflow }} ${{ github.event.workflow_run.conclusion }}",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*${{ github.repository }}* deployment ${{ github.event.workflow_run.conclusion }}"
}
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Commit:*\n<${{ github.event.workflow_run.head_commit.url }}|${{ github.event.workflow_run.head_sha }}>"
},
{
"type": "mrkdwn",
"text": "*Author:*\n${{ github.event.workflow_run.head_commit.author.name }}"
}
]
}
]
}
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
Rollback Procedure
name: Emergency Rollback
on:
workflow_dispatch:
inputs:
agent:
description: 'Agent to rollback'
required: true
revision:
description: 'Revision to rollback to (leave empty for previous)'
required: false
jobs:
rollback:
runs-on: ubuntu-latest
environment: production
steps:
- name: Configure kubectl
uses: azure/setup-kubectl@v3
- name: Rollback deployment
run: |
if [ -z "${{ github.event.inputs.revision }}" ]; then
kubectl rollout undo deployment/${{ github.event.inputs.agent }} \
-n agentstack
else
kubectl rollout undo deployment/${{ github.event.inputs.agent }} \
--to-revision=${{ github.event.inputs.revision }} \
-n agentstack
fi
- name: Wait for rollback
run: |
kubectl rollout status deployment/${{ github.event.inputs.agent }} \
-n agentstack --timeout=5m
- name: Verify
Resources
references/github-actions-patterns.md - Advanced GHA patternsreferences/argocd-patterns.md - ArgoCD best practices
