agent-deployment-pipeline

name	agent-deployment-pipeline
description	Implement CI/CD pipelines for AI agent deployment with evaluation gates. Use for GitHub Actions workflows, GitOps with ArgoCD, container image building, and automated testing. Triggers on "CI/CD", "pipeline", "GitHub Actions", "GitOps", "ArgoCD", "deployment automation", "continuous deployment", or when implementing safe agent release workflows.

Agent Deployment Pipeline

Overview

Implement CI/CD pipelines for AgentStack agents with evaluation-based safety gates, GitOps workflows, and progressive rollouts.

Pipeline Architecture

┌─────────────────────────────────────────────────────────────────┐
│                   Deployment Pipeline                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐         │
│  │  Push   │──▶│  Build  │──▶│  Test   │──▶│  Eval   │──▶      │
│  │         │   │  Image  │   │  Unit   │   │  MLflow │         │
│  └─────────┘   └─────────┘   └─────────┘   └─────────┘         │
│                                                │                │
│                                                ▼                │
│                                        ┌─────────────┐         │
│                                        │ Safety Gate │         │
│                                        │   Pass?     │         │
│                                        └─────────────┘         │
│                                           │    │               │
│                                    Yes ───┘    └─── No         │
│                                    │                │          │
│                                    ▼                ▼          │
│                            ┌─────────────┐  ┌─────────────┐    │
│                            │   Deploy    │  │   Block     │    │
│                            │   Staging   │  │   + Alert   │    │
│                            └─────────────┘  └─────────────┘    │
│                                    │                           │
│                                    ▼                           │
│                            ┌─────────────┐                     │
│                            │  Canary     │                     │
│                            │  Rollout    │                     │
│                            └─────────────┘                     │
│                                    │                           │
│                                    ▼                           │
│                            ┌─────────────┐                     │
│                            │ Production  │                     │
│                            │   100%      │                     │
│                            └─────────────┘                     │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

GitHub Actions Workflow

Main Pipeline

# .github/workflows/agent-deploy.yaml
name: Agent Deployment Pipeline

on:
  push:
    branches: [main]
    paths:
      - 'agents/**'
  pull_request:
    branches: [main]
    paths:
      - 'agents/**'

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      agents: ${{ steps.changes.outputs.agents }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: changes
        with:
          filters: |
            agents:
              - 'agents/**'
          list-files: json

  build:
    needs: detect-changes
    if: needs.detect-changes.outputs.agents != 
     
    
      
           
    
        
      
           
         
      
           
         
        
             
             
             
      
         
         
         
        
                 
                 
          
         
        
             
           
             
             
           
           

  
     
     
    
      
           
    
        
      
          
         
        
           
           
      
         
               
          
               
         
         
        
           
             

  
     [, ]
     
    
      
           
    
        
      
          
         
        
           
           
      
         
            
      
         
         
        
             
             
                   
          
            
            
          
             
             
      
          
                   
                  
             
             
          
          
                  
             
             
          
          
           
      
          
         
        
             
             

  
     []
       
     
     
    
        
      
         
         
      
         
               
          
               
          
         
  
     []
     
    
        
      
          
         
  
     []
       
     
     
    
        
      
          
         
        
                 
           
           
             
           

GitOps with ArgoCD

Application Manifest

# argocd/applications/agents.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: agentstack-agents
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: agentstack
  source:
    repoURL: https://github.com/org/agentstack-config.git
    targetRevision: HEAD
    path: environments/production/agents
    
  destination:
    server: https://kubernetes.default.svc
    namespace: agentstack
  
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ApplyOutOfSyncOnly=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        maxDuration: 3m
        factor: 2

Progressive Rollout with Argo Rollouts

# agents/customer-support/rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: customer-support-agent
  namespace: agentstack
spec:
  replicas: 5
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: customer-support-agent
  template:
    metadata:
      labels:
        app: customer-support-agent
    spec:
      containers:
        - name: agent
          image: ghcr.io/org/agents/customer-support:latest
          ports:
            - containerPort: 8080
  strategy:
    canary:
      steps:
        # Start with 10% traffic
        - setWeight: 10
        - pause: {duration: 5m}
        
        # Analysis at 10%
        - analysis:
            templates:
              - templateName: agent-success-rate
            
                
                 
        
        
          
          { }
        
        
         
            
                
        
        
          
          { }
        
        
         
            
                
                
        
        
          
      
      
       
      
      
      
        
          
             
            
               

Analysis Template

# argocd/analysis-templates/agent-success-rate.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: agent-success-rate
  namespace: agentstack
spec:
  args:
    - name: agent-name
  metrics:
    - name: success-rate
      interval: 1m
      count: 5
      successCondition: result[0] >= 0.95
      failureLimit: 2
      provider:
        prometheus:
          address: http://prometheus:9090
          query: |
            sum(rate(http_requests_total{
              app="{{args.agent-name}}",
              status=~"2.."
            }[5m])) 
            / 
            sum(rate(http_requests_total{
              app="{{args.agent-name}}"
            }[5m]))
    
    - name: error-rate
      interval: 1m
      count: 5
      successCondition: result[0] <= 0.01
      failureLimit: 2
      provider:
        prometheus:
           
           

Evaluation Script

# eval/run_evaluation.py
import argparse
import json
import subprocess
import mlflow
from typing import Dict, Any

def run_evaluation(
    agent_image: str,
    dataset_path: str,
    output_file: str
) -> Dict[str, Any]:
    """Run MLflow evaluation against agent container."""
    
    # Start agent container
    container_id = subprocess.check_output([
        "docker", "run", "-d", "-p", "8080:8080",
        agent_image
    ]).decode().strip()
    
    try:
        # Wait for agent to be ready
        wait_for_health("http://localhost:8080/health")
        
        # Load evaluation dataset
        dataset = load_dataset(dataset_path)
        
        # Run MLflow evaluation
        with mlflow.start_run():
            results = mlflow.evaluate(
                model=agent_model_wrapper,
                data=dataset,
                model_type="databricks-agent",
                evaluators="default",
                evaluator_config={
                    "databricks-agent": {
                        "metrics": [
                            "safety",
                            "groundedness", 
                            "relevance",
                            "chunk_relevance"
                        ]
                    }
                }
            )
            
            
            scores = {
                : results.metrics.get(, ),
                : calculate_quality_score(results),
                : results.metrics.get(, ),
            }
            
            
            mlflow.log_metrics(scores)
            
            
            output = {
                : scores,
                : results.to_dict(),
                : ([
                    scores[] >= ,
                    scores[] >= 
                ])
            }
            
             (output_file, )  f:
                json.dump(output, f, indent=)
            
             output
    
    :
        
        subprocess.run([, , container_id])
        subprocess.run([, , container_id])

 __name__ == :
    parser = argparse.ArgumentParser()
    parser.add_argument(, required=)
    parser.add_argument(, required=)
    parser.add_argument(, default=)
    
    args = parser.parse_args()
    run_evaluation(args.agent_image, args.dataset, args.output_file)

Notification Integration

# .github/workflows/notify.yaml
name: Deployment Notifications

on:
  workflow_run:
    workflows: ["Agent Deployment Pipeline"]
    types: [completed]

jobs:
  notify:
    runs-on: ubuntu-latest
    steps:
      - name: Slack notification
        uses: slackapi/slack-github-action@v1
        with:
          channel-id: 'deployments'
          payload: |
            {
              "text": "${{ github.workflow }} ${{ github.event.workflow_run.conclusion }}",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*${{ github.repository }}* deployment ${{ github.event.workflow_run.conclusion }}"
                  }
                },
                {
                  "type": "section",
                  "fields": [
                    {
                      "type": "mrkdwn",
                      "text": "*Commit:*\n<${{ github.event.workflow_run.head_commit.url }}|${{ github.event.workflow_run.head_sha }}>"
                    },
                    {
                      "type": "mrkdwn",
                      "text": "*Author:*\n${{ github.event.workflow_run.head_commit.author.name }}"
                    }
                  ]
                }
              ]
            }
        env:
          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}

Rollback Procedure

# .github/workflows/rollback.yaml
name: Emergency Rollback

on:
  workflow_dispatch:
    inputs:
      agent:
        description: 'Agent to rollback'
        required: true
      revision:
        description: 'Revision to rollback to (leave empty for previous)'
        required: false

jobs:
  rollback:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Configure kubectl
        uses: azure/setup-kubectl@v3
      
      - name: Rollback deployment
        run: |
          if [ -z "${{ github.event.inputs.revision }}" ]; then
            kubectl rollout undo deployment/${{ github.event.inputs.agent }} \
              -n agentstack
          else
            kubectl rollout undo deployment/${{ github.event.inputs.agent }} \
              --to-revision=${{ github.event.inputs.revision }} \
              -n agentstack
          fi
      
      - name: Wait for rollback
        run: |
          kubectl rollout status deployment/${{ github.event.inputs.agent }} \
            -n agentstack --timeout=5m
      
      - name: Verify 
         

Resources

• references/github-actions-patterns.md - Advanced GHA patterns
• references/argocd-patterns.md - ArgoCD best practices