ci-cd

name	ci-cd
description	Designs and implements CI/CD pipelines for automated testing, building, deployment, and security scanning across multiple platforms. Covers pipeline optimization, test integration, artifact management, and release automation.
trigger-keywords	CI/CD, pipeline, workflow, GitHub Actions, GitLab CI, Jenkins, CircleCI, Travis CI, Azure Pipelines, build, deploy, deployment, release, artifact, stage, job, runner, action, automation, continuous integration, continuous deployment, continuous delivery, container registry, docker build, image push, canary deployment, blue-green deployment, rolling update, rollback, integration tests, smoke tests, security scanning, SAST, DAST, dependency scanning, secrets management, cache optimization, parallelization, monorepo CI, matrix build, self-hosted runner, ML pipeline, model training pipeline, model deployment
allowed-tools	Read, Grep, Glob, Edit, Write, Bash

CI/CD

Overview

This skill covers the complete lifecycle of CI/CD pipeline design, implementation, and optimization across platforms including GitHub Actions, GitLab CI, Jenkins, CircleCI, and cloud-native solutions. It encompasses automated testing integration, security scanning, artifact management, deployment strategies, and specialized pipelines for ML workloads.

When to Use

• Implementing or migrating CI/CD pipelines
• Optimizing build and test execution times
• Integrating security scanning (SAST, DAST, dependency checks)
• Setting up deployment automation with rollback strategies
• Configuring test suites in CI environments
• Managing artifacts and container registries
• Implementing ML model training and deployment pipelines
• Troubleshooting pipeline failures and flakiness

Instructions

1. Analyze Requirements

• Identify build and test requirements
• Determine deployment targets and environments
• Assess security scanning needs (SAST, DAST, secrets, dependencies)
• Plan environment promotion strategy (dev → staging → production)
• Define quality gates and approval workflows
• Identify test suite composition (unit, integration, E2E)
• Determine artifact storage and retention policies

2. Design Pipeline Architecture

• Structure stages logically with clear dependencies
• Optimize for speed through parallelization and caching
• Design fail-fast strategy (lint → unit tests → integration tests → build)
• Plan secret management and secure credential handling
• Define deployment strategies (rolling, blue-green, canary)
• Architect for rollback and recovery procedures
• Design matrix builds for multi-platform support
• Plan monorepo CI strategies if applicable

3. Implement Testing Integration

• Configure unit test execution with coverage reporting
• Set up integration tests with service dependencies (databases, APIs)
• Implement E2E/smoke tests for critical user journeys
• Configure test parallelization and sharding
• Integrate test result reporting (JUnit, TAP, JSON)
• Set up flaky test detection and quarantine
• Configure performance/load testing stages
• Implement visual regression testing if applicable

4. Implement Security Scanning

• Integrate SAST (static analysis) tools (SonarQube, CodeQL, Semgrep)
• Configure DAST (dynamic analysis) for deployed environments
• Set up dependency/vulnerability scanning (Dependabot, Snyk, Trivy)
• Implement container image scanning
• Configure secrets detection (GitGuardian, TruffleHog)
• Set up license compliance checking
• Define security gate thresholds and failure policies

5. Implement Build and Artifact Management

• Configure dependency caching strategies
• Implement build output caching and layer caching (Docker)
• Set up artifact versioning and tagging
• Configure container registry integration
• Implement multi-stage builds for optimization
• Set up artifact signing and attestation
• Configure artifact retention and cleanup policies

6. Implement Deployment Automation

• Configure environment-specific deployments
• Implement deployment strategies (rolling, blue-green, canary)
• Set up health checks and readiness probes
• Configure smoke tests post-deployment
• Implement automated rollback on failure
• Set up deployment notifications (Slack, email, PagerDuty)
• Configure manual approval gates for production

7. Optimize Pipeline Performance

• Analyze pipeline execution times and bottlenecks
• Implement job parallelization for independent tasks
• Configure aggressive caching (dependencies, build outputs, Docker layers)
• Optimize test execution (parallel runners, test sharding)
• Use matrix builds efficiently
• Consider self-hosted runners for performance-critical workloads
• Implement conditional job execution (path filters, change detection)

8. Ensure Reliability and Observability

• Add retry logic for transient failures
• Implement comprehensive error handling
• Configure alerts for pipeline failures
• Set up metrics and dashboards for pipeline health
• Document runbooks and troubleshooting procedures
• Implement audit logging for deployments
• Configure SLO tracking for pipeline performance

Best Practices

Core Principles

1. Fail Fast: Run cheap, fast checks first (lint, type check, unit tests)
2. Parallelize Aggressively: Run independent jobs concurrently
3. Cache Everything: Dependencies, build outputs, Docker layers
4. Secure by Default: Secrets in vaults, least privilege, audit logs
5. Environment Parity: Keep dev/staging/prod as similar as possible
6. Immutable Artifacts: Build once, promote everywhere
7. Automated Rollback: Every deployment must be reversible
8. Idempotent Operations: Pipelines should be safely re-runnable

Testing in CI/CD

9. Test Pyramid: More unit tests, fewer integration tests, minimal E2E
10. Isolation: Tests should not depend on execution order
11. Determinism: Eliminate flaky tests or quarantine them
12. Fast Feedback: Unit tests < 5min, full suite < 15min target
13. Coverage Gates: Enforce minimum coverage thresholds
14. Service Mocking: Use test doubles for external dependencies

Security

15. Shift Left: Run security scans early in the pipeline
16. Dependency Scanning: Check for CVEs in all dependencies
17. Secrets Management: Never hardcode secrets, use secure vaults
18. Least Privilege: Minimal permissions for pipeline runners
19. Supply Chain Security: Verify and sign artifacts
20. Audit Trail: Log all deployments and access

Performance

21. Incremental Builds: Only rebuild changed components
22. Layer Caching: Optimize Dockerfile layer order
23. Dependency Locking: Pin versions for reproducibility
24. Resource Limits: Prevent resource exhaustion
25. Path Filtering: Skip jobs when irrelevant files change

Examples

Example 1: GitHub Actions Workflow

name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

env:
  NODE_VERSION: "20"
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Run linter
        run: npm run lint

  test:
     
     
    
      
         
        
           
           
        
           
         
    
        

         
         
        
             
           

         
          

         
            
        
           

         
         
        
           

  
     
     
    
       
       

    
        

           
         

            
         
        
             
             
             

         
         
         
        
               
           
          
         
        
           
           
             
             
           
           

  
     
     
       
     

    
        

          
         
        
           
                      
  
     
     
       
     

    
        

          
         
        
           
                                 
           

Example 2: GitLab CI Pipeline

stages:
  - validate
  - test
  - build
  - deploy

variables:
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

.node-base:
  image: node:20-alpine
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/

lint:
  stage: validate
  extends: .node-base
  script:
    - npm ci
    - npm run lint
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"

test:
  stage: test
  extends: .node-base
  services:
    - postgres:16
  variables:
    POSTGRES_DB: 
     
     
     
  
      
        
   
  
    
      
         
         
       


   
   
  
     
  
           
         
       
  
        
        


   
   
  
           
           
  
     
     
  
        


   
   
  
           
           
  
     
     
   
  
        

Example 3: Reusable Workflow (GitHub Actions)

# .github/workflows/reusable-deploy.yml
name: Reusable Deploy Workflow

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      image-tag:
        required: true
        type: string
    secrets:
      KUBE_CONFIG:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}

    steps:
      - uses: actions/checkout@v4

      - name: Set up kubectl
        uses: azure/setup-kubectl@v3

      - name: Configure kubeconfig
        run: |
          mkdir -p ~/.kube
          echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > ~/.kube/config

      - name: Deploy
        run: |
          kubectl set image deployment/app \
            app=${{ inputs.image-tag }} \
            -n ${{ inputs.environment }}

              
                
            

         
         

Example 4: Security Scanning Pipeline

name: Security Scanning

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  schedule:
    - cron: "0 0 * * 0" # Weekly scan

jobs:
  sast:
    name: Static Analysis (SAST)
    runs-on: ubuntu-latest
    permissions:
      security-events: write
      contents: read

    steps:
      - uses: actions/checkout@v4

      - name: Initialize CodeQL
        uses: github/codeql-action/init@v3
        with:
          languages: javascript, python

      - name: Autobuild
        uses: github/codeql-action/autobuild@v3

      - name: Perform CodeQL Analysis
        uses: github/codeql-action/analyze@v3

         
         
        
             
             
        
           
  
       
     

    
        

           
         
        
           
           
           
           
           

             
         
        
           

          
         
        
             
        
           

  
      
     

    
        
        
            

         
         
        
           
             
           

         
         
        
             
             
             
             

  
       
     
     [, ]

    
        

         
               

           
         
        
           
           
           

           
         
        
           
           
           

Example 5: Test Integration with Parallelization

name: Test Suite

on: [push, pull_request]

jobs:
  unit-tests:
    name: Unit Tests
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18, 20, 22]
        os: [ubuntu-latest, macos-latest, windows-latest]

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Run unit tests
        run:      

         
         
        
           
               

  
      
     
    
      
         [, , , ]

    
      
         
        
           
           
                 
           

      
         
                 
           

    
        

         
         
        
           
           

         
          

              
               
        
           
           

          
         
         
        
             
           

  
      
     

    
        

         
         
        
           
           

         
          

         
            

         
           

          
           

          
         
         
        
           
           

  
       
     
     [, , ]
     

    
        

           
         
        
           

          
         
          
         
        
           
           

Example 6: ML Pipeline (Model Training & Deployment)

name: ML Pipeline

on:
  push:
    branches: [main]
    paths:
      - "models/**"
      - "training/**"
      - "data/**"
  workflow_dispatch:
    inputs:
      model-version:
        description: "Model version to train"
        required: true
        type: string

jobs:
  data-validation:
    name: Validate Training Data
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: "pip"

      - name: Install dependencies
        run: |
          pip install pandas great-expectations dvc

      - name:    
         
          
          

          
            

  
       
     
     

    
        

         
         
        
           
           

         
         
         
         
         
                 
             

          
         
        
           
           

  
       
     
     

    
        

         
         
        
           
           

         
            

         
         
        
           
           

          
            

          
         
          
          

  
        
     
     
       
     

    
        

         
         
        
           
           

          
         
        
             
             
           

           
         
          
         
          
            

          
         

Example 7: Monorepo CI with Path Filtering

name: Monorepo CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  detect-changes:
    name: Detect Changed Services
    runs-on: ubuntu-latest
    outputs:
      api: ${{ steps.filter.outputs.api }}
      web: ${{ steps.filter.outputs.web }}
      worker: ${{ steps.filter.outputs.worker }}
      shared: ${{ steps.filter.outputs.shared }}

    steps:
      - uses: actions/checkout@v4

      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            api:
              - 'services/api/**'
              - 'packages/shared/**'
            web:
              - 'services/web/**'
              - 'packages/shared/**'
            worker:
              - 'services/worker/**'
              - 'packages/shared/**'
            shared:
              - 'packages/shared/**'

  test-api:
    name:   
     
       
     

    
      
         

    
        

         
         
        
           
           
           

         
          

         
          

  
       
     
       
     

    
      
         

    
        

         
         
        
           
           
           

         
          

         
          

        
           

  
       
     
       
     

    
      
         

    
        

         
         
        
           
           
           

         
          

         
          

  
         
     [, , , ]
          
    
      
        
            
                 
            
                 
            
                 

    
        
           

             
           
         

Example 8: Performance Optimization Pipeline

name: Optimized CI Pipeline

on: [push, pull_request]

jobs:
  # Fast feedback jobs run first
  quick-checks:
    name: Quick Checks (< 2min)
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"

      - name: Cache node_modules
        uses: actions/cache@v4
        with:
          path: node_modules
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Install dependencies
        run: npm   

            
         
  
         
     
       

    
        
        
            

         
         
        
           
           

         
           

          
         
         
             
           
              

  
        
     
     

    
        

         
         
        
           
           

          
         
        
                            
           
         
           

        
           

          
         
        
           
           
           

  
         
     
     

    
        

           
         

          
         
        
           
           
             
           
           
           

Pipeline Optimization Patterns

Caching Strategy

1. Dependency Caching: Cache node_modules, vendor/, .m2/, etc.
2. Build Output Caching: Cache compiled artifacts between runs
3. Docker Layer Caching: Use BuildKit cache mounts and GitHub Actions cache
4. Incremental Builds: Only rebuild changed modules (Nx, Turborepo)

Parallelization Strategies

1. Job-Level Parallelization: Run independent jobs concurrently
2. Test Sharding: Split test suite across multiple runners
3. Matrix Builds: Test multiple versions/platforms simultaneously
4. Monorepo Path Filtering: Only test changed services

Conditional Execution

1. Path Filters: Skip jobs when irrelevant files change
2. Changed Files Detection: Test only affected code
3. Branch-Specific Jobs: Different pipelines for different branches
4. Manual Triggers: Allow on-demand pipeline execution

ML-Specific Patterns

Model Training Pipeline

1. Data Validation: Validate schema and quality before training
2. Experiment Tracking: Log metrics to MLflow/W&B
3. Model Versioning: Tag models with git SHA or semantic version
4. Performance Gates: Enforce minimum accuracy/F1 thresholds

Model Deployment

1. A/B Testing: Deploy new model alongside existing
2. Shadow Mode: Run new model without affecting production
3. Canary Rollout: Gradually increase traffic to new model
4. Automated Rollback: Revert on performance degradation

Troubleshooting Guide

Common Issues

1. Flaky Tests: Implement retry logic, increase timeouts, fix race conditions
2. Slow Pipelines: Profile execution times, add caching, parallelize
3. Secrets Exposure: Use secret scanning, audit logs, rotate credentials
4. Resource Exhaustion: Set resource limits, use cleanup actions
5. Network Timeouts: Add retries, use artifact caching, increase timeouts

Debugging Commands

# GitHub Actions local testing
act -j test --secret-file .env.secrets

# GitLab CI local testing
gitlab-runner exec docker test

# Jenkins pipeline validation
java -jar jenkins-cli.jar declarative-linter < Jenkinsfile

# Docker build debugging
DOCKER_BUILDKIT=1 docker build --progress=plain .

# Test pipeline with dry-run
kubectl apply --dry-run=client -f k8s/

# Validate workflow syntax
actionlint .github/workflows/*.yml