| name | pipeline-patterns |
| description | Guide for designing CI/CD pipelines with artifact management, environment promotion, parallel jobs, quality gates, and notifications. Use when the user designs a CI/CD pipeline, asks about deployment workflows, needs to structure build-test-deploy stages, or wants to optimize pipeline performance. Trigger whenever CI/CD architecture or pipeline design is discussed.
|
Pipeline Patterns
Design robust CI/CD pipelines with clear stage separation, proper artifact flow,
environment promotion, quality gates, and failure handling.
When to Use
- User designs a new CI/CD pipeline
- User asks about build, test, deploy stage organization
- User needs environment promotion (dev -> staging -> prod)
- User wants to optimize pipeline speed with parallelism
- User asks about quality gates or approval workflows
Core Patterns
Pipeline Stage Architecture
Structure pipelines in clear stages with explicit dependencies and artifact passing.
Build --> Lint --|
|--> Integration Tests --> Deploy Staging --> E2E Tests --> Deploy Prod
Unit --|
jobs:
build:
runs-on: ubuntu-latest
outputs:
image-tag: ${{ steps.meta.outputs.tags }}
steps:
- uses: actions/checkout@v4
- id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/${{ github.repository }}
tags: type=sha
- uses: docker/build-push-action@v5
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run lint
unit-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm test
integration-test:
needs: [build, lint, unit-test]
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env: { POSTGRES_DB: test, POSTGRES_PASSWORD: test }
options: --health-cmd pg_isready --health-interval 5s --health-retries 5
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run test:integration
deploy-staging:
needs: integration-test
runs-on: ubuntu-latest
environment: staging
steps:
- run: echo "Deploy ${{ needs.build.outputs.image-tag }} to staging"
e2e-test:
needs: deploy-staging
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npx playwright test --project=staging
deploy-production:
needs: e2e-test
runs-on: ubuntu-latest
environment:
name: production
url: https://myapp.com
steps:
- run: echo "Deploy ${{ needs.build.outputs.image-tag }} to production"
Artifact Management
Build once, deploy everywhere. Pass artifacts between stages to ensure consistency.
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run build
- uses: actions/upload-artifact@v4
with:
name: app-build-${{ github.sha }}
path: dist/
retention-days: 7
deploy:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v4
with:
name: app-build-${{ github.sha }}
path: dist/
- run: ./scripts/deploy.sh dist/
Quality Gates
Enforce quality standards before promotion to the next environment.
jobs:
quality-gate:
needs: [unit-test, lint, security-scan]
runs-on: ubuntu-latest
steps:
- name: Check test coverage
run: |
COVERAGE=$(cat coverage/coverage-summary.json | jq '.total.lines.pct')
if (( $(echo "$COVERAGE < 80" | bc -l) )); then
echo "Coverage $COVERAGE% is below 80% threshold"
exit 1
fi
- name: Check lint results
run: |
if [ "${{ needs.lint.result }}" != "success" ]; then
echo "Lint check failed"
exit 1
fi
- name: Check security scan
run: |
if [ "${{ needs.security-scan.result }}" != "success" ]; then
echo "Security vulnerabilities found"
exit 1
fi
Environment Promotion with Approvals
Use GitHub environments to enforce manual approvals for production deployments.
jobs:
deploy-staging:
environment: staging
runs-on: ubuntu-latest
steps:
- run: deploy --env staging
smoke-test:
needs: deploy-staging
runs-on: ubuntu-latest
steps:
- run: curl -f https://staging.myapp.com/health
deploy-production:
needs: smoke-test
environment:
name: production
url: https://myapp.com
runs-on: ubuntu-latest
steps:
- run: deploy --env production
Parallel Test Splitting
Split large test suites across parallel runners to reduce pipeline duration.
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npx jest --shard=${{ matrix.shard }}/4
merge-coverage:
needs: test
runs-on: ubuntu-latest
steps:
- run: echo "Merge coverage from all shards"
Anti-Patterns
- Build per environment: Build the artifact once and promote it. Rebuilding per environment risks deploying different code to staging and production.
- No quality gates: Deploying without coverage, lint, or security checks allows regressions to reach production.
- Sequential everything: Independent jobs like lint, unit tests, and security scans should run in parallel. Only use
needs for true dependencies.
- Skipping staging: Always deploy to a staging environment before production. E2E tests against staging catch integration issues that unit tests miss.
- No rollback plan: Every deploy pipeline must have a documented and tested rollback path. If the pipeline cannot roll back, it is not production-ready.
- Secrets in logs: Never echo secrets. Use
add-mask to redact sensitive values in output.
Quick Reference
Pipeline Design Checklist:
[ ] Build once, deploy many (single artifact)
[ ] Parallel independent jobs (lint, test, scan)
[ ] Quality gates before promotion
[ ] Environment-specific configs (not rebuilt code)
[ ] Manual approval for production
[ ] Rollback procedure documented and tested
[ ] Notifications on failure
[ ] Artifact retention policy defined
[ ] Concurrency controls to prevent duplicate runs
[ ] Timeout limits on all jobs
