ci-cd

name	ci-cd
description	Use when setting up or debugging GitHub Actions pipelines — adding quality gates, configuring OIDC cloud auth, building matrix test runs, publishing artifacts to GHCR/PyPI/npm, or promoting builds from staging to production.

CI/CD with GitHub Actions

A complete reference for building robust, secure, and efficient CI/CD pipelines using GitHub Actions, covering everything from workflow anatomy to multi-environment deployment promotion.

When to Activate

• Creating or modifying a GitHub Actions workflow
• Adding a quality gate (coverage, lint, security scan) to a pipeline
• Setting up CD with environment promotion from staging to production
• Optimizing a slow CI pipeline with caching or parallelism
• Publishing a Docker image, Python package, or npm package from CI
• Configuring secrets and OIDC-based cloud authentication

---

Workflow Anatomy

Every GitHub Actions workflow is defined in .github/workflows/*.yml. Understanding the core building blocks is essential before composing pipelines.

Triggers

Trigger	Use Case
push	Run on commits to specified branches or tags
pull_request	Run on PR open, sync, or reopen
workflow_dispatch	Manual trigger with optional input parameters
schedule	Cron-based triggers (e.g., nightly security scans)
workflow_call	Reusable workflow called by another workflow

Jobs

• Jobs run in parallel by default unless needs: is specified.
• Use needs: [job-a, job-b] to declare dependencies; a job waits for all listed jobs to succeed.
• Each job runs in a fresh virtual machine (or container).

Steps

• uses: — invokes a reusable action from the marketplace or a local path.
• run: — executes a shell command directly on the runner.
• Steps within a job share the same filesystem and runner environment.

Key Contexts

Context	What It Provides
github	Repo name, SHA, ref, event name, actor, run ID
secrets	Encrypted secret values; never printed in logs
env	Environment variables set at workflow, job, or step level
matrix	Current values from the strategy matrix
job	Current job status, container info

Expression Syntax

All dynamic values use ${{ expression }} syntax:

if: github.ref == 'refs/heads/main'
run: echo "SHA is ${{ github.sha }}"
env:
  BRANCH: ${{ github.ref_name }}

Minimal Workflow Skeleton

name: CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: make test

---

Standard Pipeline Structure

The canonical stage sequence moves from fast feedback (lint) to slower gates (security), then publishing.

lint → test → build → security-scan → publish

Job Dependency Graph

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Lint
        run: make lint

  test:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Test
        run: make test

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build
        run: make build

  security-scan:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Security scan
        run: make scan

  publish:
    needs: [build, security-scan]
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Publish
        run: make publish

fail-fast Behavior

Setting	When to Use
fail-fast: true (default)	Sequential stage pipelines — stop early on first failure
fail-fast: false	Matrix builds — collect all results before deciding

Use fail-fast: false in matrix strategies so you see failures across all OS/version combinations rather than stopping at the first one.

---

Dependency Caching

Caching package manager dependencies is the single highest-impact optimization for CI speed. The key strategy: hash the lockfile so the cache automatically invalidates when dependencies change.

Cache Key Strategy

cache-key = runner-os + lockfile-hash
restore-key = runner-os (fallback to most recent cache)

Python

- uses: actions/setup-python@v5
  with:
    python-version: '3.12'
    cache: 'pip'  # built-in caching, hashes requirements*.txt

For more control with pyproject.toml:

- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml', '**/requirements*.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

Node.js

- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'  # hashes package-lock.json automatically

For pnpm or yarn, set cache: 'pnpm' or cache: 'yarn' respectively.

Go

- uses: actions/setup-go@v5
  with:
    go-version: '1.22'
    cache: true  # caches $GOPATH/pkg/mod and build cache

Cache Hit Rate Tips

• Always commit lockfiles (package-lock.json, poetry.lock, go.sum) to the repository.
• Use hashFiles('**/package-lock.json') to scope cache keys to the exact dependency set.
• Restore keys act as fallbacks: a partial hit is better than no cache.

---

Matrix Builds

Matrix builds let a single job definition run across multiple configurations (language versions, operating systems) in parallel.

Matrix Strategy

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        python-version: ['3.11', '3.12', '3.13']
        os: [ubuntu-latest, windows-latest]
        include:
          - python-version: '3.12'
            os: ubuntu-latest
            publish: true      # custom flag for conditional steps
        exclude:
          - python-version: '3.11'
            os: windows-latest  # skip known unsupported combo
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      - run: pytest
      - name: Publish (only from designated matrix entry)
        if: matrix.publish == true
        run: make publish

Matrix Reference

Feature	Syntax	Purpose
Access value	${{ matrix.python-version }}	Use current dimension value in steps
include	Adds extra key-value pairs to specific entries	Inject flags or override runner for one combo
exclude	Removes specific combinations	Skip known broken or unnecessary combos
fail-fast	false on matrix	See all failures, not just first

---

Quality Gates

Coverage Threshold

Fail the build if coverage drops below the required threshold.

- name: Run tests with coverage
  run: pytest --cov=src --cov-report=xml --cov-fail-under=80

- name: Upload coverage report
  uses: codecov/codecov-action@v4
  with:
    file: ./coverage.xml
    fail_ci_if_error: true

For Node.js with Jest:

- name: Run tests with coverage
  run: jest --coverage --coverageThreshold='{"global":{"lines":80}}'

Inline Lint Annotations with reviewdog

reviewdog posts lint results as PR review comments, making issues visible without reading raw logs.

- uses: reviewdog/action-flake8@v3
  with:
    reporter: github-pr-review
    level: warning

- uses: reviewdog/action-eslint@v1
  with:
    reporter: github-pr-review
    eslint_flags: '--ext .js,.ts src/'

Branch Protection Rules

Configure these in GitHub Settings → Branches, not in workflow YAML:

• Required status checks: list every CI job name that must pass before merge.
• Required approvals: minimum 1 reviewer for main.
• Dismiss stale approvals: re-approval required after new commits are pushed.
• Require branches to be up to date: prevents merging outdated branches.

Security Scanning

- name: Python dependency audit
  run: pip-audit

- name: Node audit
  run: npm audit --audit-level=high

- name: Go vulnerability check
  run: govulncheck ./...

For container scanning, integrate Trivy:

- uses: aquasecurity/trivy-action@master
  with:
    image-ref: ghcr.io/${{ github.repository }}:${{ github.sha }}
    severity: 'CRITICAL,HIGH'
    exit-code: '1'

---

Secrets and Environment Variables

Contexts

env:
  APP_ENV: production                    # plain env var (visible in logs)
  DB_HOST: ${{ vars.DB_HOST }}           # repository variable (not secret, visible)
  API_KEY: ${{ secrets.API_KEY }}        # encrypted secret (masked in logs)

Variable Scoping

Scope	Set In	Available To
Workflow	Top-level env: block	All jobs and steps
Job	env: under a specific job	All steps in that job
Step	env: under a specific step	That step only
Repository	GitHub Settings → Secrets and Variables	All workflows in the repo
Environment	GitHub Settings → Environments	Jobs using that environment

$GITHUB_TOKEN Permissions

Declare only what the job requires (principle of least privilege):

permissions:
  contents: read       # read repo files
  packages: write      # push to GHCR
  id-token: write      # request OIDC token for cloud auth
  pull-requests: write # post PR comments
  checks: write        # create check runs

Set permissions: {} at the workflow level, then grant specific permissions per job to minimize blast radius.

OIDC for Cloud Auth (No Long-Lived Credentials)

OIDC eliminates the need to store cloud credentials as GitHub secrets. The cloud provider verifies the workflow's identity token and grants temporary credentials.

AWS:

- uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789:role/github-actions
    aws-region: us-east-1

Requires an IAM role with a trust policy that allows token.actions.githubusercontent.com as an OIDC provider.

Google Cloud:

- uses: google-github-actions/auth@v2
  with:
    workload_identity_provider: projects/123/locations/global/workloadIdentityPools/my-pool/providers/github
    service_account: [e-mail]

Requires a Workload Identity Pool configured in GCP with a GitHub OIDC provider binding.

Azure:

- uses: azure/login@v2
  with:
    client-id: ${{ secrets.AZURE_CLIENT_ID }}
    tenant-id: ${{ secrets.AZURE_TENANT_ID }}
    subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

Uses federated credentials on the Azure App Registration — no client secret stored.

---

CD and Environment Promotion

The standard promotion pattern: build once, deploy to staging automatically, then gate production behind a manual approval.

Staging → Production Pipeline

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build artifact
        run: make build
      - uses: actions/upload-artifact@v4
        with:
          name: app-artifact
          path: dist/

  deploy-staging:
    environment: staging
    runs-on: ubuntu-latest
    needs: build
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: app-artifact
          path: dist/
      - name: Deploy to staging
        run: ./scripts/deploy.sh staging

  smoke-test:
    runs-on: ubuntu-latest
    needs: deploy-staging
    steps:
      - name: Run smoke tests against staging
        run: ./scripts/smoke-test.sh https://staging.example.com

  deploy-production:
    environment: production   # required reviewers configured in GitHub Settings
    runs-on: ubuntu-latest
    needs: smoke-test
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: app-artifact
          path: dist/
      - name: Deploy to production
        run: ./scripts/deploy.sh production
      - name: Notify on failure
        if: failure()
        run: ./scripts/notify-failure.sh "${{ job.status }}"

Environment Protection Configuration

Set these in GitHub Settings → Environments, not in YAML:

Setting	Recommended Value
Required reviewers	1–2 senior engineers for production
Wait timer	Optional: 5–10 min buffer before deployment
Deployment branches	Limit to main branch only
Environment secrets	Prod credentials scoped here, not at repo level

Deployment Status

Use job.status in post-deploy notifications:

- name: Notify Slack
  if: always()
  uses: slackapi/slack-github-action@v1
  with:
    payload: '{"text": "Deploy to production: ${{ job.status }}"}'
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

---

Artifact and Package Publishing

Docker Image to GHCR

jobs:
  publish-image:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=sha,prefix=,suffix=,format=short
            type=ref,event=branch
            type=semver,pattern={{version}}

      - uses: docker/build-push-action@v5
        with:
          context: .
          push: ${{ github.ref == 'refs/heads/main' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Always tag images with the commit SHA. Never use latest as the only tag in production.

PyPI with Trusted Publishing (No Token Needed)

Configure a Trusted Publisher in PyPI project settings pointing to this repository and workflow, then:

jobs:
  publish-pypi:
    runs-on: ubuntu-latest
    permissions:
      id-token: write  # required for trusted publishing
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install build
      - run: python -m build
      - uses: pypa/gh-action-pypi-publish@release/v1
        with:
          repository-url: https://upload.pypi.org/legacy/
          # For pre-release testing:
          # repository-url: https://test.pypi.org/legacy/

npm Publish

jobs:
  publish-npm:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          registry-url: 'https://registry.npmjs.org'
      - run: npm ci
      - run: npm run build
      - run: npm publish --access public
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

---

Complete Workflow Examples

Python Service

name: Python CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: pip
      - run: pip install -e ".[dev]"
      - run: ruff check .
      - run: mypy src/
      - run: pytest --cov=src --cov-report=xml --cov-fail-under=80
      - run: pip-audit
      - name: Upload coverage
        uses: codecov/codecov-action@v4
        with:
          file: ./coverage.xml

TypeScript/Node Service

name: Node CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck
      - run: npm test -- --coverage
      - run: npm audit --audit-level=high

Go Service

name: Go CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: '1.22'
          cache: true
      - run: go vet ./...
      - run: staticcheck ./...
      - run: go test -race -coverprofile=coverage.out ./...
      - run: go tool cover -func=coverage.out
      - run: govulncheck ./...

---

Decision Tables

Which Cache Approach to Use

Language	Recommended Approach	Lockfile to Hash
Python	setup-python built-in cache: pip	requirements*.txt, pyproject.toml
Node.js	setup-node built-in cache: npm/pnpm	package-lock.json, pnpm-lock.yaml
Go	setup-go built-in cache: true	go.sum
Rust	actions/cache on ~/.cargo	Cargo.lock
Java/Maven	actions/cache on ~/.m2	pom.xml

When to Use OIDC vs Stored Secrets

Credential Type	Use OIDC	Use Stored Secret
AWS IAM role	Yes	No — OIDC is the standard
GCP service account	Yes	No — Workload Identity preferred
Azure managed identity	Yes	No — Federated credentials preferred
NPM publish token	No	Yes — npm does not support OIDC
PyPI publish token	No (use Trusted Publishing)	Only if Trusted Publishing unavailable
Third-party API key	No	Yes

Trigger Selection Guide

Scenario	Recommended Trigger
Run CI on every PR	pull_request targeting main
Deploy on merge to main	push on main branch
Nightly dependency audit	schedule with cron
Manual production deploy	workflow_dispatch with input parameters
Shared CI logic across repos	workflow_call (reusable workflow)
Release on tag push	push with tags: ['v*']

---

Red Flags

• Storing cloud credentials as GitHub Secrets — long-lived access keys can be exfiltrated via PR log injection; use OIDC federated credentials (aws-actions/configure-aws-credentials) instead
• Hardcoding runs-on: ubuntu-latest without version pinning — ubuntu-latest can shift to a new OS major version mid-project, silently changing available toolchains; pin to ubuntu-22.04 for stability
• Single monolithic job with 20+ steps — any step failure retries the entire job from scratch; split into separate jobs with needs: so fast gates (lint) don't block slow ones (security scan)
• Pushing to main without a required status check — branch protection "Required status checks" must be configured in GitHub Settings, not just in the YAML; YAML alone can be bypassed
• Cache key without a lockfile hash — using key: ${{ runner.os }}-pip without hashFiles(...) causes stale caches after dependency updates, producing false green builds
• Tagging Docker images with only latest — latest is overwritten on every build; a failed rollback cannot target a specific previous image; always add a commit SHA tag alongside latest
• Using pull_request_target without careful filtering — this trigger runs in the context of the base branch and has access to secrets, making it exploitable by a forked PR that modifies the workflow
• Skipping --cov-fail-under or equivalent threshold — coverage upload without a fail threshold lets coverage silently drop to zero; the gate must actively fail the build

Checklist

• Workflow triggers cover both push to main and pull_request
• Jobs run in correct dependency order using needs:
• Package manager caching configured (pip/npm/go modules)
• Lint, type-check, test, and security scan are all separate steps
• Coverage threshold enforced (--cov-fail-under / --coverage)
• $GITHUB_TOKEN permissions scoped to least privilege
• Secrets accessed via ${{ secrets.X }} — never hardcoded
• OIDC used for cloud credentials — no long-lived access keys stored as secrets
• CD deploys to staging first; production requires approval via environment protection
• Docker images tagged with commit SHA, not latest
• Failed jobs produce clear error output; no silent failures
• Pipeline completes in under 10 minutes on a typical PR