containerization-for-agentic-workflows

# 🐳 Containerization for Agentic Workflows ## 🔴 AI FIRST Quality Principle > **Apply the AI FIRST principle: never accept first-pass quality. Minimum 2 iterations. Read all output, improve every section. No shortcuts.** ## 📋 Overview This skill provides comprehensive patterns for containerizing GitHub Agentic Workflows using Docker and container orchestration platforms. It covers container isolation strategies, security hardening techniques, multi-stage build optimization, Docker Compose orchestration, and Kubernetes deployment patterns for production-ready autonomous agent systems. ## 🎯 Core Concepts ### Container Architecture ```mermaid graph TB subgraph "Build Stage" A[Source Code] --> B[Multi-Stage Build] B --> C[Dependencies] B --> D[Agent Code] C --> E[Runtime Image] D --> E end subgraph "Runtime Environment" E --> F[Container Runtime] F --> G[Agent Process] G --> H[MCP Servers] G --> I[LLM APIs] G --> J[GitHub API] end subgraph "Orchestration" F --> K[Docker Compose] F --> L[Kubernetes] K --> M[Multi-Container] L --> N[Scalable Deployment] end subgraph "Security" F --> O[Non-Root User] F --> P[Read-Only FS] F --> Q[Resource Limits] end style E fill:#00d9ff style G fill:#ff006e style O fill:#ffbe0b ``` ### Key Principles 1. **Isolation**: Containers provide process and resource isolation 2. **Immutability**: Containers are built once, run anywhere 3. **Security**: Defense-in-depth with multiple security layers 4. **Efficiency**: Minimal image size and resource usage 5. **Orchestration**: Automated deployment and scaling 6. **Observability**: Comprehensive logging and monitoring ## 🏗️ Multi-Stage Builds ### 1. Optimized Node.js Agent ```dockerfile # docker/agent-node.Dockerfile # Multi-stage build for Node.js agentic workflow # Stage 1: Build dependencies FROM node:24-alpine AS deps LABEL stage=deps WORKDIR /app # Copy dependency manifests COPY package.json package-lock.json ./ # Install dependencies (production only) RUN npm ci --only=production \ && npm cache clean --force # Stage 2: Build application FROM node:24-alpine AS builder LABEL stage=builder WORKDIR /app # Copy dependencies from deps stage COPY --from=deps /app/node_modules ./node_modules # Copy source code COPY scripts/agents ./scripts/agents COPY .github/copilot-mcp.json ./config/ # Build TypeScript if needed # RUN npm run build # Stage 3: Production runtime FROM node:24-alpine AS runtime # Install security updates RUN apk update && apk upgrade \ && apk add --no-cache \ dumb-init \ ca-certificates \ && rm -rf /var/cache/apk/* # Create non-root user RUN addgroup -g 1001 -S agent \ && adduser -u 1001 -S agent -G agent WORKDIR /app # Copy production dependencies COPY --from=deps --chown=agent:agent /app/node_modules ./node_modules # Copy application code COPY --from=builder --chown=agent:agent /app/scripts/agents ./scripts/agents COPY --from=builder --chown=agent:agent /app/config ./config # Set environment variables ENV NODE_ENV=production \ NODE_OPTIONS="--max-old-space-size=2048" \ LOG_LEVEL=info # Switch to non-root user USER agent # Health check HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ CMD node -e "require('http').get('http://localhost:3000/health', (r) => { process.exit(r.statusCode === 200 ? 0 : 1); });" # Expose port (if needed) EXPOSE 3000 # Use dumb-init to handle signals properly ENTRYPOINT ["dumb-init", "--"] # Default command CMD ["node", "scripts/agents/main.js"] ``` ### 2. Optimized Python Agent ```dockerfile # docker/agent-python.Dockerfile # Multi-stage build for Python agentic workflow # Stage 1: Build dependencies FROM python:3.11-slim AS builder # Install build dependencies RUN apt-get update \ && apt-get install -y --no-install-recommends \ build-essential \ gcc \ g++ \ && rm -rf /var/lib/apt/lists/* WORKDIR /app # Copy dependency manifests COPY requirements.txt requirements-agent.txt ./ # Create virtual environment and install dependencies RUN python -m venv /opt/venv \ && /opt/venv/bin/pip install --upgrade pip setuptools wheel \ && /opt/venv/bin/pip install --no-cache-dir \ -r requirements.txt \ -r requirements-agent.txt # Stage 2: Production runtime FROM python:3.11-slim # Install security updates and runtime dependencies RUN apt-get update \ && apt-get upgrade -y \ && apt-get install -y --no-install-recommends \ ca-certificates \ dumb-init \ curl \ && rm -rf /var/lib/apt/lists/* # Create non-root user RUN useradd -m -u 1001 -s /bin/bash agent WORKDIR /app # Copy virtual environment from builder COPY --from=builder --chown=agent:agent /opt/venv /opt/venv # Copy application code COPY --chown=agent:agent scripts/agents ./scripts/agents COPY --chown=agent:agent .github/copilot-mcp.json ./config/ # Set environment variables ENV PATH="/opt/venv/bin:$PATH" \ PYTHONUNBUFFERED=1 \ PYTHONDONTWRITEBYTECODE=1 \ PYTHONHASHSEED=random \ PIP_NO_CACHE_DIR=1 \ PIP_DISABLE_PIP_VERSION_CHECK=1 # Switch to non-root user USER agent # Health check HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:3000/health')" # Expose port EXPOSE 3000 # Use dumb-init ENTRYPOINT ["dumb-init", "--"] # Default command CMD ["python", "scripts/agents/main.py"] ``` ### 3. Multi-Runtime Agent ```dockerfile # docker/agent-multi.Dockerfile # Multi-stage build with Node.js and Python # Stage 1: Node.js dependencies FROM node:24-alpine AS node-deps WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci --only=production && npm cache clean --force # Stage 2: Python dependencies FROM python:3.11-slim AS python-deps WORKDIR /app COPY requirements.txt ./ RUN python -m venv /opt/venv \ && /opt/venv/bin/pip install --no-cache-dir -r requirements.txt # Stage 3: Production runtime FROM ubuntu:22.04 # Install Node.js, Python, and runtime dependencies RUN apt-get update \ && apt-get upgrade -y \ && apt-get install -y --no-install-recommends \ ca-certificates \ curl \ dumb-init \ nodejs \ npm \ python3.11 \ python3-pip \ && rm -rf /var/lib/apt/lists/* # Create non-root user RUN useradd -m -u 1001 -s /bin/bash agent WORKDIR /app # Copy Node.js dependencies COPY --from=node-deps --chown=agent:agent /app/node_modules ./node_modules # Copy Python virtual environment COPY --from=python-deps --chown=agent:agent /opt/venv /opt/venv # Copy application code COPY --chown=agent:agent scripts/agents ./scripts/agents COPY --chown=agent:agent .github/copilot-mcp.json ./config/ # Set environment variables ENV PATH="/opt/venv/bin:$PATH" \ NODE_ENV=production \ PYTHONUNBUFFERED=1 USER agent ENTRYPOINT ["dumb-init", "--"] CMD ["node", "scripts/agents/orchestrator.js"] ``` ## 🔒 Security Hardening ### 1. Non-Root User ```dockerfile # Never run containers as root # Create user and group RUN addgroup -g 1001 -S agent \ && adduser -u 1001 -S agent -G agent # Set ownership COPY --chown=agent:agent ./app ./app # Switch to non-root user USER agent ``` ### 2. Read-Only Filesystem ```dockerfile # Make root filesystem read-only for security # In Dockerfile: specify writable volumes VOLUME ["/tmp", "/app/logs"] # In docker-compose.yml services: agent: read_only: true tmpfs: - /tmp:mode=1777 - /app/logs:mode=0755 ``` ```yaml # In Kubernetes apiVersion: v1 kind: Pod spec: containers: - name: agent securityContext: readOnlyRootFilesystem: true volumeMounts: - name: tmp mountPath: /tmp - name: logs mountPath: /app/logs volumes: - name: tmp emptyDir: {} - name: logs emptyDir: {} ``` ### 3. Resource Limits ```dockerfile # Set resource limits in Dockerfile (documentation) LABEL resource.memory="512Mi" LABEL resource.cpu="0.5" ``` ```yaml # docker-compose.yml services: agent: deploy: resources: limits: cpus: '0.5' memory: 512M reservations: cpus: '0.25' memory: 256M ``` ```yaml # Kubernetes apiVersion: v1 kind: Pod spec: containers: - name: agent resources: limits: cpu: "500m" memory: "512Mi" requests: cpu: "250m" memory: "256Mi" ``` ### 4. Security Scanning ```yaml # .github/workflows/container-security.yml name: Container Security Scan on: push: branches: [main] paths: - 'docker/**' pull_request: paths: - 'docker/**' jobs: scan: runs-on: ubuntu-latest steps: - name: Checkout Code uses: actions/checkout@v4 - name: Build Image run: | docker build -f docker/agent-node.Dockerfile \ -t agentic-workflow:${{ github.sha }} . - name: Run Trivy Vulnerability Scanner uses: aquasecurity/trivy-action@master with: image-ref: agentic-workflow:${{ github.sha }} format: 'sarif' output: 'trivy-results.sarif' severity: 'CRITICAL,HIGH' - name: Upload Trivy Results uses: github/codeql-action/upload-sarif@v3 with: sarif_file: 'trivy-results.sarif' - name: Run Dockle Security Lint run: | docker run --rm \ -v /var/run/docker.sock:/var/run/docker.sock \ goodwithtech/dockle:latest \ --exit-code 1 \ --exit-level fatal \ agentic-workflow:${{ github.sha }} ``` ### 5. Secrets Management ```dockerfile # Use BuildKit secrets for sensitive build-time data # syntax=docker/dockerfile:1 FROM node:24-alpine # Mount secret during build (never stored in image) RUN --mount=type=secret,id=npm_token \ echo "//registry.npmjs.org/:_authToken=$(cat /run/secrets/npm_token)" > ~/.npmrc \ && npm ci \ && rm ~/.npmrc ``` ```bash # Build with secrets docker build \ --secret id=npm_token,src=$HOME/.npmrc \ -f docker/agent.Dockerfile \ -t agent:latest . ``` ## 🎨 Image Optimization ### 1. Layer Caching ```dockerfile # Optimize layer caching for faster builds # ❌ Bad: Copy everything first COPY . /app RUN npm install # ✅ Good: Copy dependencies first (cached layer) COPY package*.json /app/ RUN npm ci # Then copy code (invalidates cache only if code changes) COPY . /app/ ``` ### 2. Minimize Image Size ```dockerfile # Use Alpine base images FROM node:24-alpine # ~180MB vs ~1GB for full Node.js # Remove build dependencies after installation RUN apk add --no-cache --virtual .build-deps \ python3 \ make \ g++ \ && npm ci --only=production \ && apk del .build-deps # Clean up package manager cache RUN rm -rf /var/cache/apk/* \ && npm cache clean --force # Use .dockerignore to exclude unnecessary files # .dockerignore: # node_modules # npm-debug.log # .git # .github # tests # *.md ``` ### 3. Distroless Images ```dockerfile # Use distroless images for maximum security and minimal size FROM gcr.io/distroless/nodejs24-debian12 # Copy application COPY --from=builder /app /app WORKDIR /app # Distroless images: # - No shell # - No package manager # - Minimal attack surface # - Only runtime dependencies CMD ["scripts/agents/main.js"] ``` ### 4. Build Cache Optimization ```yaml # .github/workflows/build-container.yml name: Build Container on: push: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - name: Checkout Code uses: actions/checkout@v4 - name: Set up Docker Buildx