docker-optimization

name	docker-optimization
description	Docker image optimization patterns including multi-stage builds, layer caching, security hardening, and size reduction techniques. Use when building Docker images, optimizing container size, improving build performance, or implementing Docker security best practices. Reduces image sizes by 70-90% and build times by 50-80%.

Docker Optimization Patterns

Comprehensive guide to optimizing Docker images for size, build speed, and security. Covers multi-stage builds, layer caching strategies, security hardening, and production deployment patterns.

---

Quick Reference

When to use this skill:

• Building production Docker images
• Optimizing image size (reducing from 500MB+ to <100MB)
• Improving Docker build times
• Implementing Docker security best practices
• Debugging slow builds or large images
• Setting up Docker for microservices

Common triggers:

• "My Docker image is too large"
• "Docker builds take forever"
• "How do I optimize this Dockerfile"
• "Docker security best practices"
• "Multi-stage build pattern"

---

Part 1: Multi-Stage Builds

The Problem: Bloated Images

Typical single-stage Dockerfile (800MB+ image):

FROM python:3.11
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]

Problems:

• Includes build tools (gcc, make, etc.) - 300MB+
• Includes pip cache - 100MB+
• Includes source .git directory - 50MB+
• Includes test files and dev dependencies - 50MB+
• Total: 800MB+ for simple Python app

The Solution: Multi-Stage Pattern

Optimized multi-stage Dockerfile (120MB image):

# Stage 1: Builder
FROM python:3.11-slim AS builder
WORKDIR /app

# Install build dependencies in separate layer
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    && rm -rf /var/lib/apt/lists/*

# Copy only requirements first (cache optimization)
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Stage 2: Runtime
FROM python:3.11-slim
WORKDIR /app

# Copy only Python packages from builder
COPY --from=builder /root/.local /root/.local

# Copy only application code
COPY app.py .
COPY src/ ./src/

# Make sure scripts in .local are usable
ENV PATH=/root/.local/bin:$PATH

# Run as non-root user
RUN useradd -m appuser && chown -R appuser:appuser /app
USER appuser

CMD ["python", "app.py"]

Result: 800MB → 120MB (85% reduction)

Multi-Stage Pattern Breakdown

Stage 1: Builder (Throw away after build)

• Install build tools
• Compile dependencies
• Run tests (optional)
• Generate artifacts

Stage 2: Runtime (Final image)

• Minimal base image
• Copy only artifacts from builder
• No build tools
• No source files (only compiled/necessary files)

---

Part 2: Layer Caching Optimization

Understanding Docker Layer Caching

Each instruction creates a layer. Docker caches unchanged layers.

Bad Order (cache invalidated on every code change):

FROM python:3.11-slim
COPY . .                        # ❌ Copies everything
RUN pip install -r requirements.txt  # ❌ Runs on every code change

Good Order (cache preserved):

FROM python:3.11-slim
COPY requirements.txt .         # ✅ Only requirements
RUN pip install -r requirements.txt  # ✅ Cached if requirements unchanged
COPY . .                        # ✅ Code changes don't invalidate pip cache

Layer Caching Best Practices

1. Order by change frequency (least to most):

# 1. System dependencies (rarely change)
RUN apt-get update && apt-get install -y curl

# 2. Language runtime (rarely changes)
FROM python:3.11-slim

# 3. Dependencies (change occasionally)
COPY requirements.txt .
RUN pip install -r requirements.txt

# 4. Application code (changes frequently)
COPY . .

2. Separate COPY operations:

# ❌ Bad: Invalidates cache on any file change
COPY . .

# ✅ Good: Cache preserved unless specific files change
COPY package.json package-lock.json ./
RUN npm ci
COPY src/ ./src/
COPY public/ ./public/

3. Use .dockerignore:

# .dockerignore
.git
.gitignore
node_modules
npm-debug.log
Dockerfile
.dockerignore
.env
.venv
__pycache__
*.pyc
tests/
docs/

---

Part 3: Image Size Optimization

Choose Minimal Base Images

Image Size Comparison:

python:3.11          → 1.01GB
python:3.11-slim     → 130MB   (87% smaller)
python:3.11-alpine   → 50MB    (95% smaller)

When to use each:

• Full image (python:3.11): Never for production
• Slim (python:3.11-slim): Default choice, good compatibility
• Alpine (python:3.11-alpine): Smallest, but can have glibc issues

Multi-Stage Size Optimization

Node.js Example (900MB → 150MB):

# Builder stage
FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Production stage
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]

Result: 900MB → 150MB (83% reduction)

Clean Up in Same Layer

❌ Bad (creates large intermediate layers):

RUN apt-get update
RUN apt-get install -y build-essential
RUN rm -rf /var/lib/apt/lists/*

✅ Good (single layer, no intermediate garbage):

RUN apt-get update && \
    apt-get install -y --no-install-recommends build-essential && \
    rm -rf /var/lib/apt/lists/*

Remove Build Dependencies After Use

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        gcc \
        g++ \
        make \
    && pip install --no-cache-dir -r requirements.txt \
    && apt-get purge -y --auto-remove \
        gcc \
        g++ \
        make \
    && rm -rf /var/lib/apt/lists/*

---

Part 4: Security Best Practices

Don't Run as Root

❌ Bad (runs as root):

FROM python:3.11-slim
COPY app.py .
CMD ["python", "app.py"]

✅ Good (runs as non-root user):

FROM python:3.11-slim

# Create non-root user
RUN useradd -m -u 1000 appuser && \
    mkdir -p /app && \
    chown -R appuser:appuser /app

WORKDIR /app
USER appuser

COPY --chown=appuser:appuser app.py .
CMD ["python", "app.py"]

Never Include Secrets in Image

❌ Bad (secrets baked into image):

ENV DATABASE_PASSWORD=secret123
COPY .env .

✅ Good (secrets provided at runtime):

# Pass secrets via environment variables at runtime
# docker run -e DATABASE_PASSWORD=$DB_PASS myapp

✅ Also Good (Docker secrets):

# Use Docker secrets (Swarm/Kubernetes)
CMD ["sh", "-c", "python app.py"]
# Secrets mounted at /run/secrets/

Scan Images for Vulnerabilities

# Using Docker Scout
docker scout cves myapp:latest

# Using Trivy
trivy image myapp:latest

# Using Snyk
snyk container test myapp:latest

Use Specific Image Tags

❌ Bad (unpredictable):

FROM python:latest

✅ Good (reproducible):

FROM python:3.11.9-slim-bookworm

---

Part 5: Build Performance Optimization

BuildKit (Modern Docker Builder)

Enable BuildKit for faster builds:

export DOCKER_BUILDKIT=1
docker build -t myapp .

Benefits:

• Parallel layer building
• Skip unused stages
• Better caching
• 30-50% faster builds

Build Cache Mounts

With BuildKit (cache pip downloads):

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

Benefits:

• Pip packages cached between builds
• No need to clear cache (doesn't bloat image)
• Significantly faster rebuilds

Parallel Multi-Stage Builds

BuildKit automatically parallelizes independent stages:

# Stage 1: Frontend build (runs in parallel)
FROM node:20 AS frontend
WORKDIR /app/frontend
COPY frontend/package*.json ./
RUN npm ci
COPY frontend/ ./
RUN npm run build

# Stage 2: Backend build (runs in parallel)
FROM python:3.11-slim AS backend
WORKDIR /app/backend
COPY backend/requirements.txt .
RUN pip install -r requirements.txt

# Stage 3: Final image (waits for both stages)
FROM python:3.11-slim
COPY --from=frontend /app/frontend/dist /app/static
COPY --from=backend /app/backend /app

---

Part 6: Production Patterns

Health Checks

FROM python:3.11-slim
COPY app.py .

# Add health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

CMD ["python", "app.py"]

Proper Signal Handling

# Use exec form to ensure proper signal handling
CMD ["python", "app.py"]  # ✅ Receives SIGTERM

# Not shell form
CMD python app.py  # ❌ Shell doesn't forward signals

Labels for Metadata

LABEL org.opencontainers.image.title="MyApp"
LABEL org.opencontainers.image.version="1.2.3"
LABEL org.opencontainers.image.authors="[e-mail]"
LABEL org.opencontainers.image.source="https://github.com/org/repo"

---

Part 7: Language-Specific Patterns

Python Optimization

FROM python:3.11-slim AS builder

# Prevent Python from writing pyc files
ENV PYTHONDONTWRITEBYTECODE=1
# Prevent Python from buffering stdout/stderr
ENV PYTHONUNBUFFERED=1

# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .

# Install to user site-packages
RUN pip install --user --no-cache-dir -r requirements.txt

# Runtime stage
FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PATH=/root/.local/bin:$PATH

WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .

RUN useradd -m appuser && chown -R appuser:appuser /app
USER appuser

CMD ["python", "-m", "uvicorn", "app:app", "--host", "0.0.0.0"]

Node.js Optimization

FROM node:20-alpine AS builder

WORKDIR /app
COPY package*.json ./

# Install production dependencies only
RUN npm ci --only=production && \
    # Remove npm cache
    npm cache clean --force

# Runtime stage
FROM node:20-alpine

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodeuser -u 1001

WORKDIR /app
COPY --from=builder --chown=nodeuser:nodejs /app/node_modules ./node_modules
COPY --chown=nodeuser:nodejs . .

USER nodeuser
EXPOSE 3000

CMD ["node", "server.js"]

Go Optimization

# Builder stage
FROM golang:1.21-alpine AS builder

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download

COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .

# Runtime stage - minimal scratch image
FROM scratch

# Copy CA certificates for HTTPS
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy binary
COPY --from=builder /app/main /main

EXPOSE 8080
ENTRYPOINT ["/main"]

Result: 1GB → 15MB (98.5% reduction!)

---

Part 8: Common Mistakes to Avoid

Mistake 1: Installing Recommended Packages

❌ Bad (installs hundreds of unnecessary packages):

RUN apt-get install curl

✅ Good (minimal installation):

RUN apt-get install -y --no-install-recommends curl && \
    rm -rf /var/lib/apt/lists/*

Mistake 2: Using ADD Instead of COPY

❌ Bad (ADD has implicit behavior):

ADD requirements.txt .  # Can extract tarballs, fetch URLs

✅ Good (COPY is explicit):

COPY requirements.txt .  # Only copies files

Mistake 3: Multiple FROM Without AS

❌ Bad (can't reference previous stages):

FROM python:3.11
RUN pip install -r requirements.txt
FROM python:3.11-slim
# Can't copy from previous stage!

✅ Good (named stages):

FROM python:3.11 AS builder
RUN pip install -r requirements.txt
FROM python:3.11-slim
COPY --from=builder /root/.local /root/.local

Mistake 4: Not Using .dockerignore

Without .dockerignore:

• Copies .git directory (50MB+)
• Copies node_modules (100MB+)
• Copies test files
• Invalidates cache on any file change

Mistake 5: Hardcoding Versions Incorrectly

❌ Bad (no control over patch versions):

FROM python:3.11

✅ Good (pin exact version):

FROM python:3.11.9-slim-bookworm

---

Part 9: Before/After Examples

Example 1: Python FastAPI App

Before (1.2GB image, 5min build):

FROM python:3.11
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["uvicorn", "app:app"]

After (140MB image, 2min build):

FROM python:3.11-slim AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends gcc && \
    rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

FROM python:3.11-slim
ENV PATH=/root/.local/bin:$PATH
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY app.py .
COPY src/ ./src/

RUN useradd -m appuser && chown -R appuser:appuser /app
USER appuser

CMD ["uvicorn", "app:app", "--host", "0.0.0.0"]

Results:

• Size: 1.2GB → 140MB (88% reduction)
• Build time: 5min → 2min (60% faster)
• Security: Now runs as non-root
• Cache: Code changes don't rebuild dependencies

---

Part 10: Quick Optimization Checklist

Image Size:

• Use slim or alpine base images
• Multi-stage build (build tools in first stage only)
• Clean up in same layer (apt-get install && rm -rf)
• Use --no-install-recommends with apt-get
• Remove package manager cache (pip --no-cache-dir, npm cache clean)
• Use .dockerignore

Build Speed:

• Order COPY by change frequency
• Copy dependency files before code
• Enable BuildKit
• Use build cache mounts

Security:

• Run as non-root user
• Pin specific image versions
• Scan for vulnerabilities
• Never include secrets in image
• Use minimal base images

Production:

• Add HEALTHCHECK
• Use exec form for CMD
• Add metadata labels
• Proper signal handling
• Set up proper logging

---

Resources

Official Docker Documentation:

• Multi-stage builds: https://docs.docker.com/build/building/multi-stage/
• Best practices: https://docs.docker.com/develop/dev-best-practices/
• BuildKit: https://docs.docker.com/build/buildkit/

Security Scanning:

• Docker Scout: https://docs.docker.com/scout/
• Trivy: https://github.com/aquasecurity/trivy
• Snyk: https://snyk.io/product/container-vulnerability-management/

Base Images:

• Docker Hub: https://hub.docker.com/
• Google Distroless: https://github.com/GoogleContainerTools/distroless
• Alpine Linux: https://alpinelinux.org/