loom-docker

# Docker ## Overview Production-grade containers: Dockerfiles, compose, multi-stage builds, layer/cache optimization, security hardening, registries, debugging. The **Expert Practices** section below is the highest-value part — it states the *mechanism* behind each rule, which is what separates correct-but-naive Dockerfiles from production ones. Read it before writing anything non-trivial. ## Writing Dockerfiles **Base image:** official images; `-alpine`/`-slim`/distroless/`scratch` for runtime; pin by tag AND digest (`python:3.12-slim@sha256:...`, see Currency), never `latest`; multi-stage to drop build toolchains. **Layer ordering (cache):** least-changing first (base → system deps → dependency manifests → source). **Copy dependency manifests and install BEFORE copying source** so a code edit doesn't bust the dependency layer — this single ordering rule is the biggest cache win. **Layer hygiene:** - Keep `apt-get update` and `apt-get install` in the SAME `RUN` — in separate layers Docker reuses a stale cached `update` index when only the install list changes, silently installing outdated/vulnerable packages (docs call this "cache busting"). Details in Anti-Patterns. - `--no-install-recommends`, clean caches in the same layer (`rm -rf /var/lib/apt/lists/*`). - `.dockerignore` to keep context small (excludes `.git`, `node_modules`, build artifacts) — bloated context slows builds and busts cache. **Runtime & security:** non-root numeric `USER uid:gid` (see Gotchas); `--read-only` root fs where possible; secrets via `RUN --mount=type=secret`, never `ENV`/`ARG`/`COPY` (see below); `HEALTHCHECK`; exec-form `CMD`/`ENTRYPOINT` (shell form breaks signals — see Anti-Patterns). > **Why `ARG`/`ENV` are unsafe for secrets:** `ARG` (incl. `--build-arg`) and `ENV` are recorded in image metadata, visible via `docker history --no-trunc`, `docker inspect`, and provenance attestations to anyone who can pull; `ENV` is additionally readable at runtime via `/proc/self/environ`. Unsetting in a later `RUN` does NOT scrub it — already committed in the earlier layer. Use `RUN --mount=type=secret`. `ARG`/`ENV` are safe only for non-sensitive values (versions, build flags). ## Docker Compose - **Deps:** `depends_on` with `condition:` (`service_healthy`, `service_completed_successfully`) — bare `depends_on` only orders start, doesn't wait for readiness. - **Health:** `healthcheck` on every service other services wait on. - **Restart:** `unless-stopped` / `on-failure` for production. - **Networking:** named networks for isolation; databases/caches get no host port mapping; reference services by name (DNS); listen on `0.0.0.0` not `127.0.0.1`. - **Volumes:** named volumes for data; bind mounts for dev code only. - **Config:** `${VAR:-default}` interpolation; `secrets:` (file-backed) with `*_FILE` env convention, not plaintext env. - **Limits:** `deploy.resources.limits` for prod. ## Security Hardening - Minimal base (distroless/alpine) shrinks attack surface. - Drop all Linux capabilities, add back only what's needed; pair with `no-new-privileges` (see Security). - Non-root, read-only root, no `--privileged`. - Scan in CI (trivy, grype, Docker Scout); keep bases current; generate SBOM + provenance (see Security). - Secrets: never in Dockerfile/`ENV`/`ARG`; build-time `--mount=type=secret`; runtime via orchestrator secrets/vault. ## Debugging ```bash docker logs -f --tail 100 <container> # or: docker-compose logs -f app docker exec -it <container> /bin/sh # shell into running container docker run --rm -it --entrypoint /bin/sh img # shell into a broken image docker build --progress=plain --no-cache . # full build output docker diff <container> # filesystem changes vs image ``` Tools: **dive** (layer bloat), **hadolint** (Dockerfile lint), **trivy**/**grype**/**Docker Scout** (CVEs), **ctop** (live resource monitor). For minimal/distroless images that lack a shell, debug via `docker run --entrypoint`, ephemeral debug containers (`kubectl debug`), or `docker cp` files out. Common causes: permission-denied → volume ownership vs container UID; "name not found" → wrong network / used `localhost` instead of service name; missing shared libs → glibc-vs-musl (see Gotchas); container exits immediately → check `docker logs`, verify CMD/ENTRYPOINT executable and required env present. ## Language Notes | Lang | Key points | | --- | --- | | Python | `python:3.x-slim` for glibc; `pip install --no-cache-dir`; wheel-cache in builder; `uv` for speed. | | Node | Copy `package*.json` first; `npm ci` (reproducible) not `install`; ship prod deps via `npm ci --omit=dev` or `npm prune --omit=dev` AFTER build (bundler/TS are devDeps — see Currency); `node:alpine` for size. | | Rust | `cargo chef` for dep caching; `--release` + `strip`; copy only the binary; `scratch`/distroless runtime; target musl for alpine/scratch. | | Go | `golang:alpine` builder; `CGO_ENABLED=0` static binary; `scratch`/distroless runtime; copy only the binary + CA certs. | ## Examples ### Multi-stage Python (glibc, non-root, healthcheck) ```dockerfile FROM python:3.12-slim AS builder WORKDIR /app RUN apt-get update && apt-get install -y --no-install-recommends build-essential \ && rm -rf /var/lib/apt/lists/* COPY requirements.txt . RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements.txt FROM python:3.12-slim AS runtime RUN groupadd --gid 1000 app && useradd --uid 1000 --gid app --create-home app WORKDIR /app COPY --from=builder /app/wheels /wheels RUN pip install --no-cache-dir /wheels/* && rm -rf /wheels COPY --chown=1000:1000 . . USER 1000:1000 # numeric UID:GID for Kubernetes runAsNonRoot EXPOSE 8000 HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ CMD curl -f http://localhost:8000/health || exit 1 CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "4", "app:create_app()"] ``` ### Node with BuildKit cache mounts + prune ```dockerfile # syntax=docker/dockerfile:1 FROM node:20-alpine AS base WORKDIR /app FROM base AS build COPY package*.json ./ RUN --mount=type=cache,target=/root/.npm npm ci # ALL deps: build needs devDeps COPY . . RUN npm run build && npm prune --omit=dev # strip devDeps after build FROM base AS production RUN apk update && apk upgrade && rm -rf /var/cache/apk/* \ && addgroup -g 1001 -S nodejs && adduser -S app -u 1001 COPY --from=build --chown=1001:1001 /app/dist ./dist COPY --from=build --chown=1001:1001 /app/node_modules ./node_modules COPY --from=build --chown=1001:1001 /app/package.json ./ USER 1001:1001 EXPOSE 3000 ENV NODE_ENV=production CMD ["node", "dist/server.js"] ``` ### Rust (cargo chef → distroless) ```dockerfile FROM rust:1.75-alpine AS chef RUN apk add --no-cache musl-dev && cargo install cargo-chef WORKDIR /app FROM chef AS planner COPY Cargo.toml Cargo.lock ./ COPY src ./src RUN cargo chef prepare --recipe-path recipe.json FROM chef AS builder COPY --from=planner /app/recipe.json recipe.json RUN cargo chef cook --release --recipe-path recipe.json # cached dependency layer COPY . . RUN cargo build --release && strip target/release/myapp FROM gcr.io/distroless/cc-debian12 COPY --link --from=builder /app/target/release/myapp /usr/local/bin/myapp USER 65532:65532 # distroless "nonroot"; numeric so runAsNonRoot verifies it EXPOSE 8080 ENTRYPOINT ["/usr/local/bin/myapp"] ``` ### Go (static binary → scratch) ```dockerfile FROM golang:1.22-alpine AS builder WORKDIR /app RUN apk add --no-cache ca-certificates tzdata COPY go.mod go.sum ./ RUN go mod download COPY . . RUN CGO_ENABLED=0 GOOS=linux go build -ldflags='-w -s' -o /app/bin/server ./cmd/server FROM scratch COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY --from=builder /app/bin/server /server USER 65534:65534 # scratch has no /etc/passwd — MUST be numeric ENTRYPOINT ["/server"] ``` ### Compose with secrets, healthcheck, limits ```yaml services: app: build: { context: ., args: { APP_VERSION: "${APP_VERSION:-latest}" } } ports: ["8080:8080"] environment: - DATABASE_URL_FILE=/run/secrets/db_url # *_FILE convention, not plaintext env secrets: [db_url] depends_on: db: { condition: service_healthy } # waits for readiness, not just start restart: unless-stopped deploy: resources: limits: { cpus: "1.0", memory: 512M } db: image: postgres:16-alpine volumes: [postgres_data:/var/lib/postgresql/data] environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password secrets: [db_password] healthcheck: test: ["CMD-SHELL", "pg_isready -U myapp"] interval: 10s timeout: 5s retries: 5 secrets: db_url: { file: ./secrets/db_url.txt } db_password: { file: ./secrets/db_password.txt } volumes: postgres_data: ``` ### .dockerignore ```gitignore .git node_modules __pycache__ *.pyc .venv dist build .env .env.* Dockerfile* docker-compose* *.log .DS_Store ``` ### Multi-arch build (buildx) ```dockerfile FROM --platform=${BUILDPLATFORM} golang:1.22-alpine AS builder ARG TARGETARCH TARGETOS # provided by buildx per target WORKDIR /app COPY . . RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -o /app/bin/server ./cmd/server FROM alpine:3.21 # pin runtime base (never latest); static binary could use scratch COPY --from=builder /app/bin/server /server ENTRYPOINT ["/server"] ``` ```bash docker buildx build --platform linux/amd64,linux/arm64 -t myapp:latest --push . ``` ## Expert Practices: Idioms, Anti-Patterns & Gotchas High-signal guidance. Each item states the mechanism, not just the rule. ### Anti-Patterns **Shell form silently breaks signals and drops CMD.** `CMD gunicorn ...` is wrapped as `/bin/sh -c '<string>'`, so `/bin/sh` becomes PID 1 and (docs) "does not pass signals" — your app never gets SIGTERM from `docker stop`, which waits the full timeout then SIGKILLs (no graceful shutdown, no error). A shell-form `ENTRYPOINT` additionally "ignores any CMD or docker run command line arguments" — a `CMD` beneath it is silently dropped. Use exec form; a setup wrapper must hand off PID 1 with `exec "$@"`. ```dockerfile ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:8000", "app:create_app()"] # process IS PID 1 ``` ```sh #!/bin/sh set -e do_setup exec "$@" # execve replaces the shell so the app becomes PID 1 ``` **Separate `apt-get update` / `install` layers install stale packages.** Docker caches a RUN by its literal command string, so a standalone `RUN apt-get update` is reused even after the upstream index moves; editing only the install line rebuilds against the stale index, silently reintroducing outdated/CVE-laden packages. Keep them in one RUN, pin versions as an extra cache-bust trigger, clean lists in the same layer. ```dockerfile RUN apt-get update && apt-get install -y --no-install-recommends \ curl=7.88.* git \ && rm -rf /var/lib/apt/lists/* ``` ### Gotchas **`set -o pipefail` is needed for piped RUNs — but default `/bin/sh` may lack it.** POSIX `sh` reports only the last command's exit code, so `RUN wget -O- url | sh` exits 0 even when `wget` fails, committing a broken layer. dash (Debian) and ash (Alpine) don't support `-o pipefail`; set it Dockerfile-wide via `SHELL`. ```dockerfile SHELL ["/bin/bash", "-o", "pipefail", "-c"] RUN wget -qO- https://example.com/install.sh | sh ``` **Use a numeric `USER` (uid:gid), not a username.** A username must resolve against `/etc/passwd`, which `scratch` and some distroless images lack — the container fails to start. It also breaks Kubernetes `runAsNonRoot: true`: the kubelet parses the image User field **as a number** and never reads `/etc/passwd`, rejecting a string user ("container has runAsNonRoot and image has non-numeric user"). Always `USER <uid>:<gid>` matching the uid/gid you created. Canonical numbers: `65534` = traditional `nobody`; distroless `nonroot` = `65532`. **`COPY --link --chown` requires numeric UID:GID.** `--link` copies into an isolated layer rebased on an implicit `scratch` with no `/etc/passwd`, so a named user fails with `invalid user index: -1`. Worse, behavior varies by builder: the BuildKit `docker-container` driver errors while the legacy `docker` driver may silently fall back to UID 0. Use numeric `--chown` with `--link`. **Re-declare `ARG` inside each stage — values reset at every `FROM`.** An `ARG` before the first `FROM` is global scope, usable only by `FROM` lines, NOT inside any stage; an in-stage `ARG` is inherited only by stages built `FROM` it. A reference to an undeclared `ARG` expands to an empty string with no error, so builds succeed with a blank value. ```dockerfile ARG VERSION=3.12 FROM python:${VERSION}-slim AS builder ARG VERSION # re-declare to use it inside the stage RUN echo "Building with Python ${VERSION}" ```