container-vuln-testing

name

container-vuln-testing

description

Test container runtime vulnerabilities (runc, crun, containerd, CRI-O) against the full container stack — from direct runtime invocation through Docker, containerd, Kubernetes API, and node-level access. Use when validating whether a container runtime vulnerability is exploitable through each layer of the stack, or when you need to determine which layers block an exploit and which don't. Triggers on tasks involving container escape testing, runc/crun vulnerability validation, container runtime security research, testing security boundaries between Docker/containerd/CRI-O/Kubernetes and the OCI runtime, or determining exploitability of OCI runtime bugs across different deployment contexts. Requires labctl for ephemeral test environments.

Container Vulnerability Testing

Systematically test container runtime vulnerabilities (runc, containerd, CRI-O) against each layer of the container stack to determine real-world exploitability. Uses labctl playgrounds for ephemeral, isolated test environments.

Why Layer-by-Layer Testing Matters

Container runtime vulnerabilities don't exist in isolation. A bug in runc may be:

• Exploitable via direct runc invocation but blocked by Docker's validation
• Blocked by Kubernetes API admission but exploitable from a privileged pod on the node
• Accidentally mitigated by containerd's user resolution logic without anyone realizing it
• Not applicable to CRI-O because it uses crun (a different OCI runtime) instead of runc

Each layer in the stack may independently validate, transform, or block inputs before they reach runc. Testing only one layer gives an incomplete picture. The goal is to map exactly which paths are exploitable and which aren't.

Prerequisites

• labctl installed, on PATH, and authenticated (labctl auth login)
• Use the labctl-playground skill for playground lifecycle management
• Authorization to perform security testing

Testing Strategy

Test from the most direct (closest to runc) to the most abstracted (Kubernetes API):

Layer 1: runc/crun exec directly     (the OCI runtime itself)
Layer 2: containerd ctr / crictl     (container daemon CLI)
Layer 3: Docker CLI / Docker daemon  (full Docker stack)
Layer 4: Kubernetes API              (orchestrator layer)
Layer 5: Kubernetes node-level       (escape back to runtime from inside K8s)

If a vulnerability is blocked at Layer 4, you still need to check Layer 5 — an attacker with node access can bypass the Kubernetes API entirely.

For Kubernetes testing, use k8s-omni with two runtime variants:

• containerd (default): uses runc — labctl playground start k8s-omni
• CRI-O: uses crun — labctl playground start k8s-omni -i 'Container runtime=cri-o'

Test both if the vulnerability targets the OCI runtime spec generically. If it's runc-specific, CRI-O/crun may not be affected (and vice versa).

Phase 1: Docker / containerd / Direct runc

Use a docker playground. This gives you Docker, containerd, ctr, and runc all on a single machine.

Start the environment

PLAYGROUND_ID=$(labctl playground start docker --quiet)

Identify versions

labctl ssh "$PLAYGROUND_ID" --user root -- bash -c '
    echo "Docker:     $(docker --version)"
    echo "containerd: $(containerd --version)"
    echo "runc:       $(runc --version | head -1)"
    echo "Arch:       $(uname -m) ($(getconf LONG_BIT)-bit)"
'

Layer 1: Direct runc exec

runc manages container state under a "root" directory. When Docker is the container manager, this is typically /run/docker/runtime-runc/moby.

# Write a test script and copy it to the playground
cat > /tmp/test_runc.sh << 'SCRIPT'
#!/bin/bash
# Start a container via Docker (easiest way to get a running runc container)
docker run -d --name test-target busybox sleep 3600
FULL_ID=$(docker inspect test-target --format '{{.Id}}')
RUNC_ROOT="/run/docker/runtime-runc/moby"

# Verify runc can see it
runc --root "$RUNC_ROOT" list

# Run your exploit test against the container directly
# Example: runc --root "$RUNC_ROOT" exec --user <value> "$FULL_ID" id
# Replace with your specific vulnerability test

docker rm -f test-target
SCRIPT

labctl cp /tmp/test_runc.sh "$PLAYGROUND_ID":/home/laborant/test_runc.sh
labctl ssh "$PLAYGROUND_ID" -- chmod +x /home/laborant/test_runc.sh
labctl ssh "$PLAYGROUND_ID" --user root -- /home/laborant/test_runc.sh

Key points for direct runc testing:

• Must run as root — runc state directories have restrictive permissions
• Use labctl ssh --user root to run commands as root
• The container ID for runc is the full Docker container ID (64 hex chars), not the short form
• runc exec requires the container to be in "running" state

Layer 2: containerd (ctr)

containerd has its own CLI (ctr) that bypasses Docker but still goes through containerd's API before reaching runc.

labctl ssh "$PLAYGROUND_ID" --user root -- bash -c '
    # Pull an image into containerd default namespace
    ctr images pull docker.io/library/busybox:latest

    # Run a container directly via containerd
    ctr run -d docker.io/library/busybox:latest ctr-test /bin/sleep 3600

    # Test your exploit via ctr
    # Example: ctr task exec --exec-id test1 --user <value> ctr-test id
    # Replace with your specific vulnerability test

    # Cleanup
    ctr task kill ctr-test --signal SIGKILL
    sleep 1
    ctr task delete ctr-test
    ctr containers delete ctr-test
'

Key points for ctr testing:

• ctr has a --user flag for exec but processes it differently than runc's --user
• containerd may do user resolution (mounting rootfs, reading /etc/passwd) before invoking runc
• containerd uses exec-id to track exec sessions — each must be unique per container
• Docker containers live in the moby namespace: ctr -n moby task list

Layer 3: Docker CLI

labctl ssh "$PLAYGROUND_ID" --user root -- bash -c '
    docker run -d --name docker-test busybox sleep 3600

    # Test via Docker CLI
    # Example: docker exec --user <value> docker-test id
    # Replace with your specific vulnerability test

    docker rm -f docker-test
'

Key points for Docker testing:

• Docker daemon adds validation on various fields (e.g., UID range 0-2147483647)
• Docker validation errors come from the daemon, not runc
• Errors at this layer don't mean runc is safe — only that Docker blocks it

Cleanup Phase 1

labctl playground destroy "$PLAYGROUND_ID"

Phase 2a: Kubernetes with containerd

Use a k8s-omni playground (default runtime). This gives a multi-node Kubernetes cluster with containerd and runc.

Start the environment

PLAYGROUND_ID=$(labctl playground start k8s-omni --quiet)

k8s-omni playgrounds have multiple machines: dev-machine, cplane-01, node-01, node-02. Use dev-machine for kubectl:

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- kubectl get nodes -o wide

Layer 4: Kubernetes API

Test whether the Kubernetes API server accepts or rejects the exploit values in pod specs.

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    # Test pod spec with exploit values
    # Example: runAsUser, capabilities, volumes, securityContext fields
    cat <<EOF | kubectl apply -f - 2>&1
apiVersion: v1
kind: Pod
metadata:
  name: exploit-test
spec:
  securityContext:
    runAsUser: <EXPLOIT_VALUE>
  containers:
  - name: test
    image: busybox
    command: ["sleep", "3600"]
  restartPolicy: Never
EOF

    # Check if the API accepted or rejected it
    kubectl get pod exploit-test -o wide 2>&1
    kubectl delete pod exploit-test --force --grace-period=0 2>/dev/null
'

Key points for Kubernetes API testing:

• The API server validates many security-relevant fields (runAsUser, capabilities, etc.)
• Pod Security Standards (PSS) and admission webhooks add more validation
• Even if the API blocks something, the underlying runtime may still be vulnerable via node access

Layer 5: Node-level runc from inside Kubernetes

This is the critical test: can an attacker who has gained node access in a Kubernetes cluster exploit the vulnerability directly?

Step 1: Create a target pod to test against

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: target-pod
spec:
  containers:
  - name: target
    image: busybox
    command: ["sleep", "3600"]
  restartPolicy: Never
EOF
    kubectl wait --for=condition=Ready pod/target-pod --timeout=60s
'

Step 2: Create a privileged pod on the same node to access runc

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    TARGET_NODE=$(kubectl get pod target-pod -o jsonpath="{.spec.nodeName}")

    cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: runc-tester
spec:
  nodeName: $TARGET_NODE
  hostPID: true
  containers:
  - name: tester
    image: busybox
    command: ["sleep", "3600"]
    securityContext:
      privileged: true
    volumeMounts:
    - name: host
      mountPath: /host
  volumes:
  - name: host
    hostPath:
      path: /
  restartPolicy: Never
EOF
    kubectl wait --for=condition=Ready pod/runc-tester --timeout=60s
'

Step 3: Access runc from inside the node namespace

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    kubectl exec runc-tester -- nsenter -t 1 -m -u -i -n -p -- sh -c '\''
        RUNC_ROOT="/run/containerd/runc/k8s.io"

        echo "runc: $(runc --version | head -1)"

        # Find a container with the test binary available
        for CID in $(runc --root $RUNC_ROOT list -q 2>/dev/null); do
            if runc --root $RUNC_ROOT exec $CID /bin/id >/dev/null 2>&1; then
                echo "Target container: $CID"

                # Run your exploit test
                # Example: runc --root $RUNC_ROOT exec --user <value> $CID /bin/id
                break
            fi
        done
    '\''
'

Key points for k8s-omni (containerd) node-level testing:

• runc is at /usr/local/bin/runc and is in $PATH on worker nodes
• The runc state directory is /run/containerd/runc/k8s.io
• Use nsenter -t 1 -m -u -i -n -p to enter the node's full namespace from a privileged pod
• A privileged pod with hostPID: true is required for nsenter to work
• Not all container images have the tools you need — use busybox which includes id, cat, ls, etc.

Cleanup Phase 2a

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    kubectl delete pod target-pod runc-tester --force --grace-period=0 2>/dev/null
'
labctl playground destroy "$PLAYGROUND_ID"

Phase 2b: Kubernetes with CRI-O

Use a k8s-omni playground with the CRI-O runtime. This is essential for testing runc/crun vulnerabilities through the CRI-O code path, which differs from containerd in how it processes container specs before invoking the OCI runtime.

Important: CRI-O uses crun as its OCI runtime, not runc. If your vulnerability is runc-specific, it may not apply here. If it targets the OCI runtime spec more broadly, test both.

Start the environment

PLAYGROUND_ID=$(labctl playground start k8s-omni -i 'Container runtime=cri-o' --quiet)

The -i flag passes interactive parameter overrides. The machine layout is the same as the containerd variant: dev-machine, cplane-01, node-01, node-02.

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- kubectl get nodes -o wide

Identify versions

labctl ssh "$PLAYGROUND_ID" --machine node-01 --user root -- bash -c '
    echo "crun:    $(crun --version | head -1)"
    echo "CRI-O:   $(crictl version 2>/dev/null | grep RuntimeVersion)"
    echo "Arch:    $(uname -m) ($(getconf LONG_BIT)-bit)"
'

Layer 2: CRI-O via crictl

crictl is the CRI client that talks directly to CRI-O, bypassing the Kubernetes API but still going through CRI-O's processing before reaching crun.

labctl ssh "$PLAYGROUND_ID" --machine node-01 --user root -- bash -c '
    # List running pods and containers
    crictl pods
    crictl ps

    # Pull an image via CRI-O
    crictl pull docker.io/library/busybox:latest

    # Run a pod + container via crictl for testing
    # Create pod config
    cat > /tmp/pod-config.json <<PODEOF
{
    "metadata": { "name": "crictl-test-pod", "namespace": "default", "uid": "crictl-test" },
    "log_directory": "/tmp/crictl-test-logs"
}
PODEOF

    # Create container config
    cat > /tmp/container-config.json <<CTREOF
{
    "metadata": { "name": "crictl-test" },
    "image": { "image": "docker.io/library/busybox:latest" },
    "command": ["sleep", "3600"],
    "log_path": "crictl-test.log"
}
CTREOF

    POD_ID=$(crictl runp /tmp/pod-config.json)
    CTR_ID=$(crictl create "$POD_ID" /tmp/container-config.json /tmp/pod-config.json)
    crictl start "$CTR_ID"

    # Test your exploit via crictl exec
    # Example: crictl exec --user <value> "$CTR_ID" id
    # Replace with your specific vulnerability test

    # Cleanup
    crictl stop "$CTR_ID"
    crictl rm "$CTR_ID"
    crictl stopp "$POD_ID"
    crictl rmp "$POD_ID"
'

Key points for crictl/CRI-O testing:

• crictl talks to CRI-O via the CRI gRPC API — it's the equivalent of ctr for containerd
• CRI-O may validate or transform fields differently than containerd before invoking crun
• Use crictl exec for exec-based vulnerability tests
• CRI-O processes user specs through its own logic before passing to the OCI runtime

Layer 4: Kubernetes API (CRI-O)

The Kubernetes API layer works the same as Phase 2a — use the same pod spec tests. The difference is what happens after the API accepts the spec: CRI-O processes it instead of containerd.

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    cat <<EOF | kubectl apply -f - 2>&1
apiVersion: v1
kind: Pod
metadata:
  name: exploit-test
spec:
  securityContext:
    runAsUser: <EXPLOIT_VALUE>
  containers:
  - name: test
    image: busybox
    command: ["sleep", "3600"]
  restartPolicy: Never
EOF

    kubectl get pod exploit-test -o wide 2>&1
    kubectl delete pod exploit-test --force --grace-period=0 2>/dev/null
'

Layer 5: Node-level crun from inside Kubernetes (CRI-O)

Same privileged-pod approach as Phase 2a, but targeting crun instead of runc.

Step 1: Create target and tester pods (same as Phase 2a)

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: target-pod
spec:
  containers:
  - name: target
    image: busybox
    command: ["sleep", "3600"]
  restartPolicy: Never
---
apiVersion: v1
kind: Pod
metadata:
  name: runtime-tester
spec:
  nodeName: node-01
  hostPID: true
  containers:
  - name: tester
    image: busybox
    command: ["sleep", "3600"]
    securityContext:
      privileged: true
    volumeMounts:
    - name: host
      mountPath: /host
  volumes:
  - name: host
    hostPath:
      path: /
  restartPolicy: Never
EOF
    kubectl wait --for=condition=Ready pod/target-pod pod/runtime-tester --timeout=60s
'

Step 2: Access crun from inside the node namespace

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    kubectl exec runtime-tester -- nsenter -t 1 -m -u -i -n -p -- sh -c '\''
        CRUN="/usr/bin/crun"
        CRUN_ROOT="/run/crun"

        echo "crun: $($CRUN --version | head -1)"

        # List running containers
        $CRUN --root $CRUN_ROOT list 2>/dev/null

        # Find a container with the test binary available
        for CID in $($CRUN --root $CRUN_ROOT list -q 2>/dev/null); do
            if $CRUN --root $CRUN_ROOT exec $CID /bin/id >/dev/null 2>&1; then
                echo "Target container: $CID"

                # Run your exploit test
                # Example: $CRUN --root $CRUN_ROOT exec --user <value> $CID /bin/id
                break
            fi
        done
    '\''
'

Key points for k8s-omni (CRI-O) node-level testing:

• CRI-O uses crun (/usr/bin/crun), not runc — runc-specific bugs may not apply
• The crun state directory is /run/crun
• crun is a C implementation of the OCI runtime spec, while runc is Go — they handle edge cases differently
• The same nsenter technique works: privileged pod with hostPID: true
• If testing a vulnerability that affects the OCI runtime spec generically, compare results between Phase 2a (runc) and Phase 2b (crun)

Cleanup Phase 2b

labctl ssh "$PLAYGROUND_ID" --machine dev-machine -- bash -c '
    kubectl delete pod target-pod runtime-tester --force --grace-period=0 2>/dev/null
'
labctl playground destroy "$PLAYGROUND_ID"

Practical Tips

Script management

Write test scripts locally, copy them to the playground, and run them. This is more reliable than passing complex inline bash through labctl ssh:

labctl cp ./my_test.sh "$PLAYGROUND_ID":/home/laborant/my_test.sh
labctl ssh "$PLAYGROUND_ID" -- chmod +x /home/laborant/my_test.sh
labctl ssh "$PLAYGROUND_ID" --user root -- /home/laborant/my_test.sh

The default user is laborant (home: /home/laborant). Use --user root for root access.

Handling errors

• Test scripts should use set +e (or no set -e) so they continue past individual test failures
• Capture both stdout and stderr: result=$(command 2>&1)
• Always record exit codes: echo "Exit: $?"
• Wrap each test in a function for clean output

Common pitfalls

Problem	Solution
runc list permission denied	Use --user root with labctl ssh
Container ID not found by runc	Use full 64-char Docker ID, not short form
/bin/id not found in container	Use busybox image instead of alpine
ctr exec-id conflict	Use unique exec-id strings for each exec call
bash -c not passing through labctl ssh	Write a script file, copy it, run it
CRI-O uses crun not runc	Check which OCI runtime the cluster uses before testing runc-specific bugs

Recording results

For each vulnerability test, record a table like this:

Attack Path	Exploitable?	Validated?	Notes
runc direct (version X.Y.Z)	YES/NO	Confirmed/Not tested	Details
docker exec	YES/NO	Confirmed/Not tested	Details
ctr task exec	YES/NO	Confirmed/Not tested	Details
K8s API (containerd)	YES/NO	Confirmed/Not tested	Details
K8s node-level runc (containerd)	YES/NO	Confirmed/Not tested	Details
crictl exec (CRI-O)	YES/NO	Confirmed/Not tested	Details
K8s API (CRI-O)	YES/NO	Confirmed/Not tested	Details
K8s node-level crun (CRI-O)	YES/NO	Confirmed/Not tested	Details

This makes it clear which layers provide mitigation and which don't.

Reference: Container Stack Architecture

┌─────────────────────────────────────┐
│  Kubernetes API Server              │ ← Layer 4: API validation
│  (admission, PSS, webhooks)         │   (runAsUser range, capabilities, etc.)
├─────────────────────────────────────┤
│  kubelet                            │
├─────────────────────────────────────┤
│  CRI (containerd / CRI-O)          │ ← Layer 2/3: Runtime validation
│  (user resolution, image pull,      │   (containerd user lookup, Docker
│   sandbox creation)                 │    daemon UID range check, etc.)
├─────────────────────────────────────┤
│  OCI Runtime (runc / crun)          │ ← Layer 1: The runtime itself
│  (namespace setup, cgroup config,   │   (often the weakest validation)
│   exec, signal, delete)             │
├─────────────────────────────────────┤
│  Linux Kernel                       │ ← Layer 0: Kernel enforcement
│  (namespaces, cgroups, seccomp,     │   (uid_map, capabilities, etc.)
│   LSMs, UID/GID enforcement)        │
└─────────────────────────────────────┘

A vulnerability in the OCI runtime (Layer 1) may be blocked by any layer above it, or by the kernel below it. The kernel is the ultimate enforcer — even if runc/crun passes a bad value, the kernel may reject it (e.g., unmapped UIDs in user namespaces). Full testing covers all layers and both runtimes where applicable.

Reference: OCI Runtime State Directories

Container Manager	OCI Runtime	Root Directory
Docker	runc	/run/docker/runtime-runc/moby
k8s-omni (containerd)	runc	/run/containerd/runc/k8s.io
k8s-omni (CRI-O)	crun	/run/crun
containerd (default ns)	runc	/run/containerd/runc/default
K3s (containerd CRI)	runc	/run/containerd/runc/k8s.io
Podman	runc	/run/user/<uid>/runc (rootless) or /run/runc (rootful)

Reference: OCI Runtime Binary Locations

Distribution	Runtime	Path
Docker (apt/yum)	runc	/usr/bin/runc or /usr/sbin/runc
k8s-omni (containerd)	runc	/usr/local/bin/runc
k8s-omni (CRI-O)	crun	/usr/bin/crun
K3s (bundled)	runc	/var/lib/rancher/k3s/data/<hash>/bin/runc
containerd (standalone)	runc	/usr/bin/runc or /usr/local/bin/runc
RKE2 (bundled)	runc	/var/lib/rancher/rke2/data/<hash>/bin/runc