stress-test

name	stress-test
description	Generates comprehensive smoke and stress tests for infrastructure components (sandbox runtimes, container orchestration, firewall rules, etc.), prioritizing end-to-end realism that catches real setup issues across platforms. Activate when the user asks to "stress test", "smoke test", "test the sandbox", "verify isolation", or wants comprehensive infrastructure testing for a component.

Stress Test Skill

Generate smoke and stress tests for infrastructure components (sandbox runtimes, network isolation, container orchestration, entrypoint hardening, cross-platform setup). Prioritize end-to-end realism over unit-test purity — the goal is catching setup issues that surface on real platforms (Linux, macOS Colima/OrbStack, WSL2), not exercising code paths in isolation. Also fires on "make sure this works on macOS/Linux/WSL2".

General test-quality rules — test behavior not source text, non-vacuity, parametrize repeated patterns, no duplicate coverage, fixtures for repeated I/O — come from the writing-tests skill; follow it. This skill adds the infra-specific shape on top.

Principles

1. Two tiers, always both. Static config validation (pytest, no Docker, runs everywhere) + live runtime checks (bash, needs Docker, proves isolation actually works).
2. Assert the operation, not a keyword. assert "chown -R root:root" in content catches the real behavior; assert "chown" in content also matches a comment. Prefer the invariant/relationship (e.g. "proxy env vars reference the firewall's IP", read from both sides) over a hardcoded value that breaks on a legitimate config change.
3. Guard empty-vs-empty. When comparing values that could be empty (namespace readlinks), assert non-empty first — an empty-vs-empty comparison silently passes.
4. Bash checks are self-contained. Each check passes/fails/warns independently via a FAILURES counter, not set -e (which kills the whole script on one non-critical failure). Print PASS:/FAIL: prefixes for grep-ability.
5. Test what breaks per platform. macOS: volume mounts (virtiofs/9p) and runtime install (Colima SSH). Linux: KVM availability and cgroup drivers. WSL2: nested virtualization.

Workflow

1. Identify the component and inventory existing tests

Determine what's under test (runsc/kata runtime, firewall/squid/dnsmasq, compose lifecycle, entrypoint hardening, setup scripts). Search tests/ and bin/check-*.bash for existing coverage and list it, so you extend rather than duplicate.

2. Static config tests (pytest)

Parse config files directly to catch misconfigurations without Docker: compose YAML → capabilities, network topology, resource limits, env vars; domain allowlist → access modes, no wildcards, no raw IPs; entrypoint/firewall scripts → hardening steps present; setup.bash → platform-detection logic. Parametrize service-level properties that repeat across containers.

3. Live runtime tests (bash)

Exercise the actual runtime to prove isolation end-to-end (needs Docker):

• Container starts with the sandbox runtime
• Process isolation: /proc shows only container processes
• Device isolation: no host devices visible
• Network isolation: internal network blocks egress
• Filesystem: read-only enforcement holds
• Volume mounts: bind mounts work (critical on macOS)
• Capabilities: CapEff=0 under --cap-drop=ALL

4. CI workflow

Two jobs: static tests on every PR (fast, no Docker); runtime tests on sandbox-related path changes (needs Docker, installs the runtime). Keep paths filters consistent across push and pull_request; always set timeout-minutes.

5. Critique-fix loop

Re-check each Principle against what you actually wrote (not what you intended), plus: missing coverage (an important invariant or platform-specific failure mode untested) and fragile bash (A && B || C instead of if-then-else, missing cleanup for Docker networks/temp dirs). Fix each, then start a fresh pass — fixes introduce their own bugs. Stop when a full pass finds nothing worth changing; cap at 3 passes, then list what remains and ask. Skip for a trivial one-test addition — say so explicitly.

Example: "Stress test the runsc setup"

1. Search tests/ — runsc runtime defaults already covered in the wrapper tests.
2. Add a pytest file with parametrized config checks (caps, limits, network, proxy).
3. Add a bin/check-*-smoke.bash with live isolation checks (process, device, network, mount, filesystem).
4. Add a workflow with static + runtime jobs.
5. Run pytest locally; self-critique removes 3 duplicates of the wrapper tests and parametrizes 8 similar tests into 2.