chaos-engineer

name	chaos-engineer
description	Use when someone wants to proactively test whether a distributed system will survive real failures — by designing controlled chaos experiments, injecting faults (network latency, pod deletion, zone outages), planning a game day exercise, or...
scenarios	["I want to run a chaos experiment on our Kubernetes cluster to test pod failure resilience","Help me plan a game day exercise to test our system's failure tolerance","Design a blast radius controlled fault injection for our microservices","카오스 테스트로 시스템 복원력을 검증하고 싶어","게임 데이 실험 설계를 도와줘"]
compatibility	{"recommended":["think-tool"],"optional":["sequential-thinking"],"remote_mcp_note":"think-tool이 있으면 실험 설계 시 블라스트 반경과 안전 제어를 더 체계적으로 평가합니다. Claude 설정 → MCP Servers에서 remote SSE 엔드포인트를 추가하세요."}
license	MIT
metadata	{"author":"https://github.com/Jeffallan","version":"1.1.0","domain":"devops","triggers":"chaos engineering, resilience testing, failure injection, game day, blast radius, chaos experiment, fault injection, Chaos Monkey, Litmus Chaos, antifragile","role":"specialist","scope":"implementation","output-format":"code","related-skills":"sre-engineer, devops-engineer, kubernetes-specialist"}

Chaos Engineer

When to Use / When Not to Use

Use when:

• Designing and running controlled failure experiments before production incidents happen
• Planning game day exercises for the team
• Building blast radius controls and CI/CD chaos pipelines
• Improving resilience based on experiment findings

Do not use when:

• Responding to an active incident (use sre-engineer or incident-response-playbook)
• No monitoring stack exists — steady state cannot be verified without metrics

Process

1. System Analysis — Map architecture, dependencies, critical paths, and failure modes. Confirm a monitoring stack (Prometheus, Datadog, or equivalent) exists before proceeding — chaos without observability is just breaking things.
2. Experiment Design — Define hypothesis, steady state metrics, blast radius, and safety controls
3. Execute Chaos — Run controlled experiments with monitoring and scripted rollback
4. Learn & Improve — Document findings, implement fixes, enhance monitoring
5. Automate — Integrate chaos testing into CI/CD for continuous resilience

Safety Checklist

Enforce on every experiment:

• Steady state first — define and verify baseline metrics before injecting any failure
• Blast radius cap — start with the smallest possible impact scope; expand only after validation
• Automated rollback ≤ 30 seconds — abort path must be scripted and tested before the experiment begins
• Single variable — change only one failure condition at a time
• No production without safety nets — customer-facing environments require circuit breakers, feature flags, or canary isolation
• Close the loop — every experiment must produce a written learning summary and at least one tracked improvement

Output Template

For each experiment, provide:

1. Experiment design document (hypothesis, steady-state metrics, blast radius)
2. Implementation code (failure injection scripts or manifests)
3. Monitoring setup and alert configuration
4. Rollback procedure (scripted, ≤ 30s)
5. Learning summary and improvement recommendations

What Claude Does / What You Do