메뉴
Structured incident response and root-cause analysis for production failures. Use when the user reports a production problem - crashes, latency spikes, OOM, hangs, wrong routing behavior - especially with logs, metrics, or benchmark evidence.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | incident |
| description | Structured incident response and root-cause analysis for production failures. Use when the user reports a production problem - crashes, latency spikes, OOM, hangs, wrong routing behavior - especially with logs, metrics, or benchmark evidence. |
| argument-hint | ["one-line-description"] |
Summary: $ARGUMENTS
Severity: [P0-Critical / P1-High / P2-Medium / P3-Low]
Symptoms / Timeline / Affected scope: fill in from the user's report; ask for what's missing:
Ref:
.dev-context/claude-context.mdfor build constraints, architecture, coding conventions, and the Hard Rules.
| # | Hypothesis | Supporting Evidence | Quick Test |
|---|---|---|---|
| 1 | [hypothesis] | [evidence] | [how to verify] |
Discipline: do not write a code fix until the evidence points to one hypothesis. If two hypotheses survive, design the discriminating experiment first (see the A/B methodology in /benchmark and .dev-context/next-benchmark-checklist.md for a worked example).
| Failure Mode | Symptoms | Check | Rule # |
|---|---|---|---|
| Reactor stall | High latency, timeouts, "Reactor stalled for N ms" logs | Blocking calls, unmarked seastar::async, loops without maybe_yield | #1, #12, #17 |
| Memory exhaustion | OOM, growing RSS | Unbounded queues; pinned temporary_buffer shares | #4, #23 |
| Cross-shard deadlock | Hanging requests | smp::submit_to patterns, semaphore units vs capacity | #0, #19 |
| Use-after-free | Random segfaults, often under load only | Timer/callback lifetimes, gate holder scope, lambda coroutines, do_with copies | #5, #6, #16, #20, #21 |
| Allocator corruption | free(): invalid pointer, SIGSEGV with garbage addresses | Cross-shard heap moves; FFI without local reallocation | #14, #15 |
| Connection exhaustion | Connection refused | Pool sizing, leaks | #4 |
| Future leak | "Exceptional future ignored" logs | Missing co_await / discarded futures | #18 |
| Routing degradation | Cache-hit ratio drops, TTFT regression, affinity thrashing | Route learn/evict counters, load-aware + residency override interplay (route_scorer.hpp); see .dev-context/investigation-289-routing-regression.md and the May 22 follow-ups | — |
:9180 Prometheus; reactor stall counters, ranvier_* counters)git log --oneline --since=...)Root Cause: [description]
Affected Component: [file/service/function]
Failure Mode: [which pattern from Phase 3]
Rule Violated: [#N or "New pattern"]
Action: [what to do right now — config change, rollback, disable feature flag]
Expected Result: [what should improve]
// Before (problematic)
[code]
// After (fixed)
[code]
.dev-context/investigations/<topic>.md: symptoms, hypotheses tested with evidence, reproduction steps, current best explanation, next experiments. Follow investigation-289-routing-regression.md as the exemplar — these files are how the next session picks up where you stopped./extract-pattern to add it to the Hard Rules./doc)Hostile security and robustness audit of src/ assuming malicious traffic and 100x load. Use before releases, after large feature merges, or when the user asks for a security audit, adversarial review, or "what breaks under attack".
Holistic system audit for architecture drift, Hard Rules compliance, and async integrity across recently changed code. Use after completing a batch of tasks, before a release, or when the user asks for a health check or audit.
Semantic-correctness audit of hot-path code - enumerate the invariants a component must maintain, then hunt code paths that violate them (wrong answers, not crashes). Use after changing routing, eviction, accounting, or counter logic, before a release, or when metrics drift without any crash.
Orientation for a session starting work on this codebase - reading order, where things live, which skill to use for which task, and the non-negotiable constraints. Use at the start of a session, when unsure where to look, or when the user asks how to approach a task here.
Plan, run, and interpret Locust load benchmarks (mock-backend and real-GPU) including A/B routing experiments. Use when the user wants to measure cache-hit ratio, TTFT, or throughput, compare routing modes, chase a benchmark regression, or design a benchmark experiment.
Triage build or runtime failures. Use when the user pastes a compile error, link error, runtime crash, test failure, or error log, or mentions a build failure.