メニュー
Scan a repo or diff for "vibe-coded" sections — code that lacks tests, evals, logging, error handling, or has hardcoded prompts/magic numbers. Karpathy warning lens. Triggers on "vibe check", "audit for vibes", "is this vibe-coded".
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Visualize an agent's perceive→think→act loop. Inspects tool calls, latency per step, decision branches, retry patterns. Surfaces bottlenecks. Triggers on "trace agent", "why is agent slow", "agent loop analysis", "what did the agent do".
Compute context window budget for a prompt/agent — sums system prompt, tool descriptions, examples, conversation history; flags overruns; suggests cuts. Triggers on "context budget", "how much context", "will this fit", "budget my prompt".
Treat dataset / eval JSONL as code — lint, validate schema, detect duplicates, leakage between train/eval, label imbalance, PII. Karpathy "Software 2.0" lens. Triggers on "audit dataset", "lint jsonl", "check for leakage", "validate eval data".
Enforce "evals over vibes" — before any prompt, agent, or model change, require an eval set with success criteria and baseline measurement. Blocks vibe-only changes. Triggers on "tune prompt", "improve agent", "change model" without prior eval set.
Build a 10-30 case eval dataset from a PRD, skill, or prompt — covering golden path, edge cases, and adversarial inputs. Output as JSONL with input/expected/category. Triggers on "build eval set", "create evals", "generate test cases for prompt".
Break down any ML/AI concept to first principles — strip jargon, derive from basics, build physical/mathematical analogy. Karpathy intuition-building style. Triggers on "first principles", "explain like I'm 5", "derive from scratch", "why does X actually work".
| name | vibe-detector |
| description | Scan a repo or diff for "vibe-coded" sections — code that lacks tests, evals, logging, error handling, or has hardcoded prompts/magic numbers. Karpathy warning lens. Triggers on "vibe check", "audit for vibes", "is this vibe-coded". |
Karpathy: vibe coding is fine for hobby, dangerous in prod.
| Signal | Points |
|---|---|
| Function >50 lines no test | +2 |
| Prompt string hardcoded inline | +3 |
| Magic numbers (no const) | +1 |
| No error handling on API call | +2 |
| No logging in agent loop | +2 |
| Eval missing for AI feature | +3 |
LLM call with temperature unspecified | +1 |
| TODO / FIXME / "for now" | +1 each |
| Commented-out code blocks | +1 |
any / # type: ignore clusters | +2 |
Vibe report — <path>
Score: 14 (HIGH — refactor before ship)
Hotspots:
src/agent.py:42 hardcoded prompt (3 pts)
src/agent.py:88 no error handling on openai.chat (2 pts)
src/eval/ MISSING — AI feature has no evals (3 pts)