القائمة
Run current eval suite vs stored baseline. Reports per-case delta, aggregate score, and flags regressions. Triggers on "run regression eval", "did this regress", "compare to baseline".
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Visualize an agent's perceive→think→act loop. Inspects tool calls, latency per step, decision branches, retry patterns. Surfaces bottlenecks. Triggers on "trace agent", "why is agent slow", "agent loop analysis", "what did the agent do".
Compute context window budget for a prompt/agent — sums system prompt, tool descriptions, examples, conversation history; flags overruns; suggests cuts. Triggers on "context budget", "how much context", "will this fit", "budget my prompt".
Treat dataset / eval JSONL as code — lint, validate schema, detect duplicates, leakage between train/eval, label imbalance, PII. Karpathy "Software 2.0" lens. Triggers on "audit dataset", "lint jsonl", "check for leakage", "validate eval data".
Enforce "evals over vibes" — before any prompt, agent, or model change, require an eval set with success criteria and baseline measurement. Blocks vibe-only changes. Triggers on "tune prompt", "improve agent", "change model" without prior eval set.
Build a 10-30 case eval dataset from a PRD, skill, or prompt — covering golden path, edge cases, and adversarial inputs. Output as JSONL with input/expected/category. Triggers on "build eval set", "create evals", "generate test cases for prompt".
Break down any ML/AI concept to first principles — strip jargon, derive from basics, build physical/mathematical analogy. Karpathy intuition-building style. Triggers on "first principles", "explain like I'm 5", "derive from scratch", "why does X actually work".
| name | regression-eval |
| description | Run current eval suite vs stored baseline. Reports per-case delta, aggregate score, and flags regressions. Triggers on "run regression eval", "did this regress", "compare to baseline". |
evals/*.jsonl).evals/baseline.json with scores).type).evals/run_<timestamp>.json.Eval: feature_x (n=24)
Baseline Current Δ
Golden (10) 1.00 1.00 0
Edge (8) 0.75 0.88 +0.13 ✓
Adversarial(4) 1.00 0.75 -0.25 ✗ REGRESSION
Overall 0.92 0.92 0
Regressions: a02 (injection), a04 (off-topic)
--update-baseline flag.