メニュー
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Persistent dietary safety layer for DoorDash CLI (dd-cli) ordering. Stores a personal/household dietary profile (allergens with severity tiers, diets, dislikes) that every cart is vetted against before checkout, with a deterministic tripwire — the vetting step saves the full cart contents to a vetted-cart dump, and a PreToolUse hook re-greps that dump against the anaphylaxis-severity allergen list before any checkout URL is allowed. Use when the user mentions allergies or dietary restrictions, when ordering food for someone with restrictions, or on every dd-cli cart flow while this skill is installed. A tripwire, not medical-grade — the human checkout page is the final check.
Group food ordering through the DoorDash CLI (dd-cli) from a persistent team roster. One request ("lunch for the team") fans out into a single merged cart with every line attributed to its eater via a person-to-cart-item-id ledger, per-person cost split with fee proration, payer rotation history, and a checkout gate that re-derives allergen conflicts against the roster live. Use when ordering for multiple people — team lunch, incident-response food, "collect orders from the thread" — or for /whose-turn payer rotation questions. Handles paste-a-thread intake: paste a Slack/chat thread and it builds the order ledger from it.
Accountability layer for agent-driven DoorDash ordering. Works with the doordash-audit-log hook (which appends every dd-cli invocation to an append-only audit log) and the /doordash-report command to answer "what has my AI been ordering and what did it cost" from data instead of memory. Use when the user asks about their DoorDash spending, ordering patterns, what the agent did in past sessions, or wants jq recipes for querying the audit log. Covers log schema, query patterns, rotation, and privacy guidance.
Named, context-bound saved DoorDash orders ("post-gym", "late-night deploy") recalled through the DoorDash CLI (dd-cli) with a mandatory cart-diff before any checkout link is handed over. Use when the user names a saved order ("order my post-gym bowl", "the usual", "my Friday ramen"), wants to save the order they just placed as a playbook, or asks to list/remove saved orders. Persists name→order-uuid playbooks across sessions, catches silent menu/price drift on every recall, and self-heals stale playbooks when a restaurant's menu changes. Requires dd-cli (macOS Apple Silicon, waitlist-gated).
Hard spending policy for agent-driven DoorDash ordering through the DoorDash CLI (dd-cli). Per-order ceiling, daily/weekly/monthly caps, cooldown between orders, and blocked hours — enforced deterministically by routing every cart-mutation and checkout through the dd-guard wrapper script, which prices the cart, checks the policy against a persistent spend ledger, and refuses out-of-policy checkouts with exit code 2. Use when the user wants budget limits on agent food ordering, asks "how much have I spent on DoorDash", wants to set spending caps, or whenever building carts / checking out with dd-cli while this skill is installed. Pairs with a PreToolUse hook that denies raw dd-cli checkout calls that bypass the wrapper.
Segment a SKU portfolio on value (ABC) and demand variability (XYZ), produce the 9-box with a planning policy per cell, and reallocate planner attention accordingly. Use when the user mentions ABC analysis, inventory segmentation, SKU rationalization, stok segmentasyonu, envanter sınıflandırma, or asks which items deserve forecasting effort. Differentiator - ABC alone is treated as half an answer; policy lives in the value x variability combination.
| name | agent-evaluation |
| description | Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent. |
| source | vibeship-spawner-skills (Apache 2.0) |
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
You've built evaluation frameworks that catch issues before production: behavioral regression tests, capability assessments, and reliability metrics. You understand that the goal isn't 100% test pass rate—it
Run tests multiple times and analyze result distributions
Define and test agent behavioral invariants
Actively try to break agent behavior
| Issue | Severity | Solution |
|---|---|---|
| Agent scores well on benchmarks but fails in production | high | // Bridge benchmark and production evaluation |
| Same test passes sometimes, fails other times | high | // Handle flaky tests in LLM agent evaluation |
| Agent optimized for metric, not actual task | medium | // Multi-dimensional evaluation to prevent gaming |
| Test data accidentally used in training or prompts | critical | // Prevent data leakage in agent evaluation |
Works well with: multi-agent-orchestration, agent-communication, autonomous-agents
