القائمة
Analyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
Use test-driven development for behavior-changing feature or fix work, and whenever the user mentions TDD, test-first, red-green-refactor, tracer bullets, integration tests, or public-interface behavior tests. Skip for docs-only, path-only rename, formatting-only, or purely mechanical chores unless explicitly requested.
Build command-only adapters for `eval` run mode using strict stdin/stdout JSON contracts.
End-to-end eval-suite orchestration with the `eval` command: run -> grade -> compare -> calibrate, using strict JSON contracts and schema discovery.
| name | compare-trials |
| description | Analyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing. |
| license | ISC |
Use this skill for analysis over graded JSONL outputs, typically after:
agent-eval-harness eval '{"mode":"grade",...}' > graded.jsonl
For normal baseline-vs-challenger comparisons, use:
agent-eval-harness eval '{"mode":"compare",...}'
Use custom scripts only when suite-specific analysis is required.
When writing a custom script:
trial_result.pass and score non-null).taskId, trialIndex, runId).exactPassAtK naming for combinatorics-based pass@k metrics (do not label exact values as estimatedPassAtK).Reference helpers remain in references/ for bootstrap/statistics utilities.