القائمة
جمع SkillsMP عدد ١١ من skills من opendatahub-io/agent-eval-harness. افتح أي skill لمراجعة مصدره وتفاصيله.
عرض ١١ من أصل ١١ skills مجمعة.
Generate eval.yaml for the agent eval harness. Two modes - (1) Skill-based - examines SKILL.md, sub-skills, scripts, test cases to verify implementation quality, OR (2) Prompt-based - tests agent capabilities using custom analysis prompts (documentation…
لغة النص الأصلي: الإنجليزية
Generate evaluation test cases for an eval.yaml. Sources cases per generation.strategy - skill analysis (default), synthetic LLM generation from generation prompts (documentation and agent-capability evals), or MLflow production traces. Bootstraps a starter…
لغة النص الأصلي: الإنجليزية
Execute an evaluation against test cases (skill or prompt mode), score with judges, and report results. Requires eval.yaml (generated by /eval-analyze). Use when the user wants to test a skill, run eval, benchmark, compare models, detect regressions, check…
لغة النص الأصلي: الإنجليزية
MLflow integration for evaluation — sync datasets, log run results, push/pull feedback between the harness and MLflow traces. Use when the user wants to log eval results to MLflow, sync test cases to MLflow datasets, connect judge scores to traces, pull…
لغة النص الأصلي: الإنجليزية
Run Design-of-Experiments (DoE) evaluations with ANOVA over a matrix of agent configurations — comparing models, thinking-effort levels, prompts, or other factors across shared test cases, with repeated-measures / mixed-effects statistics that account for…
لغة النص الأصلي: الإنجليزية
Evaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user…
لغة النص الأصلي: الإنجليزية
Compare evaluation results across multiple models or runs. Takes a directory of eval run artifacts (summary.yaml, run_result.json, HTML reports) and produces a tabbed HTML comparison report with model cards, quality/cost tables, per-case breakdowns, and…
لغة النص الأصلي: الإنجليزية
Automated skill improvement loop. Runs eval, identifies judge failures, reads traces and rationale, edits the SKILL.md to fix issues, re-runs to verify, and checks for regressions. Use when the user wants to automatically improve a skill based on eval…
لغة النص الأصلي: الإنجليزية
Interactive review of evaluation results. Presents judge scores and skill outputs for human feedback, then proposes SKILL.md improvements based on what the user identifies. Use when the user wants to review eval results, look at results, check scores, see…
لغة النص الأصلي: الإنجليزية
Optional environment configurator for the agent-eval-harness. Configures MLflow tracking, verifies API keys, and troubleshoots dependency issues. Detects available skills and agentic documentation (CLAUDE.md, AGENTS.md, ai-docs/) to suggest appropriate…
لغة النص الأصلي: الإنجليزية
Query Jira for bugs in a project and summarize coverage gaps. Test fixture for e2e external-state field detection.
لغة النص الأصلي: الإنجليزية