Menu
The review-first prompt is selected by default. You can switch to a direct command or download a local copy.
Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/thetahealth/mirobody-eval --skill run-benchmarkThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Use when analyzing ESL-Bench evaluation reports - extracting scores by difficulty dimension, comparing methods, checking fail rates, token costs, duration per query. Triggers on keywords like eslbench report, benchmark results, group by difficulty.
集成外部 benchmark 数据集到 HolyEval 框架 — 从论文/仓库到可执行评测的端到端流程。
Scaffold a new EvalAgent plugin (config + implementation + registration).
Based on SOC occupation classification
Showing SKILL.md
| name | run-benchmark |
| description | Run all benchmark test cases or a filtered subset. |
| argument-hint | <benchmark> <dataset> [options] |
运行 benchmark 跑分，对数据集中的用例批量执行评测并生成报告。
当用户要求运行 benchmark 跑分时使用此 skill。
# 语法: python -m benchmark.basic_runner <benchmark> <dataset> --target-type <type> --target-model <model> [options]
# HealthBench
python -m benchmark.basic_runner healthbench sample --target-type llm_api --target-model gpt-4.1
python -m benchmark.basic_runner healthbench hard --target-type llm_api --target-model gpt-4.1 -p 5
# MedCalc-Bench
python -m benchmark.basic_runner medcalc sample --target-type llm_api --target-model gpt-4.1
python -m benchmark.basic_runner medcalc full --target-type llm_api --target-model gpt-4.1 --limit 50
# Extraction（target 由 metadata 锁定）
python -m benchmark.basic_runner extraction simple
# 组合选项
python -m benchmark.basic_runner healthbench sample --target-type llm_api --target-model gpt-4.1 --limit 10 -p 3 -v
| 选项 | 说明 |
|---|---|
--target-type | 被测系统类型（llm_api / theta_api），必填 |
--target-model | 模型名称（如 gpt-4.1、gemini-3-pro） |
--limit N | 只跑前 N 条用例 |
--ids x,y,z | 指定用例 ID |
-p N | 并发数（默认 1） |
-v | 详细日志 |
benchmark/data/<benchmark>/<dataset>.jsonlbenchmark/report/<benchmark>/<dataset>_<target>_<timestamp>.json| 评测套件 | 数据集 | 数量 | 评估器 |
|---|---|---|---|
| healthbench | sample / hard / consensus / full | 100 ~ 5,000 | healthbench |
| medcalc | sample / full | 5 ~ 1,047 | medcalc |
| extraction | simple | 9 | preset_answer |
也可通过 Web UI 执行跑分，支持实时进度查看：
python -m web
# 访问 http://localhost:8000/tasks 创建任务
命令输出：
benchmark/report/ 目录