메뉴
When prompt, tool, or agent logic changes need regression testing → run eval benchmarks to detect regressions or improvements.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
When a task can be parallelized (bulk code review, multi-module refactor, batch analysis) → split into subtasks running in isolated git worktrees via multiple workers.
When the user wants to research academic papers → search ArXiv, extract PDFs, summarize/review/Q&A, generate experiment code.
When you need to interact with web pages using the user's logged-in browser → control Chrome via @playwright/cli daemon, reusing cookies/session with persistent tabs.
When user needs AnyGenIO content generation → unified CLI for task execution with progressive disclosure.
When you need to persist files (reports, docs, evidence) beyond the session → create/query/delete durable artifacts.
When the user wants text-to-speech audio → generate voice files locally (macOS say + afconvert, outputs m4a).
| name | openbench |
| description | When prompt, tool, or agent logic changes need regression testing → run eval benchmarks to detect regressions or improvements. |
| triggers | {"intent_patterns":["openbench|eval|benchmark|基准|评估|测试套件|跑个评测","回归.*测试|regression.*test|性能.*退化|performance.*degradation","prompt.*变了.*测一下|test.*after.*change|验证.*效果","跑.*评估|run.*eval|评测.*结果|eval.*result","对比.*版本|compare.*versions|前后.*差异|before.*after","质量.*评分|quality.*score|准确率|accuracy|通过率|pass.*rate"],"context_signals":{"keywords":["openbench","eval","benchmark","evaluation","评估","regression","回归","准确率","通过率","对比","评测"]},"confidence_threshold":0.7} |
| priority | 8 |
| requires_tools | ["bash"] |
| max_tokens | 200 |
| cooldown | 60 |
| capabilities | ["evaluation","benchmark"] |
| activation_mode | explicit |
| output | {"format":"markdown","artifacts":true,"artifact_type":"document"} |
运行 evaluation/agent_eval 评估套件，输出 pass rate、延迟、质量指标。
go in PATH）# 快速评估（默认 10 cases）
python3 skills/openbench/run.py run
# 指定套件和超时
python3 skills/openbench/run.py run --suite foundation --timeout 600
# 列出可用套件/数据集
python3 skills/openbench/run.py list
# 查看最近一次评估结果
python3 skills/openbench/run.py last
| 参数 | 说明 |
|---|---|
--suite | 评估套件名，默认 quick（对应 scripts/eval-quick.sh） |
--timeout | 每个 case 超时秒数，默认 300 |
--limit | 最多运行 N 个 case，默认 10 |
--output-dir | 结果输出目录，默认 .openmax/bench |
| 套件 | 命令 | 说明 |
|---|---|---|
quick | scripts/eval-quick.sh | 快速回归，~10 cases |
full | alex dev test ./evaluation/... | 完整评估，包含 agent_eval |