菜单
Evaluate whether benchmark measures its claimed capability
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | construct-validity-assessment |
| description | Evaluate whether benchmark measures its claimed capability |
| execution | subagent |
| prompt | ./prompt.md |
| input | benchmark_name, claimed_capability, task_examples |
| used-by | benchmark-archaeology |
Evaluate whether a benchmark actually measures the capability it claims to measure, using psychometric validity frameworks adapted for AI evaluation.
Validity verdict with evidence for each validity dimension.
SOTA Performance Baseline Campaign — 5 strategies for systematically collecting, standardizing, and analyzing performance data across methods. Produces standardized comparison tables, progress curves, and headroom analysis.
Assess systematic biases in the evidence body — publication bias, reporting bias, and selective outcome reporting. Budget: 40 studies, 40 effect sizes, 40 web searches.
Track evidence accumulation over time — cumulative meta-analysis protocol design. Budget: 40 studies, 40 effect sizes, 30 web searches.
Design structured data extraction form for systematic meta-analysis data collection
Systematically extract effect sizes and conditions from papers for meta-analytic synthesis
Determine effect size types and calculation methods for meta-analytic synthesis
