| name | metric-decomposition |
| description | Decompose composite metrics into constituent signals, analyze polarity and ceiling effects |
| execution | subagent |
| prompt | ./prompt.md |
| input | metric_name, metric_definition, score_distribution |
| used-by | benchmark-archaeology |
Metric Decomposition SOP
Decompose a benchmark's evaluation metric into its constituent signals to understand what is actually being measured, identify ceiling effects, and detect polarity issues.
Input
- metric_name: Name of the metric (e.g., "F1", "BLEU", "Pass@k", "ELO")
- metric_definition: Mathematical definition or description of how the metric is computed
- score_distribution: Known score distribution data (top scores, median, floor)
Procedure
- Parse metric formula into component signals
- Identify what each component rewards/penalizes
- Analyze ceiling and floor effects
- Check for known pathologies (metric gaming, Goodhart's law instances)
- Assess granularity and discriminative power at current SOTA level
Output
Structured decomposition showing constituent signals, their weights, ceiling effects, and known pathologies.