Menu
Estimate theoretical/practical ceiling vs current SOTA gap
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Basé sur la classification professionnelle SOC
Append research process and results to the current Phase's context file. Each append MUST contain >=500 lines of markdown covering both process and results. Use this skill at plan-designated checkpoint points — typically after each strategy completes or at key decision nodes within a research Phase.
Create a new context file for a research Phase. Called once at Phase start to initialize the file that subsequent context-checkpoint calls will append to. Use this skill whenever a new research Phase begins and a fresh context file is needed.
The optimizer brain for the ladder-foundry pretraining loop. Runs the two-level nested batch loop, delegates gating to gate_eval, attributes a failing batch to one weight (attribute-first), and recovers from disk after compaction. Control flow is fully scripted; only the backprop attribution is a judgment call.
Loss-2 judge (codex role). Over one topic's 6 shuffled research-design samples, pairwise-rank by quality using the D1–D5 standard. Emit the pairwise log; the harness computes the order and the ladder verdicts. Judge quality difference, never against academic standards.
Loss-1 judge (codex role). Given one sample's de-identified dialogue and its PolicyCard, decide axis-by-axis whether the user-simulator enacted the card's per-axis pressure. Judge enactment of the card, never whether the research is good.
Closing skill for the research-executor, loaded as the last step of formated-specs. Summarize the design just produced into one research-result JSON fenced block in your reply. Do not execute the research.
| name | headroom-estimation |
| description | Estimate theoretical/practical ceiling vs current SOTA gap |
| execution | subagent |
| prompt | ./prompt.md |
| input | task_name, current_sota, human_performance, theoretical_bounds |
Quantify the remaining improvement potential for a task by estimating various performance ceilings and computing the gap between current SOTA and those ceilings. Distinguishes between theoretical limits (information-theoretic), practical limits (current paradigm), and human performance baselines.
| Field | Type | Description |
|---|---|---|
| task_name | string | The target task |
| current_sota | object | {method, score, metric, dataset, date} |
| human_performance | object | {score, conditions, source} or null |
| theoretical_bounds | object | {bound_type, value, derivation} or null |
{
"task": "string",
"dataset": "string",
"metric": "string",
"ceilings": {
"theoretical": {
"value": null,
"type": "information_theoretic|bayes_optimal|combinatorial",
"derivation": "string",
"confidence": "high|medium|low|speculative"
},
"human": {
"value": null,
"conditions": "string",
"source": "string",
"is_expert": true,
"confidence": "high|medium|low"
},
"practical": {
"value": null,
"assumptions": "string",
"based_on": "string",
"confidence": "medium|low|speculative"
}
},
"headroom": {
"vs_theoretical": null,
"vs_human": null,
"vs_practical": null,
"interpretation": "string"
},
"saturation_assessment": {
"status": "saturating|active_progress|early_stage|unknown",
"evidence": "string",
"years_to_human_parity": null
}
}