audit-the-oracle-coverage

The dangerous case is not "no oracle" — it is a WEAK oracle you falsely believe is exact. Before trusting any test suite, metric, benchmark, eval, or proof as if it settled the question, state explicitly what it CERTIFIES and what it does NOT — the gap between the proxy and the target. The failure it prevents: false certainty from a passing check that silently misses the spec (tests green but the spec was wrong), proxies value (offline metric up, user value flat), or has drifted (backtest on a system that has since changed). Triggers whenever a check comes back positive and you are tempted to call the thing "verified", and as the mandatory middle step between build-the-oracle-before-the-answer and triangulate-without-oracle.

Ir a la instalación

Repositorio: XyraSinclair/ideonomy
Última actividad en el origen: 9 de julio de 2026 a las 10:46
Idioma detectado de SKILL.md: inglés
Estrellas: 0
Forks: 0

Opciones de instalación

De forma predeterminada está seleccionado el prompt que primero revisa el origen. Puedes cambiar a un comando directo o descargar una copia local.

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.

npx skills add https://github.com/XyraSinclair/ideonomy --skill audit-the-oracle-coverage

El comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.

¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.

Revisa los archivos de origen

Lee SKILL.md y los archivos complementarios que muestra SkillsMP antes de decidir si quieres instalarlo.

SKILL.md

Instrucciones de origen · Vista previa de solo lectura

Volver a la búsqueda Volver a la instalación

carry-the-residue-forward-across-sessions

Stop re-deriving from zero every session. At the close of any non-trivial work session, extract what RESISTED resolution — anomalies, reframes that failed, objections that survived, named-but-unfilled gaps, contested judgments, the unexplained core of a distillation — and persist it as the retrievable SEED for the next session. At the start of the next session, retrieve and cite it before expanding. The failure it prevents: total amnesia between sessions, where residue evaporates and nothing compounds. Triggers at session boundaries, before "I'll pick this up later", and whenever you notice you're re-discovering something a past session already surfaced.

2026-07-09

coin-the-missing-concept

When the same awkward phrase keeps recurring — "the kind of bug where...", "this class of lines that...", "that pattern we keep tripping on but have no name for" — treat the repetition as residue signaling a missing concept. Coin a precise noun, give it a decidable membership test, and keep it only if the corpus compresses. The failure it prevents: endless circumlocution standing in for a concept, where the model repeatedly gestures at the same structure but never binds a handle that lets the structure be reused or tested. Triggers on repeated locutions, taxonomic awkwardness, and any review where the same explanation is written three times with different wording.

2026-07-09

design-the-mechanism-from-the-desired-equilibrium

When the task is to set rules, incentives, or metrics — "how should we score this", "what policy makes agents tell the truth", "how do we reward the right behavior" — do not propose a rule and hope it induces the outcome. Start from the behavior you want as a best-response and build backward through a strategyproof template, treating the metric-gamer, often the model itself, as adversary. The failure it prevents: hope-based governance and gamable metrics, where the optimizer satisfies the letter of the score while defecting against the intent. Triggers on rubrics, auctions, evaluation design, agent incentives, and any metric someone will optimize against.

2026-07-09

Ver todos los de este repositorio

audit-the-oracle-coverage

The matched pair build-the-oracle-before-the-answer (oracle exists) vs triangulate-without-oracle (no oracle) is a false binary. Most real work lives in the middle: a partial or proxy oracle that certifies something but not the whole claim, and the characteristic failure is treating that partial certificate as a full one. This skill is the three-way partition the binary was missing — exact / partial-proxy / none — and it routes each part to the right discipline instead of laundering a weak signal into confidence.

Fuses build-the-oracle-before-the-answer (P-1) with prove-the-coverage-denominator (P-11) applied to the oracle itself, and routes the uncovered residual to triangulate-without-oracle (P-9). Organon: P3 fault-model + P11 + P19 oracle + P37 residue-seed. Phase: JUDGE.

Procedure

State what the oracle actually certifies. Write the precise claim the check establishes when it passes — not the claim you wish it established. "All 412 tests pass" certifies the behaviors those tests assert, not "the code is correct."

Name the gap between proxy and target (P3). What is true of the target that the oracle does not see? Spec ambiguity it never probed, inputs it never ran, the distribution it was built on but no longer matches, the value it proxies but is not. If you cannot name a single thing the oracle misses, the audit did not happen — that itself is the red flag.

Partition the claim (P11 over the oracle). Label every part of what you are asserting as:

covered — the oracle directly certifies it;
proxied — the oracle correlates with it but with a known, named gap;
uncovered — the oracle says nothing about it.

Route each part to its discipline. Covered → trust it. Proxied → either strengthen the oracle (build-the-oracle-before-the-answer on the gap) or carry the gap as explicit risk. Uncovered → if it is testable, build an oracle for it; if it is irreducibly normative/strategic, route to triangulate-without-oracle and name the owner.

Report the partition, not a binary verdict. "Verified" is replaced by "covered: X; proxied with gap: Y; uncovered, routed to Z."

The gate

You may not report a claim as "verified" until its parts are partitioned into covered / proxied / uncovered, with at least one named thing the oracle does not certify. An audit that finds the oracle covers everything is presumed to have not looked — partial oracles always miss something; name it or the gate fails. The one escape: an oracle that provably enumerates the claim's entire finite case space may be declared exact, and then the finiteness argument itself is what the audit must produce. The proxied and uncovered parts must be routed, not folded into the covered set.

Example

A migration's test suite passes, and the instinct is "settled, ship it." The audit: covered = the 412 asserted behaviors; proxied = "preserves user settings" (the tests check three settings shapes, correlated but not the full space); uncovered = users with null locale and deleted workspace memberships (no test exists). Routing the uncovered case to a targeted fixture (build an oracle for the gap) finds one settings row that silently drops — exactly the case "all tests pass" was about to certify away. The binary "verified vs not" would have shipped it; the coverage partition caught it.

audit-the-oracle-coverage

Procedure

Partition the claim (P11 over the oracle). Label every part of what you are asserting as:

covered — the oracle directly certifies it;
proxied — the oracle correlates with it but with a known, named gap;
uncovered — the oracle says nothing about it.

Report the partition, not a binary verdict. "Verified" is replaced by "covered: X; proxied with gap: Y; uncovered, routed to Z."

The gate

Example

audit-the-oracle-coverage

Datos de origen

Opciones de instalación

Revisa los archivos de origen

Más de este repositorio