audit-the-oracle-coverage

The dangerous case is not "no oracle" — it is a WEAK oracle you falsely believe is exact. Before trusting any test suite, metric, benchmark, eval, or proof as if it settled the question, state explicitly what it CERTIFIES and what it does NOT — the gap between the proxy and the target. The failure it prevents: false certainty from a passing check that silently misses the spec (tests green but the spec was wrong), proxies value (offline metric up, user value flat), or has drifted (backtest on a system that has since changed). Triggers whenever a check comes back positive and you are tempted to call the thing "verified", and as the mandatory middle step between build-the-oracle-before-the-answer and triangulate-without-oracle.

Ir para a instalação

Repositório: XyraSinclair/ideonomy
Última atividade na origem: 9 de julho de 2026 às 10:46
Idioma detectado do SKILL.md: inglês
Estrelas: 0
Forks: 0

Opções de instalação

Por padrão, está selecionado o prompt que primeiro revisa a origem. Você pode mudar para um comando direto ou baixar uma cópia local.

Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.

Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.

npx skills add https://github.com/XyraSinclair/ideonomy --skill audit-the-oracle-coverage

O comando permanece em uma só linha. Role horizontalmente para revisá-lo antes de copiar.

Prefere uma cópia local? Baixe os arquivos disponíveis atualmente no SkillsMP.

Revise os arquivos de origem

Leia o SKILL.md e os arquivos complementares exibidos pelo SkillsMP antes de decidir se vai instalar.

SKILL.md

Instruções da origem · Visualização somente leitura

Voltar à pesquisa Voltar à instalação

carry-the-residue-forward-across-sessions

Stop re-deriving from zero every session. At the close of any non-trivial work session, extract what RESISTED resolution — anomalies, reframes that failed, objections that survived, named-but-unfilled gaps, contested judgments, the unexplained core of a distillation — and persist it as the retrievable SEED for the next session. At the start of the next session, retrieve and cite it before expanding. The failure it prevents: total amnesia between sessions, where residue evaporates and nothing compounds. Triggers at session boundaries, before "I'll pick this up later", and whenever you notice you're re-discovering something a past session already surfaced.

2026-07-09

coin-the-missing-concept

When the same awkward phrase keeps recurring — "the kind of bug where...", "this class of lines that...", "that pattern we keep tripping on but have no name for" — treat the repetition as residue signaling a missing concept. Coin a precise noun, give it a decidable membership test, and keep it only if the corpus compresses. The failure it prevents: endless circumlocution standing in for a concept, where the model repeatedly gestures at the same structure but never binds a handle that lets the structure be reused or tested. Triggers on repeated locutions, taxonomic awkwardness, and any review where the same explanation is written three times with different wording.

2026-07-09

design-the-mechanism-from-the-desired-equilibrium

When the task is to set rules, incentives, or metrics — "how should we score this", "what policy makes agents tell the truth", "how do we reward the right behavior" — do not propose a rule and hope it induces the outcome. Start from the behavior you want as a best-response and build backward through a strategyproof template, treating the metric-gamer, often the model itself, as adversary. The failure it prevents: hope-based governance and gamable metrics, where the optimizer satisfies the letter of the score while defecting against the intent. Triggers on rubrics, auctions, evaluation design, agent incentives, and any metric someone will optimize against.

2026-07-09

Ver todas deste repositório

audit-the-oracle-coverage

The matched pair build-the-oracle-before-the-answer (oracle exists) vs triangulate-without-oracle (no oracle) is a false binary. Most real work lives in the middle: a partial or proxy oracle that certifies something but not the whole claim, and the characteristic failure is treating that partial certificate as a full one. This skill is the three-way partition the binary was missing — exact / partial-proxy / none — and it routes each part to the right discipline instead of laundering a weak signal into confidence.

Fuses build-the-oracle-before-the-answer (P-1) with prove-the-coverage-denominator (P-11) applied to the oracle itself, and routes the uncovered residual to triangulate-without-oracle (P-9). Organon: P3 fault-model + P11 + P19 oracle + P37 residue-seed. Phase: JUDGE.

Procedure

State what the oracle actually certifies. Write the precise claim the check establishes when it passes — not the claim you wish it established. "All 412 tests pass" certifies the behaviors those tests assert, not "the code is correct."

Name the gap between proxy and target (P3). What is true of the target that the oracle does not see? Spec ambiguity it never probed, inputs it never ran, the distribution it was built on but no longer matches, the value it proxies but is not. If you cannot name a single thing the oracle misses, the audit did not happen — that itself is the red flag.

Partition the claim (P11 over the oracle). Label every part of what you are asserting as:

covered — the oracle directly certifies it;
proxied — the oracle correlates with it but with a known, named gap;
uncovered — the oracle says nothing about it.

Route each part to its discipline. Covered → trust it. Proxied → either strengthen the oracle (build-the-oracle-before-the-answer on the gap) or carry the gap as explicit risk. Uncovered → if it is testable, build an oracle for it; if it is irreducibly normative/strategic, route to triangulate-without-oracle and name the owner.

Report the partition, not a binary verdict. "Verified" is replaced by "covered: X; proxied with gap: Y; uncovered, routed to Z."

The gate

You may not report a claim as "verified" until its parts are partitioned into covered / proxied / uncovered, with at least one named thing the oracle does not certify. An audit that finds the oracle covers everything is presumed to have not looked — partial oracles always miss something; name it or the gate fails. The one escape: an oracle that provably enumerates the claim's entire finite case space may be declared exact, and then the finiteness argument itself is what the audit must produce. The proxied and uncovered parts must be routed, not folded into the covered set.

Example

A migration's test suite passes, and the instinct is "settled, ship it." The audit: covered = the 412 asserted behaviors; proxied = "preserves user settings" (the tests check three settings shapes, correlated but not the full space); uncovered = users with null locale and deleted workspace memberships (no test exists). Routing the uncovered case to a targeted fixture (build an oracle for the gap) finds one settings row that silently drops — exactly the case "all tests pass" was about to certify away. The binary "verified vs not" would have shipped it; the coverage partition caught it.

audit-the-oracle-coverage

Procedure

Partition the claim (P11 over the oracle). Label every part of what you are asserting as:

covered — the oracle directly certifies it;
proxied — the oracle correlates with it but with a known, named gap;
uncovered — the oracle says nothing about it.

Report the partition, not a binary verdict. "Verified" is replaced by "covered: X; proxied with gap: Y; uncovered, routed to Z."

The gate

Example

audit-the-oracle-coverage

Informações da origem

Opções de instalação

Revise os arquivos de origem

Mais deste repositório

Mais deste repositório