g6-evidence

# G6 evidence: the certificate issuer The one place a mission's completion state is decided. `issue` gives each predicate a four-valued (support, refute) value from receipts taken on the exact artifact bytes and returns `verified`, `qualified-draft` or `blocked-escalated` with every gap named. `verify` decides a certificate again, offline, from the checks, receipts, gates and obligations it records, and names every field that does not follow. ## When to use it - You are building a harness or a workflow that labels results, and the label must follow from the checks, not from the producer's say-so: `issue`. - You are handed a certificate (from a `workforce` mission, another run, another agent) with the artifact: `verify(body, artifact_bytes)` catches changed bytes and a state, statement, gap or list that does not follow from what the certificate records. To rely on it, compare its recorded checks with the frozen checks you hold and re-run the checks in its `rerun` list on the bytes. - A check did not count and you need to know why: `grounding(check, artifact, predicate)` names each check-level reason; receipt-level ones are in the certificate's `gaps`. ## When not to use it - Checking a coding agent's report from the evidence it declares: `coding_agent_gate.verify_answer`. - Checking only that an envelope's fields agree with one another: `invariants.check_envelope`. - Running a coding task end to end with tests G6 runs itself: a `workforce` mission, which calls this issuer. ## Python API - `issue`: the `Certificate` for an artifact under the mission's facts, checks, receipts and the gates the caller lists as decided (`decisions`), with `ledger`, the `g6.core.ledger.ChainLedger` holding the controls and receipts G6 observed; raises `CertificateRefused` when the inputs contradict each other. - `record`: write controls and receipts G6 observed into a ledger, so `issue` can find them by id and hash. - `verify`: every inconsistency in a certificate body for the given bytes, including every field that differs when `issue` runs again on the checks, receipts, gates and obligations the body records (a body that records no checks cannot be decided again and is reported); given the mission's and the standard's facts, also a dropped predicate, an undecided gate the certificate does not name in its gaps, or a changed hash; given the `ledger`, it finds the records there again. - `grounding`: the check-level reasons a check does not meet the grounding rule for a predicate on an artifact (empty when it meets it). - `MissionFacts`: the mission as values: its hash, its standard's hash, its predicates, gates, broken obligations and cost bound. - `Artifact`: the artifact's sha256, its producer and its ledger position. - `Check`: a check's id, predicates, author, evidence kind, code hash, freeze position, negative controls, formal certificate digest and positive-control position. - `Control`: a negative control: a seeded defect for one predicate, run against one version of a check's code. - `Receipt`: one run of a check on some bytes: `pass`, `fail` or `unknown`, at a ledger position. - `Certificate`: the result; `state`, `predicates`, `gaps`, `formal`, `rerun`, `verified_predicates`, what it was decided from (`checks`, `gates`, `obligations`, `artifact_seq`), and `as_dict()` with its digest. - `PredicateStatement`: one predicate's support, refutation and caveats, and whether it is verified. - `CertificateRefused`: the error that lists every problem with the inputs. - `digest_of`: the certificate's unkeyed sha256 digest. - `catalogue_sha`: the hash of the catalogue a mission was judged against. - `EVIDENCE_KINDS`: `process`, `heuristic_review`, `partial_falsifier`, `independent_executable`, `formal_checked`, `no_evidence`. - `RESULTS`: the receipt results. - `STATES`: the three reliability states. ## Reading the result A pass receipt supports predicate `p` only when its check meets the grounding rule for `p`: - the check's kind is `independent_executable` or `formal_checked` (a formal check also names its certificate digest); - its author is not the artifact's producer; - it was frozen before the artifact existed; - it killed a negative control for `p`, generated by G6 (`generated_by="g6"`), on this exact check code, recorded at or after the freeze and before the receipt; - the receipt is on the artifact's bytes, taken once the artifact existed, with the check's frozen hash; - the receipt and a control backing it are in `ledger` with the same body (written there by `record`); one that is not is named in the gap (`not_in_ledger:...`) and in `unresolved`, and does not support. A failing receipt refutes `p` when the check killed a G6 control for `p` and passed a known-good reference (`positive_seq`), both before the receipt, whoever wrote it and whatever its kind; any other failure, or an `unknown` result, is a caveat that keeps `p` from verified without refuting it. The state is `blocked-escalated` when the mission breaks a soundness obligation or any predicate is refuted, `verified` when every predicate is supported with no caveat and every gate was decided, and `qualified-draft` otherwise. `gaps` names every predicate that is not verified, with why (`not_grounded:<check>:self_authored`, `no_receipt` and so on), every undecided gate (`human_decision_missing`) and every broken obligation. ## Alone and composed From Python only: `from g6.evidence import issue, record, verify, MissionFacts, Artifact, Check, Control, Receipt`. It has no block and no MCP op. Composed: - `workforce` builds the mission facts, checks and receipts of a coding mission and calls `issue` for its certificate; a gate approved in the mission is passed in as a decision, and `verify(body, bytes, gates=[...])` then catches a certificate that drops it without naming it in its gaps. - `g6.harness.run` (and through it the meta-harness) labels each run with `issue`. - `invariants.check_envelope` checks the envelope a result travels in; this issuer decides the state inside it. ## Failure modes it catches - `overstated_verification` (work called verified when the evidence does not support it): `verified` is not monotone in the evidence (one refutation turns support into conflict, and a conflict is never averaged away); self-authored, late-frozen or uncontrolled checks never support; and `verify` decides the certificate again from what it records, so a state, statement, gap, formal list or re-run list rewritten to say more than its checks and receipts support is caught even when the digest was recomputed, and so are changed bytes. ## Blind spots - It trusts the evidence kinds and authors it is told: an author-controlled test filed as `independent_executable` under another author's name can reach `verified`. Never label your own tests as independent. - It cannot prove that receipts and controls were honestly recorded; that is the job of the ledger and the runner that write them. Forged entries in a ledger the caller builds itself are not caught: the ledger is an auditable hash chain, not a signature. - Without the ledger, `verify` takes the certificate's own `unresolved` list, so offline it checks consistency only. - The digest is unkeyed and `verify` decides again from what the certificate records, so it cannot tell whether the recorded producer, authors, kinds, freeze and control positions, control results, receipts, gates, obligations and decisions are true. A forger who rewrites those as well (a new producer name, so a self-authored check no longer looks self-authored; obligations dropped with their gaps; a gate listed as decided) passes `verify`. Given the mission's gates, predicates and hashes it still catches a dropped gate or predicate. To rely on a certificate, compare its recorded checks with the frozen checks you hold and re-run the checks in its `rerun` list on the bytes. - `decisions` are gate names the caller asserts, and listing one clears its `human_decision_missing` gap: it cannot tell a person's decision from anyone else's. - It judges one artifact at a time; it says nothing about how the checks were chosen. - It declares no failure mode of its own. ## What it needs Nothing outside G6: pure standard-library Python (hashing and JSON) and G6's own ledger, in memory unless given a file. Nothing is run. ## Examples ```python import hashlib from g6.core.ledger import ChainLedger from g6.evidence import Artifact, Check, Control, MissionFacts, Receipt, grounding, issue, record, verify patch = b"def double(x):\n return x + x\n" artifact = Artifact(hashlib.sha256(patch).hexdigest(), producer="coding-agent", seq=10) facts = MissionFacts(mission_sha="mission-1", mission_standard_sha="standard-1", standard_sha="standard-1", predicates=("double_is_correct",), standard_predicates=("double_is_correct",)) code = hashlib.sha256(b"def test_double(): assert double(2) == 4").hexdigest() control = Control("mutant-1", code, "double_is_correct", generated_by="g6", seq=3, killed=True) # The producer's own test passes, but a self-authored check never supports. own = Check("own-test", ("double_is_correct",), author="coding-agent", kind="independent_executable", check_sha=code, frozen_seq=2, controls=(control,)) cert = issue(facts, artifact, [own], [Receipt("r1", "own-test", code, artifact.sha256, "pass", seq=11)]) assert cert.state == "qualified-draft" assert "self_authored" in grounding(own, artifact, "double_is_correct") # A check someone else froze before the patch, which killed a G6 control. Typed by hand, its records do not support. held = Check("held-out-test", ("double_is_correct",), author="reviewer", kind="independent_executable", check_sha=code, frozen_seq=2, controls=(control,)) receipt = Receipt("r2", "held-out-test", code, artifact.sha256, "pass", seq=11) cert = issue(facts, artifact, [held], [receipt]) assert cert.state == "qualified-draft" and "not_in_ledger" in cert.gaps[0][1] # The same records, written to the ledger by the runner that observed them: support. ledger = ChainLedger() record(ledger, control, receipt) cert = issue(facts, artifact, [held], [receipt], ledger=ledger) assert cert.state == "verified" and cert.gaps == () body = cert.as_dict() assert verify(body, patch) == [] and verify(body, patch, ledger=ledger) == [] assert verify(body, patch + b"# edited\n") # other bytes: the inconsistency is named # The self-authored draft rewritten as verified, digest recomputed: decided again, it does not follow. from g6.evidence import digest_of draft = issue(facts, artifact, [own], [Receipt("r1", "own-test", code, artifact.sha256, "pass", seq=11)]).as_dict() draft["predicates"][0].update(support=True, supported_by=["own-test"], caveats=[]) draft.update(gaps=[], rerun=[["own-test", code, "independent_executable"]], state="verified") draft["digest"] = digest_of(draft) assert any("issuing rules" in problem for problem in verify(draft, patch)) ``` ## The four open problems - **Learning signals**: it writes nothing. Each certificate names every gap with its reason, a labelled record a caller can keep; the meta-harness reads each run's certificate state when it scores candidates. Not yet: nothing learns from the gaps. - **Evaluators the model cannot influence**: this is what the grounding rule is for. A check supports only when it was frozen before the artifact existed, its author is not the producer, and it killed a G6-generated control on its exact code, so the producer cannot write or tune the evaluator after seeing its own work. Controls and receipts count only from G6's ledger; it still trusts the check kinds, authors and freeze positions it is told. - **Transfer by failure mode**: it is bound to `overstated_verification` and to predicates, not to a task: the same rules apply to any artifact with checks and receipts. Not yet: no measurement on unseen task families. - **Enough held-out cases**: Not yet: it judges one artifact and its checks; it does not count cases or run G6's sequential test.