preference-verify

name

preference-verify

description

Verifies that an exact artifact revision follows a scenario-matched preference record without merging preference fidelity with correctness or task fitness. Use after implementation, before release, or when checking drift against a recorded preference contract. Returns VERIFIED, NOT VERIFIED, or NO VERDICT with evidence. Do not use to generate alternatives, infer missing preferences, or produce a universal quality score.

Preference Verify

Verify one exact artifact revision against a compatible preference record and its objective gates.

Keep three judgments separate

Keep correctness, task fitness, and human preference separate. Never average them into one quality or taste score. A preference match cannot hide an objective failure.

Required inputs

Require:

• exact artifact revision and runtime environment;
• dossier and preference-rule versions;
• matching scenario and axis;
• objective correctness and task-fit contracts;
• evidence paths and hashes;
• held-out trial when the rule claims transfer.

Return NO VERDICT when any required input is missing. Never turn missing evidence into a probable pass.

Verify

1. Pin the artifact revision. Stop if it changes during review.
2. Validate the dossier and every evidence hash.
3. Confirm that artifact, rule, scenario, and axis match.
4. Classify each rule as applicable, not_applicable, or scope_conflict.
5. Run scenario-specific correctness checks.
6. Run task-fitness checks separately.
7. Map every applicable rule to direct evidence.
8. Mark each rule supported, violated, or unverifiable.
9. Confirm that the held-out artifact was not used during calibration.
10. Compare the preregistered prediction with the recorded response.
11. Preserve mismatches, abstentions, contradictions, and objective failures.
12. Bind the verdict to the exact revision and dossier hash.

From this skill directory, run the self-contained verifier:

node scripts/preference-harness.mjs validate <run.json> --json
node scripts/preference-harness.mjs report <run.json> --format markdown

The installed verifier derives the rule matrix and the exact VERIFIED, NOT VERIFIED, or NO VERDICT result. Do not replace it with an agent-authored approximation.

Use references/verification-contract.md for the rule matrix and freshness rules.

Verdict

• VERIFIED: required objective checks pass, every applicable rule is supported, required held-out evidence is present, and the held-out response matches the prediction.
• NOT VERIFIED: an objective check fails, an applicable rule is violated, or the held-out response mismatches the prediction.
• NO VERDICT: exact revision, provenance, compatible route, or required evidence is unavailable.

NO VERDICT blocks release when verification is mandatory, but it is not evidence that the artifact violated a rule.

Stop and refuse

Refuse to:

• infer evidence from intent, a prompt, commit prose, or an implementation plan;
• verify a moving target;
• borrow rules from another scenario;
• certify a rule derived from an abstention;
• accept evidence whose hash does not match;
• treat a reconstruction as a live experiment;
• publish unapproved human quotes;
• calculate an overall taste or quality score.

Output

Emit:

1. The rule-by-rule matrix from references/output-contract.md.
2. Separate correctness, task-fit, and preference assessments.
3. Verdict, blockers, missing evidence, exact revision, and freshness statement.
4. A consent-safe JSON report and dossier SHA-256 when --format json is used.

The public report omits raw audit records and evidence locators, and redacts human responses and derived rule statements unless their source responses have approved public consent. Assessment rationales remain only in the private dossier; the public matrix uses deterministic reasons derived from their statuses. The dossier file remains the private source record.