nv-reason-cxr-quality-v1

name	nv-reason-cxr-quality-v1
description	Used to verify nv-reason-cxr evidence packs for source-pack success, fixture/image hash binding, NV-Reason-CXR runtime identity, response shape, scope disclosure, and conservative forbidden overreach phrases.
license	Apache-2.0
allowed-tools	Bash
metadata	{"author":"NVIDIA MedTech <[e-mail]>","tags":["MedTech","radiology","chest-xray","verifier"]}

NV Reason CXR Quality Verifier

Purpose

• Used for deterministic second-pass review of an nv-reason-cxr evidence pack.
• Checks source-pack success, fixture case/prompt binding, image readability and hash, model/runtime identity, output response shape, limitation disclosure, and forbidden overreach phrases.
• Manifest I/O: inputs are nv_reason_cxr_evidence_pack; outputs are nv_reason_cxr_quality_report.

Instructions

• Use this verifier only on an evidence pack directory produced by skills/nv-reason-cxr.
• Run it through eval_engine/run.py when producing verifier evidence.
• The verifier entrypoint is scripts/grade.py; do not replace it with prompt-only review.
• Treat a pass as engineering evidence only, not as validation of clinical correctness or model quality.

Available Scripts

Script	Purpose	Arguments
scripts/grade.py	Primary verifier entrypoint declared by skill_manifest.yaml.	EVIDENCE_PACK_DIR

Prerequisites

• Runtime requirements: Python standard library only.
• The input directory must contain manifest.json, validation_summary.json, and output.json from an nv-reason-cxr evidence pack.
• The recorded fixture path and image artifact must be available for full fixture/image binding.

Limitations

• Does not validate chest X-ray interpretation, model correctness, report completeness, or medical actionability.
• Forbidden-phrase checks are regex-based and can miss paraphrases.
• Mock-mode packs prove wrapper and gate wiring; they do not prove live NV-Reason-CXR-3B model behavior.

Troubleshooting

Error	Cause	Fix
target_skill_matches fails	The fixture is not an nv-reason-cxr evidence pack.	Re-run the verifier against the source skill pack directory.
fixture_loaded fails	The source pack references a fixture that is unavailable in this checkout.	Regenerate the source pack with a committed synthetic fixture.
image_sha256_matches fails	The recorded image artifact changed or is unavailable.	Regenerate the source pack or inspect artifact provenance.
forbidden_phrases_absent fails	Output contains treatment, regulatory, patient-directed, or absolute-certainty language.	Fix the source skill prompt/runtime guard and regenerate evidence.

Example

MOCK_NV_REASON_CXR=1 python eval_engine/run.py skills/nv-reason-cxr \
  --fixture skills/nv-reason-cxr/fixtures/synthetic_cxr_input.json \
  --out runs/nv_reason_cxr_demo

python eval_engine/run.py verifiers/nv_reason_cxr_quality_v1 \
  --fixture runs/nv_reason_cxr_demo \
  --out runs/nv_reason_cxr_quality

python verifiers/nv_reason_cxr_quality_v1/scripts/grade.py \
  runs/nv_reason_cxr_demo