verdict-by-oe

name	verdict-by-oe
description	Use when auditing or fixing hema-2026 question answers with OpenEvidence MCP: read the question from D1, ask OpenEvidence for a verdict, report pending answer/explanation changes for user approval, then update D1 with history.

Verdict By OE

Use this skill in the hema-2026 repo when the user asks to audit, re-check, verdict, or fix a question answer using OpenEvidence.

Guardrails

• Do not update D1 on the first pass. Read the DB, ask OpenEvidence, and report the pending verdict first.
• Only update D1 after the user explicitly approves the specific answer and/or explanation change.
• Use [e-mail] as changed_by / updated_by for approved admin corrections unless the user provides another identity.
• Prefer the production D1 database: wrangler d1 execute hema-2026-db --remote. Use local D1 only when the user asks for local work.
• Treat medical conclusions as evidence-supported verdicts, not blind MCP output. If OE misses a NOT/EXCEPT stem, contradicts itself, lacks the needed option detail, or the prompt data is incomplete, mark the verdict needs_review.
• Do not touch unrelated git changes. DB-only fixes usually do not require a commit unless repo files changed.

Workflow

1. Read the question and current explanation from D1.

   wrangler d1 execute hema-2026-db --remote --json --command \
     "SELECT q.id, q.year, q.answer, q.stem, q.options_json,
             e.version AS explanation_version, e.content_json AS explanation_json
      FROM questions q
      LEFT JOIN explanations e ON e.question_id = q.id
      WHERE q.id = '114-007';"
   

   Confirm that the stem and options are complete. If coded options refer to missing numbered statements, report data_quality_missing_stem and stop before asking for an answer correction.

2. Ask OpenEvidence MCP for a focused verdict.

   Use the OpenEvidence MCP oe_ask tool. Keep the prompt focused and include: question id, exact stem, all options, current DB answer, and the requested output schema.

   Request structured output like:

   {
     "id": "114-007",
     "current_answer": "B",
     "best_answer": "E",
     "verdict": "agree | disagree | uncertain | needs_review",
     "confidence": "high | medium | low",
     "reason": "short rationale",
     "option_notes": {
       "A": "why right/wrong",
       "B": "why right/wrong"
     },
     "article_id": "OpenEvidence article id if available"
   }
   

   For many questions, batch only when the combined prompt stays easy to audit; otherwise handle questions one at a time.

3. Normalize the verdict before reporting it.

   • agree: OE supports the current answer with coherent reasoning.
   • disagree: OE identifies a different valid option with coherent reasoning.
   • uncertain: evidence is incomplete, conflicting, or low confidence.
   • needs_review: the question text/options are incomplete, OE likely missed a logical qualifier, or the answer depends on a local exam convention.

4. Report the pending result to the user before any write.

   Include:

   • Question id.
   • Current DB answer.
   • Proposed answer, if any.
   • Confidence and verdict.
   • One concise paragraph of medical/option reasoning.
   • OpenEvidence article id or citation handle, if present.
   • Proposed explanation changes when the explanation should change.
   • The explicit line: No DB change has been made yet.

   Ask for explicit approval only when a DB write is needed.

5. Update D1 after approval.

   Before writing, re-read the current row so the update is based on the latest DB state. Use a temp SQL file for multi-statement updates and escape strings through a real JSON/SQL helper, not manual editing.

   Minimum answer-change pattern:

   INSERT INTO answer_history
     (question_id, previous_answer, new_answer, source, challenge_id, changed_by, changed_at)
   SELECT id, answer, answer, 'original', NULL, NULL,
          strftime('%s','now') * 1000
   FROM questions
   WHERE id = '<QID>'
     AND NOT EXISTS (SELECT 1 FROM answer_history WHERE question_id = '<QID>');
   
   INSERT INTO answer_history
     (question_id, previous_answer, new_answer, source, challenge_id, changed_by, changed_at)
   SELECT id, answer, '<NEW_ANSWER>', 'admin', NULL, '[e-mail]',
          strftime('%s','now') * 1000
   FROM questions
   WHERE id = '<QID>' AND answer <> '<NEW_ANSWER>';
   
   UPDATE questions
   SET answer = '<NEW_ANSWER>'
   WHERE id =   answer  ;
   
    answer_challenges
    status  ,
       resolved_at  strftime(,)  ,
       resolution_reason  
    question_id    status  (, );
   

6. Verify the write.

   Query the changed row, answer history, and explanation history:

   wrangler d1 execute hema-2026-db --remote --json --command \
      "SELECT id, answer FROM questions WHERE id = '<QID>';
      SELECT question_id, previous_answer, new_answer, source, challenge_id, changed_by, changed_at
      FROM answer_history WHERE question_id = '<QID>' ORDER BY changed_at;
      SELECT question_id, version, updated_by, updated_at
      FROM explanations WHERE question_id = '<QID>';
      SELECT question_id, version, updated_by, updated_at
      FROM explanation_history WHERE question_id = '<QID>' ORDER BY updated_at;"
   

   Also inspect the rendered TipTap text if explanation JSON changed.

7. Final response.

   State exactly what changed, the verified answer/version, the OE article id, and whether any repo files were changed. Mention unrelated staged files only if they remain present and were intentionally untouched.