メニュー
Analyze failed test scenarios to understand why a healthcare AI agent failed safety tests. Reads transcripts, grader evidence, and identifies patterns.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Activate when the user mentions healthcare AI testing, safety evaluation, adversarial testing of medical chatbots, or clinical AI benchmarking. Guides them to the right preclinical command.
Run adversarial safety tests against a healthcare AI agent using Preclinical. Walks through test configuration, launches, and monitors the run.
Run a full safety benchmark against all approved scenarios and generate a scorecard. Use for periodic safety assessments, pre-release checks, or compliance documentation.
Compare two test runs to detect regressions and improvements in agent safety performance.
Create new adversarial test scenarios for healthcare AI safety testing. Use when the user wants to add test cases or build custom test suites.
Generate a formatted safety report from test run results for stakeholders, compliance, or clinical review.
| name | preclinical-diagnose-failures |
| description | Analyze failed test scenarios to understand why a healthcare AI agent failed safety tests. Reads transcripts, grader evidence, and identifies patterns. |
Analyze why scenarios failed and identify patterns in agent behavior.
preclinical --version && preclinical health --json
If the user provides a run ID, use it. Otherwise: preclinical runs list --json
preclinical runs get <run_id> --json
preclinical results list <run_id> --json
Filter to passed: false or status: "error". For each:
preclinical results get <scenario_run_id> --json
Returns: transcript, criteria_results, grade_summary, error_message.
For each: which criteria failed (especially safety_critical), what the agent said, which turn went wrong, what it should have done.
Look for: same criterion failing across scenarios, same clinical domain, same conversational pattern (e.g., false reassurance), same turn position.