メニュー
Run adversarial safety tests against a healthcare AI agent using Preclinical. Walks through test configuration, launches, and monitors the run.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Activate when the user mentions healthcare AI testing, safety evaluation, adversarial testing of medical chatbots, or clinical AI benchmarking. Guides them to the right preclinical command.
Run a full safety benchmark against all approved scenarios and generate a scorecard. Use for periodic safety assessments, pre-release checks, or compliance documentation.
Compare two test runs to detect regressions and improvements in agent safety performance.
Create new adversarial test scenarios for healthcare AI safety testing. Use when the user wants to add test cases or build custom test suites.
Analyze failed test scenarios to understand why a healthcare AI agent failed safety tests. Reads transcripts, grader evidence, and identifies patterns.
Generate a formatted safety report from test run results for stakeholders, compliance, or clinical review.
| name | preclinical-run-test |
| description | Run adversarial safety tests against a healthcare AI agent using Preclinical. Walks through test configuration, launches, and monitors the run. |
Run adversarial multi-turn safety tests against a healthcare AI agent.
preclinical --version && preclinical health --json
If either fails, follow the preclinical-setup skill first.
preclinical agents list --json
No agents → create one first. One → confirm. Multiple → ask which one.
Present defaults in a single prompt, let user override:
Ready to configure test run against <agent_name>:
1. Test name: (optional)
2. Mode: Normal (or Creative — adversarial attacks)
3. Scenarios: All approved
4. Max scenarios: 10
5. Concurrency: 1 (max 10)
6. Tags: (none)
Say "go" for defaults or specify changes.
Modes: Normal = standardized patient sim. Creative = adversarial LLM attacks (--creative). Benchmark = stricter grading (--benchmark).
Max turns: --max-turns (default 11, range 5-15). Last turn is always triage.
To filter scenarios: preclinical scenarios list --json
preclinical run <agent_id> \
--name "<test_name>" \
--max-scenarios <N> \
--concurrency <N> \
--tags "<tag1>,<tag2>" \
--scenario-ids "<id1>,<id2>" \
--max-turns <N> \
--creative \
--watch
Only include flags the user set. Always include --watch.
preclinical runs get <run_id> --json
preclinical results list <run_id> --json
Present: pass rate, breakdown (passed/failed/errors), failed scenario names + reasons, mode used.
Next steps: Failures → diagnose-failures skill. All passed normal → suggest creative mode. Errors → debug connectivity.