メニュー
Cryptographic integrity verification for AI safety evaluations using BLAKE3 hashing and Ed25519 signatures. Ensures scenarios haven't been tampered with and results are exactly reproducible.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Run trajectory-level healthcare AI safety evaluations using the ScribeGOAT2 framework. Use this skill when asked to evaluate medical AI safety persistence, run multi-turn trajectory analysis, detect Turn 2 cliff vulnerabilities, or generate safety disclosure reports for frontier lab review. This skill enforces deterministic execution, two-stage grading, healthcare context conditioning, and audit-grade reproducibility. All runs produce cryptographically verifiable evidence chains.
Multi-turn safety persistence evaluation for crisis scenarios. Tests whether AI models maintain appropriate safety responses when users express barriers, minimize distress, or attempt to disengage.
Anthropic-aligned medical safety evaluation with pass^k metrics, failure taxonomy, and anti-gaming graders
Run HealthBench Hard benchmark evaluation using multi-specialist council architecture with deterministic safety stack.
Scan repository for Protected Health Information (PHI) using HIPAA Safe Harbor patterns. Ensures evaluation data remains synthetic-only.
Generate frontier lab-specific evaluator briefs from ScribeGOAT2 evaluation results. Use this skill when asked to create technical safety briefs, disclosure documents, or presentation materials for OpenAI, Anthropic, DeepMind, or xAI safety teams. Produces audit-grade documentation calibrated to each lab's review culture, technical vocabulary, and safety priorities.
| name | bloom_integrity_verification |
| description | Cryptographic integrity verification for AI safety evaluations using BLAKE3 hashing and Ed25519 signatures. Ensures scenarios haven't been tampered with and results are exactly reproducible. |
| version | 1.0.0 |
| author | ScribeGoat2 Team |
| license | MIT |
| safety_level | medium |
Ensure evaluation scenarios and results maintain cryptographic integrity for reproducible safety research. Provides tamper-evident audit trails for regulatory compliance.
# Verify scenarios with signature
bloom-verify check scenarios/ \
--sig scenarios.sig \
--pubkey bloom.pub \
--fail-closed
# Generate audit log
bloom-verify audit results.json --output audit.json
# Hash directory for manifest
bloom-verify hash scenarios/ > manifest.json
# Create signed manifest
bloom-verify sign manifest.json --key bloom.key --output manifest.sig
# From source
cd bloom_medical_eval/bloom_verify
cargo build --release
# Add to PATH
export PATH="$PATH:$(pwd)/target/release"
# Or install globally
cargo install --path bloom_medical_eval/bloom_verify
command:
type: enum
values: [check, hash, sign, verify, audit]
required: true
path:
type: path
required: true
description: File or directory to process
signature:
type: path
description: Signature file (for check/verify)
pubkey:
type: path
description: Public key file (for check/verify)
privkey:
type: path
description: Private key file (for sign)
output:
type: path
description: Output file path
fail_closed:
type: boolean
default: true
description: Exit non-zero on any failure
status: enum # pass, fail
hash: string # BLAKE3 hash (64 hex chars)
signature_valid: boolean
files_verified: integer
audit_entries: array
timestamp: string # ISO 8601
| Property | Implementation | Notes |
|---|---|---|
| Hashing | BLAKE3 | 10x faster than SHA-256, secure |
| Signing | Ed25519 via ring | Fast, constant-time, secure |
| Audit Chains | Hash-chained entries | Blockchain-style integrity |
| Key Format | PEM | Standard, portable |
| Check | Requirement |
|---|---|
| Scenario verification | Exit code 0 |
| Signature validity | Ed25519 verification passes |
| Audit chain integrity | All entry hashes valid |
| No modified files | Hash matches manifest |
- gate: scenario_integrity
metric: verification_passed
operator: "=="
threshold: true
action: block_execution
severity: medium
description: |
Evaluation cannot proceed if scenario integrity fails.
Prevents running on tampered or corrupted data.
- gate: signature_valid
metric: signature_valid
operator: "=="
threshold: true
action: warn
severity: low
description: |
Missing or invalid signature triggers warning.
May indicate unsigned development data.
| Regulation | Feature |
|---|---|
| FDA 21 CFR Part 11 | Audit trails, electronic signatures |
| EU AI Act | Traceability, reproducibility |
| HIPAA | Data integrity, access logging |
| ISO 27001 | Information security controls |
# Before running crisis evaluation
bloom-verify check \
bloom_medical_eval/experiments/crisis_pilot/.private/scenarios_v2.json \
--sig scenarios_v2.sig \
--pubkey bloom.pub \
--fail-closed
# If verification fails, abort
if [ $? -ne 0 ]; then
echo "Scenario integrity check failed. Aborting."
exit 1
fi
# After evaluation completes
bloom-verify audit \
results/crisis_pilot/pilot_gpt52_n30_*.json \
--output audit_trail.json
# Sign the audit trail
bloom-verify sign audit_trail.json \
--key bloom.key \
--output audit_trail.sig
crisis_persistence_eval - Uses bloom-verify for scenario integrityphi_detection - Run before bloom-verify to ensure data is clean