菜单
Run Pulse evaluation suites quickly. Use when the user says "run evals", "check evals", "run the gate", "check routing", "run release suite", or wants to verify scores. Supports all 11 suites + selector + baseline comparison.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Cluster capacity analysis, resource forecasting, and scaling recommendations
Builds investigation plans and custom skills from user requirements
Auto-generates structured postmortem reports from incident resolution data
Security scanning, RBAC analysis, and compliance checks
SLO/SLI tracking, error budget analysis, and burn rate alerting
Cluster diagnostics, incident triage, and resource management
| name | run-evals |
| description | Run Pulse evaluation suites quickly. Use when the user says "run evals", "check evals", "run the gate", "check routing", "run release suite", or wants to verify scores. Supports all 11 suites + selector + baseline comparison. |
Quick access to all evaluation suites.
/run-evals <suite> where suite is one of:
| Suite | Gate? | What it tests |
|---|---|---|
release | YES | 12 core SRE/security scenarios |
view_designer | YES | 7 dashboard/view scenarios |
selector | YES | 55 ORCA routing scenarios (deterministic) |
core | no | 6 fundamental scenarios |
safety | no | 3 safety/guardrail scenarios |
integration | no | 7 cross-system scenarios |
adversarial | no | 5 adversarial input scenarios |
all | -- | Run all suites |
baseline | -- | Compare current vs saved baseline |
audit | -- | Prompt token cost breakdown |
python3 -m sre_agent.evals.cli --suite <suite>
python3 -m sre_agent.evals.cli --suite <suite> --fail-on-gate
python3 -c "
import sre_agent.skill_loader as sl
sl._skills = {}; sl._keyword_index = []; sl._selector = None; sl._HARD_PRE_ROUTE.clear()
from sre_agent.evals.selector_eval import run_selector_eval
r = run_selector_eval()
print(f'Selector: {r.passed}/{r.total_scenarios} ({r.passed/r.total_scenarios:.0%})')
if r.failed_scenarios:
for f in r.failed_scenarios:
print(f' FAIL: {f[\"id\"]}: got {f[\"got\"]} expected {f[\"expected\"]}')
"
python3 -m sre_agent.evals.cli --suite release --compare-baseline
python3 -m sre_agent.evals.cli --suite view_designer --compare-baseline
python3 -m sre_agent.evals.cli --suite release --save-baseline
python3 -m sre_agent.evals.cli --suite view_designer --save-baseline
python3 -m sre_agent.evals.cli --audit-prompt --mode sre
python3 -m sre_agent.evals.cli --audit-prompt --mode security
Run gating suites first, then informational:
python3 -m sre_agent.evals.cli --suite release --fail-on-gate
python3 -m sre_agent.evals.cli --suite view_designer --fail-on-gate
python3 -m sre_agent.evals.cli --suite core
python3 -m sre_agent.evals.cli --suite safety
python3 -m sre_agent.evals.cli --suite integration
python3 -m sre_agent.evals.cli --suite adversarial
policy_violation, hallucinated_tool, missing_confirmation