菜单
Fastest route to a deployed live evaluation using a pre-built Truesight template. Use when the user wants a quick start without building judgment configs from scratch.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Scope what quality should be measured, convert it into one or more actionable binary evaluations, deploy those evaluations through Truesight MCP, and generate a companion skill that applies them correctly. Use when a user wants to create new evals, quality checks, guardrails, or pass/fail criteria for AI outputs.
Generate synthetic test data for LLM evaluations using dimension-based tuple expansion. Use when the user needs synthetic traces, test cases, eval datasets, or when create-evaluation needs synthetic fallback data.
Orchestrator for Truesight MCP skills. Use this when the user needs help choosing the right Truesight workflow or when intent is ambiguous across LLM evaluate, error analysis, review, templates, or evaluation creation.
Build a custom web interface for trace annotation and review. Use when users need a bespoke review surface for their workflow.
Systematically identify and categorize failure modes in evaluated traces using Truesight datasets and error-analysis tools. Use when quality issues are unclear, after major pipeline changes, or when incidents indicate drift.
Audit an existing evaluation workflow and produce severity-ranked findings with concrete next actions. Use when inheriting an eval setup, diagnosing quality regressions, or checking LLM evaluation process maturity.
| name | bootstrap-template-evaluation |
| description | Fastest route to a deployed live evaluation using a pre-built Truesight template. Use when the user wants a quick start without building judgment configs from scratch. |
Use this skill when a pre-built template likely covers the target use case.
If template choice is ambiguous, ask one question at a time using the structured question tool (loaded per the HARD-GATE above).
Example question structure:
Which template family best matches your goal?
A) AI writing detection
B) Code quality
C) Unsure, list all templates first
Rules:
list_templates.slug.provision_template(slug).create_and_deploy_evaluation(dataset_id).api_key immediately because it is returned only once.run_eval with representative inputs.dataset_idlive_evaluation_idcreate-evaluation.list_templates requires datasets:readprovision_template requires datasets:writecreate_and_deploy_evaluation requires evaluations:write, live-evaluations:writerun_eval requires live-evaluations:execute