| name | ai-evals-regression-suite |
| description | Use this skill for LLM evals, golden datasets, structured outputs, drift detection, regression gates. Trigger when the task involves ai engineering work related to AI Evals Regression Suite, implementation, audits, debugging, strategy, or validation. |
AI Evals Regression Suite
Use this skill for AI Engineering tasks focused on LLM evals, golden datasets, structured outputs, drift detection, regression gates.
Workflow
- Clarify the user outcome, constraints, current stack, and definition of done.
- Inspect local source, artifacts, analytics exports, logs, designs, or docs before changing anything.
- Check official documentation when APIs, SDKs, policies, search behavior, accessibility rules, or production behavior may have changed.
- Compare proven open-source patterns for non-trivial choices and adapt ideas without copying incompatible code.
- Produce the smallest production-ready change or recommendation that satisfies the request.
- Validate with relevant evidence: tests, lint, typecheck, build, screenshots, crawl output, logs, analytics, or manual scenario.
- Report commands, files, findings, risks, and next steps clearly.
Focus Checklist
- Map inputs, owners, dependencies, and constraints.
- Prefer existing project conventions, design systems, and platform primitives.
- Include failure states, privacy/security implications, performance impact, and rollback where relevant.
- Separate facts from assumptions and mark any unverified claims.
- Keep recommendations actionable and prioritized by impact and risk.
Guardrails
- Do not judge AI quality only from a single hand-picked example.
- Do not perform destructive, paid, production, publishing, account-changing, or externally visible actions without explicit user confirmation.
- Do not expose secrets, private keys, tokens, cookies, private analytics, personal data, or confidential business data.
- If validation is impossible, state why and provide a concrete manual verification path.
