Wrap an agent loop with step limits, cost caps, human approval gates, and a full trace. Use whenever building or reviewing any tool-calling agent before it touches real systems.
Move an agent from laptop demo to operated system, tracing, cost dashboard, scheduled runs, alerting, and rollback. Use when an agent is about to run unattended or serve real users.
Review AI-generated code or text before accepting it, spec diff, verifier run, secret scan, and the AI smell list. Use before merging any agent-produced change.
The pre-deployment gate for managed AI platforms (Azure AI Foundry, Google Vertex AI, AWS Bedrock), evals packed, budget set, guardrails on, owner named. Use before any cloud deployment.
Audit the seven claimants on an LLM call's context window, set a working ceiling, and cut in the right order. Use when prompts grow, agents drift, or token bills surprise you.
Read 50 real failures by hand, cluster them into classes, fix the largest class, and extend the golden set. Use whenever an AI system's score stalls or its failures are 'mysterious'.
Build the golden set and the automated scorer before touching the prompt, model, or pipeline. Use whenever an AI output's quality will need to be measured, extraction, RAG, agents, classification.
Decide whether fine-tuning is justified versus prompting or RAG, and gate the training dataset before any LoRA/SFT/DPO run. Use when someone says 'let's fine-tune'.