latitude-dev/eval-skills

llm-annotation-guide

Use this skill when a developer wants to annotate their LLM outputs, set up an annotation process, or improve existing annotations. Triggers on: "help me annotate my LLM outputs", "review my annotations", "set up an annotation rubric", "my annotations feel…

원문 언어: 영어

llm-eval-type-selector

Use this skill when a developer wants to decide what type of evaluation to build for their AI system. Triggers on: "should I use a rule or a judge", "what type of eval should I build", "decide eval type", "judge vs programmatic rule", "LLM-as-judge vs…

원문 언어: 영어

llm-evals-audit

Use this skill when a developer wants to check whether their existing evaluations are trustworthy and well-targeted. Triggers on: "audit my evals", "are my evals any good", "review my evaluation setup", "check my LLM judges", "are my evaluations reliable",…

원문 언어: 영어

llm-evals-checklist

Use this skill when a developer wants to check if they're ready to build evaluations for their AI system. Triggers on: "am I ready to build evals", "can I start building evaluations", "eval readiness check", "pre-eval checklist", "what do I need before…

원문 언어: 영어

llm-golden-dataset-builder

Use this skill when a developer wants to build or expand a golden dataset for regression testing. Triggers on: "build a golden dataset", "create a test dataset", "curate eval examples", "promote logs to dataset", "build regression test cases", "I need a…

원문 언어: 영어

llm-issue-discovery

Use this skill when a developer wants to find out what's going wrong with their LLM or AI system in production. Triggers on: "analyze my LLM logs", "find issues in my AI outputs", "my LLM is giving bad responses", "find patterns in my AI failures", "review my…

원문 언어: 영어

llm-judge-alignment

Use this skill when a developer wants to validate how well their LLM judge aligns with human judgment. Triggers on: "validate my LLM judge", "check if my judge is accurate", "my judge scores don't match human ratings", "calibrate my evaluator", "how reliable…

원문 언어: 영어

llm-judge-creator

Use this skill when a developer wants to create LLM-as-a-judge evaluators for their AI system. Triggers on: "create an LLM judge", "build an eval for my AI", "automate my evaluations", "create a judge prompt", "build LLM-as-a-judge", "turn my annotations into…

원문 언어: 영어

llm-regression-runner

Use this skill when a developer wants to test a prompt change against a golden dataset and see what broke. Triggers on: "run my evals", "test this prompt change", "check for regressions", "did I break anything", "run regression tests", "test against golden…

원문 언어: 영어