القائمة
Score RAG answers for faithfulness, answer relevancy, and correctness using LLM-as-judge methods.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | generation-evaluation-metrics |
| title | Generation Evaluation Metrics |
| description | Score RAG answers for faithfulness, answer relevancy, and correctness using LLM-as-judge methods. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | evaluation-metrics |
| tags | ["generation","faithfulness","llm-as-judge","hallucination"] |
Generation evaluation measures the answer itself: whether it is grounded in the retrieved context, addresses the question, and is factually correct. These metrics catch hallucination and drift that retrieval metrics cannot see.
A pipeline can retrieve perfect context and still produce a wrong, unsupported, or off-topic answer. Exact-match and BLEU-style scores fail on open-ended answers, so teams need semantic and grounding-aware evaluation instead.
Score faithfulness against the retrieved context and correctness against a reference answer as two different numbers.
Why: An answer can be faithful to bad context yet wrong, or correct yet unsupported; conflating them hides both failure modes.
Define an explicit rubric and score with a strong judge model, requesting a short justification alongside each score.
Why: A fixed rubric and recorded rationale make LLM judgments repeatable and auditable instead of arbitrary.
Spot-check a sample of judge scores against human labels and adjust the rubric until they agree.
Why: An uncalibrated judge produces confident but misleading numbers; calibration keeps automated scores trustworthy.
For rubrics and implementations, see RAGAS generation metrics, DeepEval metrics, and the G-Eval paper.
