메뉴
Measure retrieval quality with recall, precision, MRR, nDCG, and context precision and recall.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | retrieval-evaluation-metrics |
| title | Retrieval Evaluation Metrics |
| description | Measure retrieval quality with recall, precision, MRR, nDCG, and context precision and recall. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | evaluation-metrics |
| tags | ["retrieval","recall","ndcg","mrr","context-precision"] |
Retrieval evaluation measures whether the pipeline fetched the right chunks before the LLM ever sees them. Because generation quality is capped by retrieval quality, these metrics isolate the most common and most fixable source of bad RAG answers.
Teams judge RAG only by final answer quality, which hides where the failure is. When the right chunk is never retrieved, no prompt or model change can recover, yet the pipeline is often blamed on generation instead of retrieval.
Collect representative questions and mark which document chunks are relevant to each. This golden set is the ground truth every metric compares against.
Why: Rank-based metrics are meaningless without known-relevant targets to score against.
Start with recall@k across candidate k values to confirm relevant chunks are retrieved at all, then check precision and rank quality with MRR and nDCG.
Why: A relevant chunk that is never retrieved cannot be re-ranked into place, so recall is the ceiling for everything downstream.
Use LLM-judged context precision and context recall when human relevance labels are scarce, to score whether retrieved context supports the reference answer.
Why: Manual relevance labels are expensive; reference-free metrics scale evaluation to larger query sets.
For definitions and implementations, see RAGAS context metrics, the RAGAS paper, and scikit-learn ranking metrics.
