メニュー
Choose and wire up a RAG evaluation framework and turn it into a CI regression gate.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | rag-evaluation-frameworks |
| title | RAG Evaluation Frameworks |
| description | Choose and wire up a RAG evaluation framework and turn it into a CI regression gate. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | evaluation-metrics |
| tags | ["ragas","deepeval","trulens","ci","regression"] |
This skill covers the tooling layer of RAG evaluation: which framework to adopt, how to structure a golden set, and how to run evaluation automatically on every change so quality regressions are blocked before they ship.
Metrics are only useful if they run consistently. Teams compute scores once in a notebook, never automate them, and lose the baseline. Ad hoc evaluation cannot catch the slow regressions that come from data drift, prompt edits, or model swaps.
Pick the framework that matches your stack rather than the most popular one.
Why: Each tool has a different data model and integration story, and switching later is costly.
Default mapping:
Curate representative questions with reference answers and relevant contexts, and commit it to the repository.
Why: A fixed, version-controlled dataset makes scores comparable across runs and contributors.
Run the framework on the golden set in CI and fail the build when scores regress past a threshold.
Why: An automated gate turns evaluation from a one-time report into a standing guardrail.
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
