rag-retrieval-pattern-design

license	Apache-2.0
name	rag-retrieval-pattern-design
description	Use when designing or fixing the retrieval side of a RAG system, choosing chunking strategy (fixed-size / recursive / semantic), implementing hybrid search (BM25 + dense) with RRF fusion, adding a cross-encoder reranker, evaluating with RAGAS, or running an index-freshness pipeline. Triggers: "RAG keeps citing the wrong doc", chunk size 512 tokens with overlap, RRF reciprocal rank fusion, dense+sparse hybrid, cross-encoder rerank top-5, RAGAS faithfulness / context precision, voyage-3-large vs text-embedding-3-large, daily / hourly reindex cadence. NOT for fine-tuning vs RAG decision (separate skill), agentic tool-use designs, vector DB operational tuning, or general LLM prompt engineering.
allowed-tools	Read,Grep,Glob,Edit,Write,Bash
metadata	{"category":"AI & Machine Learning","tags":["rag","retrieval","llm","vector-search","bm25","reranking","embeddings"],"provenance":{"kind":"first-party","owners":["port-daddy"]},"pairs-with":[{"skill":"llm-router","reason":"Owns the model/cost routing for the generation stage that sits downstream of the retrieval pipeline designed here."},{"skill":"logging-observability","reason":"The per-query structured logs and OTel spans (embed, retrieve, rerank, generate) that this skill's quality gates require."}],"io-contract":{"kind":"deliverable","consumes":["[Truncated]","[Truncated]"],"produces":["[Truncated]","[Truncated]"]}}

RAG Retrieval Pattern Design

When RAG fails, retrieval is the bug ~73% of the time — and ~80% of those trace to the ingestion / chunking layer. (premai.io — Building Production RAG (2026)) The LLM is faithfully reporting what you handed it; the problem is what you handed it. So the order of work is fixed: chunking → embeddings → hybrid search → rerank → eval → freshness. Get those right and you're at 90% of the achievable quality. Spend the same week on prompt engineering and you'll move 5%.

The compressed 2026 stack:

chunking            recursive 512-token chunks, 10-20% overlap
embeddings          voyage-3-large (or text-embedding-3-large; pick by MTEB on YOUR domain)
retrieval           hybrid: BM25 + dense, fused via RRF (k=60), top-20
rerank              cross-encoder, return top-5
eval                RAGAS — faithfulness ≥ 0.85, context precision ≥ 0.75
freshness           daily reindex; hourly for real-time domains

Jump to your fire:

• "RAG returns the wrong doc" → Retrieval failure mode triage
• Choosing chunk size → Chunking
• BM25 vs dense vs hybrid → Hybrid retrieval with RRF
• Adding a reranker → Cross-encoder reranking
• "Faithfulness is 0.95 but answers are still wrong" → Freshness + monitoring
• Running offline evals → RAGAS evaluation
• Latency budget for real-time → Latency budget

When to use

• Building RAG for the first time and need a defensible default architecture.
• Existing RAG cites the wrong documents; need to triage where the failure is.
• Choosing an embedding model and not sure what beats text-embedding-3-large in 2026.
• Adding a reranker to an existing system.
• Setting up evaluation so changes can be measured.
• Designing an index-freshness pipeline (daily / hourly).

Core capabilities

Retrieval failure mode triage

The standard decomposition before debugging anything else:

Symptom	Likely layer	First-line check
Wrong document retrieved