| name | rag-deep-dive |
| description | Retrieval-Augmented Generation deep dive — chunking strategies, embedding models, vector databases, retrieval quality, reranking, hybrid search, and evaluation. Use this skill when building a RAG pipeline, debugging poor retrieval quality, or choosing between RAG approaches.
|
| category | ai-engineering |
| tags | ["rag","retrieval","embedding","vector-db","llm","search"] |
| related | ["context-engineering","prompt-engineering","agent-evaluation","gcp-vertex-ai-rag","graphrag-multi-service","query-rewriting-rag","rag-for-code","rag-ingestion-pipeline","spring-ai-rag"] |
RAG Deep Dive
RAG turns a general model into a domain expert by grounding its answers in your data. The retrieval quality is the ceiling — no amount of prompting fixes bad retrieval.
When to Use This Skill
- Building a knowledge-base Q&A system
- Grounding an agent's answers in documentation or internal data
- Debugging "the model doesn't know about X" (it's a retrieval miss)
- Choosing chunking / embedding / vector DB strategy
- Evaluating RAG pipeline quality
The RAG Pipeline
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Documents│────►│ Chunking │────►│ Embedding│────►│ Vector DB│
└──────────┘ └──────────┘ └──────────┘ └──────────┘
│
User query ─► Embed query ─► Similarity search ──────────┘
│
Top-K chunks
│
┌───────▼───────┐
│ (Reranker) │ ← optional
└───────┬───────┘
│
┌───────▼───────┐
│ LLM + context│
└───────┬───────┘
│
Answer
1. Chunking
Strategy comparison
| Strategy | Chunk size | Pros | Cons |
|---|
| Fixed-size | 500-1000 tokens, 100-200 overlap | Simple, predictable | Cuts mid-sentence / mid-idea |
| Semantic (paragraph/section) | Natural boundaries (headings, \n\n) | Coherent chunks | Uneven sizes |
| Recursive | Split by \n\n, then \n, then sentence, then word | Balances coherence and size | More complex |
| Document-aware | Markdown headings, code blocks, API endpoints | Best coherence per domain | Requires format-specific logic |
Rules
- Chunk size: 200-800 tokens is the sweet spot. Smaller = more precise but noisier. Larger = more context but diluted relevance.
- Overlap by 10-20%. Prevents cutting important ideas at boundaries.
- Include metadata. Each chunk carries:
source, section, page, url. Critical for citations.
- Don't chunk code the same as prose. Code needs function-level or file-level chunks, not token-based splitting.
2. Embedding
Model selection
| Model | Dims | Speed | Quality |
|---|
OpenAI text-embedding-3-small | 1536 | Fast | Good |
OpenAI text-embedding-3-large | 3072 | Medium | Better |
Cohere embed-english-v3 | 1024 | Fast | Very good |
bge-large-en-v1.5 (open source) | 1024 | Self-host | Good |
Voyage AI voyage-3 | 1024 | Fast | Excellent for code |
- Match embedding model to your domain. Code-focused → Voyage AI or CodeBERT. General → OpenAI or Cohere.
- Same model for indexing and querying. Different models produce incompatible vectors.
- Normalize embeddings if your DB supports cosine similarity natively.
3. Vector Database
| DB | Type | Strengths |
|---|
| pgvector (PostgreSQL) | Extension | Familiar SQL, no new infra, good for < 10M vectors |
| Pinecone | Managed SaaS | Zero-ops, fast, metadata filtering |
| Weaviate | Self-hosted/cloud | Hybrid search (vector + keyword), rich schema |
| Qdrant | Self-hosted/cloud | Rust, fast, good filtering |
| ChromaDB | Embedded | Prototyping, single-machine |
- For < 1M vectors and a PostgreSQL stack, start with pgvector. No new infra.
- For > 10M vectors or extreme latency needs, use a dedicated vector DB.
- Store metadata alongside vectors. Filter by source, date, category before similarity.
4. Retrieval
Similarity search
query_embedding = embed(user_query)
results = vector_db.search(query_embedding, top_k=20, filter={"source": "docs"})
Hybrid search (vector + keyword)
Combine semantic similarity with BM25 (keyword) search:
Vector results (semantic meaning) + BM25 results (exact keywords)
→ Merge with Reciprocal Rank Fusion (RRF)
→ Top-K final results
- Hybrid search consistently outperforms vector-only. Especially for proper nouns, acronyms, error codes.
- Top-K for retrieval: 10-20. More than needed; the reranker narrows down.
5. Reranking
A reranker scores each retrieved chunk against the query with a cross-encoder (much more accurate than embedding similarity, but slower).
Retrieved 20 chunks → Reranker scores each → Take top 5
| Reranker | Quality |
|---|
| Cohere Rerank | Excellent, API |
bge-reranker-v2-m3 | Good, self-host |
| Voyage Rerank | Excellent for code |
- Always rerank for production RAG. Retrieval recall is high but noisy; reranking improves precision significantly.
- Rerank is expensive — apply on top-20, not top-1000.
6. Context Assembly
After retrieval, assemble the prompt:
System: You are a helpful assistant. Answer based only on the provided context.
If the answer is not in the context, say "I don't know."
Context:
[chunk 1: {source: docs/auth.md, content: "..."}]
[chunk 2: {source: docs/api.md, content: "..."}]
User: {query}
- Include source attribution in chunks. The model can cite: "According to docs/auth.md..."
- Cap context at ~30% of the window. Leave room for system prompt, conversation, and response.
- Order chunks by relevance (best first). Primacy effect helps.
- Tell the model to refuse if context doesn't answer. Reduces hallucination.
Evaluation
What to measure
| Metric | What it measures |
|---|
| Retrieval recall@K | % of relevant chunks in top-K |
| Retrieval precision@K | % of top-K that are relevant |
| MRR (Mean Reciprocal Rank) | How high is the first relevant result? |
| Answer correctness | Does the final answer match the expected answer? |
| Faithfulness | Is the answer grounded in the retrieved context (no hallucination)? |
| Answer relevance | Does the answer address the question? |
Building an eval set
- 20-50 question-answer pairs with tagged relevant chunks. Covers retrieval and generation quality.
- Include "unanswerable" questions. The system should say "I don't know", not hallucinate.
- Measure retrieval and generation separately. Bad retrieval + good generation = wrong diagnosis.
Common Failure Modes
| Failure | Cause | Fix |
|---|
| Correct info exists but not retrieved | Embedding mismatch; query phrasing differs from doc | Hybrid search; query rewriting |
| Retrieved but wrong chunk | Chunks too large; relevant info diluted | Smaller chunks; better boundaries |
| Retrieved but model ignores it | Context too long; critical info buried | Rerank; put best chunks first |
| Model hallucinates despite context | Weak "answer from context" instruction | Stronger grounding prompt; add "I don't know" examples |
| Proper nouns / codes not found | Embedding-only search misses exact matches | Hybrid search (BM25 + vector) |
| Stale information | Docs updated but index not | Incremental re-indexing pipeline |
Anti-Patterns
| Anti-pattern | Fix |
|---|
| Fixed 1000-token chunks for everything | Domain-aware chunking |
| No metadata on chunks | Include source, section, date |
| Vector-only search | Add BM25 keyword search (hybrid) |
| Top-3 without reranking | Retrieve top-20, rerank to top-5 |
| "Answer from context" without examples | Add few-shot examples of grounded answers + refusals |
| No retrieval evaluation | Build a golden set; measure recall and precision |
| Same embedding model for code and prose | Use domain-specific models |
| Re-embedding everything daily | Incremental: only changed docs |
Checklist
Related Skills
