rag-pipeline-optimizer

name

rag-pipeline-optimizer

description

Audit and tune RAG (retrieval-augmented generation) pipelines for compute efficiency — chunk sizing, embedding-model right-sizing, retrieval-k tuning, rerank-only-when-needed, context stuffing, and index refresh cadence. Use this skill whenever the user shares a RAG setup (LangChain/LlamaIndex configs, vector DB settings, retrieval code), complains RAG answers are slow or costly, or is designing document Q&A / knowledge-base search over an LLM. Part of Lean Agentic AI Skills; emits lean-findings.json.

RAG Pipeline Optimizer

Producer skill. Input: pipeline configs/code, sample queries with retrieved contexts, index stats. Output: lean-findings.json.

RAG multiplies every user question into embedding calls, vector searches, optional reranks, and a generation whose prompt carries everything retrieved. Each stage has a dial that teams leave at the default — and the defaults are generous.

Subject type: emit subject.type: "ai-inference" in findings.

Signatures

• Context stuffing — retrieved chunks pasted into the prompt beyond what the answer uses; top-k high "to be safe". High: generation tokens are the expensive stage, and prompt size scales linearly with k × chunk size. Evidence: actual k, chunk size, and resulting prompt token counts from samples. Fix: lower k with an eval; retrieval-confidence cutoffs (drop chunks below similarity threshold rather than always sending k).
• Oversized chunks — chunks so large that one retrieval fills the context; or so small that k must be huge. Medium-high. Fix: chunk-size sweep on the user's own eval set; sentence-aware splitting.
• Frontier embeddings for retrieval — largest embedding model where a small one holds recall on the corpus. Medium-high; embedding runs on every query AND every indexed document. Fix: benchmark small vs large embedding recall on a labeled sample; smaller dimension also shrinks the index (M↓).
• Rerank everything — cross-encoder rerank on every query, including ones where first-stage scores are already decisive. Medium. Fix: rerank only when top-k scores are close (margin gate).
• Full re-embedding on every refresh — index rebuilds embedding unchanged documents (no content hashing). High on large corpora. Fix: delta indexing by content hash.
• No retrieval cache — repeated/near-duplicate queries re-embed and re-search; route design to llm-cache-designer (partial-pipeline caching), record the finding here.
• Answering without retrieval need — every query goes through RAG including chit-chat/formatting requests; a router should bypass retrieval when the query doesn't need documents. Medium.
• Generation without citations budget — unbounded answer length compounding the stuffed context; pair with prompt-carbon-optimizer output caps.

Honesty rules

Every dial change (k, chunk size, embedding model, rerank gate) ships with the same instruction: measure answer quality on the user's eval set before/after — retrieval quality regressions are silent and this skill must never cause one invisibly. Token counts real; energy claims never.

Cost signal (countable)

Cost drivers: generation tokens per query (dominant), embedding tokens per indexed doc + per query, and rerank calls per query. Every dial change (k, chunk size, rerank gate) should map to one of these in cost_signal.

Not this skill's job

Prompt body optimization (prompt-carbon-optimizer), generation model choice (model-right-sizer), cache implementation (llm-cache-designer).