agentsop-llamaindex

# LlamaIndex · SOP > Third-person analytical view of how LlamaIndex *thinks* about turning private > documents into a grounded answering system. The skill is for an LLM agent that > writes / reviews / debugs RAG code — not for an end user reading docs. --- ## 何时激活 (Activation Rules) Activate this skill when any of the following holds: 1. The user's request involves building, modifying, or debugging a **RAG pipeline** (retrieval over private/unstructured data + LLM synthesis). 2. The user mentions **LlamaIndex** (`from llama_index...`), **LlamaParse**, **LlamaCloud**, or a LlamaIndex-style primitive (`VectorStoreIndex`, `SummaryIndex`, `IngestionPipeline`, `QueryEngine`, `SubQuestionQueryEngine`, `RouterQueryEngine`, `Settings`, `Workflows`). 3. The user is **comparing RAG frameworks** (LlamaIndex vs LangChain vs Haystack vs raw vector store). 4. The user is choosing between **stuffing context, RAG, or an agent** for a knowledge task. 5. The user is debugging retrieval quality (hallucinations, wrong chunks, stale data, embedding drift) — even if the codebase predates LlamaIndex, the failure-mode taxonomy applies. 6. The user is **evaluating** a RAG system (faithfulness, relevancy, MRR, hit-rate). Do **not** activate when: - The task is pure agent orchestration with no retrieval (use LangGraph/CrewAI skill instead). - The corpus is tiny (<100k tokens, static) and prompt-stuffing is the correct answer. - The data is pure SQL/tabular with no unstructured component. --- ## 核心心智模型 (Core Mental Model) LlamaIndex's design rests on three principles that distinguish it from "vector DB SDK + custom glue": ### Principle 1 — The Index is a noun, not a verb > In LangChain, "indexing" is something you do to a vector store. In LlamaIndex, an `Index` is a first-class typed object with its own retrieval semantics. Picking the right Index is half the architecture decision. The 5-layer pipeline: ``` Documents → Nodes → Index → Retriever → Query Engine → Response ↓ ↓ ↓ ↓ ↓ parsing chunking storage filters synthesis metadata graph primitive rerank (refine/tree_sum/compact) ``` Each layer has a **distinct failure mode** and a **distinct optimization knob**. See `references/R1-architecture.md` for the layer-failure-knob mapping. ### Principle 2 — A Node is a graph node, not a chunk A `Node` carries: `text`, `metadata`, `embedding`, **`relationships`** (PREV/NEXT/PARENT/CHILD links), and lifecycle ids. The `relationships` field is what enables Hierarchical, Auto-Merging, and Sentence-Window retrieval. The mental flip: **don't think "split into chunks", think "build a chunk-graph"**. ### Principle 3 — Indices are not interchangeable | Index | Pick when | |---|---| | `VectorStoreIndex` | Default; semantic Q&A over chunks; ~90% of RAG cases | | `SummaryIndex` | "Summarize this whole doc" — small, fan-out synthesis | | `TreeIndex` | Hierarchical content with progressive zoom-in | | `KeywordTableIndex` | Keyword-heavy queries, no embeddings budget | | `PropertyGraphIndex` | Multi-hop reasoning over entities | | `DocumentSummaryIndex` | Mixed corpora needing document-level routing first | A `RouterQueryEngine` over multiple per-task indices is often the correct top-level shape, not a single monolithic `VectorStoreIndex`. ### The 2025 shift LlamaIndex now positions as **"the leading document agent and OCR platform"** (README). `LlamaParse v2` + `Workflows 1.0` (June 2025) + `LlamaCloud` mark a strategic move from "RAG framework" to "platform between messy documents and document-grounded agents". For a coder agent: assume Workflows for any new agentic code (QueryPipeline is deprecated). --- ## SOP 工作流 (Agentic Protocol) The protocol every RAG implementation must walk through. Each stage gates on the next. ### Stage 0 — Frame the problem Before code, answer: 1. Is the corpus **unstructured + non-trivial size (>100k tokens) + growing**? If not → see `R4` boundaries; LlamaIndex may be the wrong tool. 2. Is **retrieval quality** the bottleneck (not orchestration)? If orchestration dominates → LangGraph leads, LlamaIndex becomes a retrieval tool *inside* it. 3. What is the **query distribution**? (lookup-only / summary / compare-contrast / mixed). This decides whether a single Index or a Router is needed. ### Stage 1 — Baseline (cheap, fast, observable) ```python from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings from llama_index.core.node_parser import SentenceSplitter Settings.llm = OpenAI(model="gpt-4o-mini") Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small") Settings.node_parser = SentenceSplitter(chunk_size=1024, chunk_overlap=20) docs = SimpleDirectoryReader("./data").load_data() index = VectorStoreIndex.from_documents(docs) qe = index.as_query_engine(similarity_top_k=4) ``` Pin `Settings` **once at app boot**, never inline. This eliminates the entire embedding-mismatch failure class (failure #4). ### Stage 2 — Build the eval loop **before** optimizing anything ```python from llama_index.core.evaluation import ( DatasetGenerator, FaithfulnessEvaluator, RelevancyEvaluator, RetrieverEvaluator, ) qa = DatasetGenerator.from_documents(docs).generate_dataset_from_nodes(num=50) ``` Track {MRR, hit-rate, faithfulness, relevancy, p95 latency}. **Every** subsequent change must be gated on these numbers. > Most RAG failures in production trace to weak retrieval or sloppy ingestion — not the LLM. The eval loop is what surfaces them. ### Stage 3 — Optimize in LlamaIndex's recommended order From the official `basic_strategies` guide: 1. **Prompt engineering** (cheapest) 2. **Embedding model** (pick from MTEB; full re-embed if you change it) 3. **Chunk size sweep** ({256, 512, 1024, 2048}; default 1024 for prose, 80-160 for code) 4. **Hybrid search** (BM25 + dense) — *only if traffic contains lexical-identity queries* 5. **Metadata filters** — for multi-tenant / multi-collection corpora 6. **Document/chunk decoupling** — `HierarchicalNodeParser`+`AutoMergingRetriever` *or* `SentenceWindowNodeParser` 7. **Reranking** (Cohere / SentenceTransformer / ColBERT) — widen top_k to 20-50, rerank to 3-5 Note the order: **prompts first, reranking last**. Reranking is high-impact but expensive — exhaust cheap knobs first. ### Stage 4 — Compose for query heterogeneity | Query shape | Right primitive | |---|---| | "Summarize doc X" | `SummaryIndex` per doc, routed | | "Find the clause about X" | `VectorStoreIndex` + metadata filters | | "Compare X and Y across docs" | `SubQuestionQueryEngine` | | "What entities relate to X?" | `PropertyGraphIndex` | | Mixed | `RouterQueryEngine` over per-task engines | ### Stage 5 — Production hardening Apply the failure-mode checklist (`R4`). Top 5 non-negotiables: - `IngestionPipeline` with `docstore` + `UPSERTS_AND_DELETE` for any live corpus. - `Settings.embed_model` pinned at boot; embedding model name in index metadata. - `tree_summarize` synthesizer when packing many chunks (mitigates lost-in-the-middle). - Tracing/observability captures `query + retrieved_nodes + scores + index_id + LLM prompt` for every failure. - Indices versioned as deployment artifacts; ingestion completes before traffic routing. ### Stage 6 — Escalate to Workflows / Agents (only when justified) Escalate when at least one of: - A retrieval loop is needed ("retrieve → check → re-query"). - Tool calls beyond retrieval (calculator, web, code-exec). - State surviving across query turns. - Multiple specialized retrievers chosen at runtime. Use **Workflows 1.0** (event-driven), not deprecated `QueryPipeline`. Wrap query engines as `QueryEngineTool`s and tune the `description=` carefully — it is the only signal the router/agent reads. --- ## 操作模型 (Operation Models) Each operation: **Trigger / Action / Output / Evidence**. ### OP-01 BaselineVectorIndex - **Trigger**: First-pass RAG over a new corpus; retrieval-quality baseline unknown. - **Action**: `VectorStoreIndex.from_documents()` with `SentenceSplitter(1024, 20)`, `top_k=4`, default synthesizer. Ship to eval bench *before* tuning. - **Output**: Working RAG endpoint + baseline {MRR, hit-rate, faithfulness, relevancy, p95}. - **Evidence**: `developers.llamaindex.ai/python/framework/optimizing/basic_strategies/basic_strategies/` ### OP-02 TuneChunkSize - **Trigger**: Faithfulness below target OR retrieved chunks visibly truncated/incomplete. - **Action**: Sweep `chunk_size ∈ {256, 512, 1024, 2048}` with overlap at ~10-20%; re-evaluate faithfulness + relevancy + latency. Default land: **1024 for prose, 80-160 for code**. - **Output**: Optimal `chunk_size` pinned + embedding model version locked in index metadata. - **Evidence**: `llamaindex.ai/blog/evaluating-the-ideal-chunk-size-for-a-rag-system-using-llamaindex-6207e5d3fec5` (faithfulness peaked at 1024 in LlamaIndex's own eval on Uber 10-K). ### OP-03 AddReranker - **Trigger**: Top-1 wrong but relevant docs appear in top-k (failure #1 / #10). - **Action**: Add `CohereRerank` or `SentenceTransformerRerank` as a NodePostprocessor; widen retrieval top_k to 20-50, narrow to `top_n=3-5` after rerank. - **Output**: Faithfulness lift typically 5-15pp on noisy corpora; lower context-window pressure. - **Evidence**: `developers.llamaindex.ai/python/framework/optimizing/rag_failure_mode_checklist/` (#1, #10). ### OP-04 AddHybridBM25 - **Trigger**: Traffic contains exact identifiers, error codes, SKUs, code symbols, rare jargon — pure dense silently misses them. - **Action**: `QueryFusionRetriever([vector_retriever, BM25Retriever])` *or* vendor hybrid (Qdrant/Milvus alpha). Tune alpha **per query type**, not globally. - **Output**: Recall lift on lexical-identity queries with no degradation on semantic queries. - **Evidence**: `llamaindex.ai/blog/llamaindex-enhancing-retrieval-performance-with-alpha-tuning-in-hybrid-search-in-rag-135d0c9b8a00`; BM25Retriever docs. ### OP-05 DecoupleChunkScope - **Trigger**: Chunk-size sweep produces no single winner (small wins precision, large wins context). - **Action**: `HierarchicalNodeParser` + `AutoMergingRetriever` (for structured docs) *or* `SentenceWindowNodeParser` + `MetadataReplacementPostProcessor` (for flat prose). Embed small, return large. - **Output**: Precision-recall pareto improvement; LLM gets surrounding context that small chunks alone lost. - **Evidence**: AutoMergingRetriever / Hierarchical / SentenceWindow docs on `developers.llamaindex.ai`. ### OP-06 RouteByQueryType - **Trigger**: Corpus serves heterogeneous tasks (summary / lookup / compare) from one entry point. - **Action**: Build per-task `QueryEngines` (`SummaryIndex` for digest, `VectorStoreIndex` for lookup, `SubQuestionQueryEngine` for compare) + a `RouterQueryEngine` with LLM or Pydantic selector. Carefully author each `QueryEngineTool.description`. - **Output**: Each query lands on the structurally-correct retrieval primitive; latency stays bounded. - **Evidence**: DeepLearning.AI *Building Agentic RAG with LlamaIndex*; router docs. ### OP-07 DecomposeMultiHop - **Trigger**: Compare/contrast queries; queries needing facts from >1 document; "what changed between X and Y?". - **Action**: `SubQuestionQueryEngine` decomposes query → dispatches sub-questions to sub-engines → synthesizes. - **Output**: Multi-hop answers a single retrieval cannot assemble. - **Evidence**: `developers.llamaindex.ai` sub-question query engine docs. ### OP-08 IngestionWithDocstore - **Trigger**: Documents will update/delete over time (any production system). - **Action**: `IngestionPipeline(transformations=..., docstore=..., vector_store=..., docstore_strategy=UPSERTS_AND_DELETE)`. Run on a schedule, not manually. - **Output**: Idempotent re-ingestion; no duplicate vectors; deletes propagate. - **Evidence**: `developers.llamaindex.ai/python/framework/module_guides/loading/ingestion_pipeline/`; failure #3. ### OP-09 MetadataFilters - **Trigger**: Multi-tenant corpus; cross-contamination between sub-collections; access control needed. - **Action**: Inject structured metadata at ingestion (`tenant`, `doc_type`, `date`); apply `MetadataFilters` at query time OR enable auto-retrieval to let an LLM emit filters. - **Output**: Hard isolation between tenants; targeted retrieval without expensive rerank. - **Evidence**: Failure #7; `basic_strategies` metadata filters section. ### OP-10 EvalLoop - **Trigger**: Any non-trivial RAG, pre-deploy AND continuously in production. - **Action**: `DatasetGenerator` → labeled QA pairs; run `FaithfulnessEvaluator` + `RelevancyEvaluator` + `RetrieverEvaluator(["mrr","hit_rate"])`. Gate every change. - **Output**: Quantitative regression test for every chunking / embedding / retriever / prompt change. - **Evidence**: `developers.llamaindex.ai/python/framework-api-reference/evaluation/`; `cookbook.openai.com/examples/evaluation/evaluate_rag_with_llamaindex`. ### OP-11 LockGlobalSettings - **Trigger**: Multiple modules each instantiate LLM/embed independently — drift risk. - **Action**: Set `Settings.llm` and `Settings.embed_model` once in app bootstrap. Forbid inline overrides in PR review. - **Output**: Eliminates failure #4 (config drift) and #5 (embedding mismatch). - **Evidence**: `docs.llamaindex.ai/en/stable/module_guides/supporting_modules/service_context_migration/`. ### OP-12 AgenticWorkflow - **Trigger**: Need loops, tool calls beyond retrieval, multi-step reasoning, or state across turns. - **Action**: Build a Workflows 1.0 event-driven workflow OR a `FunctionAgent`/`ReActAgent` with `QueryEngineTool`s. Do NOT use the deprecated `QueryPipeline`. - **Output**: Cycle-capable agentic system with retrieval as one tool among many. - **Evidence**: `llamaindex.ai/blog/announcing-workflows-1-0-a-lightweight-framework-for-agentic-systems`. --- ## 困境决策案例 (Dilemma Cases) (Full text in `references/R3-dilemma-cases.md`. Summarized here.) ### Dilemma 1 — Chunk size: precision vs context **困境**: Small chunks → precise embeddings, fragmented context for the LLM. Large chunks → rich context, embeddings become "topic averages", recall on specific queries drops. Failure modes #2 and #6 are the two poles. **约束**: Embedding model has a fixed input window; metadata is propagated into payload (so very small chunks become all-metadata — GitHub `#12200`, `#13792`); token budget caps how many chunks fit downstream. **决策步骤**: 1. Generate ~20 eval QA pairs. 2. Sweep `chunk_size ∈ {128, 256, 512, 1024, 2048}` with overlap = 10-20%. 3. Build a VectorStoreIndex per config; record faithfulness, relevancy, latency. 4. If a single winner emerges → pin it. 5. If the frontier is non-flat → **do not compromise**; switch to small-embed/large-return via Hierarchical+AutoMerging *or* SentenceWindow. **结果**: LlamaIndex's own published study (Uber 10-K) peaked at **1024** on both faithfulness and relevancy → 1024 became the framework default for prose. For code: 80-160 tokens. When the eval doesn't converge, decoupling wins; never average two bad chunk_sizes.