Menu
Use hierarchical clustering and summarization to create multi-level document trees for improved retrieval.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | raptor-hierarchical-retrieval |
| title | RAPTOR - Hierarchical Abstractive Retrieval |
| description | Use hierarchical clustering and summarization to create multi-level document trees for improved retrieval. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | retrieval-strategies |
| tags | ["raptor","hierarchical","clustering","summarization","tree-structure"] |
RAPTOR (Recursive Abstractive Processing and Tree-Organized Retrieval) creates a hierarchical tree of document summaries, allowing retrieval at multiple levels of abstraction. This approach balances broad overviews with specific details, enabling more nuanced information retrieval from large document collections.
Large document collections present retrieval challenges:
Embed all documents and cluster them by semantic similarity.
Why: Clustering groups related documents together, enabling summary creation that captures themes across multiple documents rather than individual pieces.
Use your preferred embedding model to create vector representations, then apply clustering algorithms (e.g., Gaussian Mixture Models, k-means).
Generate summaries for each cluster of documents.
Why: Summaries capture the key information from multiple documents, creating a higher-level abstraction that preserves important themes while reducing volume.
For each cluster, concatenate the documents and use an LLM to generate a concise summary.
Recursively apply embedding, clustering, and summarization to build multiple levels.
Why: Multiple levels of abstraction allow the system to retrieve at appropriate granularity - broad context from high levels, specific details from low levels.
Apply the same process to the summaries created in the previous level, building up the tree until reaching a single root summary or maximum levels.
Index all documents and summaries at all levels.
Why: A unified vector index allows efficient similarity search across all abstraction levels, enabling the retriever to select the most appropriate level for each query.
Embed and index all content from all tree levels with metadata indicating the level and origin.
Query across all levels and combine results intelligently.
Why: Multi-level retrieval ensures both broad context and specific details are considered. Combining results from different levels provides comprehensive information.
Retrieve from multiple levels and balance the results to ensure both breadth and depth in the retrieved information.
Extract only relevant parts from retrieved content.
Why: High-level summaries may contain more information than needed for a specific query. Contextual compression focuses on the most relevant portions.
Use an LLM or compression technique to extract the most relevant portions of retrieved content for the specific query.
Combine all components into a cohesive system.
Why: A complete pipeline with timing and explanation allows optimization, debugging, and production deployment.
For implementation patterns, see the RAPTOR paper, LlamaIndex RAPTOR pack, and scikit-learn clustering documentation.
