Menu
Use LLM-generated hypothetical documents to bridge the query-document gap and improve retrieval relevance.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | hyde-hypothetical-document-embeddings |
| title | HyDE - Hypothetical Document Embeddings |
| description | Use LLM-generated hypothetical documents to bridge the query-document gap and improve retrieval relevance. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | retrieval-strategies |
| tags | ["hyde","query-expansion","llm-generation","semantic-gap"] |
HyDE (Hypothetical Document Embeddings) is a query expansion technique that generates a hypothetical document answering the user's query, then uses this synthetic document as the query for vector search. This bridges the semantic gap between short queries and longer, more detailed document representations.
Traditional RAG systems face a fundamental mismatch:
Use an LLM to generate a detailed document answering the user's query.
Why: The generated hypothetical document shares characteristics (length, style, content structure) with actual documents in your corpus, enabling better similarity matching in embedding space.
Direct the LLM to create a comprehensive answer that matches the typical length and style of documents in your collection.
Match the hypothetical document size to your average chunk size.
Why: Documents of similar length will have more comparable embedding representations. Mismatched lengths can create distance artifacts unrelated to semantic meaning.
Target a specific character count or token count that matches your chunking strategy (typically 400-1000 characters).
Use the generated document as the query for similarity search instead of the original query.
Why: The hypothetical document is in the same style and semantic space as your indexed documents, allowing the vector database to find semantically similar content more effectively.
Replace the original query with the generated hypothetical document when performing vector similarity search.
Filter retrieved documents for actual relevance to the original query.
Why: While HyDE improves retrieval, some results may be semantically similar but factually irrelevant. Relevance filtering ensures quality.
Evaluate retrieved documents against the original query (not the hypothetical one) to filter out irrelevant matches.
Combine generation, retrieval, and filtering into a cohesive system.
Why: A complete pipeline with timing and explanation allows optimization, debugging, and production deployment.
For implementation patterns, see the HyDE paper, LlamaIndex HyDE Query Transform, and LangChain HyDE reference.
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
