Menu
Precompute hypothetical questions during indexing to eliminate runtime query expansion overhead.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | hype-hypothetical-prompt-embeddings |
| title | HyPE - Hypothetical Prompt Embeddings |
| description | Precompute hypothetical questions during indexing to eliminate runtime query expansion overhead. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | retrieval-strategies |
| tags | ["hype","precomputed-queries","indexing-time","query-alignment"] |
HyPE (Hypothetical Prompt Embeddings) transforms retrieval from query-document matching to question-question matching by generating multiple hypothetical questions for each document chunk during the indexing phase. This eliminates runtime LLM calls while significantly improving retrieval precision.
Hybrid and enhanced retrieval techniques often require runtime query expansion:
For each document chunk, generate 3-5 hypothetical questions that would retrieve it.
Why: Multiple questions per chunk create diverse entry points for retrieval, increasing the likelihood of matching various user query formulations.
Direct an LLM to create questions that, when answered, capture the main points of each chunk.
Generate embeddings for all hypothetical questions and store them with their source chunks.
Why: Question embeddings align with user query embeddings better than raw document embeddings, while maintaining a link to original content for answer generation.
Embed each question using your preferred embedding model and maintain a mapping to the source chunk.
Create an index structure that maps question embeddings to their source chunks.
Why: The index needs to efficiently retrieve the original chunk content after finding matching question embeddings. This requires maintaining proper mapping metadata.
Build a vector index where each question embedding points to its corresponding source chunk.
Query the index directly without runtime question generation.
Why: Since questions are precomputed, retrieval is as fast as standard vector search - no LLM latency or cost at query time.
Perform standard vector similarity search using the user's query against the precomputed question embeddings.
Use parallel processing and batching for efficient index building.
Why: Index construction is one-time but can be time-consuming. Efficient processing reduces deployment time and resource usage.
Process multiple chunks in parallel and batch embedding calls when possible.
Combine all components into a cohesive system.
Why: A complete pipeline with configurable parameters allows for optimization and experimentation with different window strategies.
For implementation patterns, see the HyPE paper, FAISS documentation, and vector database documentation for multi-vector indexing.
