Menu
Combine BM25 and dense retrieval signals to improve search quality across keyword and semantic queries.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | hybrid-search-bm25-dense |
| title | Hybrid Search: BM25 + Dense |
| description | Combine BM25 and dense retrieval signals to improve search quality across keyword and semantic queries. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | retrieval-strategies |
| tags | ["hybrid","bm25","dense","fusion","keyword"] |
Hybrid search combines BM25 (keyword search) with dense vector embeddings (semantic search) to leverage both exact term matching and semantic understanding. This dual approach improves retrieval quality across diverse query types - keyword queries get exact matches, while semantic queries capture conceptual relationships.
Pure semantic search or pure keyword search each have limitations:
Implement vector-based semantic search with your chosen embedding model.
Why: Dense search captures semantic meaning but may miss exact term matches that are crucial for many queries.
Implement keyword-based search using BM25.
Why: BM25 excels at exact term matching, domain-specific terminology, and proper noun searches where semantic models might struggle.
Normalize scores from different ranking methods to comparable scales.
Why: Different scoring systems (cosine similarity vs. BM25 scores) can't be directly combined - normalization creates a common scale.
Combine results from multiple search methods.
Why: RRF is robust to different scoring scales and doesn't require explicit score normalization, making it ideal for fusion.
Combine all components into a single interface.
Why: A unified interface simplifies usage and allows easy experimentation with different fusion strategies and weights.
For implementation patterns, compare OpenSearch hybrid search docs, OpenSearch score ranker processor, the reciprocal rank fusion paper, and Sentence Transformers retrieve & re-rank.
