build-vector-search

name	build-vector-search
description	Builds semantic/vector search — pick an embedding model + dimensionality (and whether to truncate Matryoshka dims) and the matching distance metric (cosine/dot/L2, normalize to unit length so cosine == dot and IP is correct), an ANN index with the recall/latency/memory tradeoff understood (HNSW M/efConstruction/efSearch for low-latency RAM-resident; IVF-PQ nlist/nprobe/PQ for billion-scale compressed; flat/exact for <100k) in pgvector/Qdrant/Milvus/FAISS/Pinecone, chunking + overlap + per-chunk metadata for filtering, HYBRID retrieval fusing BM25 + dense by Reciprocal Rank Fusion (RRF, k≈60) not score addition, a cross-encoder/Cohere reranker over the top-50→k, correct pre-filter-vs-ANN interaction (filterable HNSW, not post-filter that starves k), and offline eval with recall@k / nDCG@10 / MRR against a labeled qrels set. Quantize (scalar/PQ) only after measuring recall loss; tune efSearch/nprobe to a recall target, not a guess.
when_to_use	Building or tuning the embedding + vector-index + retrieval-quality core — choosing an embedding model/dim/metric, sizing/tuning an HNSW or IVF-PQ index for a recall@k target, adding hybrid (BM25+vector via RRF) or a reranker, fixing pre-filtering that tanks recall, or running a recall@k/nDCG eval. Distinct from rag-pipeline (the full retrieve-augment-generate app — prompt assembly, grounding, citations, hallucination control; this skill is the retrieval engine it embeds) and design-search-index-infra (the lexical/inverted-index + cluster topology + zero-downtime reindex infra; this skill owns the embedding model, distance metric, ANN params, and relevance eval rather than shard/analyzer/capacity design).

When to Use

Reach for this skill when the task is the quality and mechanics of vector retrieval itself — embeddings, the ANN index, hybrid/rerank, and measuring relevance:

• "Pick an embedding model + dimensionality + distance metric for semantic search"
• "Our ANN search misses obvious matches" / "tune HNSW/IVF for recall@10 without blowing latency"
• "pgvector / Qdrant / Milvus / FAISS / Pinecone — which index and what parameters?"
• "Add hybrid search (BM25 + vector) and a reranker" / "results are semantically close but wrong-ranked"
• "Filtering by metadata returns too few results / wrong ones" (pre-filter vs ANN)
• "How do I know retrieval got better?" → recall@k / nDCG / MRR eval
• "Quantize to fit in RAM" / "embeddings cost/latency too high"

NOT this skill:

• The end-to-end retrieve→augment→generate app — prompt assembly, context packing, grounding, citations, hallucination control → rag-pipeline (this skill is the retrieval core it calls; tune retrieval here, wire the LLM there)
• Lexical/inverted-index search infra — Elasticsearch/OpenSearch analyzers & mappings, shard/replica topology, capacity sizing, alias-based zero-downtime reindex → design-search-index-infra (it owns BM25 analyzer config + cluster ops; this skill owns the embedding model, metric, ANN params, and relevance eval)
• Measuring LLM answer quality (faithfulness, answer correctness, LLM-as-judge) → llm-eval-harness (this skill evals retrieval — recall@k/nDCG — not generation)
• Cutting embedding/inference cost & latency at the model/serving layer (batching, caching, model size) → optimize-llm-cost-latency
• The BM25/keyword half as a standalone full-text feature with no vectors → design-search-index-infra
• Picking a document/KV store schema unrelated to vectors → model-nosql-data; relational schema for the metadata table → design-relational-schema
• Profiling the corpus before indexing (length distribution, dupes, language mix) → profile-dataset

Steps

1. Pick the embedding model, dimensionality, and distance metric together — they're coupled. Don't default to text-embedding-ada-002 (legacy). 2025-2026 strong choices:

   Model	Dim	Notes
   OpenAI text-embedding-3-large	3072 (truncatable to 256/1024)	Matryoshka — truncate then re-normalize; strong general
   OpenAI text-embedding-3-small	1536 (truncatable)	cheap, good baseline
   Cohere embed-v3 / embed-v4	1024	has input_type (query vs document) — use it
   BAAI/bge-large-en-v1.5, intfloat/e5-large-v2	1024	open, self-host; require a prefix (query: / passage:) — omitting it craters recall
   BAAI/bge-m3	1024	multilingual + multi-vector
   Voyage voyage-3	1024	strong retrieval, code/domain variants

   Rules: embed the query and the document with the SAME model (and the right input_type/prefix). Higher dim ≈ better recall but more RAM/latency — Matryoshka models let you truncate (e.g. 3072→1024) and trade recall for cost; re-normalize after truncating. Metric choice:

   Metric	Use when	pgvector op	Note
   Cosine	text embeddings (default)	<=> (vector_cosine_ops)	direction only
   Dot / inner product	already unit-normalized vectors	<#> (negative IP)	== cosine when normalized; faster
   L2 / Euclidean	rarely for text; some image models	<-> (vector_l2_ops)	magnitude matters

Common Errors

• Embedding query and documents with different models (or wrong input_type/prefix). Vectors live in different spaces → garbage similarity. Fix: same model both sides; set Cohere input_type, E5/BGE query:/passage: prefixes.
• Metric/opclass mismatch or un-normalized vectors with cosine/IP. A cosine index on un-normalized vectors mis-ranks; IP on un-normalized ≠ cosine. Fix: normalize to unit length at write time, pick the matching opclass (vector_cosine_ops etc.).
• Tuning by feel instead of to a recall target. Picking efSearch/nprobe "that seems fine" hides recall cliffs. Fix: exact search as ground truth, raise the knob until recall@k ≥ target, then stop.
• Post-filtering a selective metadata filter. ANN returns k, the filter drops most → too few/empty results. Fix: filterable ANN (payload/WHERE index) or pre-filter/partition per tenant.
• Weighted score-sum hybrid instead of RRF. BM25 and cosine scales differ wildly; one dominates. Fix: fuse by rank with RRF (k≈60) — no score normalization needed.
• Building an IVFFlat index before loading data. It clusters on existing rows; empty → degenerate. Fix: load data, then build IVFFlat (HNSW is fine on empty).
• No overlap / mid-sentence chunking. Facts split across boundaries become unretrievable. Fix: 10–15% overlap, split on semantic boundaries.
• Reranking the whole index. Cross-encoders are O(N) per query → unusable latency. Fix: rerank only the top-50–100 shortlist.
• Quantizing without measuring. Silent recall drop in prod. Fix: measure recall@k before/after; add a full-precision rescore pass.
• Mixing embedding spaces after a model upgrade. New and old vectors are incomparable. Fix: store model+dim in metadata; re-embed the whole corpus on upgrade.
• HNSW out-of-memory at scale. The graph is RAM-resident; tens of millions × high M × float32 blows the budget. Fix: lower M, scalar-quantize, or switch to IVF-PQ / DiskANN.

Verify

1. Metric/normalization correct: vectors are unit-normalized; the index opclass matches the metric; a known query returns its known-relevant doc as a top hit.
2. Same-model invariant: grep the pipeline — query and document embeddings use the identical model + correct input_type/prefix.
3. Recall measured against exact search: flat/brute-force gives the ground truth; ANN recall@k is computed and meets target (e.g. ≥0.95) at the chosen efSearch/nprobe, with latency recorded.
4. Filter recall holds: run the eval with the production metadata filter applied; recall doesn't collapse (no post-filter starvation), and selective filters use pre-filter/partition.
5. Hybrid fuses by RRF: BM25 and dense both contribute; fusion is rank-based (RRF k≈60), and hybrid recall@k ≥ either retriever alone on the eval set.
6. Rerank improves nDCG, not latency-killing: cross-encoder runs over the top-50–100 only; nDCG@10 improves vs pre-rerank; added latency is within budget.
7. Chunking sound: chunks are 256–512 tokens with 10–15% overlap on semantic boundaries, each carrying filter/citation metadata; a boundary-straddling fact is retrievable.
8. Quantization is net-positive: recall@k before/after quantization is measured; any drop is recovered by a full-precision rescore pass and is within tolerance.
9. Index choice fits scale/memory: the index type (flat/HNSW/IVF-PQ/DiskANN) matches corpus size and the RAM budget; HNSW graph fits in memory or a compressed index was chosen.

Done = query and documents share one normalized embedding model with a matching distance metric/opclass, the ANN index is chosen for the corpus's scale/latency/memory budget and tuned to a measured recall@k target against exact search, hybrid retrieval fuses BM25 + dense by RRF, a cross-encoder reranks the shortlist, metadata filtering uses filterable/pre-filter (not post-filter starvation), and every change is validated by the recall@k / nDCG / MRR eval in checks 3–8.