agentsop-reranker-stage

name	agentsop-reranker-stage
version	0.1.0
description	Enhancement-overlay SOP for the reranker stage of a RAG pipeline — the "retrieve wide, rerank narrow" discipline. Activate when a calling agent owns a retrieval pipeline whose answers have plateaued: top-k contains the right document but it is buried below noise, or the context window is under pressure from too many marginal chunks. Encodes the one non- negotiable insight — a cheap bi-encoder retrieves *wide* for recall, then a more expensive cross-encoder (which reads query + document *together*) reranks *narrow* for precision; keep top-N=20-50, rerank to top-k=3-5. Covers when to add a reranker (and when not to), N-vs-k tuning, model choice (Cohere/Voyage API vs bge-reranker local vs SentenceTransformer cross-encoder), latency/cost budgeting, and the cross-framework mapping (LlamaIndex node postprocessors, LangChain ContextualCompressionRetriever, Cohere/Voyage rerank APIs, local cross-encoders). This is an ENHANCE overlay over the per- framework skills — cross-link `[[llamaindex]]` and `[[agentsop-hybrid-retrieval]]` for the deep API. Search keywords: cross-encoder, Cohere rerank, bge reranker, rerank RAG, ColBERT, retrieval reranking, improve RAG precision, retrieve wide rerank narrow.trigger_keywords: - "reranker" - "rerank" - "cross-encoder" - "CohereRerank" - "bge-reranker" - "SentenceTransformerRerank" - "node postprocessor" - "ContextualCompressionRetriever" - "voyage rerank" - "top_n after retrieval" - "retrieve wide rerank narrow"
when_to_use	["a RAG pipeline's answer quality has plateaued and the relevant doc is present in top-k but not ranked first","the LLM context window is pressured by too many marginal chunks and you want fewer, higher-precision chunks","retrieval recall is already good (hit-rate high) but MRR / top-1 precision is low","a user asks where to add a reranker, how to tune N vs k, or which rerank model to use (API vs local)","reviewing a pipeline with `similarity_top_k=20` and no postprocessor — a reranker is missing"]
when_not_to_use	["retrieval RECALL is the problem (the right doc is NOT in top-N at all) — fix retrieval/chunking/hybrid first","top-k already small (<=5) and answers are correct — no plateau, no lever to pull","hard real-time path where the extra rerank round-trip blows the latency budget and quality is already acceptable"]
overlay	true
cross_links	["llamaindex","hybrid-retrieval"]

Reranker Stage · SOP

Third-person analytical view of how a mature RAG pipeline thinks about the reranker. The skill is for an LLM agent that writes / reviews / debugs retrieval code — it teaches the cross-framework reranking discipline, not one vendor's API. For the per-framework API, descend to [[llamaindex]] (node postprocessors) or [[agentsop-hybrid-retrieval]] (the recall stage that feeds the reranker).

This is the C4 gap skill in the Phase-D enhance pass. The reranker SOP existed only buried inside [[llamaindex]] (OP-03 AddReranker, Stage 3 step 7, anti-pattern A6). It is the highest-ROI single addition to a naive RAG pipeline, so it earns a standalone overlay.

---

1 · 何时激活 (Activation Rules)

Activate when any holds:

1. A RAG pipeline's answer quality has plateaued after the cheap knobs (prompt, embedding model, chunk size) are exhausted — [[llamaindex]] Stage 3 lists reranking as the last optimization step, deliberately.
2. Diagnostics show the relevant document is in top-k but buried — high hit-rate, low MRR, wrong top-1. This is LlamaIndex failure modes #1 / #10 ([[llamaindex]] OP-03).
3. The LLM context window is under pressure — too many marginal chunks inflate cost, latency, and "lost-in-the-middle" degradation. A reranker lets you retrieve 50 and feed 5.
4. A user asks where to add a reranker, how to tune N vs k, or API vs local.

Do not activate (boundary — see §6):

• Recall is the bottleneck: the right doc is not in top-N at all. A reranker can only reorder what retrieval already found — fix retrieval, hybrid ([[agentsop-hybrid-retrieval]]), or chunking first.
• top-k is already small (≤5) and answers are correct — no plateau.
• A hard sub-100ms path where the extra round-trip is unaffordable and quality is already acceptable.

---

2 · 核心心智模型 (Core Mental Model)

The one sentence

Retrieve wide for recall with a cheap bi-encoder; rerank narrow for precision with an expensive cross-encoder that sees query + document together — something the bi-encoder structurally could not do.

Why two stages exist at all

The retriever (bi-encoder / vector search) embeds the query and every document . Similarity is a dot product of two vectors that never met. This is (vectors are precomputed; ANN search is sub-linear) but : the document's vector is a single "topic average" computed without knowledge of the query.