Cohere RAG Pipeline (Core Workflow A)
Overview
End-to-end Retrieval-Augmented Generation using Cohere's three core endpoints: Embed (vectorize), Rerank (sort by relevance), Chat (generate grounded answer with citations).
Prerequisites
- Completed
cohere-install-auth setup
cohere-ai package installed- Understanding of vector similarity search
Instructions
Step 1: Embed Your Documents
import { CohereClientV2 } from 'cohere-ai';
const cohere = new CohereClientV2();
const documents = [
{ id: 'doc1', text: 'Cohere Command A has 256K context and supports tool use.' },
{ id: 'doc2', text: 'Embed v4 generates 1024-dim vectors with 128K token context.' },
{ id: 'doc3', text: 'Rerank v3.5 scores relevance from 0 to 1 across 100+ languages.' },
{ id: 'doc4', text: 'The Chat API v2 requires model as a mandatory parameter.' },
{ id: 'doc5', text: 'Cohere supports structured JSON output via response_format.' },
];
const docEmbeddings = await cohere.embed({
model: 'embed-v4.0',
texts: documents.map(d => d.text),
inputType: 'search_document',
embeddingTypes: ['float'],
});
const vectors = docEmbeddings.embeddings.float;
console.log(`Embedded ${vectors.length} docs, ${vectors[0].length} dimensions each`);
Step 2: Search — Embed the Query
async function searchDocuments(query: string, topK = 10) {
const queryEmbedding = await cohere.embed({
model: 'embed-v4.0',
texts: [query],
inputType: 'search_query',
embeddingTypes: ['float'],
});
const queryVector = queryEmbedding.embeddings.float[0];
const scores = vectors.map((vec, i) => ({
index: i,
score: cosineSimilarity(queryVector, vec),
}));
return scores
.sort((a, b) => b.score - a.score)
.slice(0, topK)
.map(s => documents[s.index]);
}
function cosineSimilarity(a: number[], b: number[]): number {
let dot = , magA = , magB = ;
( i = ; i < a.; i++) {
dot += a[i] * b[i];
magA += a[i] * a[i];
magB += b[i] * b[i];
}
dot / (.(magA) * .(magB));
}
Step 3: Rerank Retrieved Documents
async function rerankResults(query: string, candidates: typeof documents) {
const response = await cohere.rerank({
model: 'rerank-v3.5',
query,
documents: candidates.map(d => d.text),
topN: 3,
});
return response.results.map(r => ({
...candidates[r.index],
relevanceScore: r.relevanceScore,
}));
}
Step 4: Generate Grounded Answer with Citations
async function ragAnswer(query: string) {
const candidates = await searchDocuments(query);
const topDocs = await rerankResults(query, candidates);
const response = await cohere.chat({
model: 'command-a-03-2025',
messages: [{ role: 'user', content: query }],
documents: topDocs.map(d => ({
id: d.id,
data: { text: d.text },
})),
});
const answer = response.message?.content?.[0]?.text ?? '';
const citations = response.message?.citations ?? [];
return { answer, citations, sources: topDocs };
}
const result = await ragAnswer('What context length does Command A support?');
console.log(, result.);
.(, result..);
Complete Pipeline (Copy-Paste Ready)
import { CohereClientV2 } from 'cohere-ai';
const cohere = new CohereClientV2();
async function rag(query: string, knowledgeBase: string[]) {
const ranked = await cohere.rerank({
model: 'rerank-v3.5',
query,
documents: knowledgeBase,
topN: 5,
});
const docs = ranked.results.map((r, i) => ({
id: `doc-${i}`,
data: { text: knowledgeBase[r.index] },
}));
const response = await cohere.chat({
model: 'command-a-03-2025',
messages: [{ role: 'user', content: query }],
documents: docs,
});
return response.message?.content?.[0]?.text ?? '';
}
Output
- Embedded document vectors (float, int8, or binary)
- Reranked candidates with relevance scores (0.0-1.0)
- Grounded answer with fine-grained citations pointing to source documents
Error Handling
| Error | Cause | Solution |
|---|
input_type is required | Missing embed inputType | Use search_document or search_query |
embedding_types required | Missing for v3+ models | Add embeddingTypes: ['float'] |
| Empty citations | Docs too short/irrelevant | Improve document quality or chunking |
too many documents | >1000 rerank docs | Batch into groups of 1000 |
Resources
Next Steps
For tool-use and agents workflow, see cohere-core-workflow-b.
