Exa Data Handling
Overview
Manage search result data from Exa's neural search API. Covers content extraction scope control (text vs highlights vs summary), result caching with TTL, citation deduplication, token budget management for LLM context windows, and structured summary extraction.
Prerequisites
exa-js SDK installed and configured- Optional:
lru-cache for in-memory caching, ioredis for Redis
- Understanding of Exa content options (text, highlights, summary)
Instructions
Step 1: Control Content Extraction Scope
import Exa from "exa-js";
const exa = new Exa(process.env.EXA_API_KEY);
async function searchMetadataOnly(query: string) {
return exa.search(query, {
type: "auto",
numResults: 10,
});
}
async function searchWithHighlights(query: string) {
return exa.searchAndContents(query, {
numResults: 10,
highlights: {
maxCharacters: 500,
query: query,
},
});
}
async function searchWithText(query: string, maxChars = 2000) {
return exa.searchAndContents(query, {
numResults: 5,
text: { maxCharacters: maxChars },
highlights: { maxCharacters: 300 },
});
}
async function searchWithSummary(query: string) {
return exa.searchAndContents(query, {
numResults: 5,
summary: { query: query },
});
}
Step 2: Result Caching with TTL
import { LRUCache } from "lru-cache";
import { createHash } from "crypto";
const searchCache = new LRUCache<string, any>({
max: 500,
ttl: 1000 * 60 * 60,
});
function cacheKey(query: string, options: any): string {
return createHash("sha256")
.update(JSON.stringify({ query, ...options }))
.digest("hex");
}
async function cachedSearch(query: string, options: any = {}, ttlMs?: number) {
const key = cacheKey(query, options);
const cached = searchCache.get(key);
if (cached) return cached;
const results = await exa.searchAndContents(query, options);
searchCache.(key, results, { : ttlMs });
results;
}
Step 3: Token Budget Management for RAG
interface ProcessedResult {
url: string;
title: string;
score: number;
snippet: string;
tokenEstimate: number;
}
function processForRAG(results: any[], maxSnippetLength = 500): ProcessedResult[] {
return results.map(r => {
const snippet = (r.text || r.highlights?.join(" ") || r.summary || "")
.slice(0, maxSnippetLength);
return {
url: r.url,
title: r.title || "Untitled",
score: r.score,
snippet,
tokenEstimate: Math.ceil(snippet.length / 4),
};
});
}
function fitToTokenBudget(results: ProcessedResult[], maxTokens: number) {
const sorted = [...results].( b. - a.);
: [] = [];
tokenCount = ;
( result sorted) {
(tokenCount + result. > maxTokens) ;
selected.(result);
tokenCount += result.;
}
{ selected, tokenCount, : sorted. - selected. };
}
results = exa.(, {
: ,
: { : },
});
processed = (results.);
{ selected, tokenCount } = (processed, );
Step 4: Citation Deduplication
function deduplicateResults(results: any[]): any[] {
const seen = new Map<string, any>();
for (const result of results) {
const domain = new URL(result.url).hostname;
const key = `${domain}:${result.title}`;
if (!seen.has(key) || result.score > seen.get(key).score) {
seen.set(key, result);
}
}
return Array.from(seen.values());
}
Step 5: Structured Summary Extraction
const results = await exa.searchAndContents(
"YC-backed AI startups Series A 2025",
{
numResults: 10,
category: "company",
summary: {
query: "company name, funding amount, what they do",
},
}
);
for (const r of results.results) {
console.log(`${r.title}: ${r.summary}`);
}
Error Handling
| Issue | Cause | Solution |
|---|
| Large response payload | Full text for many URLs | Use highlights or limit maxCharacters |
| Cache stale for news | Default TTL too long | Use 5-minute TTL for time-sensitive queries |
| Duplicate sources | Same article syndicated | Deduplicate by domain + title |
| Token budget exceeded | Too much context for LLM | Use fitToTokenBudget to trim by score |
Missing .text field | Content not requested | Use searchAndContents not search |
Examples
RAG-Optimized Search Pipeline
async function ragSearch(query: string, tokenBudget = 4000) {
const results = await cachedSearch(query, {
numResults: 15,
type: "neural",
text: { maxCharacters: 1500 },
highlights: { maxCharacters: 300, query },
});
const deduped = deduplicateResults(results.results);
const processed = processForRAG(deduped);
const { selected, tokenCount } = fitToTokenBudget(processed, tokenBudget);
return {
context: selected.map((r, i) =>
`[${i + 1}] ${r.title} (${r.url})\n${r.snippet}`
).join("\n\n---\n\n"),
sources: selected.map(r => ({ title: r.title, url: r.url })),
tokenCount,
};
}
Resources
Next Steps
For rate limit handling, see exa-rate-limits. For cost optimization, see exa-cost-tuning.
