redis-semantic-cache

# Redis Semantic Cache Semantic caching for LLM responses with Redis Cloud's LangCache service. Stores prompts as embeddings; subsequent semantically-similar prompts return the cached response without re-calling the model. > LangCache is currently in **preview** on Redis Cloud. Features and behavior may change. ## When to apply - Wrapping an LLM call (OpenAI, Anthropic, etc.) with a cache layer to cut cost and latency. - Caching RAG answers, classification outputs, or any deterministic LLM workload. - Tuning the precision/hit-rate trade-off for a semantic cache. - Splitting one application's LLM workloads across multiple cache instances. ## 1. The cache-aside flow LangCache fits in front of any LLM call as a standard cache-aside pattern: 1. Send the user's prompt to LangCache's `search`. 2. **Cache hit** — return the stored response directly. 3. **Cache miss** — call the LLM, then `set` the response so future similar prompts hit. ```python from langcache import LangCache import os lang_cache = LangCache( server_url=f"https://{os.getenv('HOST')}", cache_id=os.getenv("CACHE_ID"), api_key=os.getenv("API_KEY"), ) result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.9) if result: response = result[0]["response"] else: response = llm.generate("What is Redis?") lang_cache.set(prompt="What is Redis?", response=response) ``` The same operations are available via REST (`POST /v1/caches/{cacheId}/entries/search` and `POST /v1/caches/{cacheId}/entries`) when an SDK isn't an option. See [references/langcache-usage.md](references/langcache-usage.md) for full SDK + REST samples and attribute-based storage. ## 2. Tune the similarity threshold The threshold controls how close (in embedding cosine distance) a new prompt must be to a cached one to count as a hit. Higher = stricter match, fewer false positives. Lower = more hits, more risk of returning an off-topic answer. | Threshold | Behavior | Use when | |---|---|---| | 0.95+ | Near-exact match required | Customer-facing answers where wrong responses are costly | | 0.9 | Balanced default | Most workloads — start here | | 0.8 | Loose semantic match | Internal tools, exploratory queries, FAQ deduplication | ```python # Stricter — fewer false positives result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.95) # Looser — higher hit rate result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.8) ``` Adjust by watching the actual cache-hit rate and spot-checking that returned answers are still relevant. See [references/best-practices.md](references/best-practices.md). ## 3. Separate caches per task type Different LLM workloads should not share one cache — a "code question" prompt is semantically close to other code questions but has nothing to do with a password-reset support query, and crossing them returns garbage. ```python support_cache = LangCache(server_url=..., cache_id="support-cache-id", api_key=...) code_cache = LangCache(server_url=..., cache_id="code-cache-id", api_key=...) ``` Create distinct cache IDs in Redis Cloud per task, and route each call to the right one. As a finer-grained alternative, store and search with **custom attributes** (e.g. `{"category": "database"}`) to keep tasks in the same cache but isolated by attribute filter — useful when the same prompt format spans subtopics. ## References - [LangCache documentation](https://redis.io/docs/latest/develop/ai/langcache/)