prompt-caching

name	prompt-caching
description	Prompt caching strategies for LLM APIs — cache breakpoints, system prompt caching, and cost optimization.
layer	utility
category	ai-ml
triggers	["prompt cache","prompt caching","cache breakpoint","llm caching","cached prompt"]
inputs	["LLM API usage patterns and cost concerns","System prompt optimization questions","Cache configuration for multi-turn conversations","Cost analysis for cached vs uncached calls"]
outputs	["Cache-optimized prompt structures","Breakpoint placement strategies","Cost comparison calculations","Provider-specific caching configurations"]
linksTo	["claude-api","openai","caching"]
linkedFrom	[]
riskLevel	low
memoryReadPolicy	selective
memoryWritePolicy	none
sideEffects	[]

Prompt Caching Strategies for LLM APIs

Purpose

Optimize LLM API costs and latency by leveraging prompt caching features across providers. Covers Anthropic's cache breakpoints, OpenAI's automatic caching, cache-friendly prompt architecture, and cost modeling.

Key Patterns

Anthropic Prompt Caching

Anthropic supports explicit cache breakpoints on content blocks. Cached content is billed at a reduced rate on cache hits and a small write premium on cache misses.

System prompt caching — Place cache_control on the system message:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const response = await client.messages.create({
  model: 'claude-sonnet-4-20250514',
  max_tokens: 1024,
  system: [
    {
      type: 'text',
      text: `You are an expert assistant with deep knowledge of our codebase.
Here is the full project documentation:
${largeDocumentation}`, // Large static content
      cache_control: { type: 'ephemeral' },
    },
  ],
  messages: [{ role: 'user', content: 'How do I add a new API endpoint?' }],
});

// Check cache performance in response headers
// response.usage.cache_creation_input_tokens — tokens written to cache
// response.usage.cache_read_input_tokens — tokens read from cache

Multi-turn conversation caching — Cache the conversation prefix:

  () {
  
  
   : ..[] = [
    ...conversationHistory.( {
       (i === conversationHistory. - ) {
        
         {
          ...msg,
          :
             msg. === 
              ? [
                  {
                    :   ,
                    : msg.,
                    : { :    },
                  },
                ]
              : msg.,
        };
      }
       msg;
    }),
    { : , : newMessage },
  ];

   client..({
    : ,
    : ,
    : [
      {
        : ,
        : systemPrompt,
        : { :  },
      },
    ],
    messages,
  });
}