llm-patterns

name	LLM Patterns
description	LLM integration patterns — RAG, embeddings, streaming, tool use. Triggers on rag/embeddings/vector/llm/prompt keywords.

LLM Patterns Skill

Load this skill when working with LLMs, RAG systems, embeddings, or AI integrations.

---

Rule

LLM INTEGRATIONS MUST BE ROBUST AND COST-EFFECTIVE!

• Always handle streaming and errors
• Optimize tokens for cost
• Use appropriate models for tasks

---

Model Selection Guide

Task	Model	Why
Complex reasoning	Claude Opus 4.5 / GPT-4	Best quality
General tasks	Claude Sonnet 4.5 / GPT-4o	Balance
Simple tasks	Claude Haiku 4.5 / GPT-4o-mini	Fast & cheap
Embeddings	text-embedding-3-small	Cost-effective
Classification	Fine-tuned small model	Fastest

---

RAG Architecture

Basic RAG Flow

Query → Embed Query → Vector Search → Retrieve Chunks → Augment Prompt → LLM → Response

Implementation

async function ragQuery(query: string): Promise<string> {
  // 1. Embed the query
  const queryEmbedding = await embedText(query);

  // 2. Vector search for relevant chunks
  const chunks = await vectorStore.similaritySearch(queryEmbedding, {
    topK: 5,
    minScore: 0.7,
  });

  // 3. Build context from chunks
  const context = chunks.map((c) => c.content).join('\n\n---\n\n');

  // 4. Augment prompt
  const prompt = `Answer based on the following context:

Context:
${context}

Question: ${query}

Answer:`;

  // 5. Get LLM response
  return await llm.complete(prompt);
}

---

Chunking Strategies

Size Guidelines

Document Type	Chunk Size	Overlap
Dense text (legal, technical)	256-512 tokens	50 tokens
General content	512-1024 tokens	100 tokens
Conversational	1024-2048 tokens	200 tokens

Chunking Methods

// 1. Fixed size chunking
function fixedChunks(text: string, size: number, overlap: number): string[] {
  const chunks = [];
  for (let i = 0; i < text.length; i += size - overlap) {
    chunks.push(text.slice(i, i + size));
  }
  return chunks;
}

// 2. Semantic chunking (by paragraph/section)
function semanticChunks(text: string): string[] {
  return text
    .split(/\n\n+/)
    .filter((chunk) => chunk.trim().length > 50);
}

// 3. Recursive character splitting (LangChain style)
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
  separators: ['\n\n', '\n', '. ', ' ', ''],
});

---

Streaming Responses

Server-Sent Events (SSE)

// Server (Express)
app.get('/api/chat', async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  const stream = await anthropic.messages.stream({
    model: 'claude-sonnet-4-5-20250929',
    max_tokens: 1024,
    messages: [{ role: 'user', content: req.query.message }],
  });

  for await (const event of stream) {
    if (event.type === 'content_block_delta') {
      res.write(`data: ${JSON.stringify({ text: event.delta.text })}\n\n`);
    }
  }

  res.write('data: [DONE]\n\n');
  res.end();
});

// Client
const eventSource = new EventSource('/api/chat?message=Hello');

eventSource.onmessage = (event) => {
  if (event.data === '[DONE]') {
    eventSource.close();
    return;
  }
  const { text } = JSON.parse(event.data);
  appendToOutput(text);
};

WebSocket Streaming

// Server
ws.on('message', async (data) => {
  const { message } = JSON.parse(data);

  const stream = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: message }],
    stream: true,
  });

  for await (const chunk of stream) {
    const text = chunk.choices[0]?.delta?.content || '';
    ws.send(JSON.stringify({ type: 'chunk', text }));
  }

  ws.send(JSON.stringify({ type: 'done' }));
});

---

Token Optimization

Counting Tokens

import { encoding_for_model } from 'tiktoken';

function countTokens(text: string, model = 'gpt-4'): number {
  const enc = encoding_for_model(model);
  return enc.encode(text).length;
}

Cost Optimization Tips

1. Use smaller models for simple tasks
2. Cache responses for repeated queries
3. Truncate context to essential information
4. Batch requests when possible
5. Use embeddings for similarity instead of LLM

Context Window Management

function trimContext(messages: Message[], maxTokens: number): Message[] {
  let totalTokens = 0;
  const trimmed = [];

  // Always keep system message
  const systemMsg = messages.find((m) => m.role === 'system');
  if (systemMsg) {
    totalTokens += countTokens(systemMsg.content);
    trimmed.push(systemMsg);
  }

  // Add messages from most recent
  for (const msg of messages.reverse()) {
    if (msg.role === 'system') continue;

    const tokens = countTokens(msg.content);
    if (totalTokens + tokens > maxTokens) break;

    totalTokens += tokens;
    trimmed.unshift(msg);
  }

  return trimmed;
}

---

Tool Use / Function Calling

Defining Tools

const tools = [
  {
    name: 'get_weather',
    description: 'Get current weather for a location',
    input_schema: {
      type: 'object',
      properties: {
        location: {
          type: 'string',
          description: 'City name, e.g., "San Francisco, CA"',
        },
      },
      required: ['location'],
    },
  },
];

const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-5-20250929',
  max_tokens: 1024,
  tools,
  messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
});

// Handle tool use
if (response.stop_reason === 'tool_use') {
  const toolUse = response.content.find((c) => c.type === 'tool_use');
  const result = await executeTools(toolUse.name, toolUse.input);

  // Continue conversation with tool result
  const finalResponse = await anthropic.messages.create({
    model: 'claude-sonnet-4-5-20250929',
    max_tokens: 1024,
    tools,
    messages: [
      { role: 'user', content: 'What is the weather in Tokyo?' },
      { role: 'assistant', content: response.content },
      { role: 'user', content: [{ type: 'tool_result', tool_use_id: toolUse.id, content: result }] },
    ],
  });
}

---

Error Handling

Retry Strategy

async function llmWithRetry<T>(
  fn: () => Promise<T>,
  maxRetries = 3
): Promise<T> {
  let lastError: Error;

  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      lastError = error;

      if (error.status === 429) {
        // Rate limit - exponential backoff
        const delay = Math.min(1000 * Math.pow(2, attempt), 30000);
        await sleep(delay);
        continue;
      }

      if (error.status >= 500) {
        // Server error - retry
        await sleep(1000 * attempt);
        continue;
      }

      // Client error - don't retry
      throw error;
    }
  }

  throw lastError;
}

Common Error Codes

Code	Meaning	Action
400	Bad request	Fix prompt/parameters
401	Invalid API key	Check credentials
429	Rate limited	Backoff and retry
500	Server error	Retry with backoff
529	Overloaded	Retry with longer backoff

---

Prompt Engineering Tips

System Prompts

const systemPrompt = `You are a helpful assistant that answers questions based on the provided context.

Rules:
1. Only use information from the context
2. If the answer is not in the context, say "I don't have information about that"
3. Be concise but complete
4. Cite relevant parts of the context`;

Structured Output

const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-5-20250929',
  max_tokens: 1024,
  messages: [
    {
      role: 'user',
      content: `Extract entities from this text and return as JSON:

Text: "John Smith, CEO of Acme Corp, announced the merger on January 15, 2024."

Return format:
{
  "people": [{"name": string, "role": string}],
  "organizations": [string],
  "dates": [string]
}`,
    },
  ],
});

---

Cost Estimation

Token Pricing (approximate)

Model	Input (1M tokens)	Output (1M tokens)
GPT-4o	$2.50	$10.00
GPT-4o-mini	$0.15	$0.60
Claude Sonnet 4.5	$3.00	$15.00
Claude Haiku 4.5	$0.25	$1.25

Estimation Formula

function estimateCost(inputTokens: number, outputTokens: number, model: string): number {
  const pricing = {
    'gpt-4o': { input: 0.0000025, output: 0.00001 },
    'claude-sonnet-4-5': { input: 0.000003, output: 0.000015 },
  };

  const p = pricing[model];
  return inputTokens * p.input + outputTokens * p.output;
}

---

When to Use This Skill

• Implementing RAG systems
• Setting up vector search
• Adding streaming to chat
• Optimizing LLM costs
• Building tool-using agents
• Handling LLM errors gracefully