DSPy: Declarative Language Model Programming
Framework for programming — not prompting — language models. Build modular AI systems with automatic prompt optimization, RL training, and reflective prompt evolution.
GitHub: 22,000+ stars | By: Stanford NLP | Current: DSPy 3.1+ (Feb 2026)
Installation
pip install dspy
pip install dspy[all]
pip install git+https://github.com/stanfordnlp/dspy.git
For RLM/ProgramOfThought/CodeAct (sandboxed code execution):
curl -fsSL https://deno.land/install.sh | sh
Python: 3.10+ required (3.9 dropped in 3.0)
Quick Start
import dspy
lm = dspy.LM("anthropic/claude-sonnet-4-5-20250929", max_tokens=1000)
dspy.configure(lm=lm)
class QA(dspy.Signature):
"""Answer questions with short factual answers."""
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 1 and 5 words")
qa = dspy.Predict(QA)
result = qa(question="What is the capital of France?")
print(result.answer)
Chain of Thought
cot = dspy.ChainOfThought("question -> answer")
result = cot(question="If John has 5 apples and gives 2 to Mary, how many remain?")
print(result.rationale)
print(result.answer)
RLM — Recursive Language Model (3.1+)
Process documents far beyond context window limits via programmatic exploration:
rlm = dspy.RLM("context, query -> answer", max_iterations=20)
result = rlm(
context="...massive document (100k+ tokens)...",
query="What was Q3 revenue?"
)
Core Concepts
1. LM Configuration
import dspy
lm = dspy.LM("openai/gpt-4o-mini", temperature=0.7, cache=True)
dspy.configure(lm=lm)
lm = dspy.LM("anthropic/claude-sonnet-4-5-20250929", max_tokens=2000)
lm = dspy.LM("ollama_chat/llama3.1", api_base="http://localhost:11434")
cheap = dspy.LM("openai/gpt-4o-mini")
strong = dspy.LM("anthropic/claude-sonnet-4-5-20250929")
dspy.configure(lm=cheap)
with dspy.context(lm=strong):
result = expensive_module(question=q)
dspy.configure(lm=lm, adapter=dspy.ChatAdapter())
dspy.configure(lm=lm, adapter=dspy.JSONAdapter())
dspy.configure(lm=lm, adapter=dspy.XMLAdapter())
dspy.configure(lm=lm, track_usage=True)
result = qa(question="...")
print(result.get_lm_usage())
2. Signatures
Define task structure as input-output contracts:
qa = dspy.Predict("question -> answer")
summarizer = dspy.ChainOfThought("text -> summary")
class ExtractEntities(dspy.Signature):
"""Extract named entities from text."""
text = dspy.InputField(desc="raw text to analyze")
entities = dspy.OutputField(desc="comma-separated list of entities")
class DescribeImage(dspy.Signature):
"""Describe an image."""
image: dspy.Image = dspy.InputField()
description = dspy.OutputField()
3. Modules
Composable building blocks (like PyTorch nn.Module):
| Module | Purpose | When to Use |
|---|
dspy.Predict | Direct prediction | Simple tasks, speed critical |
dspy.ChainOfThought | Step-by-step reasoning | Complex reasoning, math |
dspy.ProgramOfThought | Code-based reasoning | Calculations, data transforms |
dspy.ReAct | Tool-using agent | Multi-step research, API calls |
dspy.RLM | Recursive context exploration | Long documents beyond context window (3.1+) |
dspy.CodeAct | Code generation + tool execution | Dynamic tool use, self-learning (3.0+) |
dspy.TypedPredictor | Pydantic structured output | Extraction, typed responses |
dspy.Refine | Iterative self-refinement | Quality-critical outputs |
dspy.BestofN | N-sample selection | High-stakes decisions |
dspy.MultiChainComparison | Compare multiple chains | Ambiguous questions |
See references/modules.md for complete API and usage patterns for each module.
4. Optimizers
Automatically improve prompts using training data:
| Optimizer | Best For | Speed | Data Needed |
|---|
BootstrapFewShot | General purpose first try | Fast | 10-50 examples |
dspy.MIPROv2 | Reliable joint optimization | Medium | 50-200 examples |
dspy.GEPA | Reflective prompt evolution (3.0+) | Medium | 20-100 examples |
dspy.SIMBA | Self-reflection with feedback (3.0+) | Medium | 20-100 examples |
ArborGRPO | RL-based weight training (3.0+) | Slow | 100+ examples |
BootstrapFinetune | Model fine-tuning | Slow | 100+ examples |
COPRO | Prompt search | Medium | 20-100 examples |
tp = dspy.MIPROv2(metric=my_metric, auto="medium", num_threads=8)
optimized = tp.compile(my_module, trainset=trainset)
optimizer = dspy.GEPA(
metric=metric_with_feedback, auto="light", num_threads=32,
reflection_lm=dspy.LM("openai/gpt-4o", temperature=1.0, max_tokens=32000)
)
optimized = optimizer.compile(program, trainset=trainset, valset=valset)
See references/optimizers.md for complete optimizer guide with metrics and evaluation patterns.
5. Building Custom Modules
class RAG(dspy.Module):
def __init__(self, num_passages=3):
super().__init__()
self.retrieve = dspy.Retrieve(k=num_passages)
self.generate = dspy.ChainOfThought("context, question -> answer")
def forward(self, question):
passages = self.retrieve(question).passages
context = "\n".join(passages)
return self.generate(context=context, question=question)
Key Patterns
Structured Output with Pydantic
from pydantic import BaseModel, Field
class PersonInfo(BaseModel):
name: str = Field(description="Full name")
age: int = Field(description="Age in years")
class ExtractPerson(dspy.Signature):
"""Extract person information from text."""
text = dspy.InputField()
person: PersonInfo = dspy.OutputField()
extractor = dspy.TypedPredictor(ExtractPerson)
result = extractor(text="John Doe, 35, is a software engineer.")
print(result.person.name)
Async and Streaming
async_cot = dspy.asyncify(dspy.ChainOfThought("question -> answer"))
result = await async_cot(question="What is DSPy?")
stream_predict = dspy.streamify(my_module)
for chunk in stream_predict(question="Explain quantum computing"):
print(chunk, end="")
Thread-Safe Batch Processing (3.0+)
results = my_module.batch(
inputs_list,
num_threads=8,
return_failed_examples=True,
max_errors=5
)
Save and Load (Stable in 3.0+)
optimized.save("models/qa_v2", save_program=True)
loaded = dspy.ChainOfThought("question -> answer")
loaded.load("models/qa_v2.json")
Evaluation
from dspy.evaluate import Evaluate
def exact_match(example, pred, trace=None):
return example.answer.lower() == pred.answer.lower()
evaluator = Evaluate(devset=testset, metric=exact_match, num_threads=4)
score = evaluator(optimized_module)
print(f"Accuracy: {score}")
Best Practices
- Start simple —
dspy.Predict first, add ChainOfThought only if accuracy matters
- Use descriptive signatures — docstrings and field descriptions guide the LM
- Optimize with representative data — cover edge cases in training examples
- Evaluate on held-out test set — avoid overfitting to training data
- Use
dspy.MIPROv2(auto="light") as first optimizer — fast and effective
- Try GEPA for agentic tasks — reflective evolution with custom feedback (3.0+)
- Track usage —
dspy.configure(track_usage=True) to monitor costs
- Cache during development — enabled by default, saves API calls
- Use Module.batch for throughput — thread-safe concurrent processing (3.0+)
Additional Resources
Reference Files
Detailed documentation for each area:
references/modules.md — Complete module API: Predict, ChainOfThought, ReAct, RLM, CodeAct, Refine, BestofN, adapters, types, and composition patternsreferences/optimizers.md — All optimizers: MIPROv2, GEPA, SIMBA, ArborGRPO, BootstrapFewShot, Ensemble, metrics, evaluation, and optimization workflowsreferences/examples.md — Production examples: RAG, agents, classifiers, RLM long-context, multi-modal, pipelines, async patterns, teacher-student distillationreferences/migration.md — Breaking changes from DSPy 1.x/2.0 to 2.6, and from 2.6 to 3.xREADME.md — High-level DSPy overview for quick familiarization
External Resources
