Control LLM output with regex and grammars, guarantee valid JSON/XML/code generation, enforce structured formats, and build multi-step workflows with Guidance - Microsoft Research's constrained generation framework
Control LLM output with regex and grammars, guarantee valid JSON/XML/code generation, enforce structured formats, and build multi-step workflows with Guidance - Microsoft Research's constrained generation framework
Build multi-step workflows with Pythonic control flow
Prevent invalid outputs through grammatical constraints
GitHub Stars: 18,000+ | From: Microsoft Research
Installation
# Base installation
pip install guidance
# With specific backends
pip install guidance[transformers] # Hugging Face models
pip install guidance[llama_cpp] # llama.cpp models
Quick Start
Basic Example: Structured Generation
from guidance import models, gen
# Load model (supports OpenAI, Transformers, llama.cpp)
lm = models.OpenAI("gpt-4")
# Generate with constraints
result = lm + "The capital of France is " + gen("capital", max_tokens=5)
print(result["capital"]) # "Paris"
With Anthropic Claude
from guidance import models, gen, system, user, assistant
# Configure Claude
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Use context managers for chat formatwith system():
lm += "You are a helpful assistant."with user():
lm += "What is the capital of France?"with assistant():
lm += gen(max_tokens=20)
Core Concepts
1. Context Managers
Guidance uses Pythonic context managers for chat-style interactions.
from guidance import system, user, assistant, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# System messagewith system():
lm += "You are a JSON generation expert."# User messagewith user():
lm += "Generate a person object with name and age."# Assistant responsewith assistant():
lm += gen("response", max_tokens=100)
print(lm["response"])
Benefits:
Natural chat flow
Clear role separation
Easy to read and maintain
2. Constrained Generation
Guidance ensures outputs match specified patterns using regex or grammars.
Regex Constraints
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Constrain to valid email format
lm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
# Constrain to date format (YYYY-MM-DD)
lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}")
# Constrain to phone number
lm += "Phone: " + gen("phone", regex=r"\d{3}-\d{3}-\d{4}")
print(lm["email"]) # Guaranteed valid emailprint(lm["date"]) # Guaranteed YYYY-MM-DD format
How it works:
Regex converted to grammar at token level
Invalid tokens filtered during generation
Model can only produce matching outputs
Selection Constraints
from guidance import models, gen, select
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Constrain to specific choices
lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")
# Multiple-choice selection
lm += "Best answer: " + select(
["A) Paris", "B) London", "C) Berlin", "D) Madrid"],
name="answer"
)
print(lm["sentiment"]) # One of: positive, negative, neutralprint(lm["answer"]) # One of: A, B, C, or D
3. Token Healing
Guidance automatically "heals" token boundaries between prompt and generation.
# Without token healing
prompt = "The capital of France is "# Last token: " is "# First generated token might be " Par" (with leading space)# Result: "The capital of France is Paris" (double space!)
Solution: Guidance backs up one token and regenerates.
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Token healing enabled by default
lm += "The capital of France is " + gen("capital", max_tokens=5)
# Result: "The capital of France is Paris" (correct spacing)
Benefits:
Natural text boundaries
No awkward spacing issues
Better model performance (sees natural token sequences)
4. Grammar-Based Generation
Define complex structures using context-free grammars.
Create reusable generation patterns with the @guidance decorator.
from guidance import guidance, gen, models
@guidancedefgenerate_person(lm):
"""Generate a person with name and age."""
lm += "Name: " + gen("name", max_tokens=20, stop="\n")
lm += "\nAge: " + gen("age", regex=r"[0-9]+", max_tokens=3)
return lm
# Use the function
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = generate_person(lm)
print(lm["name"])
print(lm["age"])
# Token healing is enabled by default# No special action needed - just concatenate naturally
lm += "The capital is " + gen("capital") # Automatic healing
4. Use stop Sequences
# ✅ Good: Stop at newline for single-line outputs
lm += "Name: " + gen("name", stop="\n")
# ❌ Bad: May generate multiple lines
lm += "Name: " + gen("name", max_tokens=50)
