Langfuse Cost Tuning
Overview
Track, analyze, and optimize LLM costs using Langfuse's built-in token/cost tracking, the Metrics API for programmatic cost analysis, model routing for cost reduction, and automated budget alerts.
Prerequisites
- Langfuse tracing with token usage captured (via
observeOpenAI or manual usage fields)
- For Metrics API:
@langfuse/client installed
- Understanding of LLM pricing models
How Langfuse Tracks Costs
Langfuse automatically calculates costs for supported models (OpenAI, Anthropic, Google) when token usage is captured. For custom models, you can configure pricing in the Langfuse UI under Settings > Model Definitions.
Cost tracking works on observations of type generation and embedding. The observeOpenAI wrapper captures usage automatically; for manual tracing, include usage in your observation updates.
Instructions
Step 1: Ensure Token Usage is Captured
import { observeOpenAI } from "@langfuse/openai";
const openai = observeOpenAI(new OpenAI());
import { startActiveObservation, updateActiveObservation } from "@langfuse/tracing";
await startActiveObservation(
{ name: "llm-call", asType: "generation" },
async () => {
updateActiveObservation({ model: "gpt-4o" });
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: prompt }],
});
updateActiveObservation({
output: response.choices[0].message.content,
usage: {
promptTokens: response.usage?.prompt_tokens,
completionTokens: response.usage?.completion_tokens,
totalTokens: response.usage?.total_tokens,
},
});
}
);
Step 2: Query Costs via Metrics API
import { LangfuseClient } from "@langfuse/client";
const langfuse = new LangfuseClient();
async function getCostReport(days: number) {
const fromTimestamp = new Date(Date.now() - days * 86400000).toISOString();
const traces = await langfuse.api.traces.list({
fromTimestamp,
limit: 1000,
orderBy: "timestamp",
});
const costByModel = new Map<string, { cost: number; tokens: number; count: number }>();
for (const trace of traces.data) {
const observations = await langfuse.api.observations.list({
traceId: trace.id,
: ,
});
( obs observations.) {
model = obs. || ;
existing = costByModel.(model) || { : , : , : };
existing. += obs. || ;
existing. += obs. || ;
existing. += ;
costByModel.(model, existing);
}
}
.();
.();
totalCost = ;
( [model, data] costByModel.()) {
.();
.();
.();
.();
totalCost += data.;
}
.();
}
();
Step 3: Implement Smart Model Routing
Route requests to cheaper models when appropriate:
import { observe, updateActiveObservation } from "@langfuse/tracing";
interface ModelConfig {
model: string;
costPer1MInput: number;
costPer1MOutput: number;
maxComplexity: "simple" | "moderate" | "complex";
}
const MODELS: ModelConfig[] = [
{ model: "gpt-4o-mini", costPer1MInput: 0.15, costPer1MOutput: 0.60, maxComplexity: "simple" },
{ model: "gpt-4o", costPer1MInput: 2.50, costPer1MOutput: 10.00, maxComplexity: "moderate" },
{ model: "claude-sonnet-4-20250514", costPer1MInput: 3.00, costPer1MOutput: 15.00, maxComplexity: "complex" },
];
function selectModel(task: string, inputLength: number): ModelConfig {
const simpleTasks = ["classify", "extract", , ];
isSimple = simpleTasks.( task.(t));
isShort = inputLength < ;
(isSimple && isShort) [];
(isSimple || inputLength < ) [];
[];
}
costOptimizedLLM = (
{ : , : },
(: , : ) => {
config = (task, input.);
({
: config.,
: {
task,
: ,
: config.,
},
});
response = (config., input);
({
: response.,
: response.,
});
response;
}
);
Step 4: Budget Alerts
import { LangfuseClient } from "@langfuse/client";
const langfuse = new LangfuseClient();
const ALERT_THRESHOLDS = {
dailyWarn: 50,
dailyCritical: 200,
perRequestWarn: 1,
};
async function checkCostAlerts() {
const since = new Date(Date.now() - 86400000).toISOString();
const traces = await langfuse.api.traces.list({
fromTimestamp: since,
limit: 500,
});
let dailyCost = 0;
let maxRequestCost = 0;
for (const trace of traces.data) {
const observations = await langfuse.api.observations.list({
: trace.,
: ,
});
traceCost = observations..(
sum + (obs. || ),
);
dailyCost += traceCost;
maxRequestCost = .(maxRequestCost, traceCost);
}
.();
.();
(dailyCost > .) {
(, );
} (dailyCost > .) {
(, );
}
}
();
Langfuse Dashboard Features
Langfuse provides built-in cost analytics in the UI:
- Cost Dashboard: Tracks token usage and costs over time by model, user, and session
- Latency Dashboard: Response times across models and user segments
- Custom Dashboards: Build custom views with multi-level aggregations
- Pricing Tiers: Supports complex pricing (cached tokens, audio tokens, per-model tiers)
Cost Optimization Strategies
| Strategy | Savings | Effort | How |
|---|
| Model downgrade | 50-95% | Low | Route simple tasks to gpt-4o-mini |
| Prompt optimization | 10-30% | Low | Remove filler words, use structured prompts |
| Response caching | 20-80% | Medium | Cache identical prompts with TTL |
| Batch processing | 50% | Medium | Use OpenAI Batch API for offline tasks |
| Token limits | 10-40% | Low | Set max_tokens on all calls |
Error Handling
| Issue | Cause | Solution |
|---|
| Missing cost data | No usage in generation | Ensure usage is included with promptTokens/completionTokens |
| Wrong cost calculation | Model name mismatch | Use exact model ID (e.g., gpt-4o-2024-08-06) |
| Custom model no cost | No pricing configured | Add model pricing in Langfuse Settings > Model Definitions |
| Stale pricing | Model prices changed | Update model definitions periodically |
Resources
