langfuse-reference-architecture

name	langfuse-reference-architecture
description	Production-grade Langfuse architecture patterns and best practices. Use when designing LLM observability infrastructure, planning Langfuse deployment, or implementing enterprise-grade tracing architecture. Trigger with phrases like "langfuse architecture", "langfuse design", "langfuse infrastructure", "langfuse enterprise", "langfuse at scale".
allowed-tools	Read, Write, Edit
version	1.0.0
license	MIT
author	Jeremy Longshore <[e-mail]>

Langfuse Reference Architecture

Overview

Production-grade architecture patterns for Langfuse LLM observability at scale.

Prerequisites

• Understanding of distributed systems
• Knowledge of cloud infrastructure
• Familiarity with observability patterns

Architecture Patterns

Pattern 1: Basic Cloud Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Application Layer                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────┐    ┌──────────┐    ┌──────────┐              │
│  │ API      │    │ Worker   │    │ Cron     │              │
│  │ Service  │    │ Service  │    │ Jobs     │              │
│  └────┬─────┘    └────┬─────┘    └────┬─────┘              │
│       │               │               │                     │
│       └───────────────┴───────────────┘                     │
│                       │                                      │
│              ┌────────┴────────┐                            │
│              │ Langfuse SDK    │                            │
│              │ (Singleton)     │                            │
│              └────────┬────────┘                            │
│                       │                                      │
└───────────────────────┼──────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────────┐
│                  Langfuse Cloud                              │
│  ┌─────────────────────────────────────────────────────┐   │
│  │ Ingestion API → Processing → PostgreSQL → Dashboard  │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

Pattern 2: Self-Hosted Architecture

┌─────────────────────────────────────────────────────────────┐
│                        VPC                                   │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌─────────────────────────────────────────────────────┐   │
│  │                 Application Cluster                  │   │
│  │  ┌───────┐  ┌───────┐  ┌───────┐  ┌───────┐       │   │
│  │  │ Pod 1 │  │ Pod 2 │  │ Pod 3 │  │ Pod N │       │   │
│  │  └───┬───┘  └───┬───┘  └───┬───┘  └───┬───┘       │   │
│  │      └──────────┴─────┬────┴──────────┘            │   │
│  └─────────────────────────┼──────────────────────────┘   │
│                            │                               │
│                   ┌────────┴────────┐                     │
│                   │ Internal LB     │                     │
│                   └────────┬────────┘                     │
│                            │                               │
│  ┌─────────────────────────┼──────────────────────────┐   │
│  │         Langfuse Self-Hosted Cluster                │   │
│  │  ┌───────────┐    ┌───────────┐    ┌───────────┐  │   │
│  │  │ Langfuse  │    │ Langfuse  │    │ Langfuse  │  │   │
│  │  │ Instance 1│    │ Instance 2│    │ Instance 3│  │   │
│  │  └─────┬─────┘    └─────┬─────┘    └─────┬─────┘  │   │
│  │        └────────────────┼────────────────┘         │   │
│  │                         │                          │   │
│  │                ┌────────┴────────┐                 │   │
│  │                │ PostgreSQL RDS  │                 │   │
│  │                │ (Multi-AZ)      │                 │   │
│  │                └─────────────────┘                 │   │
│  └────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

Pattern 3: High-Scale Architecture with Buffer

┌─────────────────────────────────────────────────────────────┐
│                    Application Layer                         │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌────────────────────────────────────────────────────┐    │
│  │              Regional Application Clusters          │    │
│  │  ┌─────────┐  ┌─────────┐  ┌─────────┐            │    │
│  │  │ US-East │  │ EU-West │  │ AP-South│            │    │
│  │  └────┬────┘  └────┬────┘  └────┬────┘            │    │
│  │       └────────────┼────────────┘                  │    │
│  └────────────────────┼───────────────────────────────┘    │
│                       │                                     │
│              ┌────────┴────────┐                           │
│              │ Langfuse SDK    │                           │
│              │ (Batched)       │                           │
│              └────────┬────────┘                           │
│                       │                                     │
│              ┌────────┴────────┐                           │
│              │  Message Queue  │  ← Buffer for high volume │
│              │  (SQS/Kafka)    │                           │
│              └────────┬────────┘                           │
│                       │                                     │
│              ┌────────┴────────┐                           │
│              │ Ingestion       │  ← Async workers          │
│              │ Workers         │                           │
│              └────────┬────────┘                           │
│                       │                                     │
└───────────────────────┼─────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────────┐
│                  Langfuse (Cloud/Self-Hosted)                │
└─────────────────────────────────────────────────────────────┘

Instructions

Step 1: Implement Singleton SDK Pattern

// lib/langfuse/client.ts
import { Langfuse } from "langfuse";

class LangfuseClient {
  private static instance: Langfuse | null = null;
  private static shutdownRegistered = false;

  static getInstance(): Langfuse {
    if (!LangfuseClient.instance) {
      LangfuseClient.instance = new Langfuse({
        publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
        secretKey: process.env.LANGFUSE_SECRET_KEY!,
        baseUrl: process.env.LANGFUSE_HOST,
        // Production settings
        flushAt: parseInt(process.env.LANGFUSE_FLUSH_AT || "25"),
        flushInterval: parseInt(process.env.LANGFUSE_FLUSH_INTERVAL || "5000"),
        requestTimeout: 15000,
      });

      if (!LangfuseClient.shutdownRegistered) {
        .();
      }
    }

     .;
  }

    () {
      =  () => {
      .();
       (.) {
         ..();
        . = ;
      }
    };

    process.(,  ());
    process.(,  ());
    process.(,  ());

    . = ;
  }
}

  langfuse = .();

Step 2: Implement Trace Context Propagation

// lib/langfuse/context.ts
import { AsyncLocalStorage } from "async_hooks";

interface TraceContext {
  traceId: string;
  parentSpanId?: string;
  userId?: string;
  sessionId?: string;
}

const traceStorage = new AsyncLocalStorage<TraceContext>();

export function withTraceContext<T>(
  context: TraceContext,
  fn: () => T
): T {
  return traceStorage.run(context, fn);
}

export function getTraceContext(): TraceContext | undefined {
  return traceStorage.getStore();
}

// Middleware for Express
export function langfuseMiddleware() {
  return (req: Request, res: Response, next: NextFunction) => {
    const trace = langfuse.trace({
      name: ,
      : req.?.,
      : req.?.,
      : {
        : req.,
        : req.,
        : req.[],
      },
    });

     :  = {
      : trace.,
      : req.?.,
      : req.?.,
    };

    (context,  {
      
      req. = trace;

      
      res.(,  {
        trace.({
          : { : res. },
          : res. >=  ?  : ,
        });
      });

      ();
    });
  };
}

Step 3: Implement Queue-Based Ingestion

// lib/langfuse/queue.ts
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
import { Langfuse } from "langfuse";

interface QueuedTrace {
  name: string;
  input?: any;
  output?: any;
  metadata?: Record<string, any>;
  userId?: string;
  sessionId?: string;
  timestamp: string;
}

// Producer: Application sends to queue
class QueuedLangfuseProducer {
  private sqs: SQSClient;
  private queueUrl: string;

  constructor() {
    this.sqs = new SQSClient({});
    this.queueUrl = process.env.LANGFUSE_QUEUE_URL!;
  }

  async trace(params: Omit<QueuedTrace, >) {
     :  = {
      ...params,
      :  ().(),
    };

     ..(
       ({
        : .,
        : .(message),
        : params. || ,
      })
    );
  }
}


  {
   : ;

  () {
    . =  ();
  }

   () {
     trace = ..({
      : message.,
      : message.,
      : message.,
      : message.,
      : message.,
      : message.,
      :  (message.),
    });

     trace.;
  }

   () {
     ( message  messages) {
       .(message);
    }
     ..();
  }
}

Step 4: Multi-Environment Configuration

// config/langfuse.ts
type Environment = "development" | "staging" | "production";

interface LangfuseEnvironmentConfig {
  publicKey: string;
  secretKey: string;
  host: string;
  flushAt: number;
  flushInterval: number;
  enabled: boolean;
  sampling: {
    rate: number;
    alwaysSampleErrors: boolean;
  };
}

const ENVIRONMENT_CONFIGS: Record<Environment, LangfuseEnvironmentConfig> = {
  development: {
    publicKey: process.env.LANGFUSE_PUBLIC_KEY_DEV!,
    secretKey: process.env.LANGFUSE_SECRET_KEY_DEV!,
    host: process.env.LANGFUSE_HOST_DEV || "http://localhost:3000",
    flushAt: 1,
    flushInterval: 1000,
    enabled: true,
    sampling: { rate: 1.0, alwaysSampleErrors: true },
  },
  : {
    : process..!,
    : process..!,
    : process.. || ,
    : ,
    : ,
    : ,
    : { : , :  },
  },
  : {
    : process..!,
    : process..!,
    : process.. || ,
    : ,
    : ,
    : ,
    : { : , :  },
  },
};

  ():  {
   env = (process.. || )  ;
   [env] || .;
}

Step 5: Implement Service Mesh Tracing

// For microservices: propagate trace context across services
interface TraceHeaders {
  "x-langfuse-trace-id": string;
  "x-langfuse-parent-id"?: string;
  "x-langfuse-session-id"?: string;
}

// Outgoing request: inject headers
function injectTraceHeaders(headers: Headers) {
  const context = getTraceContext();
  if (context) {
    headers.set("x-langfuse-trace-id", context.traceId);
    if (context.parentSpanId) {
      headers.set("x-langfuse-parent-id", context.parentSpanId);
    }
    if (context.sessionId) {
      headers.set("x-langfuse-session-id", context.sessionId);
    }
  }
}

// Incoming request: extract headers and continue trace
function extractTraceContext(request: Request): TraceContext | null {
  const traceId = request.headers.get("x-langfuse-trace-id");
  if (!traceId) return null;

  return {
    traceId,
    : request..() || ,
    : request..() || ,
  };
}


 () {
   langfuse.({
    name,
    : parentContext.,
    : {
      : parentContext.,
      : parentContext.,
    },
  });
}

Output

• Singleton SDK pattern with graceful shutdown
• Trace context propagation
• Queue-based async ingestion
• Multi-environment configuration
• Service mesh integration

Architecture Decision Matrix

Pattern	Use Case	Complexity	Scale
Basic Cloud	Small apps	Low	100K traces/day
Self-Hosted	Data privacy	Medium	1M traces/day
Queue-Based	High volume	High	10M+ traces/day

Error Handling

Issue	Cause	Solution
Multiple instances	No singleton	Use singleton pattern
Lost traces	No shutdown	Register shutdown handlers
Cross-service gaps	No propagation	Implement header injection
Scale issues	Direct ingestion	Add message queue buffer

Resources

• Langfuse Self-Hosting
• Langfuse Architecture
• OpenTelemetry Context

Next Steps

For multi-environment setup, see langfuse-multi-env-setup.