A reference layered architecture for production LangChain 1.0 / LangGraph 1.0
services — LLM factory with version-safe defaults, chain/graph registry,
retriever and tool DI, Pydantic-validated config, per-request tenant scoping,
middleware ordering, checkpointer selection per environment. Use when starting
a new service, refactoring a tangled chain, or onboarding a team to existing code.
Trigger with "langchain architecture", "langchain llm factory",
"langchain chain registry", "langchain dependency injection",
"langchain project structure".
A reference layered architecture for production LangChain 1.0 / LangGraph 1.0
services — LLM factory with version-safe defaults, chain/graph registry,
retriever and tool DI, Pydantic-validated config, per-request tenant scoping,
middleware ordering, checkpointer selection per environment. Use when starting
a new service, refactoring a tangled chain, or onboarding a team to existing code.
Trigger with "langchain architecture", "langchain llm factory",
"langchain chain registry", "langchain dependency injection",
"langchain project structure".
Designed for Claude Code, also compatible with Codex
LangChain Reference Architecture (Python)
Overview
Eight months into a LangChain service, a code review surfaces the mess.
Twelve chain definitions live inlined inside FastAPI route handlers. Three
retrievers are constructed at module-global scope, one bound to
tenant_id="acme" because that was the first tenant in the pilot —
that retriever now returns Acme's documents to every other tenant, a P33
leak that has been live in production for six weeks.
max_retries=6 is hardcoded at four separate call sites. A
RunnableWithMessageHistory backed by the default
InMemoryChatMessageHistory loses every conversation on pod restart
(P22) — which is most days, because Cloud Run scales to zero.
Config is read from os.environ in three modules with three different
fallback strategies. There is no place to put a new provider without
touching seven files, and nobody remembers why the retriever is built
at import time.
The fix is not "rename a variable." The fix is an architecture that made
every one of those mistakes hard to write. This skill is the target
layered architecture:
app/ — FastAPI routes. Thin. Parses HTTP, calls into services,
serializes response. No chain logic, no vendor clients, no env vars.
services/ — chain and graph definitions. Take dependencies through
constructor args, not module-level imports.
adapters/ — vendor clients, LLM factory, retriever factory, tool
factory. This is where langchain-anthropic is imported. Nowhere else.
config/ — one Pydantic Settings class. SecretStr for keys,
Literal["dev","staging","prod"] for env names, .env file loader.
domain/ — Pydantic models, typed LangGraph state, enums. No I/O.
Five layers, five imports deep at most. Dependency direction is
strictly downward. app imports services; services imports
adapters; adapters imports config and domain. Never the reverse.
Import-linter enforces this in CI. Pain-catalog anchors: P22 (in-memory
history loses messages — architectural fix is persistent history
injected via DI) and P33 (per-tenant vector stores leak if retriever
bound at import — architectural fix is per-request factory). Adjacent:
P10 (recursion limits), P24 (middleware order), P28 (callback
inheritance). Pin: , ,
, , ,
.
The max_retries=6 scatter in the mess-case becomes max_retries=2 in exactly one file. Services that want a longer timeout pass timeout=60 — but they never set max_retries=6 by accident. Cross-reference langchain-model-inference Step 3 for the factory pattern's provenance; see LLM Factory Pattern for per-provider variants and caching.
Step 3 — Replace scattered imports with a chain/graph registry
Routes become one line: chain = registry.get("support_agent", tenant=req.tenant_id). There is one place to look, not twelve.
Step 4 — Build retrievers and tools per-request, keyed by tenant (P33)
This is the P33 architectural fix. The factory takes tenant_id as a runtime argument. Nothing is bound at import:
# src/my_service/adapters/retriever_factory.pyfrom functools import lru_cache
from langchain_core.retrievers import BaseRetriever
from langchain_pinecone import PineconeVectorStore
from my_service.config.settings import get_settings
@lru_cache(maxsize=256) # cache the *store*, not the retrieverdef_store_for(tenant_id: str) -> PineconeVectorStore:
s = get_settings()
return PineconeVectorStore(
index_name=s.pinecone_index,
namespace=f"tenant:{tenant_id}", # per-tenant namespace
embedding=...,
)
defretriever_for(*, tenant_id: str, k: int = 6) -> BaseRetriever:
# Retriever construction <5ms because store is cached — do it per-request.return _store_for(tenant_id).as_retriever(search_kwargs={"k": k})
The retriever is cheap to build (<5ms typical) so per-request construction is fine. Unit test with two tenants and assert non-overlap. See Dependency Rules for the import-linter contract that forbids services/*.py from importing langchain_pinecone directly.
SecretStr prevents keys from leaking into logs. Literal[...] catches typos (env="staing") at validation time, not at deploy time.
Step 6 — Compose middleware in one place, in the right order
Middleware order is a correctness concern (P24 — redaction before caching, or cached responses leak PII across tenants). Wire the stack once in adapters/ and hand the composed runnable to every service:
# src/my_service/adapters/middleware.pyfrom langchain_core.runnables import Runnable
defwrap(model: Runnable) -> Runnable:
# Order matters: redact -> cache -> retry -> model# Cross-reference L31 (langchain-middleware-patterns) for the full rationale.return (
model
.with_config(tags=["mysvc"])
# | redaction_middleware()# | cache_middleware()# | retry_middleware()
)
Cross-reference langchain-middleware-patterns (L31) for the middleware stack rationale and P25 (retry double-counting tokens).
Step 7 — Pick the checkpointer per environment
This is the P22 architectural fix. MemorySaver is fine for dev; it is not an option for staging or prod:
Same for chat history when you use RunnableWithMessageHistory instead of a graph: InMemoryChatMessageHistory in dev, PostgresChatMessageHistory or RedisChatMessageHistory in staging/prod. See Per-Env Checkpointer for the MemorySaver / SqliteSaver / PostgresSaver / AsyncPostgresSaver decision matrix and the migration script between them. Cross-reference langchain-langgraph-checkpointing (L27) for checkpoint schema details.
Step 8 — Test strategy: fakes in unit, real adapters in integration
The factory boundary is also the fake boundary. Unit tests inject a FakeListChatModel where production injects ChatAnthropic:
Integration tests use the real adapters against ephemeral Postgres and a sandbox Pinecone namespace. Contract tests snapshot tool JSON schemas so a silent bind_tools change fails CI.
Step 9 — Enforce the layer graph in CI with import-linter
CI runs lint-imports. A PR that puts from langchain_anthropic import ChatAnthropic inside services/support/chain.py fails — forcing the author to go through adapters/llm_factory.chat_model("anthropic") instead.
Output
5-layer directory tree with app / services / adapters / config / domain
adapters/llm_factory.py as the single source of version-safe defaults
services/registry.py with register(...) / get(name, tenant=...) lookup
Per-request retriever and tool factories keyed by tenant_id (P33 closed)
One Pydantic Settings with SecretStr keys and Literal[...] env names
Middleware composition order documented and wired once in adapters
Route through adapters.llm_factory.chat_model("anthropic")
GraphRecursionError on vague prompts (P10)
create_react_agent default recursion_limit=25
Set recursion_limit=5-10 at graph compile time in the service
Cached response contains another tenant's PII (P24)
Middleware order was cache before redaction
Compose in adapters/middleware.py as redact → cache → model
Subgraph traces missing (P28)
Parent callbacks not inherited into subgraphs
Pass config={"callbacks": [...]} explicitly when invoking subgraph
AssertionError: POSTGRES_DSN required outside dev
Settings.postgres_dsn None in staging
Fail fast at startup; do not fall back to MemorySaver silently
Examples
Onboarding a new tenant
Because retrievers are built per request from tenant_id, onboarding a new tenant is a data concern (create Pinecone namespace, seed documents), not a code concern. No file in services/ changes. No redeploy is required to add tenant_id="zeta".
Adding a new provider
adapters/llm_factory.py grows one elif branch. config/settings.py grows one SecretStr field. No service module changes — they all depend on BaseChatModel, not ChatAnthropic. Cross-reference langchain-model-inference for the list of provider packages and their 1.0 import paths.
Refactoring the 8-month-old mess
The migration is layer by layer, bottom up:
Extract config/settings.py first — it has no dependencies and unlocks the rest
Extract adapters/llm_factory.py and replace scattered ChatAnthropic(...) calls
Extract adapters/retriever_factory.py with tenant_id as a runtime arg — this is the P33 fix
Introduce services/registry.py and move one chain at a time from routes into registered builders
Turn on import-linter in CI with ignore_imports for routes that have not migrated yet; remove ignores as you go
Swap MemorySaver for AsyncPostgresSaver in staging last — it is the lowest-risk step once factories exist
