A reference layered architecture for production LangChain 1.0 / LangGraph 1.0
services — LLM factory with version-safe defaults, chain/graph registry,
retriever and tool DI, Pydantic-validated config, per-request tenant scoping,
middleware ordering, checkpointer selection per environment. Use when starting
a new service, refactoring a tangled chain, or onboarding a team to existing code.
Trigger with "langchain architecture", "langchain llm factory",
"langchain chain registry", "langchain dependency injection",
"langchain project structure".
A reference layered architecture for production LangChain 1.0 / LangGraph 1.0
services — LLM factory with version-safe defaults, chain/graph registry,
retriever and tool DI, Pydantic-validated config, per-request tenant scoping,
middleware ordering, checkpointer selection per environment. Use when starting
a new service, refactoring a tangled chain, or onboarding a team to existing code.
Trigger with "langchain architecture", "langchain llm factory",
"langchain chain registry", "langchain dependency injection",
"langchain project structure".
Eight months into a LangChain service, a code review surfaces the mess.
Twelve chain definitions live inlined inside FastAPI route handlers. Three
retrievers are constructed at module-global scope, one bound to
tenant_id="acme" because that was the first tenant in the pilot —
that retriever now returns Acme's documents to every other tenant, a P33
leak that has been live in production for six weeks.
max_retries=6 is hardcoded at four separate call sites. A
RunnableWithMessageHistory backed by the default
InMemoryChatMessageHistory loses every conversation on pod restart
(P22) — which is most days, because Cloud Run scales to zero.
Config is read from os.environ in three modules with three different
fallback strategies. There is no place to put a new provider without
touching seven files, and nobody remembers why the retriever is built
at import time.
The fix is not "rename a variable." The fix is an architecture that made
every one of those mistakes hard to write. This skill is the target
layered architecture:
app/ — FastAPI routes. Thin. Parses HTTP, calls into services,
serializes response. No chain logic, no vendor clients, no env vars.
services/ — chain and graph definitions. Take dependencies through
constructor args, not module-level imports.
adapters/ — vendor clients, LLM factory, retriever factory, tool
factory. This is where langchain-anthropic is imported. Nowhere else.
config/ — one Pydantic Settings class. SecretStr for keys,
Literal["dev","staging","prod"] for env names, .env file loader.
domain/ — Pydantic models, typed LangGraph state, enums. No I/O.
Five layers, five imports deep at most. Dependency direction is
strictly downward. app imports services; services imports
adapters; adapters imports config and domain. Never the reverse.
Import-linter enforces this in CI. Pain-catalog anchors: P22 (in-memory
history loses messages — architectural fix is persistent history
injected via DI) and P33 (per-tenant vector stores leak if retriever
bound at import — architectural fix is per-request factory). Adjacent:
P10 (recursion limits), P24 (middleware order), P28 (callback
inheritance). Pin: , ,
, , ,
.
The max_retries=6 scatter in the mess-case becomes max_retries=2 in exactly one file. Services that want a longer timeout pass timeout=60 — but they never set max_retries=6 by accident. Cross-reference langchain-model-inference Step 3 for the factory pattern's provenance; see LLM Factory Pattern for per-provider variants and caching.
Step 3 — Replace scattered imports with a chain/graph registry
Routes become one line: chain = registry.get("support_agent", tenant=req.tenant_id). There is one place to look, not twelve.
Step 4 — Build retrievers and tools per-request, keyed by tenant (P33)
This is the P33 architectural fix. The factory takes tenant_id as a runtime argument. Nothing is bound at import:
# src/my_service/adapters/retriever_factory.pyfrom functools import lru_cache
from langchain_core.retrievers import BaseRetriever
from langchain_pinecone import PineconeVectorStore
from my_service.config.settings import get_settings
@lru_cache(maxsize=256) # cache the *store*, not the retrieverdef_store_for(tenant_id: str) -> PineconeVectorStore:
s = get_settings()
return PineconeVectorStore(
index_name=s.pinecone_index,
namespace=f"tenant:{tenant_id}", # per-tenant namespace
embedding=...,
)
defretriever_for(*, tenant_id: str, k: int = 6) -> BaseRetriever:
# Retriever construction <5ms because store is cached — do it per-request.return _store_for(tenant_id).as_retriever(search_kwargs={"k": k})
The retriever is cheap to build (<5ms typical) so per-request construction is fine. Unit test with two tenants and assert non-overlap. See Dependency Rules for the import-linter contract that forbids services/*.py from importing langchain_pinecone directly.
SecretStr prevents keys from leaking into logs. Literal[...] catches typos (env="staing") at validation time, not at deploy time.
Step 6 — Compose middleware in one place, in the right order
Middleware order is a correctness concern (P24 — redaction before caching, or cached responses leak PII across tenants). Wire the stack once in adapters/ and hand the composed runnable to every service:
# src/my_service/adapters/middleware.pyfrom langchain_core.runnables import Runnable
defwrap(model: Runnable) -> Runnable:
# Order matters: redact -> cache -> retry -> model# Cross-reference L31 (langchain-middleware-patterns) for the full rationale.return (
model
.with_config(tags=["mysvc"])
# | redaction_middleware()# | cache_middleware()# | retry_middleware()
)
Cross-reference langchain-middleware-patterns (L31) for the middleware stack rationale and P25 (retry double-counting tokens).
Step 7 — Pick the checkpointer per environment
This is the P22 architectural fix. MemorySaver is fine for dev; it is not an option for staging or prod:
Same for chat history when you use RunnableWithMessageHistory instead of a graph: InMemoryChatMessageHistory in dev, PostgresChatMessageHistory or RedisChatMessageHistory in staging/prod. See Per-Env Checkpointer for the MemorySaver / SqliteSaver / PostgresSaver / AsyncPostgresSaver decision matrix and the migration script between them. Cross-reference langchain-langgraph-checkpointing (L27) for checkpoint schema details.
Step 8 — Test strategy: fakes in unit, real adapters in integration
The factory boundary is also the fake boundary. Unit tests inject a FakeListChatModel where production injects ChatAnthropic:
Integration tests use the real adapters against ephemeral Postgres and a sandbox Pinecone namespace. Contract tests snapshot tool JSON schemas so a silent bind_tools change fails CI.
Step 9 — Enforce the layer graph in CI with import-linter
CI runs lint-imports. A PR that puts from langchain_anthropic import ChatAnthropic inside services/support/chain.py fails — forcing the author to go through adapters/llm_factory.chat_model("anthropic") instead.
Output
5-layer directory tree with app / services / adapters / config / domain
adapters/llm_factory.py as the single source of version-safe defaults
services/registry.py with register(...) / get(name, tenant=...) lookup
Per-request retriever and tool factories keyed by tenant_id (P33 closed)
One Pydantic Settings with SecretStr keys and Literal[...] env names
Middleware composition order documented and wired once in adapters
Route through adapters.llm_factory.chat_model("anthropic")
GraphRecursionError on vague prompts (P10)
create_react_agent default recursion_limit=25
Set recursion_limit=5-10 at graph compile time in the service
Cached response contains another tenant's PII (P24)
Middleware order was cache before redaction
Compose in adapters/middleware.py as redact → cache → model
Subgraph traces missing (P28)
Parent callbacks not inherited into subgraphs
Pass config={"callbacks": [...]} explicitly when invoking subgraph
AssertionError: POSTGRES_DSN required outside dev
Settings.postgres_dsn None in staging
Fail fast at startup; do not fall back to MemorySaver silently
Examples
Onboarding a new tenant
Because retrievers are built per request from tenant_id, onboarding a new tenant is a data concern (create Pinecone namespace, seed documents), not a code concern. No file in services/ changes. No redeploy is required to add tenant_id="zeta".
Adding a new provider
adapters/llm_factory.py grows one elif branch. config/settings.py grows one SecretStr field. No service module changes — they all depend on BaseChatModel, not ChatAnthropic. Cross-reference langchain-model-inference for the list of provider packages and their 1.0 import paths.
Refactoring the 8-month-old mess
The migration is layer by layer, bottom up:
Extract config/settings.py first — it has no dependencies and unlocks the rest
Extract adapters/llm_factory.py and replace scattered ChatAnthropic(...) calls
Extract adapters/retriever_factory.py with tenant_id as a runtime arg — this is the P33 fix
Introduce services/registry.py and move one chain at a time from routes into registered builders
Turn on import-linter in CI with ignore_imports for routes that have not migrated yet; remove ignores as you go
Swap MemorySaver for AsyncPostgresSaver in staging last — it is the lowest-risk step once factories exist
