Compose LangChain 1.0 Python runnables with the production defaults the docs
do not warn about: parallel batching, narrow fallbacks, and brace-safe prompts.
Use when building an LCEL chain with RunnableSequence / RunnableParallel,
adding resilience via `.with_fallbacks()`, tuning throughput with `.batch()`
or `.abatch()`, or wrapping user input in a prompt template.
Trigger with "langchain runnable", "with_fallbacks", "langchain batch",
"runnable sequence", "lcel", "runnableparallel", "chain composition".
Compose LangChain 1.0 Python runnables with the production defaults the docs
do not warn about: parallel batching, narrow fallbacks, and brace-safe prompts.
Use when building an LCEL chain with RunnableSequence / RunnableParallel,
adding resilience via `.with_fallbacks()`, tuning throughput with `.batch()`
or `.abatch()`, or wrapping user input in a prompt template.
Trigger with "langchain runnable", "with_fallbacks", "langchain batch",
"runnable sequence", "lcel", "runnableparallel", "chain composition".
Designed for Claude Code, also compatible with Codex
LangChain SDK Patterns (Python)
Overview
chain.batch(inputs) in LangChain 1.0 does not parallelize by default. The
max_concurrency parameter defaults to 1 in several provider packages
(notably older langchain-openai), so a call like chain.batch(inputs_1000)
runs 1,000 sequential round-trips — same wall-clock time as a for loop, plus
the overhead of the batch machinery. Users file "batch is slow" tickets,
benchmark it against asyncio, and move to a different framework — when the fix
is two lines:
# BAD — silently serializes (P08)
chain.batch(inputs_1000)
# GOOD — 10 in flight at once
chain.batch(inputs_1000, config={"max_concurrency": 10})
Then three more traps wait:
P07 — .with_fallbacks([backup]) defaults exceptions_to_handle=(Exception,),
and on Python <3.12 that tuple includes KeyboardInterrupt. A Ctrl+C during
a long run does not stop the process — it silently hands off to the fallback
chain and keeps billing.
P57 — ChatPromptTemplate.from_messages(..., template_format="f-string")
(the default) parses every { in every string, including user input. A user
who pastes {"error": "..."} raises KeyError: 'error' at invoke time.
P53 — Pydantic v2 rejects extra fields by default; models cheerfully add
summary or confidence to your Plan schema and with_structured_output
crashes with ValidationError: extra fields not permitted.
This skill walks through LCEL composition (RunnableSequence, RunnableParallel,
RunnableBranch, RunnablePassthrough, RunnableLambda); the correct
exceptions_to_handle whitelist per provider; max_concurrency tuning with
safe ceilings (10 for most providers, 20+ with a semaphore); and prompt
templates that survive untrusted input. Pin: langchain-core 1.0.x,
langchain-anthropic 1.0.x, . Pain-catalog anchors:
P07, P08, P53, P57.
langchain-openai 1.0.x
Prerequisites
Python 3.10+ (3.12+ fixes the KeyboardInterrupt half of P07 — upgrade if you can)
langchain-core >= 1.0, < 2.0
At least one provider: pip install langchain-anthropic langchain-openai
pydantic >= 2.0 for schema-aware composition
Completed langchain-model-inference — the chat-model factory from that skill is reused here
Instructions
Step 1 — Compose with typed runnables, not lambdas
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableParallel, RunnablePassthrough
llm = ChatAnthropic(model="claude-sonnet-4-6", timeout=30, max_retries=2)
prompt = ChatPromptTemplate.from_messages(
[("system", "You are a summarizer."), ("human", "{text}")],
template_format="jinja2", # P57 — see Step 4
)
# Sequence: prompt -> llm -> str
chain = prompt | llm | StrOutputParser()
# Parallel: run two sub-chains and merge
enriched = RunnableParallel(
summary=chain,
original=RunnablePassthrough(),
)
The | operator creates a RunnableSequence. Each step has a declared input
and output shape — swap a concrete model for a router and the type contract
holds. See Runnable Composition Matrix
for when to reach for RunnableSequence vs RunnableParallel vs RunnableBranch
vs RunnableLambda, with input/output shape conventions for each.
Step 2 — Add fallbacks with a narrow exception whitelist
from anthropic import APIError, APITimeoutError, RateLimitError
from langchain_openai import ChatOpenAI
backup = ChatOpenAI(model="gpt-4o", timeout=30, max_retries=2)
backup_chain = prompt | backup | StrOutputParser()
# GOOD — only retry on transient provider errors
resilient = chain.with_fallbacks(
[backup_chain],
exceptions_to_handle=(RateLimitError, APIError, APITimeoutError),
)
# BAD — default `(Exception,)` catches KeyboardInterrupt on Python <3.12 (P07)# resilient_bad = chain.with_fallbacks([backup_chain])
The default exceptions_to_handle=(Exception,) on Python <3.12 inherits
KeyboardInterrupt and SystemExit into the caught set — which means a
Ctrl+C during a long .batch() run falls through to the backup instead of
stopping. Python 3.12+ moved these under BaseException directly, which fixes
the inheritance path, but the default is still too broad: a Pydantic
ValidationError or a ToolException will trigger a pointless backup call.
See Fallback Exception List for the
curated whitelist per provider with concrete imports.
Safe ceilings: 10 for Anthropic and OpenAI at default tier; 20+ only
behind an asyncio.Semaphore if you are also tracking rate-limit headers.
Claude TPM/RPM limits vary by tier; OpenAI's TPD (tokens per day) is the
binding limit at scale. See Batch Concurrency Tuning
for per-provider ceilings and the semaphore pattern.
invoke vs batch vs stream — when each is correct:
Method
Input shape
Concurrency
Error behavior
When to use
.invoke(x)
Single
1
Raises on failure
One-shot call, interactive, tests
.batch(xs, config={"max_concurrency": N})
List
N parallel
Raises on first failure unless return_exceptions=True
Bulk sync workloads, ETL, eval harnesses
.abatch(xs, config={"max_concurrency": N})
List
N parallel (async)
Same as .batch
Event loops, async web servers, LangGraph nodes
.stream(x)
Single
1, chunked
Raises on failure
Interactive UI, live token display
.astream(x) / .astream_events(x, version="v2")
Single
1, chunked (async)
Raises on failure
Async UIs, event-driven pipelines, token metering (see langchain-model-inference)
Pass return_exceptions=True in the config to keep a batch from aborting on
the first failure — exceptions come back in the result list instead of raising.
Step 4 — Escape prompt templates for untrusted input
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
# BAD — default f-string format crashes on literal `{` in user input (P57)
bad = ChatPromptTemplate.from_messages(
[("system", "Reply in JSON"), ("human", "{user_text}")]
)
bad.invoke({"user_text": '{"error": "oops"}'}) # KeyError: 'error'# GOOD — jinja2 treats `{...}` as literal, uses `{{ var }}` for substitution
good = ChatPromptTemplate.from_messages(
[("system", "Reply in JSON"), ("human", "{{ user_text }}")],
template_format="jinja2",
)
good.invoke({"user_text": '{"error": "oops"}'}) # OK# MIXED — message history is a list, use MessagesPlaceholder
with_history = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant."),
MessagesPlaceholder("history"),
("human", "{{ question }}"),
], template_format="jinja2")
Rule of thumb: if any variable can contain user-provided free text (a paste,
a transcript, a code block), use template_format="jinja2". The f-string
format is fine for trusted template authors composing fixed instructions, but
it is the wrong tool for user input. See Prompt Template Escaping
for the full brace-escaping rules and a MessagesPlaceholder reference.
Step 5 — Validate structured output with extra="ignore"
from pydantic import BaseModel, ConfigDict, Field
classPlan(BaseModel):
# P53 — without this, the chain crashes when the model adds extra fields
model_config = ConfigDict(extra="ignore")
steps: list[str] = Field(default_factory=list)
estimated_minutes: int
structured_chain = prompt | llm.with_structured_output(Plan, method="json_schema")
Pydantic v2 rejects unknown fields by default. Models trained on "be helpful"
add summary, confidence, rationale — the schema crashes instead of
dropping them. extra="ignore" is the right default for model outputs.
Output
RunnableSequence / RunnableParallel composition with declared input/output shapes
.with_fallbacks(exceptions_to_handle=(...)) with a narrow, provider-specific whitelist
ChatPromptTemplate.from_messages(..., template_format="jinja2") for any template touching user input
Pydantic schemas with ConfigDict(extra="ignore") for structured output
A clear invoke / batch / abatch / stream / astream decision matrix for each chain stage
Error Handling
Error
Cause
Fix
Ctrl+C does not stop a long .batch(); fallback keeps running
exceptions_to_handle=(Exception,) swallows KeyboardInterrupt on Python <3.12 (P07)
Pass a narrow tuple: exceptions_to_handle=(RateLimitError, APIError, APITimeoutError)
.batch(inputs) takes same time as sequential loop
max_concurrency defaults to 1 (P08)
config={"max_concurrency": 10}; raise to 20+ only with a semaphore
KeyError: '<some-token>' when invoking a ChatPromptTemplate
f-string parser reads user input's { as a variable (P57)
template_format="jinja2"; escape literals as {{/}} in f-string mode
ValidationError: extra fields not permitted on structured output
Pydantic v2 strict-by-default (P53)
model_config = ConfigDict(extra="ignore") on the schema
ValidationError caught by fallback and treated as transient
Fallback whitelist too broad
Remove ValidationError from exceptions_to_handle so it surfaces
.batch aborts on the first failure, losing all results
Default raises on first error
Pass config={"max_concurrency": 10, "return_exceptions": True} and filter
Fallback chain never fires even on genuine RateLimitError
Provider's own max_retries consumes the error first
Lower max_retries=0 on the primary when a fallback chain is the retry strategy
Examples
Fan-out enrichment with RunnableParallel
A common pattern — given a document, produce a summary, extracted entities,
and sentiment in parallel. RunnableParallel runs sub-chains concurrently and
merges results into a dict. Combined with .batch() at the outer level, you
get N documents times 3 sub-chains in flight up to max_concurrency.
See Runnable Composition Matrix
for the fan-out/fan-in pattern and the input/output shape of each runnable type.
Resilient chain with per-provider fallback
Primary: Claude Sonnet 4.6. Fallback: GPT-4o. Catch only RateLimitError,
APIError, and APITimeoutError from each SDK — let AuthenticationError
and ValidationError crash the process so they get debugged, not masked.
See Fallback Exception List for the
concrete imports per provider and a note on why BadRequestError should not
be in the whitelist.
High-throughput batch with semaphore-bounded concurrency
At N >= 20 concurrent in-flight calls, provider rate-limit headers become the
bottleneck. Wrap .abatch() in an asyncio.Semaphore and honor the
retry-after header on 429 responses.
See Batch Concurrency Tuning for the
semaphore pattern and a table of provider TPM/RPM limits per tier.
Prompt template over user-pasted JSON payload
Support ticket triage where users paste arbitrary JSON from their app's error
log. Without template_format="jinja2", every single ticket with a JSON body
crashes the chain at template-render time.
See Prompt Template Escaping for the
worked example and the MessagesPlaceholder pattern for chat history.
