Skip to main content ホーム クリエイター jeremylongshore tons-of-skills-marketplace clay-reliability-patterns
clay-reliability-patterns Build fault-tolerant Clay integrations with circuit breakers, dead letter queues, and graceful degradation.
Use when building production Clay pipelines that need resilience,
implementing retry strategies, or adding fault tolerance to enrichment workflows.
Trigger with phrases like "clay reliability", "clay circuit breaker", "clay resilience",
"clay fallback", "clay fault tolerance", "clay dead letter queue".
インストールへ移動 Skills Marketplace コミュニティが作成したAIスキルを発見・探索
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/jeremylongshore/tons-of-skills-marketplace --skill clay-reliability-patternsコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか?SkillsMP が現在取得できるファイルをダウンロードできます。
Zipをダウンロード ダウンロード中... このリポジトリの他の Skills langchain-deploy-integration Deploy a LangChain 1.0 / LangGraph 1.0 app to Cloud Run, Vercel, or LangServe correctly — with timeouts sized for chain length, cold-start mitigation, SSE anti-buffering headers, and Secret Manager over .env. Use when prepping a first production deploy, debugging a stream that hangs behind a proxy, or diagnosing p99 latency spikes. Trigger with "langchain deploy", "langchain cloud run", "langchain vercel python", "langchain langserve", or "langchain docker".
langchain-langgraph-agents Build a correct LangGraph 1.0 ReAct agent with create_react_agent — typed tools, error propagation, recursion caps, and stop conditions that actually stop. Use when writing a first tool-calling agent, migrating from AgentExecutor or initialize_agent, or diagnosing an agent that loops on vague prompts. Trigger with "langgraph agent", "create_react_agent", "langgraph tool calling", "AgentExecutor migration", or "agent loop cost".
langchain-langgraph-human-in-loop Build LangGraph 1.0 human-in-the-loop approval flows with interrupt_before /
interrupt_after and Command(resume=...) — JSON-serializable state, clean
resume semantics, and UI wiring for approval decisions. Use when adding an
approval gate before an expensive tool call, wiring a Slack/web UI for agent
approvals, or debugging a graph that crashes on interrupt.
Trigger with "langgraph human in loop", "langgraph interrupt_before",
"langgraph approval flow", "Command resume", "langgraph HITL".
name clay-reliability-patterns description Build fault-tolerant Clay integrations with circuit breakers, dead letter queues, and graceful degradation.
Use when building production Clay pipelines that need resilience,
implementing retry strategies, or adding fault tolerance to enrichment workflows.
Trigger with phrases like "clay reliability", "clay circuit breaker", "clay resilience",
"clay fallback", "clay fault tolerance", "clay dead letter queue".
allowed-tools Read, Write, Edit, Bash(curl:*) version 1.14.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","clay","clay-reliability"] compatibility Designed for Claude Code
Clay Reliability Patterns
Overview
Production reliability patterns for Clay data enrichment pipelines. Clay's async enrichment model, credit-based billing, and dependency on 150+ external data providers require specific resilience strategies: credit budget circuit breakers, webhook delivery tracking, dead letter queues for failed batches, and graceful degradation when Clay is unavailable.
Prerequisites
Clay integration in production or pre-production
Redis or similar for state tracking
Understanding of Clay's async enrichment model
Monitoring infrastructure (see clay-observability)
Instructions
Step 1: Credit Budget Circuit Breaker
Stop processing when credit burn exceeds budget to prevent runaway costs:
class CreditCircuitBreaker {
private state : 'closed' | 'open' | 'half-open' = 'closed' ;
private dailyCreditsUsed = 0 ;
private failureCount = 0 ;
private lastFailureAt : Date | null = null ;
private readonly cooldownMs : number ;
constructor (
private dailyLimit : number ,
private failureThreshold : number = 5 ,
cooldownMinutes : number = 15 ,
) {
this .cooldownMs = cooldownMinutes * * ;
}
( : ): { : ; ?: } {
( . === ) {
( . && . () - . . () > . ) {
. = ;
. ( );
} {
{ : , : };
}
}
( . + estimatedCredits > . ) {
{ : , : };
}
{ : };
}
( ) {
. += creditsUsed;
( . === ) {
. = ;
. = ;
. ( );
}
}
( ) {
. ++;
. = ();
( . >= . ) {
. = ;
. ( );
}
}
( ) {
. = ;
}
}
60
1000
canProcess
estimatedCredits
number
allowed
boolean
reason
string
if
this
state
'open'
if
this
lastFailureAt
Date
now
this
lastFailureAt
getTime
this
cooldownMs
this
state
'half-open'
console
log
'Circuit breaker: half-open (testing)'
else
return
allowed
false
reason
`Circuit OPEN. Cooldown until ${new Date (this .lastFailureAt!.getTime() + this .cooldownMs).toISOString()} `
if
this
dailyCreditsUsed
this
dailyLimit
return
allowed
false
reason
`Daily credit limit reached: ${this .dailyCreditsUsed} /${this .dailyLimit} `
return
allowed
true
recordSuccess
creditsUsed : number
this
dailyCreditsUsed
if
this
state
'half-open'
this
state
'closed'
this
failureCount
0
console
log
'Circuit breaker: closed (recovered)'
recordFailure
this
failureCount
this
lastFailureAt
new
Date
if
this
failureCount
this
failureThreshold
this
state
'open'
console
error
`Circuit breaker: OPEN after ${this .failureCount} failures`
resetDaily
this
dailyCreditsUsed
0
Step 2: Dead Letter Queue for Failed Submissions
interface DLQEntry {
row : Record <string , unknown >;
error : string ;
webhookUrl : string ;
failedAt : string ;
retryCount : number ;
maxRetries : number ;
}
class ClayDLQ {
private entries : DLQEntry [] = [];
addToQueue (row : Record <string , unknown >, error : string , webhookUrl : string ): void {
this .entries .push ({
row,
error,
webhookUrl,
failedAt : new Date ().toISOString (),
retryCount : 0 ,
maxRetries : 3 ,
});
console .warn (`DLQ: Added row (${this .entries.length} total). Error: ${error} ` );
}
async retryAll (): Promise <{ retried : number ; succeeded : number ; permanentFailures : number }> {
let succeeded = 0 , permanentFailures = 0 ;
const remaining : DLQEntry [] = [];
for (const entry of this .entries ) {
if (entry.retryCount >= entry.maxRetries ) {
permanentFailures++;
continue ;
}
try {
const res = await fetch (entry.webhookUrl , {
method : 'POST' ,
headers : { 'Content-Type' : 'application/json' },
body : JSON .stringify (entry.row ),
});
if (res.ok ) {
succeeded++;
} else {
entry.retryCount ++;
remaining.push (entry);
}
} catch {
entry.retryCount ++;
remaining.push (entry);
}
await new Promise (r => setTimeout (r, 500 ));
}
this .entries = remaining;
return { retried : this .entries .length + succeeded + permanentFailures, succeeded, permanentFailures };
}
getStats ( ) {
return {
pending : this .entries .length ,
byError : this .entries .reduce ((acc, e ) => {
acc[e.error ] = (acc[e.error ] || 0 ) + 1 ;
return acc;
}, {} as Record <string , number >),
};
}
}
Step 3: Webhook Health Monitor
class WebhookHealthMonitor {
private successCount = 0 ;
private failureCount = 0 ;
private lastCheck : Date = new Date ();
private readonly windowMs = 5 * 60 * 1000 ;
record (success : boolean ) {
if (success) this .successCount ++;
else this .failureCount ++;
}
getHealthScore (): { score : number ; status : 'healthy' | 'degraded' | 'unhealthy' } {
const total = this .successCount + this .failureCount ;
if (total === 0 ) return { score : 100 , status : 'healthy' };
const score = (this .successCount / total) * 100 ;
if (Date .now () - this .lastCheck .getTime () > this .windowMs ) {
this .successCount = 0 ;
this .failureCount = 0 ;
this .lastCheck = new Date ();
}
return {
score,
status : score > 95 ? 'healthy' : score > 80 ? 'degraded' : 'unhealthy' ,
};
}
}
Step 4: Graceful Degradation When Clay Is Down
interface FallbackConfig {
cacheEnrichedData : boolean ;
queueForLater : boolean ;
useLocalFallback : boolean ;
}
class ClayWithFallback {
private cache = new Map <string , Record <string , unknown >>();
private offlineQueue : Record <string , unknown >[] = [];
async enrichOrFallback (
lead : Record <string , unknown >,
webhookUrl : string ,
config : FallbackConfig ,
): Promise <{ data : Record <string , unknown >; source : 'clay' | 'cache' | 'queued' | 'local' }> {
try {
const res = await fetch (webhookUrl, {
method : 'POST' ,
headers : { 'Content-Type' : 'application/json' },
body : JSON .stringify (lead),
signal : AbortSignal .timeout (10_000 ),
});
if (res.ok ) {
return { data : lead, source : 'clay' };
}
} catch {
console .warn ('Clay webhook unavailable — using fallback' );
}
const domain = lead.domain as string ;
if (config.cacheEnrichedData && this .cache .has (domain)) {
return { data : { ...lead, ...this .cache .get (domain) }, source : 'cache' };
}
if (config.queueForLater ) {
this .offlineQueue .push (lead);
return { data : lead, source : 'queued' };
}
if (config.useLocalFallback ) {
return {
data : { ...lead, company_name : domain.replace (/\.\w+$/ , '' ).replace (/-/g , ' ' ) },
source : 'local' ,
};
}
return { data : lead, source : 'local' };
}
async drainOfflineQueue (webhookUrl : string ): Promise <number > {
let drained = 0 ;
while (this .offlineQueue .length > 0 ) {
const lead = this .offlineQueue .shift ()!;
try {
await fetch (webhookUrl, {
method : 'POST' ,
headers : { 'Content-Type' : 'application/json' },
body : JSON .stringify (lead),
});
drained++;
await new Promise (r => setTimeout (r, 200 ));
} catch {
this .offlineQueue .unshift (lead);
break ;
}
}
return drained;
}
}
Step 5: Combine All Patterns
const circuitBreaker = new CreditCircuitBreaker (500 );
const dlq = new ClayDLQ ();
const healthMonitor = new WebhookHealthMonitor ();
async function reliableEnrich (lead : Record <string , unknown >, webhookUrl : string ): Promise <void > {
const { allowed, reason } = circuitBreaker.canProcess (6 );
if (!allowed) {
dlq.addToQueue (lead, `Circuit breaker: ${reason} ` , webhookUrl);
return ;
}
try {
const res = await fetch (webhookUrl, {
method : 'POST' ,
headers : { 'Content-Type' : 'application/json' },
body : JSON .stringify (lead),
});
if (res.ok ) {
circuitBreaker.recordSuccess (6 );
healthMonitor.record (true );
} else {
throw new Error (`HTTP ${res.status} ` );
}
} catch (err) {
circuitBreaker.recordFailure ();
healthMonitor.record (false );
dlq.addToQueue (lead, (err as Error ).message , webhookUrl);
}
}
Error Handling Issue Cause Solution Runaway credit spend No budget circuit breaker Implement credit budget limiter Lost leads during outage No DLQ Queue failed submissions for retry Silent webhook failures No health monitoring Track success/failure rates Clay outage blocks pipeline No fallback Implement cache + queue fallback
Resources
Next Steps For policy guardrails, see clay-policy-guardrails.