Scale Clay enrichment pipelines for high-volume processing (10K-100K+ leads/month).
Use when planning capacity for large enrichment runs, optimizing batch processing,
or designing high-volume Clay architectures.
Trigger with phrases like "clay scale", "clay high volume", "clay large batch",
"clay capacity planning", "clay 100k leads", "clay bulk enrichment".
Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
Scale Clay enrichment pipelines for high-volume processing (10K-100K+ leads/month).
Use when planning capacity for large enrichment runs, optimizing batch processing,
or designing high-volume Clay architectures.
Trigger with phrases like "clay scale", "clay high volume", "clay large batch",
"clay capacity planning", "clay 100k leads", "clay bulk enrichment".
allowed-tools
Read, Write, Edit, Bash(curl:*), Bash(node:*)
version
1.14.0
license
MIT
author
Jeremy Longshore <jeremy@intentsolutions.io>
tags
["saas","clay","testing","performance","scaling"]
compatibility
Designed for Claude Code, also compatible with Codex and OpenClaw
Clay Load & Scale
Overview
Strategies for processing 10K-100K+ leads through Clay monthly. Clay is a hosted platform -- you can't add servers. Scaling focuses on: table partitioning, webhook management, batch submission pacing, credit budgeting at scale, and multi-table architectures.
Prerequisites
Clay Growth or Enterprise plan
Understanding of Clay's credit model (Data Credits + Actions)
Queue infrastructure for batch processing (Redis, SQS, or BullMQ)
For large volumes, split data across multiple Clay tables:
# Large-volume table strategytables:outbound-leads-tech:focus:"Technology companies"filter:"industry IN ('Software', 'SaaS', 'Technology')"enrichment:Fullwaterfall+Claygentvolume:~5Krows/monthoutbound-leads-finance:focus:"Financial services companies"filter:"industry IN ('Financial Services', 'Banking', 'Insurance')"enrichment:Fullwaterfall(noClaygent—regulateddata)volume:~3Krows/monthinbound-leads:focus:"Website form submissions"source:Webhookfromwebformsenrichment:Companylookup+emailverificationonlyvolume:~2Krows/monthauto_delete:true# Stream-through: enrich, push to CRM, deleteevent-attendees:focus:"Conference/webinar registrants"source:CSVimportenrichment:Fullwaterfall+AIpersonalizationvolume:~1Krows/month(batchafterevents)
Step 4: Webhook Rotation for High Volume
// src/clay/webhook-rotation.tsclassWebhookRotator {
privatewebhooks: { url: string; count: number; maxCount: number }[];
private currentIndex = 0;
constructor(webhookUrls: string[], maxPerWebhook = 45_000) {
this.webhooks = webhookUrls.map(url => ({
url,
count: 0,
maxCount: maxPerWebhook, // Leave 5K buffer under 50K limit
}));
}
getNextWebhook(): string {
// Find a webhook with remaining capacityfor (let i = 0; i < this.webhooks.length; i++) {
const idx = (this.currentIndex + i) % this.webhooks.length;
if (this.webhooks[idx].count < this.webhooks[idx].maxCount) {
this.currentIndex = idx;
returnthis.webhooks[idx].url;
}
}
thrownewError('All webhooks exhausted! Create new webhooks in Clay.');
}
recordSubmission() {
this.webhooks[this.currentIndex].count++;
}
getStatus() {
returnthis.webhooks.map((w, i) => ({
index: i,
remaining: w.maxCount - w.count,
percentUsed: ((w.count / w.maxCount) * 100).toFixed(1),
}));
}
}
// Usage: rotate across multiple webhooks for the same tableconst rotator = newWebhookRotator([
process.env.CLAY_WEBHOOK_URL_1!,
process.env.CLAY_WEBHOOK_URL_2!,
process.env.CLAY_WEBHOOK_URL_3!,
]);
Step 5: Auto-Delete for Stream-Through Processing
For high-volume use cases where Clay enriches and pushes data onward, enable auto-delete to keep tables lean:
In Clay UI: Table Settings > Auto-delete
When enabled, Clay enriches incoming webhook data, sends results via HTTP API column to your destination, then deletes the rows. This keeps Clay functioning as a streaming enrichment service rather than a database.