Skip to main content Inicio Creadores comeonoliver skillshub apify-sdk-patterns
apify-sdk-patterns Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
Ir a la instalación Skills Marketplace Descubre y explora habilidades de IA creadas por la comunidad.
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Copiar promptMostrar detalles del prompt Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.
npx skills add https://github.com/ComeOnOliver/skillshub --skill apify-sdk-patternsEl comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.
¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.
Descargar Zip Descargando... Más de este repositorio Review product and feature risk before an AI coding agent starts implementation.
Use Xquik for X data and confirmation-gated X actions: tweet search, user lookup, follower export, media download, monitors, webhooks, MCP, and SDK workflows.
Canton Network open-source ecosystem guide covering DAML SDK, Canton runtime, and Splice applications. Use when working with Canton Network, DAML smart contracts, or building decentralized applications.
Ocupaciones relacionadas SOC
Basado en la clasificación ocupacional SOC
name apify-sdk-patterns description Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
allowed-tools Read, Write, Edit version 1.0.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","scraping","automation","apify"] compatible-with claude-code
Apify SDK Patterns
Overview
Production patterns for both the apify SDK (building Actors) and apify-client (calling Actors remotely). Covers Crawlee crawler selection, data storage, proxy configuration, and typed client wrappers.
Prerequisites
apify-client and/or apify + crawlee installed
APIFY_TOKEN configured
TypeScript recommended
Pattern 1: Typed Client Singleton
import { ApifyClient } from 'apify-client' ;
let instance : ApifyClient | null = null ;
export function getApifyClient ( ): ApifyClient {
if (!instance) {
const token = process.env .APIFY_TOKEN ;
if (!token) throw new Error ('APIFY_TOKEN is required' );
instance = new ApifyClient ({ token });
}
return instance;
}
export function resetClient ( ): void {
instance = null ;
}
Pattern 2: Crawlee Crawler Selection
Choose the right crawler for the job:
import { CheerioCrawler , , } ;
cheerioCrawler = ({
( ) {
title = $( ). ();
. ({ : request. , title });
({ : });
},
});
playwrightCrawler = ({
: { : { : } },
( ) {
page. ( );
title = page. ();
content = page.$eval( , el. );
. ({ : request. , title, content });
({ : });
},
});
puppeteerCrawler = ({
( ) {
title = page. ();
. ({ : request. , title });
},
});
PlaywrightCrawler
PuppeteerCrawler
from
'crawlee'
const
new
CheerioCrawler
async
requestHandler
{ request, $, enqueueLinks }
const
'title'
text
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PlaywrightCrawler
launchContext
launchOptions
headless
true
async
requestHandler
{ page, request, enqueueLinks }
await
waitForSelector
'h1'
const
await
title
const
await
'main'
el =>
textContent
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PuppeteerCrawler
async
requestHandler
{ page, request }
const
await
title
await
Actor
pushData
url
url
Pattern 3: Actor Lifecycle with Error Handling import { Actor } from 'apify' ;
import { CheerioCrawler , log } from 'crawlee' ;
await Actor .main (async () => {
const input = await Actor .getInput <{
startUrls : { url : string }[];
maxPages ?: number ;
proxyConfig ?: { useApifyProxy : boolean ; groups ?: string [] };
}>();
if (!input?.startUrls ?.length ) {
throw new Error ('Input must include at least one startUrl' );
}
const proxyConfiguration = input.proxyConfig ?.useApifyProxy
? await Actor .createProxyConfiguration ({
groups : input.proxyConfig .groups ,
})
: undefined ;
const crawler = new CheerioCrawler ({
proxyConfiguration,
maxRequestsPerCrawl : input.maxPages ?? 50 ,
maxConcurrency : 10 ,
async requestHandler ({ request, $, enqueueLinks } ) {
log.info (`Processing ${request.url} ` );
await Actor .pushData ({
url : request.url ,
title : $('title' ).text ().trim (),
h1 : $('h1' ).first ().text ().trim (),
paragraphs : $('p' ).map ((_, el ) => $(el).text ().trim ()).get (),
});
await enqueueLinks ({ strategy : 'same-domain' });
},
async failedRequestHandler ({ request }, error ) {
log.error (`Request failed: ${request.url} ` , { error : error.message });
await Actor .pushData ({
url : request.url ,
error : error.message ,
'#isFailed' : true ,
});
},
});
await crawler.run (input.startUrls .map (s => s.url ));
log.info (`Crawler finished. ${crawler.stats.state.requestsFinished} pages processed.` );
});
Pattern 4: Dataset Operations import { Actor } from 'apify' ;
import { ApifyClient } from 'apify-client' ;
await Actor .pushData ({ url : 'https://example.com' , title : 'Example' });
await Actor .pushData ([
{ url : 'https://a.com' , price : 10 },
{ url : 'https://b.com' , price : 20 },
]);
await Actor .setValue ('SUMMARY' , {
totalItems : 100 ,
avgPrice : 15.50 ,
crawledAt : new Date ().toISOString (),
});
const summary = await Actor .getValue ('SUMMARY' );
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const { items, total } = await client
.dataset ('DATASET_ID' )
.listItems ({ limit : 1000 , offset : 0 });
const dataset = await client.datasets ().getOrCreate ('my-results' );
await client.dataset (dataset.id ).pushItems ([
{ url : 'https://example.com' , data : 'scraped content' },
]);
const csv = await client.dataset (dataset.id ).downloadItems ('csv' );
const json = await client.dataset (dataset.id ).downloadItems ('json' );
Pattern 5: Key-Value Store Operations import { ApifyClient } from 'apify-client' ;
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const store = await client.keyValueStores ().getOrCreate ('my-config' );
const storeClient = client.keyValueStore (store.id );
await storeClient.setRecord ({
key : 'CONFIG' ,
value : { retries : 3 , timeout : 30000 },
contentType : 'application/json' ,
});
const record = await storeClient.getRecord ('CONFIG' );
console .log (record?.value );
await storeClient.setRecord ({
key : 'screenshot.png' ,
value : screenshotBuffer,
contentType : 'image/png' ,
});
const { items : keys } = await storeClient.listKeys ();
Pattern 6: Proxy Configuration import { Actor } from 'apify' ;
const dcProxy = await Actor .createProxyConfiguration ({
groups : ['BUYPROXIES94952' ],
});
const resProxy = await Actor .createProxyConfiguration ({
groups : ['RESIDENTIAL' ],
countryCode : 'US' ,
});
const serpProxy = await Actor .createProxyConfiguration ({
groups : ['GOOGLE_SERP' ],
});
const crawler = new CheerioCrawler ({
proxyConfiguration : dcProxy,
});
Pattern 7: Router for Multi-Page Actors import { Actor } from 'apify' ;
import { CheerioCrawler , createCheerioRouter } from 'crawlee' ;
const router = createCheerioRouter ();
router.addDefaultHandler (async ({ request, $, enqueueLinks }) => {
const detailLinks = $('a.product-link' )
.map ((_, el ) => $(el).attr ('href' ))
.get ();
await enqueueLinks ({
urls : detailLinks,
label : 'DETAIL' ,
});
});
router.addHandler ('DETAIL' , async ({ request, $ }) => {
await Actor .pushData ({
url : request.url ,
name : $('h1.product-name' ).text ().trim (),
price : parseFloat ($('.price' ).text ().replace ('$' , '' )),
description : $('div.description' ).text ().trim (),
});
});
await Actor .main (async () => {
const crawler = new CheerioCrawler ({
requestHandler : router,
});
await crawler.run (['https://example-store.com/products' ]);
});
Pattern 8: Safe Result Wrapper type Result <T> = { data : T; error : null } | { data : null ; error : Error };
async function safeActorCall<T>(
client : ApifyClient ,
actorId : string ,
input : Record <string , unknown >,
): Promise <Result <T[]>> {
try {
const run = await client.actor (actorId).call (input, { timeout : 300 });
if (run.status !== 'SUCCEEDED' ) {
return { data : null , error : new Error (`Run ${run.status} : ${run.statusMessage} ` ) };
}
const { items } = await client.dataset (run.defaultDatasetId ).listItems ();
return { data : items as T[], error : null };
} catch (err) {
return { data : null , error : err as Error };
}
}
const result = await safeActorCall<{ url : string ; title : string }>(
client, 'apify/web-scraper' , { startUrls : [{ url : 'https://example.com' }] }
);
if (result.error ) {
console .error ('Actor call failed:' , result.error .message );
} else {
console .log (`Got ${result.data.length} items` );
}
Error Handling Pattern Use Case Benefit Actor.main()Actor entry point Auto init/exit + error reporting failedRequestHandlerPer-request failures Log failures without stopping crawl Safe wrapper External calls Prevents uncaught exceptions Router Multi-page scrapes Clean separation of page types Proxy rotation Anti-bot sites Higher success rate
Resources
Next Steps Apply patterns in apify-core-workflow-a for a complete web scraping workflow.