Skip to main content Accueil Créateurs comeonoliver skillshub apify-sdk-patterns
apify-sdk-patterns Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
Aller à l'installation Skills Marketplace Découvrez et explorez les compétences IA créées par la communauté.
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Copier le promptAfficher les détails du prompt Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
npx skills add https://github.com/ComeOnOliver/skillshub --skill apify-sdk-patternsLa commande reste sur une seule ligne. Faites défiler horizontalement pour la vérifier avant de la copier.
Vous préférez une copie locale ? Téléchargez les fichiers actuellement disponibles dans SkillsMP.
Télécharger Zip Téléchargement... Plus depuis ce dépôt Review product and feature risk before an AI coding agent starts implementation.
Use Xquik for X data and confirmation-gated X actions: tweet search, user lookup, follower export, media download, monitors, webhooks, MCP, and SDK workflows.
Canton Network open-source ecosystem guide covering DAML SDK, Canton runtime, and Splice applications. Use when working with Canton Network, DAML smart contracts, or building decentralized applications.
Métiers associés SOC
Basé sur la classification professionnelle SOC
name apify-sdk-patterns description Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
allowed-tools Read, Write, Edit version 1.0.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","scraping","automation","apify"] compatible-with claude-code
Apify SDK Patterns
Overview
Production patterns for both the apify SDK (building Actors) and apify-client (calling Actors remotely). Covers Crawlee crawler selection, data storage, proxy configuration, and typed client wrappers.
Prerequisites
apify-client and/or apify + crawlee installed
APIFY_TOKEN configured
TypeScript recommended
Pattern 1: Typed Client Singleton
import { ApifyClient } from 'apify-client' ;
let instance : ApifyClient | null = null ;
export function getApifyClient ( ): ApifyClient {
if (!instance) {
const token = process.env .APIFY_TOKEN ;
if (!token) throw new Error ('APIFY_TOKEN is required' );
instance = new ApifyClient ({ token });
}
return instance;
}
export function resetClient ( ): void {
instance = null ;
}
Pattern 2: Crawlee Crawler Selection
Choose the right crawler for the job:
import { CheerioCrawler , , } ;
cheerioCrawler = ({
( ) {
title = $( ). ();
. ({ : request. , title });
({ : });
},
});
playwrightCrawler = ({
: { : { : } },
( ) {
page. ( );
title = page. ();
content = page.$eval( , el. );
. ({ : request. , title, content });
({ : });
},
});
puppeteerCrawler = ({
( ) {
title = page. ();
. ({ : request. , title });
},
});
PlaywrightCrawler
PuppeteerCrawler
from
'crawlee'
const
new
CheerioCrawler
async
requestHandler
{ request, $, enqueueLinks }
const
'title'
text
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PlaywrightCrawler
launchContext
launchOptions
headless
true
async
requestHandler
{ page, request, enqueueLinks }
await
waitForSelector
'h1'
const
await
title
const
await
'main'
el =>
textContent
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PuppeteerCrawler
async
requestHandler
{ page, request }
const
await
title
await
Actor
pushData
url
url
Pattern 3: Actor Lifecycle with Error Handling import { Actor } from 'apify' ;
import { CheerioCrawler , log } from 'crawlee' ;
await Actor .main (async () => {
const input = await Actor .getInput <{
startUrls : { url : string }[];
maxPages ?: number ;
proxyConfig ?: { useApifyProxy : boolean ; groups ?: string [] };
}>();
if (!input?.startUrls ?.length ) {
throw new Error ('Input must include at least one startUrl' );
}
const proxyConfiguration = input.proxyConfig ?.useApifyProxy
? await Actor .createProxyConfiguration ({
groups : input.proxyConfig .groups ,
})
: undefined ;
const crawler = new CheerioCrawler ({
proxyConfiguration,
maxRequestsPerCrawl : input.maxPages ?? 50 ,
maxConcurrency : 10 ,
async requestHandler ({ request, $, enqueueLinks } ) {
log.info (`Processing ${request.url} ` );
await Actor .pushData ({
url : request.url ,
title : $('title' ).text ().trim (),
h1 : $('h1' ).first ().text ().trim (),
paragraphs : $('p' ).map ((_, el ) => $(el).text ().trim ()).get (),
});
await enqueueLinks ({ strategy : 'same-domain' });
},
async failedRequestHandler ({ request }, error ) {
log.error (`Request failed: ${request.url} ` , { error : error.message });
await Actor .pushData ({
url : request.url ,
error : error.message ,
'#isFailed' : true ,
});
},
});
await crawler.run (input.startUrls .map (s => s.url ));
log.info (`Crawler finished. ${crawler.stats.state.requestsFinished} pages processed.` );
});
Pattern 4: Dataset Operations import { Actor } from 'apify' ;
import { ApifyClient } from 'apify-client' ;
await Actor .pushData ({ url : 'https://example.com' , title : 'Example' });
await Actor .pushData ([
{ url : 'https://a.com' , price : 10 },
{ url : 'https://b.com' , price : 20 },
]);
await Actor .setValue ('SUMMARY' , {
totalItems : 100 ,
avgPrice : 15.50 ,
crawledAt : new Date ().toISOString (),
});
const summary = await Actor .getValue ('SUMMARY' );
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const { items, total } = await client
.dataset ('DATASET_ID' )
.listItems ({ limit : 1000 , offset : 0 });
const dataset = await client.datasets ().getOrCreate ('my-results' );
await client.dataset (dataset.id ).pushItems ([
{ url : 'https://example.com' , data : 'scraped content' },
]);
const csv = await client.dataset (dataset.id ).downloadItems ('csv' );
const json = await client.dataset (dataset.id ).downloadItems ('json' );
Pattern 5: Key-Value Store Operations import { ApifyClient } from 'apify-client' ;
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const store = await client.keyValueStores ().getOrCreate ('my-config' );
const storeClient = client.keyValueStore (store.id );
await storeClient.setRecord ({
key : 'CONFIG' ,
value : { retries : 3 , timeout : 30000 },
contentType : 'application/json' ,
});
const record = await storeClient.getRecord ('CONFIG' );
console .log (record?.value );
await storeClient.setRecord ({
key : 'screenshot.png' ,
value : screenshotBuffer,
contentType : 'image/png' ,
});
const { items : keys } = await storeClient.listKeys ();
Pattern 6: Proxy Configuration import { Actor } from 'apify' ;
const dcProxy = await Actor .createProxyConfiguration ({
groups : ['BUYPROXIES94952' ],
});
const resProxy = await Actor .createProxyConfiguration ({
groups : ['RESIDENTIAL' ],
countryCode : 'US' ,
});
const serpProxy = await Actor .createProxyConfiguration ({
groups : ['GOOGLE_SERP' ],
});
const crawler = new CheerioCrawler ({
proxyConfiguration : dcProxy,
});
Pattern 7: Router for Multi-Page Actors import { Actor } from 'apify' ;
import { CheerioCrawler , createCheerioRouter } from 'crawlee' ;
const router = createCheerioRouter ();
router.addDefaultHandler (async ({ request, $, enqueueLinks }) => {
const detailLinks = $('a.product-link' )
.map ((_, el ) => $(el).attr ('href' ))
.get ();
await enqueueLinks ({
urls : detailLinks,
label : 'DETAIL' ,
});
});
router.addHandler ('DETAIL' , async ({ request, $ }) => {
await Actor .pushData ({
url : request.url ,
name : $('h1.product-name' ).text ().trim (),
price : parseFloat ($('.price' ).text ().replace ('$' , '' )),
description : $('div.description' ).text ().trim (),
});
});
await Actor .main (async () => {
const crawler = new CheerioCrawler ({
requestHandler : router,
});
await crawler.run (['https://example-store.com/products' ]);
});
Pattern 8: Safe Result Wrapper type Result <T> = { data : T; error : null } | { data : null ; error : Error };
async function safeActorCall<T>(
client : ApifyClient ,
actorId : string ,
input : Record <string , unknown >,
): Promise <Result <T[]>> {
try {
const run = await client.actor (actorId).call (input, { timeout : 300 });
if (run.status !== 'SUCCEEDED' ) {
return { data : null , error : new Error (`Run ${run.status} : ${run.statusMessage} ` ) };
}
const { items } = await client.dataset (run.defaultDatasetId ).listItems ();
return { data : items as T[], error : null };
} catch (err) {
return { data : null , error : err as Error };
}
}
const result = await safeActorCall<{ url : string ; title : string }>(
client, 'apify/web-scraper' , { startUrls : [{ url : 'https://example.com' }] }
);
if (result.error ) {
console .error ('Actor call failed:' , result.error .message );
} else {
console .log (`Got ${result.data.length} items` );
}
Error Handling Pattern Use Case Benefit Actor.main()Actor entry point Auto init/exit + error reporting failedRequestHandlerPer-request failures Log failures without stopping crawl Safe wrapper External calls Prevents uncaught exceptions Router Multi-page scrapes Clean separation of page types Proxy rotation Anti-bot sites Higher success rate
Resources
Next Steps Apply patterns in apify-core-workflow-a for a complete web scraping workflow.