Skip to main content Startseite Ersteller comeonoliver skillshub apify-sdk-patterns
apify-sdk-patterns Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
Zur Installation springen Skills Marktplatz Entdecken und erkunden Sie KI-Skills, die von der Community erstellt wurden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Prompt kopierenPrompt-Details anzeigen Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
npx skills add https://github.com/ComeOnOliver/skillshub --skill apify-sdk-patternsDer Befehl bleibt in einer Zeile. Scrollen Sie horizontal, um ihn vor dem Kopieren vollständig zu prüfen.
Sie bevorzugen eine lokale Kopie? Laden Sie die Dateien herunter, die SkillsMP derzeit vorliegen.
ZIP herunterladen Herunterladen... Mehr aus diesem Repository
Verwandte Berufe SOC
Basierend auf der SOC-Berufsklassifikation
name apify-sdk-patterns description Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
allowed-tools Read, Write, Edit version 1.0.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","scraping","automation","apify"] compatible-with claude-code
Apify SDK Patterns
Overview
Production patterns for both the apify SDK (building Actors) and apify-client (calling Actors remotely). Covers Crawlee crawler selection, data storage, proxy configuration, and typed client wrappers.
Prerequisites
apify-client and/or apify + crawlee installed
APIFY_TOKEN configured
TypeScript recommended
Pattern 1: Typed Client Singleton
import { ApifyClient } from 'apify-client' ;
let instance : ApifyClient | null = null ;
export function getApifyClient ( ): ApifyClient {
if (!instance) {
const token = process.env .APIFY_TOKEN ;
if (!token) throw new Error ('APIFY_TOKEN is required' );
instance = new ApifyClient ({ token });
}
return instance;
}
export function resetClient ( ): void {
instance = null ;
}
Pattern 2: Crawlee Crawler Selection
Choose the right crawler for the job:
import { CheerioCrawler , , } ;
cheerioCrawler = ({
( ) {
title = $( ). ();
. ({ : request. , title });
({ : });
},
});
playwrightCrawler = ({
: { : { : } },
( ) {
page. ( );
title = page. ();
content = page.$eval( , el. );
. ({ : request. , title, content });
({ : });
},
});
puppeteerCrawler = ({
( ) {
title = page. ();
. ({ : request. , title });
},
});
PlaywrightCrawler
PuppeteerCrawler
from
'crawlee'
const
new
CheerioCrawler
async
requestHandler
{ request, $, enqueueLinks }
const
'title'
text
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PlaywrightCrawler
launchContext
launchOptions
headless
true
async
requestHandler
{ page, request, enqueueLinks }
await
waitForSelector
'h1'
const
await
title
const
await
'main'
el =>
textContent
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PuppeteerCrawler
async
requestHandler
{ page, request }
const
await
title
await
Actor
pushData
url
url
Pattern 3: Actor Lifecycle with Error Handling import { Actor } from 'apify' ;
import { CheerioCrawler , log } from 'crawlee' ;
await Actor .main (async () => {
const input = await Actor .getInput <{
startUrls : { url : string }[];
maxPages ?: number ;
proxyConfig ?: { useApifyProxy : boolean ; groups ?: string [] };
}>();
if (!input?.startUrls ?.length ) {
throw new Error ('Input must include at least one startUrl' );
}
const proxyConfiguration = input.proxyConfig ?.useApifyProxy
? await Actor .createProxyConfiguration ({
groups : input.proxyConfig .groups ,
})
: undefined ;
const crawler = new CheerioCrawler ({
proxyConfiguration,
maxRequestsPerCrawl : input.maxPages ?? 50 ,
maxConcurrency : 10 ,
async requestHandler ({ request, $, enqueueLinks } ) {
log.info (`Processing ${request.url} ` );
await Actor .pushData ({
url : request.url ,
title : $('title' ).text ().trim (),
h1 : $('h1' ).first ().text ().trim (),
paragraphs : $('p' ).map ((_, el ) => $(el).text ().trim ()).get (),
});
await enqueueLinks ({ strategy : 'same-domain' });
},
async failedRequestHandler ({ request }, error ) {
log.error (`Request failed: ${request.url} ` , { error : error.message });
await Actor .pushData ({
url : request.url ,
error : error.message ,
'#isFailed' : true ,
});
},
});
await crawler.run (input.startUrls .map (s => s.url ));
log.info (`Crawler finished. ${crawler.stats.state.requestsFinished} pages processed.` );
});
Pattern 4: Dataset Operations import { Actor } from 'apify' ;
import { ApifyClient } from 'apify-client' ;
await Actor .pushData ({ url : 'https://example.com' , title : 'Example' });
await Actor .pushData ([
{ url : 'https://a.com' , price : 10 },
{ url : 'https://b.com' , price : 20 },
]);
await Actor .setValue ('SUMMARY' , {
totalItems : 100 ,
avgPrice : 15.50 ,
crawledAt : new Date ().toISOString (),
});
const summary = await Actor .getValue ('SUMMARY' );
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const { items, total } = await client
.dataset ('DATASET_ID' )
.listItems ({ limit : 1000 , offset : 0 });
const dataset = await client.datasets ().getOrCreate ('my-results' );
await client.dataset (dataset.id ).pushItems ([
{ url : 'https://example.com' , data : 'scraped content' },
]);
const csv = await client.dataset (dataset.id ).downloadItems ('csv' );
const json = await client.dataset (dataset.id ).downloadItems ('json' );
Pattern 5: Key-Value Store Operations import { ApifyClient } from 'apify-client' ;
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const store = await client.keyValueStores ().getOrCreate ('my-config' );
const storeClient = client.keyValueStore (store.id );
await storeClient.setRecord ({
key : 'CONFIG' ,
value : { retries : 3 , timeout : 30000 },
contentType : 'application/json' ,
});
const record = await storeClient.getRecord ('CONFIG' );
console .log (record?.value );
await storeClient.setRecord ({
key : 'screenshot.png' ,
value : screenshotBuffer,
contentType : 'image/png' ,
});
const { items : keys } = await storeClient.listKeys ();
Pattern 6: Proxy Configuration import { Actor } from 'apify' ;
const dcProxy = await Actor .createProxyConfiguration ({
groups : ['BUYPROXIES94952' ],
});
const resProxy = await Actor .createProxyConfiguration ({
groups : ['RESIDENTIAL' ],
countryCode : 'US' ,
});
const serpProxy = await Actor .createProxyConfiguration ({
groups : ['GOOGLE_SERP' ],
});
const crawler = new CheerioCrawler ({
proxyConfiguration : dcProxy,
});
Pattern 7: Router for Multi-Page Actors import { Actor } from 'apify' ;
import { CheerioCrawler , createCheerioRouter } from 'crawlee' ;
const router = createCheerioRouter ();
router.addDefaultHandler (async ({ request, $, enqueueLinks }) => {
const detailLinks = $('a.product-link' )
.map ((_, el ) => $(el).attr ('href' ))
.get ();
await enqueueLinks ({
urls : detailLinks,
label : 'DETAIL' ,
});
});
router.addHandler ('DETAIL' , async ({ request, $ }) => {
await Actor .pushData ({
url : request.url ,
name : $('h1.product-name' ).text ().trim (),
price : parseFloat ($('.price' ).text ().replace ('$' , '' )),
description : $('div.description' ).text ().trim (),
});
});
await Actor .main (async () => {
const crawler = new CheerioCrawler ({
requestHandler : router,
});
await crawler.run (['https://example-store.com/products' ]);
});
Pattern 8: Safe Result Wrapper type Result <T> = { data : T; error : null } | { data : null ; error : Error };
async function safeActorCall<T>(
client : ApifyClient ,
actorId : string ,
input : Record <string , unknown >,
): Promise <Result <T[]>> {
try {
const run = await client.actor (actorId).call (input, { timeout : 300 });
if (run.status !== 'SUCCEEDED' ) {
return { data : null , error : new Error (`Run ${run.status} : ${run.statusMessage} ` ) };
}
const { items } = await client.dataset (run.defaultDatasetId ).listItems ();
return { data : items as T[], error : null };
} catch (err) {
return { data : null , error : err as Error };
}
}
const result = await safeActorCall<{ url : string ; title : string }>(
client, 'apify/web-scraper' , { startUrls : [{ url : 'https://example.com' }] }
);
if (result.error ) {
console .error ('Actor call failed:' , result.error .message );
} else {
console .log (`Got ${result.data.length} items` );
}
Error Handling Pattern Use Case Benefit Actor.main()Actor entry point Auto init/exit + error reporting failedRequestHandlerPer-request failures Log failures without stopping crawl Safe wrapper External calls Prevents uncaught exceptions Router Multi-page scrapes Clean separation of page types Proxy rotation Anti-bot sites Higher success rate
Resources
Next Steps Apply patterns in apify-core-workflow-a for a complete web scraping workflow.