Skip to main content الرئيسية المنشئون comeonoliver skillshub apify-sdk-patterns
apify-sdk-patterns Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
الانتقال إلى التثبيت سوق المهارات اكتشف واستكشف مهارات الذكاء الاصطناعي التي بناها المجتمع.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
نسخ Promptعرض تفاصيل Prompt يتجاوز الأمر المباشر Prompt المخصّص للمراجعة. افحص المصدر قبل تشغيله.
npx skills add https://github.com/ComeOnOliver/skillshub --skill apify-sdk-patternsيبقى الأمر في سطر واحد. مرّر أفقيًا لمراجعته كاملًا قبل النسخ.
تفضّل نسخة محلية؟ نزّل الملفات المتاحة حاليًا لدى SkillsMP.
تحميل Zip جاري التحميل... المهن ذات الصلة SOC
استنادا إلى تصنيف SOC المهني
name apify-sdk-patterns description Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
allowed-tools Read, Write, Edit version 1.0.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","scraping","automation","apify"] compatible-with claude-code
Apify SDK Patterns
Overview
Production patterns for both the apify SDK (building Actors) and apify-client (calling Actors remotely). Covers Crawlee crawler selection, data storage, proxy configuration, and typed client wrappers.
Prerequisites
apify-client and/or apify + crawlee installed
APIFY_TOKEN configured
TypeScript recommended
Pattern 1: Typed Client Singleton
import { ApifyClient } from 'apify-client' ;
let instance : ApifyClient | null = null ;
export function getApifyClient ( ): ApifyClient {
if (!instance) {
const token = process.env .APIFY_TOKEN ;
if (!token) throw new Error ('APIFY_TOKEN is required' );
instance = new ApifyClient ({ token });
}
return instance;
}
export function resetClient ( ): void {
instance = null ;
}
Pattern 2: Crawlee Crawler Selection
Choose the right crawler for the job:
import { CheerioCrawler , , } ;
cheerioCrawler = ({
( ) {
title = $( ). ();
. ({ : request. , title });
({ : });
},
});
playwrightCrawler = ({
: { : { : } },
( ) {
page. ( );
title = page. ();
content = page.$eval( , el. );
. ({ : request. , title, content });
({ : });
},
});
puppeteerCrawler = ({
( ) {
title = page. ();
. ({ : request. , title });
},
});
PlaywrightCrawler
PuppeteerCrawler
from
'crawlee'
const
new
CheerioCrawler
async
requestHandler
{ request, $, enqueueLinks }
const
'title'
text
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PlaywrightCrawler
launchContext
launchOptions
headless
true
async
requestHandler
{ page, request, enqueueLinks }
await
waitForSelector
'h1'
const
await
title
const
await
'main'
el =>
textContent
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PuppeteerCrawler
async
requestHandler
{ page, request }
const
await
title
await
Actor
pushData
url
url
Pattern 3: Actor Lifecycle with Error Handling import { Actor } from 'apify' ;
import { CheerioCrawler , log } from 'crawlee' ;
await Actor .main (async () => {
const input = await Actor .getInput <{
startUrls : { url : string }[];
maxPages ?: number ;
proxyConfig ?: { useApifyProxy : boolean ; groups ?: string [] };
}>();
if (!input?.startUrls ?.length ) {
throw new Error ('Input must include at least one startUrl' );
}
const proxyConfiguration = input.proxyConfig ?.useApifyProxy
? await Actor .createProxyConfiguration ({
groups : input.proxyConfig .groups ,
})
: undefined ;
const crawler = new CheerioCrawler ({
proxyConfiguration,
maxRequestsPerCrawl : input.maxPages ?? 50 ,
maxConcurrency : 10 ,
async requestHandler ({ request, $, enqueueLinks } ) {
log.info (`Processing ${request.url} ` );
await Actor .pushData ({
url : request.url ,
title : $('title' ).text ().trim (),
h1 : $('h1' ).first ().text ().trim (),
paragraphs : $('p' ).map ((_, el ) => $(el).text ().trim ()).get (),
});
await enqueueLinks ({ strategy : 'same-domain' });
},
async failedRequestHandler ({ request }, error ) {
log.error (`Request failed: ${request.url} ` , { error : error.message });
await Actor .pushData ({
url : request.url ,
error : error.message ,
'#isFailed' : true ,
});
},
});
await crawler.run (input.startUrls .map (s => s.url ));
log.info (`Crawler finished. ${crawler.stats.state.requestsFinished} pages processed.` );
});
Pattern 4: Dataset Operations import { Actor } from 'apify' ;
import { ApifyClient } from 'apify-client' ;
await Actor .pushData ({ url : 'https://example.com' , title : 'Example' });
await Actor .pushData ([
{ url : 'https://a.com' , price : 10 },
{ url : 'https://b.com' , price : 20 },
]);
await Actor .setValue ('SUMMARY' , {
totalItems : 100 ,
avgPrice : 15.50 ,
crawledAt : new Date ().toISOString (),
});
const summary = await Actor .getValue ('SUMMARY' );
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const { items, total } = await client
.dataset ('DATASET_ID' )
.listItems ({ limit : 1000 , offset : 0 });
const dataset = await client.datasets ().getOrCreate ('my-results' );
await client.dataset (dataset.id ).pushItems ([
{ url : 'https://example.com' , data : 'scraped content' },
]);
const csv = await client.dataset (dataset.id ).downloadItems ('csv' );
const json = await client.dataset (dataset.id ).downloadItems ('json' );
Pattern 5: Key-Value Store Operations import { ApifyClient } from 'apify-client' ;
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const store = await client.keyValueStores ().getOrCreate ('my-config' );
const storeClient = client.keyValueStore (store.id );
await storeClient.setRecord ({
key : 'CONFIG' ,
value : { retries : 3 , timeout : 30000 },
contentType : 'application/json' ,
});
const record = await storeClient.getRecord ('CONFIG' );
console .log (record?.value );
await storeClient.setRecord ({
key : 'screenshot.png' ,
value : screenshotBuffer,
contentType : 'image/png' ,
});
const { items : keys } = await storeClient.listKeys ();
Pattern 6: Proxy Configuration import { Actor } from 'apify' ;
const dcProxy = await Actor .createProxyConfiguration ({
groups : ['BUYPROXIES94952' ],
});
const resProxy = await Actor .createProxyConfiguration ({
groups : ['RESIDENTIAL' ],
countryCode : 'US' ,
});
const serpProxy = await Actor .createProxyConfiguration ({
groups : ['GOOGLE_SERP' ],
});
const crawler = new CheerioCrawler ({
proxyConfiguration : dcProxy,
});
Pattern 7: Router for Multi-Page Actors import { Actor } from 'apify' ;
import { CheerioCrawler , createCheerioRouter } from 'crawlee' ;
const router = createCheerioRouter ();
router.addDefaultHandler (async ({ request, $, enqueueLinks }) => {
const detailLinks = $('a.product-link' )
.map ((_, el ) => $(el).attr ('href' ))
.get ();
await enqueueLinks ({
urls : detailLinks,
label : 'DETAIL' ,
});
});
router.addHandler ('DETAIL' , async ({ request, $ }) => {
await Actor .pushData ({
url : request.url ,
name : $('h1.product-name' ).text ().trim (),
price : parseFloat ($('.price' ).text ().replace ('$' , '' )),
description : $('div.description' ).text ().trim (),
});
});
await Actor .main (async () => {
const crawler = new CheerioCrawler ({
requestHandler : router,
});
await crawler.run (['https://example-store.com/products' ]);
});
Pattern 8: Safe Result Wrapper type Result <T> = { data : T; error : null } | { data : null ; error : Error };
async function safeActorCall<T>(
client : ApifyClient ,
actorId : string ,
input : Record <string , unknown >,
): Promise <Result <T[]>> {
try {
const run = await client.actor (actorId).call (input, { timeout : 300 });
if (run.status !== 'SUCCEEDED' ) {
return { data : null , error : new Error (`Run ${run.status} : ${run.statusMessage} ` ) };
}
const { items } = await client.dataset (run.defaultDatasetId ).listItems ();
return { data : items as T[], error : null };
} catch (err) {
return { data : null , error : err as Error };
}
}
const result = await safeActorCall<{ url : string ; title : string }>(
client, 'apify/web-scraper' , { startUrls : [{ url : 'https://example.com' }] }
);
if (result.error ) {
console .error ('Actor call failed:' , result.error .message );
} else {
console .log (`Got ${result.data.length} items` );
}
Error Handling Pattern Use Case Benefit Actor.main()Actor entry point Auto init/exit + error reporting failedRequestHandlerPer-request failures Log failures without stopping crawl Safe wrapper External calls Prevents uncaught exceptions Router Multi-page scrapes Clean separation of page types Proxy rotation Anti-bot sites Higher success rate
Resources
Next Steps Apply patterns in apify-core-workflow-a for a complete web scraping workflow.