Skip to main content 홈 크리에이터 comeonoliver skillshub apify-sdk-patterns
apify-sdk-patterns Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
설치로 이동 Skills Marketplace 커뮤니티가 만든 AI 스킬을 발견하고 탐색하세요.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
직접 명령은 검토 Prompt를 거치지 않습니다. 실행하기 전에 소스를 확인하세요.
npx skills add https://github.com/ComeOnOliver/skillshub --skill apify-sdk-patterns명령은 한 줄로 유지됩니다. 복사하기 전에 가로로 스크롤해 전체 내용을 확인하세요.
로컬 사본을 원하시나요? SkillsMP에서 현재 제공할 수 있는 파일을 다운로드하세요.
Zip 다운로드 다운로드 중... name apify-sdk-patterns description Production-ready patterns for Apify SDK and apify-client in TypeScript.
Use when building Actors with Crawlee, managing datasets/KV stores,
or implementing robust client wrappers with retry and validation.
Trigger: "apify SDK patterns", "apify best practices",
"apify client wrapper", "crawlee patterns", "idiomatic apify".
allowed-tools Read, Write, Edit version 1.0.0 license MIT author Jeremy Longshore <jeremy@intentsolutions.io> tags ["saas","scraping","automation","apify"] compatible-with claude-code
Apify SDK Patterns
Overview
Production patterns for both the apify SDK (building Actors) and apify-client (calling Actors remotely). Covers Crawlee crawler selection, data storage, proxy configuration, and typed client wrappers.
Prerequisites
apify-client and/or apify + crawlee installed
APIFY_TOKEN configured
TypeScript recommended
Pattern 1: Typed Client Singleton
import { ApifyClient } from 'apify-client' ;
let instance : ApifyClient | null = null ;
export function getApifyClient ( ): ApifyClient {
if (!instance) {
const token = process.env .APIFY_TOKEN ;
if (!token) throw new Error ('APIFY_TOKEN is required' );
instance = new ApifyClient ({ token });
}
return instance;
}
export function resetClient ( ): void {
instance = null ;
}
Pattern 2: Crawlee Crawler Selection
Choose the right crawler for the job:
import { CheerioCrawler , , } ;
cheerioCrawler = ({
( ) {
title = $( ). ();
. ({ : request. , title });
({ : });
},
});
playwrightCrawler = ({
: { : { : } },
( ) {
page. ( );
title = page. ();
content = page.$eval( , el. );
. ({ : request. , title, content });
({ : });
},
});
puppeteerCrawler = ({
( ) {
title = page. ();
. ({ : request. , title });
},
});
PlaywrightCrawler
PuppeteerCrawler
from
'crawlee'
const
new
CheerioCrawler
async
requestHandler
{ request, $, enqueueLinks }
const
'title'
text
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PlaywrightCrawler
launchContext
launchOptions
headless
true
async
requestHandler
{ page, request, enqueueLinks }
await
waitForSelector
'h1'
const
await
title
const
await
'main'
el =>
textContent
await
Actor
pushData
url
url
await
enqueueLinks
strategy
'same-domain'
const
new
PuppeteerCrawler
async
requestHandler
{ page, request }
const
await
title
await
Actor
pushData
url
url
Pattern 3: Actor Lifecycle with Error Handling import { Actor } from 'apify' ;
import { CheerioCrawler , log } from 'crawlee' ;
await Actor .main (async () => {
const input = await Actor .getInput <{
startUrls : { url : string }[];
maxPages ?: number ;
proxyConfig ?: { useApifyProxy : boolean ; groups ?: string [] };
}>();
if (!input?.startUrls ?.length ) {
throw new Error ('Input must include at least one startUrl' );
}
const proxyConfiguration = input.proxyConfig ?.useApifyProxy
? await Actor .createProxyConfiguration ({
groups : input.proxyConfig .groups ,
})
: undefined ;
const crawler = new CheerioCrawler ({
proxyConfiguration,
maxRequestsPerCrawl : input.maxPages ?? 50 ,
maxConcurrency : 10 ,
async requestHandler ({ request, $, enqueueLinks } ) {
log.info (`Processing ${request.url} ` );
await Actor .pushData ({
url : request.url ,
title : $('title' ).text ().trim (),
h1 : $('h1' ).first ().text ().trim (),
paragraphs : $('p' ).map ((_, el ) => $(el).text ().trim ()).get (),
});
await enqueueLinks ({ strategy : 'same-domain' });
},
async failedRequestHandler ({ request }, error ) {
log.error (`Request failed: ${request.url} ` , { error : error.message });
await Actor .pushData ({
url : request.url ,
error : error.message ,
'#isFailed' : true ,
});
},
});
await crawler.run (input.startUrls .map (s => s.url ));
log.info (`Crawler finished. ${crawler.stats.state.requestsFinished} pages processed.` );
});
Pattern 4: Dataset Operations import { Actor } from 'apify' ;
import { ApifyClient } from 'apify-client' ;
await Actor .pushData ({ url : 'https://example.com' , title : 'Example' });
await Actor .pushData ([
{ url : 'https://a.com' , price : 10 },
{ url : 'https://b.com' , price : 20 },
]);
await Actor .setValue ('SUMMARY' , {
totalItems : 100 ,
avgPrice : 15.50 ,
crawledAt : new Date ().toISOString (),
});
const summary = await Actor .getValue ('SUMMARY' );
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const { items, total } = await client
.dataset ('DATASET_ID' )
.listItems ({ limit : 1000 , offset : 0 });
const dataset = await client.datasets ().getOrCreate ('my-results' );
await client.dataset (dataset.id ).pushItems ([
{ url : 'https://example.com' , data : 'scraped content' },
]);
const csv = await client.dataset (dataset.id ).downloadItems ('csv' );
const json = await client.dataset (dataset.id ).downloadItems ('json' );
Pattern 5: Key-Value Store Operations import { ApifyClient } from 'apify-client' ;
const client = new ApifyClient ({ token : process.env .APIFY_TOKEN });
const store = await client.keyValueStores ().getOrCreate ('my-config' );
const storeClient = client.keyValueStore (store.id );
await storeClient.setRecord ({
key : 'CONFIG' ,
value : { retries : 3 , timeout : 30000 },
contentType : 'application/json' ,
});
const record = await storeClient.getRecord ('CONFIG' );
console .log (record?.value );
await storeClient.setRecord ({
key : 'screenshot.png' ,
value : screenshotBuffer,
contentType : 'image/png' ,
});
const { items : keys } = await storeClient.listKeys ();
Pattern 6: Proxy Configuration import { Actor } from 'apify' ;
const dcProxy = await Actor .createProxyConfiguration ({
groups : ['BUYPROXIES94952' ],
});
const resProxy = await Actor .createProxyConfiguration ({
groups : ['RESIDENTIAL' ],
countryCode : 'US' ,
});
const serpProxy = await Actor .createProxyConfiguration ({
groups : ['GOOGLE_SERP' ],
});
const crawler = new CheerioCrawler ({
proxyConfiguration : dcProxy,
});
Pattern 7: Router for Multi-Page Actors import { Actor } from 'apify' ;
import { CheerioCrawler , createCheerioRouter } from 'crawlee' ;
const router = createCheerioRouter ();
router.addDefaultHandler (async ({ request, $, enqueueLinks }) => {
const detailLinks = $('a.product-link' )
.map ((_, el ) => $(el).attr ('href' ))
.get ();
await enqueueLinks ({
urls : detailLinks,
label : 'DETAIL' ,
});
});
router.addHandler ('DETAIL' , async ({ request, $ }) => {
await Actor .pushData ({
url : request.url ,
name : $('h1.product-name' ).text ().trim (),
price : parseFloat ($('.price' ).text ().replace ('$' , '' )),
description : $('div.description' ).text ().trim (),
});
});
await Actor .main (async () => {
const crawler = new CheerioCrawler ({
requestHandler : router,
});
await crawler.run (['https://example-store.com/products' ]);
});
Pattern 8: Safe Result Wrapper type Result <T> = { data : T; error : null } | { data : null ; error : Error };
async function safeActorCall<T>(
client : ApifyClient ,
actorId : string ,
input : Record <string , unknown >,
): Promise <Result <T[]>> {
try {
const run = await client.actor (actorId).call (input, { timeout : 300 });
if (run.status !== 'SUCCEEDED' ) {
return { data : null , error : new Error (`Run ${run.status} : ${run.statusMessage} ` ) };
}
const { items } = await client.dataset (run.defaultDatasetId ).listItems ();
return { data : items as T[], error : null };
} catch (err) {
return { data : null , error : err as Error };
}
}
const result = await safeActorCall<{ url : string ; title : string }>(
client, 'apify/web-scraper' , { startUrls : [{ url : 'https://example.com' }] }
);
if (result.error ) {
console .error ('Actor call failed:' , result.error .message );
} else {
console .log (`Got ${result.data.length} items` );
}
Error Handling Pattern Use Case Benefit Actor.main()Actor entry point Auto init/exit + error reporting failedRequestHandlerPer-request failures Log failures without stopping crawl Safe wrapper External calls Prevents uncaught exceptions Router Multi-page scrapes Clean separation of page types Proxy rotation Anti-bot sites Higher success rate
Resources
Next Steps Apply patterns in apify-core-workflow-a for a complete web scraping workflow.