apify-core-workflow-b

name	apify-core-workflow-b
description	Manage Apify datasets, key-value stores, and request queues programmatically, and orchestrate multi-Actor pipelines. Use when you need to read or write Apify datasets, export scraped data to CSV/JSON/XLSX, store config or binary artifacts in a key-value store, manage a resumable request queue, chain Actors into a scrape → transform → export pipeline, or monitor Actor run status and cost. Trigger with "apify dataset", "apify key-value store", "apify storage", "export apify data", "apify pipeline", "apify request queue".
allowed-tools	Read, Write, Edit, Bash(npm:*), Bash(npx:*), Grep
version	1.5.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","scraping","automation","apify"]
compatibility	Designed for Claude Code

Apify Core Workflow B — Storage & Pipelines

Overview

Manage Apify's three storage types (datasets, key-value stores, request queues) and orchestrate multi-Actor pipelines using the apify-client JS SDK. Covers CRUD operations, data export, automatic pagination, and chaining Actors together (scrape → transform → export).

This SKILL.md gives you the high-level workflow plus the essential first example for each storage type. Drill into the reference files for the complete, copy-ready code:

• Storage operations — full reference — every dataset, key-value store, and request queue operation with pagination, format export, and binary records.
• Pipelines & run monitoring — full reference — the multi-Actor pipeline function and Actor-run status/cost/abort monitoring.

Prerequisites

• Node.js with apify-client installed (npm install apify-client).
• An Apify account token exported as APIFY_TOKEN (see Authentication below).
• Familiarity with apify-core-workflow-a (Actor invocation and run lifecycle), since pipelines chain Actor runs and read their default storages.

Authentication

All operations authenticate with an Apify API token. Never hard-code it — read it from the environment and construct the client once:

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

Generate a token at Apify Console → Settings → Integrations, then export it (export APIFY_TOKEN=apify_api_...) or load it from your secrets manager.

Storage Types at a Glance

Storage	Best For	Analogy	Retention
Dataset	Lists of similar items (products, pages)	Append-only table	7 days (unnamed)
Key-Value Store	Config, screenshots, summaries, any file