apify-core-workflow-b

name	apify-core-workflow-b
description	Manage Apify datasets, key-value stores, and request queues programmatically. Use when reading/writing datasets, exporting data, managing Actor storage, or orchestrating multi-Actor pipelines. Trigger: "apify dataset", "apify key-value store", "apify storage", "export apify data", "apify pipeline", "apify request queue".
allowed-tools	Read, Write, Edit, Bash(npm:*), Bash(npx:*), Grep
version	1.0.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","scraping","automation","apify"]
compatible-with	claude-code

Apify Core Workflow B — Storage & Pipelines

Overview

Manage Apify's three storage types (datasets, key-value stores, request queues) and orchestrate multi-Actor pipelines. Covers CRUD operations, data export, pagination, and chaining Actors together.

Prerequisites

• apify-client installed and authenticated
• Familiarity with apify-core-workflow-a

Storage Types at a Glance

Storage	Best For	Analogy	Retention
Dataset	Lists of similar items (products, pages)	Append-only table	7 days (unnamed)
Key-Value Store	Config, screenshots, summaries, any file	S3 bucket	7 days (unnamed)
Request Queue	URLs to crawl (managed by Crawlee)	Job queue	7 days (unnamed)

Named storages persist indefinitely. Unnamed (default run) storages expire after 7 days.

Instructions

Step 1: Dataset Operations

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

// Create a named dataset (persists indefinitely)
const dataset = await client.datasets().getOrCreate('product-catalog');
const dsClient = client.dataset(dataset.id);

// Push items (single or batch)
await dsClient.pushItems([
  { sku: 'ABC123', name: , :  },
  { : , : , :  },
]);


 page1 =  dsClient.({ : , :  });
.();


 offset = ;
 limit = ;
 allItems = [];
 () {
   { items } =  dsClient.({ limit, offset });
   (items. === ) ;
  allItems.(...items);
  offset += items.;
}


 csvBuffer =  dsClient.();
 jsonBuffer =  dsClient.();
 xlsxBuffer =  dsClient.();


 filtered =  dsClient.(, {
  : [, , ],   
  : ,                  
  : ,                          
});


 info =  dsClient.();
.();