apify-actor-runtime

# Local Apify Actor runtime These are the operating instructions for the Actor runtime the reader is talking to: a local Apify platform serving an Apify-compatible API on `http://localhost:3333` and a console UI on `http://localhost:3000` (the defaults - see "Other ports" below). It emulates the subset of the Apify API needed to develop, run and debug Actors locally, so no change needs a rebuild on the real platform to be tried out. It is not the Apify platform. Actors, builds, runs and storages created here exist only in this runtime's data directory, and disappear when that directory is removed. ## Point the Apify CLI at it Everything below is done through `apify`. Two ways to aim it: ```sh apify runtime connect # every command, every terminal, until 'apify runtime disconnect' ``` or, for one shell only (these take precedence over `connect` wherever they are set): ```sh export APIFY_CLIENT_BASE_URL=http://localhost:3333 export APIFY_CONSOLE_URL=http://localhost:3000 ``` Unset both, or run `apify runtime disconnect`, to talk to the real Apify platform again. Check which one the CLI is currently aimed at with `apify runtime status`. Any non-empty token authenticates - `apify login --token local-dev-token` is enough. Set `APIFY_DISABLE_KEYRING=1` first in a sandbox with no OS keyring. To act as a second user, pass a different token on a single call: `apify api v2/datasets -H '{"authorization": "Bearer OTHER"}'`. ### Other ports When 3333 or 3000 is taken on this machine, start the runtime on other ports: `apify runtime start --api-port 4333 --console-port 4000`. The CLI remembers them for later starts and for `apify runtime connect`; with a hand-started container, set `-e ACTOR_RUNTIME_API_PORT=4333 -e ACTOR_RUNTIME_CONSOLE_PORT=4000` and publish the same numbers (`-p 4333:4333 -p 4000:4000`). Every URL in this document then uses those ports instead, `apify-api:3333` included. ## The normal loop ```sh cd my-actor apify push # upload + build, once apify call --input '{"maxPages":3}' apify runs ls --json apify runs log <runId> apify datasets get-items <datasetId> --format json ``` ### Apple Silicon and other arm64 hosts Builds run for the host's architecture. An Actor whose base image has no build for it (the Playwright images are `linux/amd64` only) is built and run for `linux/amd64` instead, as on the platform, under the engine's emulation: it works, just slower, and the build log says so. ## Monorepo Actors (`dockerContextDir`) Push from the Actor's own folder; it builds as the platform builds it from Git with that folder selected: ```sh cd monorepo/actors/my-actor apify push apify call ``` - The push sends your local working copy, uncommitted changes included, with a warning: the platform builds only what is committed and pushed. `.actorignore` does not apply, as for Git builds. - No Git access or credentials are needed; check out the branch you want, then push. - `apify push` of such an Actor to the Apify platform stops with an error: it is not supported there yet (https://github.com/apify/apify-core/issues/28685). Build it from the Git repository instead. ## Dataset schema validation A dataset schema in `.actor/actor.json` is enforced as on the platform. It comes from the **build**, so an edited schema takes effect only after `apify push`. ## Input schema: defaults and validation If the Actor declares an input schema - the `input` field of `.actor/actor.json`, or `.actor/INPUT_SCHEMA.json`, or `INPUT_SCHEMA.json` at its root - the runtime uses it the way the platform does: ```sh apify call # no input: the run gets the schema's defaults apify call --input '{"maxPages":0}' # rejected before anything starts, if the schema forbids it ``` - Every field left out of `--input` is filled from the schema's `default`, and that filled-in input is what the run reads back as `INPUT`. - An input the schema rejects fails the call with the same message the real API gives (`Input is not valid: Field input.maxPages must be >= 1`); no run is created and no container starts. - The schema comes from the **build**, so a locally edited `INPUT_SCHEMA.json` needs an `apify push` to take effect - unlike a source edit under a registered dev folder. - A schema the Apify input-schema meta-schema rejects fails the **build**, with the defect in the build log, instead of being silently ignored. - Secret input fields (`isSecret`) are stored encrypted in the run's `INPUT`, as on the platform; the SDK's `Actor.getInput()` returns them decrypted, while reading the `INPUT` record directly shows `ENCRYPTED_VALUE:...`. - One local difference: Apify Proxy groups are not checked (any `apifyProxyGroups` selection is accepted). - Apify Proxy is **real**: when a proxy password is known (the runtime's `APIFY_PROXY_PASSWORD`, or the real account behind the token), runs get it, so the Actor can use the real proxy, billed to that Apify account; the run log carries a warning saying so. Unchecking "Use Apify Proxy" on the console's Settings page (on by default) sets `APIFY_PROXY_PASSWORD` to an empty string instead. ## Saved inputs (tasks) Tasks work as on the platform, for your own Actors: save an input and run options once, then run from them. ```sh apify api POST v2/actor-tasks --body '{"actId": "my-actor", "name": "small-crawl", "input": {"maxPages": 3}, "options": {"memoryMbytes": 2048}}' apify api POST v2/actor-tasks/~small-crawl/runs # runs my-actor with the saved input apify api POST v2/actor-tasks/~small-crawl/runs --body '{"maxPages": 5}' # merged over the saved input apify api GET v2/actor-tasks/~small-crawl/runs/last/dataset/items ``` - The SDKs' and `apify-client`'s task calls (`client.task(id).call()`, `getInput()`, `updateInput()`) work against it, `run-sync` included. A task run's `actorTaskId`, and `ACTOR_TASK_ID` in its container, name the task. - The saved input is validated against the input schema of the build the task runs, as on the platform. - A task of a Standby Actor has its own `standbyUrl` (see "Run an Actor server"). - The console's Tasks page shows each task's input, run options, standby URL and runs. - Tasks cannot be published here. ## Webhooks Webhooks work as on the platform, including `apify-client`'s webhook calls and `Actor.addWebhook()`. ```sh apify api POST v2/webhooks --body '{"eventTypes": ["ACTOR.RUN.SUCCEEDED"], "condition": {"actorId": "<actorId>"}, "requestUrl": "https://example.com/hook"}' apify api GET v2/webhooks/<webhookId>/dispatches # each delivery, with every attempt's status and answer ``` - Requests are sent from the runtime, where `localhost` is the runtime itself: reach a server on your machine through `host.docker.internal` (Docker Desktop) or `host.containers.internal` (Podman). - A URL naming the runtime's own API (`http://localhost:3333/v2/...`) is authenticated as you unless it has a `token`, so a webhook can start another local Actor. - Only HTTP request webhooks: Slack, email, Drive and GitHub actions are refused. ## Schedules Schedules work as on the platform, including `apify-client`'s schedule calls, while the runtime is running. ```sh apify api POST v2/schedules --body '{"cronExpression": "0 2 * * *", "isEnabled": true, "actions": [{"type": "RUN_ACTOR", "actorId": "<actorId>"}]}' apify api POST v2/schedules/<scheduleId>/invoke # run its actions now apify api GET v2/schedules/<scheduleId>/log ``` - A run time that passed while the runtime was down is skipped, not made up. - No email is sent when a run fails to start; the schedule's log says so instead. ## Run memory With no `-m`/`--memory`, a run gets `defaultMemoryMbytes` from `.actor/actor.json` - a number or a memory expression over the input, evaluated as the platform does - or the Actor's default run memory. Any memory, explicit or not, is then clamped to the Actor's `minMemoryMbytes` / `maxMemoryMbytes`, and the run log says so. An expression that fails falls back to the default run memory with a warning in the run log. The fields come from the build, so changing them needs an `apify push`. ## Environment variables and secrets Declared in `.actor/actor.json` and sent by `apify push`, as on the platform: ```sh apify secrets add myApiKey 'sk-...' # stored by the CLI on your machine # .actor/actor.json: "environmentVariables": { "MODE": "dev", "API_KEY": "@myApiKey" } apify push ``` - Every run gets the version's env vars; the runtime's own `APIFY_*`/`ACTOR_*` vars always win over one with the same name. - A secret's value is never returned by the API (a short `valueHash` changes when it is set again). It is stored encrypted with a key pair of its own Actor, but that key pair sits in the same data directory: a simplified scheme that stops accidental reads, not someone with access to that directory. - As on the platform, the run log shows `*********` for the exact value of a secret env var or of the run's `APIFY_TOKEN`. Anything else is printed as is, secret input fields included, so do not log secrets in any other form either. - Every run gets its Actor's private key as `APIFY_INPUT_SECRETS_PRIVATE_KEY_FILE` and `APIFY_INPUT_SECRETS_PRIVATE_KEY_PASSPHRASE`, which the SDKs' `getInput()` uses to decrypt secret input fields. - They reach the **build** only with `applyEnvVarsToBuild` on, as Docker build arguments, secrets included; the Dockerfile reads each with `ARG NAME`: `apify api PUT v2/actors/<actorId>/versions/0.0 --body '{"applyEnvVarsToBuild": true}'`. A later `apify push` keeps the setting; with no local edits since, it needs `--force`. - `v2/actors/<actorId>/versions/<version>/env-vars` manages them one at a time. A change applies to the next run, or to the next build for build arguments. ## Iterate without rebuilding (dev folder) After that first `apify push`, the runtime registers the pushed directory as the Actor's **dev folder**. The live dev folder is **off by default**; once it is enabled for the Actor, the runtime bind-mounts that folder into every later run (the build log's `Registered live dev folder` line confirms it). Edit locally, recompile locally (`tsc`, or the language equivalent), and `apify call` again - no `apify push`, no rebuild. It pays off most for plain Python or JavaScript Actors whose source runs as-is, with no compile or other build step. For Actors that compile, generate files at build time, or use a non-standard image layout, the mounted folder has to match what the image expects at run time, which is easy to get wrong - there, `apify push` after each change is often the more reliable loop. ```sh apify api POST /actor-runtime/live-dev-folder/<actorId> --body '{"enabled": true}' # once per Actor # edit src/main.ts npm run build apify call --input '{"maxPages":3}' # picks up the new dist/ ``` - A local recompile is picked up by the **next** run's container start, not by a run already going. - `node_modules` still comes from the built image, so a new dependency in `package.json` does need a real `apify push`. Only source edits skip the rebuild. - Disable it with `--body '{"enabled": false}'`; the Actor's console page has the same checkbox. While it is off, each run's log says the registered folder is not mounted. - `apify call --no-dev-folder` runs once from the built image alone, leaving the registration alone. - Register a different folder by hand with `apify api POST /actor-runtime/dev-folder/<actorId> --body '"/abs/path/to/src"'`; clear it with `--body '""'`. The path must be absolute and must exist on the machine the container engine runs on - the runtime rejects the call otherwise, and re-checks at every run start. ## Debug an Actor with a real IDE debugger Turn debug mode on for the Actor once; every later `apify call` then starts paused, waiting for a debugger, with the attach address in the run's log. ```sh apify api POST /actor-runtime/debug/<actorId> --body '{"enabled": true}' apify api POST /actor-runtime/debug/<actorId> --body '{"enabled": false}' # back to normal ``` Add `"language": "node"` or `"python"` when auto-detection cannot classify the image, and `"port": <n>` to override the default (`9229` Node, `5678` Python). Attach VS Code's "Attach" (Node) or PyCharm's "Attach to DAP" / VS Code's "Python: Remote Attach" (Python) to `127.0.0.1:<port>`. Two things that bite: - **An Actor pushed with no `Dockerfile` of its own needs one.** The injected default inherits `CMD ["npm", "start", "--silent"]`, which debug mode refuses, because it would attach to npm rather than the Actor. Give the Actor a `Dockerfile` whose `CMD` invokes the interpreter directly, e.g. `CMD ["node", "dist/main.js"]`. - The run's `--timeout` is not extended while you attach. Pass a larger one for a slow attach. ## Watch a Playwright/Puppeteer browser ```sh apify api POST /actor-runtime/browser-view/<actorId> --body '{"enabled": true}' ``` Every later run prints a viewer URL, `http://localhost:3000/runs/<runId>/browser` - a live view of the display the Actor's browser draws on. Add `"interactive": true` to also send mouse and keyboard input. The browser must run **headful** to show anything; Apify's templates default to headless, which shows as a black display. Disable with `{"enabled": false}`. ## See the web page an Actor serves (live view) A run's web server (`ACTOR_WEB_SERVER_PORT`) and `containerUrl` work as on the platform; the run log and the run's console page name the URL. Local differences: - The URL is `http://<runid>.runs.localhost:3333` (the run owns `/`), or `http://localhost:3333/actor-runtime/container/<runId>` for clients that do not resolve `*.localhost`; from another Actor's container, `http://apify-api:3333/actor-runtime/container/<runId>`. - Under rootless Podman 3.x the runtime container needs `--network slirp4netns:allow_host_loopback=true` (`apify runtime start` adds it); without it only a standby run's server is reachable, and the run log says so. - `samples/actor_ts` serves a progress page to try it with. ## Test pay-per-event pricing and see what a run costs Give the Actor a pricing exactly the way the platform stores it - the `pricingInfos` array on the Actor object, which only an update takes, never the call that creates the Actor. Only `FREE` and `PAY_PER_EVENT` are emulated: ```sh apify api PUT /v2/actors/<actorId> --body '{"pricingInfos":[{"pricingModel":"PAY_PER_EVENT", "pricingPerEvent":{"actorChargeEvents":{ "page-scraped":{"eventTitle":"Page scraped","eventPriceUsd":0.002}, "apify-actor-start":{"eventTitle":"Actor start","eventPriceUsd":0.005,"isOneTimeEvent":true}}}}]}' ``` Both bundled samples already charge `page-scraped` per page and `crawl-finished` once at the end, and carry the matching pricing in `pricing.json` - including both synthetic events, so one run shows all four being charged: `apify api PUT /v2/actors/<actorId> --body "$(cat samples/actor_ts/pricing.json)"` prices one in a single call. From then on every run of the Actor is a pay-per-event run: the SDKs read `pricingInfo` and `chargedEventCounts` off the run object exactly as on the platform, and `Actor.charge()` (or `apify actor charge`) posts to `POST /v2/actor-runs/<runId>/charge`. Do not set `ACTOR_TEST_PAY_PER_EVENT` - the runtime already looks like the platform to the SDK, which refuses that variable together with `APIFY_IS_AT_HOME`. The synthetic events work too: `apify-actor-start` is charged at run start (once per GB of memory) and `apify-default-dataset-item` once per item pushed to the run's default dataset, when the pricing defines them. The start pre-charge scales with the run's memory and its count is written to the run log, so `-m 8192` charges eight start events, not one; a memory the platform would refuse (anything but a power of two between 128 MB and 32 GB) is warned about in the log and then used anyway. Tiered event prices resolve to the `BRONZE` (Starter) tier. The array is append-only, as on the platform: send the entries the Actor already has, unchanged, plus at most one new one starting after all of them - so making the Actor free again means appending a `{"pricingModel":"FREE"}` entry, not sending `[]`. Re-sending a file that was already applied is refused for the same reason (`pricingInfos[0] differs ...`): the stored entries carry timestamps the file does not, so append its entry to what `GET /v2/actors/<actorId>` returns rather than sending the file again. The same form is on the Actor's console page.