| name | data-programs |
| description | Save a run-code fetch/join/aggregate script as a stored, refreshable data source any app's own charts/tables can render, instead of a hardcoded provider action or a per-view re-fetch. Use when an ad-hoc run-code or provider-api-request analysis should become a live, cached data source other users or panels can reuse. |
Data Programs
A data program is a named, stored, agent-authored JS script that fetches,
joins, filters, or aggregates data — provider APIs via providerFetch /
providerFetchAll / providerSearchAll, app data via appAction, or
Resources-backed workspace files — and emits a small { rows, schema }
result that gets cached in SQL. Any app can render that result through its
own table/chart components; nothing about the primitive is analytics-specific.
This ships automatically wherever run-code is available — there is nothing
to mount, wire, or configure per app. If your app already has run-code
enabled, save-data-program / preview-data-program / run-data-program /
list-data-programs / get-data-program / delete-data-program are already
registered actions.
Why this exists
Without it, "build a live view of X" tends to end in one of two bad places:
a hardcoded action per vendor/query shape (brittle, doesn't scale to every
customer's custom fields), or re-running the same fetch/join logic from
scratch on every page view (slow, and it re-spends the same provider quota
every time someone looks at a chart). A data program is the middle ground:
write the fetch/join/aggregate logic once as ordinary run-code, save it, and
let the cache + refresh policy handle "how often does this actually need to
re-run."
Authoring workflow
- Prototype the fetch/join/aggregate logic as a normal
run-code script and
confirm it returns the rows you expect.
- Change it to call
emit(rows, schema?) exactly once at the end instead of
returning or printing the result.
- Save it with
save-data-program({ name, title, description, code, defaultParams?, refreshMode?, refreshTtlMs?, background? }). This
dry-runs the code with defaultParams before persisting and rejects
the save with a structured error if it fails, so a broken program is never
stored. On success it returns { programId, rowCount, columns, sampleRows } — use that as proof the program produces the rows you
expect.
- Bind whatever UI needs the result. Your app's own read path calls
run-data-program({ programId, params?, forceRefresh?, includeRows? })
and gets back rows/schema/cache metadata to hand to your own table/chart
components — there is no dashboard-specific plumbing baked into the
primitive itself. (The analytics template's "program" dashboard panel
source is one such adoption, not a required pattern — see its
data-programs skill if you want a concrete UI wiring reference.)
- Iterate without persisting via
preview-data-program({ code, params? }) —
same dry-run path, no stored row.
The sandbox surface
Exactly the run-code globals (providerFetch, providerFetchAll,
providerSearchAll, appAction, webFetch, workspace* Resources helpers),
plus:
params — a frozen global object holding the params for this run.
Read-only; mutating it throws.emit(rows, schema?) — call exactly once, at the end. rows must be an
array of plain objects. schema, when provided, is
{ name: string, type: string }[]; when omitted it's inferred from the
first 50 rows (a column is "json" if rows disagree on primitive type,
otherwise "number" / "string" / "boolean"). console.log remains free
for debugging and never interferes with parsing the emitted result.
Caps enforced on every run:
| Limit | Value |
|---|
| Max emitted rows | 10,000 (excess dropped, truncated: true) |
| Max emitted result size | 4 MiB (rows dropped from the end to fit; a single oversized row is a hard failure) |
| Max active programs per app | 200 |
| Minimum refresh TTL | 60,000 ms |
| Run rows kept per (program, params) | 5 most recent |
Truncation is always explicit (truncated: true) — never a silent drop.
Caching and refresh model
Every (programId, paramsHash) pair has its own run history:
| Situation | Behavior |
|---|
Fresh cache (younger than refreshTtlMs, or refreshMode: "manual" with a prior success) | Returns cached rows instantly, cacheHit: true |
| Stale cache | Re-executes synchronously (25s budget for view-triggered reads, 120s for agent/manual calls), replacing the cache on success |
refreshMode: "manual" | Only refreshes on explicit forceRefresh: true |
| No cache yet | Executes synchronously like a normal first fetch |
background: true program with no fresh cache | Serves the last good run with stale: true and enqueues a durable background execution; a later call finalizes it once complete |
| Background program still running, no prior success | background_pending failure — surface an explicit "still computing" state, never a blank result |
| Execution fails | Structured { code, message } failure; a previous success is attached as lastGoodRun for stale-serve |
| Program archived (soft-deleted) | Explicit archived failure |
Use background: true only for scripts that routinely exceed the foreground
timeout (large multi-provider joins, deep pagination). Everything else should
stay foreground.
Security notes
- Credentials are always the calling viewer's, never the program author's.
providerFetch inside a program resolves auth using the caller's own
request context. A viewer without a configured provider key sees that
provider's normal auth error on that program's result — never another
user's data.
- Programs are shareable org-internal only — never public. Sharing a
program means teammates can view its cached result; because execution uses
the viewer's credentials, only bind to or view a program you trust.
- Raw provider tokens never reach the model or the browser. Only the
{ rows, schema } an emit() call produces ever leaves the sandbox.
- The mutating/executing actions (
save-data-program, preview-data-program,
run-data-program, delete-data-program) are not callable from a
sandboxed extension/iframe bridge — only from the agent or a server-side
read path.
Related skills
actions for run-code conventions and the sandbox globals this primitive
is built on.sharing for the org-internal, never-public sharing model programs use.security for credential-handling and access-scoping invariants.
