bulk-operations

name	bulk-operations
description	Foundation patterns for the `hubspot` CLI — JSONL piping, batch read, pagination, dry-run/digest/confirm for destructive ops, and `hubspot history` for recovery. Every other skill builds on this one.
triggers	["bulk update","bulk create","bulk delete","process in bulk","JSONL pipe","pagination","dry-run","history","undo"]

Resources

File	When to use
resources/json-patterns.md	Reshape patterns for turning a read into an update payload, a search into a delete list, a CSV into an upsert stream.

Source of truth

hubspot <command> --help is authoritative. If anything in this file contradicts --help, trust --help and tell the user. Run hubspot objects types once at the start of a session to see what object types exist in this portal (standard + custom).

Output shape

Every read command (list, search, get) emits JSONL — one JSON object per line:

{"id":"123","properties":{"email":"[e-mail]","firstname":"Jane"},"createdAt":"...","updatedAt":"...","archived":false,"url":"..."}

--properties email,firstname limits which fields the server returns under .properties. Downstream jq should use .properties.email, not .prop_email.

Write commands (create, update, upsert, delete, merge, associations create) accept JSONL on stdin and emit JSONL — one result per input line: {"id":"123","ok":true,"data":{...}} or {"id":"123","ok":false,"error":{"status":...,"message":"..."}}. Order of results matches input order.

Read in batch — never one-by-one

The CLI accepts multiple IDs natively. Never pipe IDs into xargs -I{} hubspot objects get ... — that spawns one CLI process per record.

# Positional args (small, known list)
hubspot objects get --type contacts 12345 67890 23456 --properties email,firstname

# Stdin from another command — one CLI call total
hubspot associations list --from companies:67890 --to contacts \
| jq -c '{id}' \
| hubspot objects get --type contacts --properties email,firstname,jobtitle

# Bare IDs on stdin also work
printf '12345\n67890\n23456\n' | hubspot objects get --type contacts --properties email

A single hubspot objects get reads up to ~100 IDs per call via the batch endpoint. For more, page in chunks of 100.

Bulk flow: paginate first, then reshape, then write

When operating on all records of a type (or all matches of a filter), always start with pagination-loop.sh — never run a bare list or search to "check how many there are." A bare call returns at most 100 records and you will have to re-fetch them anyway.

The canonical bulk pattern is:

1. Paginate all records to a JSONL file
2. Reshape with jq into the write payload
3. Pipe to the write command (update, delete, etc.) with --dry-run first

Pagination

list and search return at most 100 records per call. Use resources/pagination-loop.sh to collect all pages into a single JSONL file:

bash resources/pagination-loop.sh <object_type> <output_file> [properties] [extra_flags...]

Examples:

# All contacts with specific properties
bash resources/pagination-loop.sh contacts /tmp/contacts.jsonl email,firstname,lastname

# Search with a filter (passes extra flags through to the CLI)
bash resources/pagination-loop.sh contacts /tmp/leads.jsonl email,firstname '--filter' 'lifecyclestage=lead'

# All deals, default properties
bash resources/pagination-loop.sh deals /tmp/deals.jsonl

The script pages through --after cursors automatically, prints progress to stderr, and writes JSONL to the output file. Run it as a single foreground command — do not background it or reconstruct the loop inline.

Write in batch — always pipe

Write commands accept JSONL on stdin. The transformation between a read shape and a write shape is a jq reshape:

Write command	Required per-line shape
objects create	{"properties":{"field":"value"}}
objects update	{"id":"123","properties":{"field":"value"}}
objects upsert	{"idProperty":"email","id":"[e-mail]","properties":{...}} (or use --id-property email once)
objects delete	{"id":"123"}
objects merge	{"primary":"123","secondary":"456"}
associations create	{"from":"contacts:123","to":"companies:456"}

Use plural object names in from/to (contacts:, not contact:).

Safe destructive workflow

Every destructive op (delete, merge, bulk update) supports --dry-run. The gating depends on row count:

≤100 rows — dry-run emits one preview line per record:

{"ok":true,"dry_run":true,"executed":false,"mutation_kind":"RecordMutation","command":"objects delete contacts","target":{"kind":"contacts_record","id":"123","name":"123"}}

Re-run without --dry-run to execute.

>100 rows — dry-run emits a single BulkData line with a digest and an apply_command_hint:

{"ok":true,"dry_run":true,"executed":false,"mutation_kind":"BulkData","portal":"123456","target":{"name":"202 records"},"impact":{"records_affected":202,"reversible":false},"digest":"blast-29cfdd48b583","expires_in_seconds":300,"apply_command_hint":"hubspot objects delete contacts --digest blast-29cfdd48b583 --confirm '202'"}

You must re-run with --digest <hash> --confirm <value> within 5 minutes. The confirm value is the record count (deletes) or the secondary ID (merge). Read it off apply_command_hint.

Three-step pattern:

# 1. Preview
hubspot objects search --type contacts --filter "lifecyclestage=subscriber" \
| jq -c '{id}' \
| hubspot objects delete --type contacts --dry-run \
| tee /tmp/preview.jsonl

# 2. Lift the digest + confirm value (only present for >100 rows)
digest=$(jq -r 'select(.mutation_kind=="BulkData") | .digest' /tmp/preview.jsonl)
confirm=$(jq -r 'select(.mutation_kind=="BulkData") | .impact.records_affected' /tmp/preview.jsonl)

# 3. Execute — re-pipe the SAME inputs
hubspot objects search --type contacts --filter "lifecyclestage=subscriber" \
| jq -c '{id}' \
| hubspot objects delete --type contacts --digest "$digest" --confirm "$confirm"

Recovery via hubspot history

Every destructive op (and its dry-run) is logged locally. Check what happened in the last hour and what's reversible:

hubspot history --since 1h --format table
hubspot history --since 24h --kind BulkData       # only bulk ops
hubspot history --since 7d --kind MetadataDestroy # schema deletes

history does not currently restore records — it's an audit log. If you deleted something by mistake, capture the history line and tell the user to restore via the UI.

Upsert beats search-then-create

For "create if missing, update if present" (the enrichment pattern), use upsert — one CLI call per record, no race condition:

cat external.jsonl \
| jq -c '{idProperty:"email", id:.email, properties:{firstname:.first, lastname:.last, company:.company}}' \
| hubspot objects upsert --type contacts --dry-run

# Or set idProperty once:
cat external.jsonl \
| jq -c '{id:.email, properties:{firstname:.first}}' \
| hubspot objects upsert --type contacts --id-property email

Rate-limit hygiene

There is no true batch endpoint behind update/delete/upsert — the CLI issues one API call per stdin line. Test with head -n 50 before piping a 50k-row file. If the API starts 429ing, the per-line output will show {"ok":false,"error":{"status":429,...}} — split your input file and retry the failed lines.

Common reshapes

See resources/json-patterns.md for the full set. The two you need 90% of the time:

# Read → update payload
hubspot objects search --type contacts --filter "industry=Tech" \
| jq -c '{id, properties:{lifecyclestage:"marketingqualifiedlead"}}' \
| hubspot objects update --type contacts

# Search → delete list
hubspot objects search --type contacts --filter "!email" \
| jq -c '{id}' \
| hubspot objects delete --type contacts --dry-run

Known constraints

• Some destructive operations may be blocked under user-OAuth (browser login); set HUBSPOT_ACCESS_TOKEN (private app token) when running deletes if the CLI returns a permission error.
• hubspot owners list returns CRM users; there is no teams object. For team-level operations, group by hubspot_owner_id client-side.
• No Lists API, no sequences/cadences API in the current CLI surface.