ingestr

How the ingestr BigQuery destination works — the staging→swap write path, load methods (load_job/storage_write), where dedup happens, copy-job vs CTAS swap selection, partition/cluster change handling (rename-aside), and merge partition pruning. Use when changing anything in pkg/destination/bigquery/ or BigQuery-affecting behavior in the replace/merge strategies (pkg/strategy/replace.go, pkg/strategy/merge.go). If your change alters behavior described here, update this file in the same change.

Implement a new ingestr source (API/SaaS connector) from scratch against a vendor's HTTP API. Use when a task asks to add a new source/connector, wire it into the URI registry, design its tables and incremental behavior, add its rate limiting, or write its tests. Covers researching the vendor API, creating the source package, the post-implementation review checklist, and verifying the output against the live account.