| name | connector-configuration |
| description | Connector-specific configuration patterns for ELT pipelines — QuickBooks OAuth + rate-limit handling, Stripe webhook + batch hybrid, Salesforce Bulk API 2.0, HubSpot API v3, GA4 BigQuery export, Shopify GraphQL Admin API. Used by `etl-pipeline-engineer` when configuring an Airbyte / Fivetran / n8n connector against a real source. |
Skill: connector-configuration
Invoked by: etl-pipeline-engineer (primary), connector-developer (custom-connector path).
When to invoke: configuring any pre-built ELT connector (QBO, Stripe, Salesforce, HubSpot, GA4, Shopify, HRIS); diagnosing a rate-limit-related failure; deciding on sync frequency / state-management posture.
Output: a working connector config + rate-limit-aware retry plan + state-management strategy + handoff documentation for engagement end.
The 6-source toolkit (per the consulting practice's most common ingestion targets)
QuickBooks Online
- Auth: OAuth 2.0 Authorization Code; 1-hour access tokens; refresh tokens 100-day rolling with notifications at 30 and 7 days
- Register the app: developer.intuit.com → My Apps → create an app, enable the Accounting scope, copy the client ID/secret, set the redirect URI to your ELT vendor's callback. Who can do it: any Intuit developer account creates the app; connecting production to a company needs that company's admin to authorize. Walkthrough:
../../knowledge/quickbooks-online-integration.md. [verify-at-build]
- Rate limits: 10 req/s per realm-ID + app; batch endpoint 120/min; concurrent ~10. HTTP 429 / 403 → honor
Retry-After
- Connector availability: Fivetran ✅, Airbyte ✅, Hevo ✅, Stitch ✅ (in maintenance)
- QBO vs QB Desktop: Desktop is materially different — usually requires QB Web Connector + CData / Transaction Pro. QB Desktop is v0.2.0+ scope for this plugin.
- 2026 watch: "Reconnect URL" mandatory field in Intuit developer portal (single-source claim; verify before implementing)
Stripe
- Auth: API key (restricted keys for production) — not an OAuth app. No developer-portal app registration: create a restricted key in the Stripe Dashboard → Developers → API keys (scoped read-only for ELT). Who can do it: a Stripe account member with Developer/Administrator access.
[verify-at-build]
- Connector availability: Fivetran ✅, Airbyte ✅, Hevo ✅, Estuary ✅
- Pattern: batch for history + webhooks for real-time — don't try to ELT a webhook-driven event source as if it's a CRUD database
- Common error: ELT vendors are batch-only; client expects real-time fraud signals — webhook handlers run separately
Salesforce
- Auth: Connected App + OAuth 2.0 JWT-bearer flow (server-side) or Web Server flow (interactive)
- Register the app: Setup → App Manager → New Connected App → enable OAuth, add the ELT vendor's callback URL, select scopes (
api, refresh_token), then copy the Consumer Key/Secret. Who can do it: a System Administrator (or "Customize Application" + "Manage Connected Apps" perms). Walkthrough: ../../knowledge/salesforce-integration.md. [verify-at-build]
- Bulk API 2.0 ceilings: 150M records/day; 15k batch submissions/24hr; 10k records/batch; 10MB payload (per developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/bulk_common_limits.htm)
- Connector availability: Universal — all major ELT vendors. Bulk API 2.0 is the standard backbone.
- Common error: custom SObject field selection — limit query result columns to avoid 10MB payload pings
HubSpot
- Register the app: single-org ELT → Settings → Integrations → Private Apps → Create (copy the access token, select CRM read scopes); distributed/marketplace → an app in developers.hubspot.com (OAuth client ID/secret + redirect URI). Who can do it: a HubSpot super admin for a private app; a developer account for a marketplace app (install still needs the target portal's super admin). Walkthrough:
../../knowledge/hubspot-integration.md. [verify-at-build]
- Rate limits: OAuth marketplace apps 110/10s per installed account; private apps 100/10s (Free/Starter) → 190/10s (Pro/Enterprise); CRM Search API capped at 4/sec; OAuth via developer accounts up to 1M/day (per developers.hubspot.com/docs/developer-tooling/platform/usage-guidelines)
- Connector availability: Fivetran ✅, Airbyte ✅, Hevo ✅
- Common error: sync designed against private-app rate limits then deployed via OAuth marketplace (or vice versa) — different ceilings
Google Analytics 4
- Native BigQuery export: FREE, daily + streaming/intraday — this is the recommended path when destination is BigQuery. No app registration — configure in GA4 → Admin → Product Links → BigQuery Links (needs GA4 property Editor + the BigQuery project owner).
- Register the app (Data API path only): when the destination isn't BigQuery, the ELT vendor authenticates via a Google Cloud service-account JSON key (Google Cloud Console → IAM & Admin → Service Accounts), or an OAuth client (same flow as Google SSO). Who can do it: Google Cloud project Owner/Editor; grant the service account GA4 property access. Walkthroughs:
../../knowledge/ga4-integration.md + ../../../auth-identity/skills/google-sso-setup/SKILL.md. [verify-at-build]
- UA is gone. GA4 only. Historical UA windows have closed for most properties.
- ELT vendor connectors: useful when destination is not BigQuery, or when blending with other sources
Shopify
- Auth: OAuth 2.0
- Register the app: single-store ELT → Shopify admin → Settings → Apps and sales channels → Develop apps → Create an app (Admin API access token + scopes like
read_orders, read_products); distribution → an app in the Shopify Partner Dashboard (OAuth client ID/secret + redirect URI). Who can do it: the store owner (or staff with the Apps and channels → Develop apps permission) for a custom app; Partner-account access for a Partner app. Walkthrough: ../../knowledge/shopify-integration.md. [verify-at-build]
- API contract: REST Admin API is legacy as of Oct 1, 2024. GraphQL Admin API required for new apps since April 1, 2025. All new connector work uses GraphQL.
- Webhooks: via GraphQL
webhookSubscriptionCreate; HTTPS / Google Pub/Sub / AWS EventBridge destinations
- Pattern: ELT for orders / customers / products (historical) + webhooks for inventory + order events (real-time)
HRIS (Workday, BambooHR, ADP)
- Workday HCM / Adaptive / Financial / Strategic Sourcing / RaaS: Fivetran ✅; setup requires Workday Integration System User with appropriate domain permissions
- BambooHR: Airbyte ✅ (both source and destination)
- ADP: thinner native ELT coverage; usually via Flexspring (API-to-API connector) or Merge.dev unified HRIS API
Sync-frequency decision
| Frequency | When to pick | Cost implication |
|---|
| Real-time (webhooks) | Operational dashboards, fraud, inventory | Webhook handler runs separately from ELT |
| Hourly | Recent-activity reporting | Mid-tier per-MAR; high credit usage |
| Daily | Most analytics dashboards | Industry-standard; cheapest defensible cadence |
| Weekly | Trend / retention / cohort | Cheapest; rare for SMB dashboards |
Heuristic: the dashboard's refresh expectation is the cadence ceiling. Daily ELT for a quarterly QBR is wasted MAR.
The Fivetran 2026 cost-predictability warning
As of Jan 1, 2026, Fivetran counts deletes toward paid MAR (previously only inserts/updates). For change-heavy sources (Salesforce, HubSpot deal updates, ticketing systems), this is a material cost increase per fivetran.com/docs/usage-based-pricing/pricing-updates/2026-pricing-updates. Flag this when proposing Fivetran for fixed-fee consulting engagements — invoice surprises break the fixed-fee model.
Workflow vs ELT decision (when n8n / Zapier vs Airbyte)
- Workflow (n8n / Zapier / Make / Activepieces): SaaS-to-SaaS, event-driven, low-volume (thousands of events/day), destination is another SaaS app
- ELT (Fivetran / Airbyte): destination is a warehouse, schema-managed, hundreds-of-thousands to millions of rows
Anti-patterns this skill flags
- Rate-limit-naive code that doesn't honor
Retry-After
- ELT pipeline scheduled more frequently than the dashboard needs
- Salesforce / HubSpot pipeline without naming the daily-API-cap math
- Fivetran proposed for change-heavy source without flagging the 2026 deletes-count change
- Stripe batch-only when client expects real-time signals (need webhooks)
- GA4 connector spend when native BigQuery export is free
- Workflow tool (n8n / Zapier) proposed for warehouse-bound ELT at million-row volume
- Custom integration when Airbyte ships a connector (re-inventing maintenance burden)
- ELT pipeline with no documented data-handoff plan
- Storing OAuth refresh tokens in plain text in a config repo (use a secret store)
References
