agent-skills

Create a HubSpot Real-Time capture using flowctl to stream CRM data into Estuary collections. Use when setting up a HubSpot source for contacts, companies, deals, tickets, or other HubSpot resources. Use when user says "capture HubSpot", "stream from HubSpot", "HubSpot CDC", or "connect HubSpot to Estuary".

Create a ClickHouse materialization to stream Estuary collections into ClickHouse tables. Use when setting up ClickHouse as a destination for captured data. Use when user says "send to ClickHouse", "materialize to ClickHouse", "ClickHouse destination", or "load into ClickHouse".

Override default column types in materializations using castToString or custom DDL. Use when column type needs different precision (e.g., NUMERIC(22,6)), VARCHAR length is wrong, need JSONB instead of JSON, integers lose precision, or need to force a string representation. Use when user says "change column type", "wrong precision", "VARCHAR too short", "integer truncated", "cast to string", or "custom DDL".

Add or define specific fields in a collection's readSchema. Use when fields are missing from materialization field selection, pruned by complexity limit, need type refinement, need to be protected from future inference changes, or need to be required for downstream consumers. Use when user says "missing fields", "pruned fields", "field not showing up", "complexity limit", "define field in read schema", "restrict field type", or "protect field from inference".

Add redaction annotations to sensitive fields in schemas to block or hash PII. Use when configuring GDPR/HIPAA compliance, protecting credentials, or blocking internal fields from materializations. Use when user says "protect PII", "hide sensitive data", "hash field", "redact field", "GDPR compliance", "don't capture field", "remove field from Estuary", or "prevent Estuary from capturing field".

Control which fields are included or excluded in materializations using depth modes and field overrides. Use when reducing table width, including nested fields, excluding PII, or optimizing performance. Use when user says "exclude fields", "include nested field", "too many columns", "select specific fields", or "field depth".

Handle NULL values and set default values for missing fields in collections and materializations. Use when fields need defaults, nullable primary key errors occur, or source data has gaps. Use when user says "NULL values", "default value", "missing field", "nullable primary key error", or "fill in missing data".

Add projections to collections to rename fields or map nested JSON paths to flat column names. Use when renaming fields for destination conventions, flattening nested paths to columns, or creating field aliases. Use when user says "rename field", "flatten nested JSON", "map JSON path to column", "field alias", or "projection".

Create an Estuary derivation that continuously aggregates metrics (daily totals, running counts, min/max, lifetime revenue) using reduction annotations. Use for real-time dashboards, continuous materialised views, and replacing batch ETL aggregations with streaming. Derivations add complexity and cost — confirm the user wants a derivation before reaching for this. Use when user says "aggregation derivation", "derivation to sum daily totals", "running-count derivation", "transformation for min/max per group", "derive lifetime metrics", "aggregate-by-date derivation", or "real-time-dashboard derivation".

Foundation for writing Estuary derivations — what they are, language choice (SQL/TypeScript/Python), project layout, flowctl workflow, stateless vs stateful shuffling, and universal gotchas. Read this before any other derivation-* skill. Use when user says "write a derivation", "how do derivations work", "create a derivation", "derivation setup", or before starting any transformation work.

Create an Estuary derivation for stateless filtering, field selection, and per-document field transformation. Use for WHERE clauses, projecting fields, computing new fields from existing ones, or data cleansing. Derivations add complexity and cost — confirm the user wants a derivation before reaching for this (renaming alone is usually a projection, not a derivation). Use when user says "filter-transform derivation", "derivation to filter by status", "transformation with WHERE clause", "derive computed fields", "cleanse data with a derivation", or "stateless derivation".

Create an Estuary derivation that flattens a nested array into one output document per array element. Use for normalising nested JSON, exploding line items, unpacking tags, or producing one-row-per-item tables from arrays. Derivations add complexity and cost — confirm the user wants a derivation before reaching for this (some destinations can `UNNEST` server-side without one). Use when user says "flatten-array derivation", "derivation to explode an array", "transformation to unnest", "derive one row per item", or "flatten nested data with a derivation".

Create an Estuary derivation that joins two or more collections on a shared key, merging their fields into a single enriched collection. Use for data enrichment, denormalisation, building wide tables for BI, and real-time CDC joins. Derivations add complexity and cost — confirm the user wants a derivation before reaching for this (a destination-side SQL join or BI tool view is often simpler). Use when user says "join-collections derivation", "derivation to join two collections", "transformation to enrich with customer data", "derive a wide table", "lookup-join derivation", or "build a join derivation".

Write Estuary derivations in Python — async-generator transforms, Pydantic-typed source/output, optional persistent state, and dependency installation via uv. Use for ML/embeddings, async API calls, or any transform where SQL and TypeScript are awkward. Derivations add complexity and cost (and Python derivations require a private/BYOC data plane) — confirm the user wants a Python derivation before reaching for this. Use when user says "python derivation", "write a derivation in python", "embed with a python derivation", "ML derivation", "call an API from a derivation", or anything mentioning model2vec / huggingface / pydantic in a derivation context.

Create an Estuary derivation with custom SQLite state (internal tables) to make per-document decisions based on accumulated history — account balances, inventory levels, approval workflows, deduplication. Derivations add complexity and cost — confirm the user wants a derivation before reaching for this. Use when user says "stateful derivation", "derivation to track balance", "transformation to approve based on state", "derive an approval workflow", "deduplication derivation", or "build a stateful derivation".

Create an Estuary derivation that maintains a sliding time window of events (e.g. "transfers in the last 24h", "failed logins in the last 5 minutes") using internal SQLite state and readDelay for time-based expiration. Derivations add complexity and cost — confirm the user wants a derivation before reaching for this. Use when user says "windowing derivation", "derivation for last-24h events", "rolling-window derivation", "sliding-window transformation", "fraud-detection derivation", "rate-limit derivation", or "time-window derivation".

Create an Alpaca capture using flowctl to stream stock trade data into Estuary collections. Use when setting up an Alpaca Market Data source for historical and real-time stock trades. Use when user says "capture Alpaca", "stream stock trades", "Alpaca market data", or "connect Alpaca to Estuary".

Run flowctl catalog status to check the control-plane state of an Estuary resource, or list catalog specs to see which connectors exist and which are active. Use when checking whether a task is running, disabled, or failing, interpreting status output, or enumerating connectors. Use when user says "check task status", "is my capture running", "catalog status", "flowctl catalog status", "what's wrong with my capture", "task failing", "check catalog status", "collection status", "derivation status", "why is my task disabled", "list my connectors", "show me my connectors", "list captures", "list materializations", "what connectors do I have", or "show me all my active connectors".

Create a capture for ANY Estuary connector using dynamic schema discovery. Use when the user wants to capture from a source that doesn't have a dedicated skill (e.g., Kafka, Salesforce, HubSpot, Stripe, S3, GCS, Kinesis, or any of the 148+ connectors). Use when user says "capture from <source>", "stream from <source>", "connect <source> to Estuary", or "set up <source> capture" and no connector-specific skill exists.

Create an HTTP Ingest (webhook) capture to receive data via HTTP POST requests. Use when setting up webhooks from GitHub, Shopify, Stripe, or any JSON source. Use when user says "set up webhook capture", "HTTP ingest to Estuary", "receive POST requests to Estuary", or "webhook capture".

Create a MongoDB CDC capture using flowctl. Use when setting up real-time streaming from MongoDB Atlas, DocumentDB, or self-hosted MongoDB. Use when user says "capture MongoDB", "stream from Mongo", "MongoDB CDC", or "connect MongoDB to Estuary".

Create a MySQL CDC capture using flowctl with binlog replication. Use when setting up streaming from MySQL, Amazon RDS MySQL, or Aurora MySQL. Use when user says "capture MySQL", "stream from MySQL", "MySQL CDC", "binlog replication", or "connect MySQL to Estuary".

Create a new PostgreSQL CDC capture from scratch using flowctl. Covers all Postgres variants (vanilla, Aurora, RDS, Cloud SQL, Supabase, Neon). Use when user says "capture Postgres", "stream from Postgres", "Postgres CDC", "set up replication slot", or "connect Postgres to Estuary".

Create a SQL Server CDC capture using flowctl. Use when setting up real-time streaming from SQL Server, Azure SQL, Amazon RDS SQL Server, or Google Cloud SQL. Use when user says "capture SQL Server", "stream from MSSQL", "SQL Server CDC", or "connect SQL Server to Estuary".

View publication history for catalog resources — who changed what and when. Use when investigating timeline of changes, finding who modified a connector, or correlating changes across capture/collection/materialization chains. Use when user says "what changed", "who changed", "publication history", "catalog history", "when was it published", "timeline of changes", or "what happened to my connector".

Pause and restart Estuary connectors by disabling and re-enabling shards via flowctl. Use when temporarily stopping a connector, performing source/destination maintenance, or forcing a fresh restart to clear transient errors or load updated configs immediately. Use when user says "pause connector", "stop connector", "restart connector", "disable connector", "enable connector", "re-enable task", "disable shards", "stop my capture", "stop my materialization", "disable my materialization", "force restart connector", "connector maintenance", "temporarily stop processing", "shards disable true", "enable shards", "resume connector", or "unpause connector".

Install and authenticate the flowctl CLI for Estuary. Use when setting up flowctl for the first time, troubleshooting authentication, upgrading to the latest version, or configuring programmatic access for CI/CD. Use when user says "install flowctl", "set up flowctl", "flowctl auth", "how to authenticate", "token expired", "not authenticated", "FLOW_AUTH_TOKEN", "CI/CD setup", "programmatic access", "upgrade flowctl", "flowctl not found", "command not found flowctl", "Exec format error", or "flowctl on Windows".

Search and analyze Estuary task logs with jq filtering. Use when investigating task failures, debugging connector errors, analyzing behavior patterns, or streaming live logs. Use when user says "check logs", "show me logs", "task failing", "why is my capture failing", "why is my materialization failing", "error messages", "why did it fail", "search task logs", "shard failed", "follow logs", "increase log level", "debug logging", "is this log message normal", "what does this message mean", "connection timeout", or "i/o timeout".

Diagnose and fix SSH tunnel connection issues for Estuary captures and materializations. Use when seeing tunnel failures, connection timeouts, or permission denied errors. Use when user says "SSH tunnel failed", "connection timeout via tunnel", "permission denied SSH", "tunnel not working", "network-tunnel failed", "no pubkey loaded", "bastion host not connecting", or "RSA key format".

Run a comprehensive health check on a single Estuary task by combining catalog status, data flow stats, recent errors, and publication history. Use when diagnosing a task that seems broken, slow, or stuck. Use when user says "is my task healthy", "something seems wrong", "check my capture", "check my materialization", "why is data not flowing", "task health check", "diagnose my task", or "what's wrong with my pipeline".

Check task processing statistics to see how much data and how many docs are moving. Use when checking throughput, diagnosing a stalled task, verifying data movement, or reporting on task activity. Use when user says "is data flowing", "how much data", "check throughput", "how many documents", "task not processing", "data not arriving", "stats", "bytes processed", "stalled capture", "no data in destination", "data volume", "task report", "usage report", or "how much data did we process".

Create a BigQuery materialization to stream Estuary collections into BigQuery tables. Use when setting up BigQuery as a destination for captured data. Use when user says "send to BigQuery", "load into BigQuery", "materialize to BigQuery", or "BigQuery destination".

Create a Databricks materialization to stream Estuary collections into Databricks tables via Unity Catalog. Use when setting up Databricks as a destination for captured data. Use when user says "send to Databricks", "materialize to Databricks", "Databricks destination", or "load into Databricks".

Create a materialization for any Estuary destination connector. Use this generic skill when no dedicated skill exists for the target system. Use when user says "materialize to", "send data to", "destination connector", or "set up materialization".

Create a PostgreSQL materialization to stream Estuary collections into PostgreSQL tables. Use when setting up PostgreSQL as a destination for captured data. Use when user says "send to Postgres", "materialize to PostgreSQL", "Postgres destination", or "load into Postgres".

Create a Redshift materialization to stream Estuary collections into Amazon Redshift tables. Use when setting up Redshift as a destination for captured data. Use when user says "send to Redshift", "materialize to Redshift", "Redshift destination", or "load into Redshift".

Create a Snowflake materialization to stream Estuary collections into Snowflake tables. Use when setting up Snowflake as a destination for captured data. Use when user says "send to Snowflake", "materialize to Snowflake", "Snowflake destination", or "load into Snowflake".