agentic-data-engineering

Create a dlt REST API pipeline to run as a Tower app. Use for the rest_api core source, or any generic REST/HTTP API source. Not for sql_database or filesystem sources.

Explore and profile loaded data from a data consumer perspective. Scores 5 dimensions (completeness, freshness, consistency, queryability, discoverability). Absorbs validate-data. Modes — PROFILE (automated after load), VALIDATE (cross-reference against source), EXPLORE (interactive queries). Routes issues to responsible persona.

Debug and inspect a Tower app after running it. Supports dlt pipelines, ASGI apps, and plain Python scripts. Use after a run (success or failure) to inspect logs, diagnose errors, and fix issues.

Find a dlt destination for a given storage provider. Use when the user asks about a destination, wants to find a connector, or asks to implement a pipeline for a specific data destination.

Find a dlt source for a given API or data provider. Use when the user asks about a source, wants to find a connector, or asks to implement a pipeline for a specific data source.

Detect existing project stack, learn conventions from code, and produce a project profile. Run before other skills to give them richer context, or let skills invoke it automatically when no profile exists.

Create a Tower app — either a hello-world scaffold (greenfield) or a wrapper around existing Python pipeline code.

Review data app requirements from a business analyst perspective. Auto-detects intent — SCOPE CHECK (30s) for high-intent users, DISCOVERY (5min, 6 scored dimensions) for low-intent. Gates entry into pipeline development. Use before find-source or when adding endpoints.

Review data model design for schema quality, dimensional modeling, and dbt best practices. Scores 6 dimensions (key integrity, normalization, naming, type correctness, join readiness, evolution safety). Modes — PRE-LOAD (config review), POST-LOAD (materialized schema), MODEL (dbt dimensional modeling). Use when schema quality needs deeper review.

Review a Tower data app for reliability and production readiness. Scores 5 gradient dimensions (incremental strategy, error resilience, resource efficiency, observability, test coverage) + 6 pass/fail checks. Modes — DEV REVIEW (read-only), PROD READINESS (makes changes, absorbs adjust-endpoint), INCIDENT (root cause), OPTIMIZATION (performance). Use after debug-pipeline or before tower_deploy.

Pre-deploy operational review for a Tower data app. Scores 4 dimensions (schedule appropriateness, failure recovery, cost awareness, monitoring) + 3 pass/fail checks (Towerfile resources, schedule configured, runbook). Single mode — PRE-DEPLOY. Use before tower_deploy.

Review credential handling, data sensitivity, and access control for a Tower data app. 5 pass/fail checks + 2 scored dimensions (PII awareness, secret rotation readiness). Modes — AUDIT (before first deploy), INCIDENT (credential compromise response). Use before tower_deploy or when credential issues arise.

Safely manage secrets as runtime environment variables. Use when the user directly asks to set up, configure, or inspect credentials (API keys, database passwords, tokens). Do NOT use when in need for reading secrets, for pipeline creation, source discovery, or debugging pipeline execution — those skills call setup-secrets when they need credentials configured.