| name | airflow |
| version | 1.0.0 |
| description | Python DAG workflow orchestration using Apache Airflow for data pipelines, ETL processes, and scheduled task automation |
| author | workspace-hub |
| category | operations |
| type | skill |
| capabilities | ["dag_authoring","task_orchestration","scheduling","sensors","operators","hooks","xcoms","variables","connections","kubernetes_deployment","docker_deployment"] |
| tools | ["airflow-cli","docker","kubernetes","helm"] |
| tags | ["airflow","dag","workflow","orchestration","etl","data-pipeline","scheduling","python","automation"] |
| platforms | ["linux","macos","docker","kubernetes"] |
| related_skills | ["yaml-configuration","python-scientific-computing","pandas-data-processing"] |
| scripts_exempt | true |
Airflow
When to Use This Skill
USE when:
- Building complex data pipelines with task dependencies
- Orchestrating ETL/ELT workflows
- Scheduling recurring batch jobs
- Managing workflows with retries and error handling
- Coordinating tasks across multiple systems
- Need visibility into workflow execution history
- Requiring audit trails and lineage tracking
- Building ML pipeline orchestration
DON'T USE when:
- Real-time streaming data (use Kafka, Flink)
- Simple cron jobs (use systemd timers, crontab)
- CI/CD pipelines (use GitHub Actions, Jenkins)
- Low-latency requirements (Airflow has scheduler overhead)
- Simple single-task automation (overkill)
- Need visual workflow design for non-developers (use n8n)
Prerequisites
Installation Options
Option 1: pip (Development)
python -m venv airflow-env
source airflow-env/bin/activate
export AIRFLOW_HOME=~/airflow
*See sub-skills for full details.*
```bash
pip install apache-airflow[dev,postgres,celery,kubernetes]
pip install pytest pytest-airflow
pip install ruff
Version History
| Version | Date | Changes |
|---|
| 1.0.0 | 2026-01-17 | Initial release with comprehensive workflow patterns |
Resources
