| name | create-orchestra-pipeline |
| description | Create, validate, and remediate Orchestra pipeline YAML files. Use when asked to build a new pipeline, add tasks to an existing pipeline, fix pipeline validation errors, or author Orchestra workflow definitions from a description. Trigger on phrases like "create a pipeline", "add a dbt task", "write orchestra yaml", "fix validate errors", or when editing files under orchestra/ or similar pipeline directories.
|
Create Orchestra Pipeline
Author or update an Orchestra version: v1 pipeline YAML, validate it, and fix validation errors.
References
../../references/orchestra/pipeline/yaml-authoring.md — schema, integrations, variables, optional sections../../references/orchestra/pipeline/examples.md — multi-stage patterns (warehouse → LLM → messaging, agents)../../references/orchestra/mcp/tools-quick-ref.md — validate_pipeline, create_pipeline, update_pipeline- Orchestra docs for integration parameters not covered in the reference
Workflow
Step 1 — Understand the request
From the user message, determine:
- Purpose, integrations, and data flow
- Target file path (default:
orchestra/<descriptive-name>.yml in the current repo)
- Connections, schedules, inputs, alerts, or matrix requirements
If no filename is given, derive a short kebab-case name from the pipeline purpose.
When editing an existing pipeline, check whether it's Git-backed or Orchestra-backed
before deciding how to apply the change — list_pipelines metadata includes
storageProvider. For Git-backed pipelines, edit the repo YAML directly (the repo is
the source of truth); update_pipeline cannot write to them. For Orchestra-backed
pipelines with no local YAML, use the MCP tools instead of authoring a file.
Step 2 — Match repo conventions
List existing pipeline YAML (typically orchestra/, or paths the user names). Read one or two
pipelines that use similar integrations before writing — match task group and task ID style,
connection references, and schedule format.
Step 3 — Write or edit the YAML
Follow ../../references/orchestra/pipeline/yaml-authoring.md for structure, required fields,
integration table, and variable syntax. Omit empty tags arrays.
Editorial defaults, unless the user asks otherwise:
- Prefer
cron or webhook triggers over sensors where the source integration supports it.
- Don't hardcode a
connection; omit it and let Orchestra pick the default, or reference an
existing environment variable (${{ ENV.VAR }}) when one already covers that connection.
- Add a failure alert as a matter of course — most pipelines should have one.
- For matrices, define the
inputs list once on the task group, then reference values in task
parameters as ${{ MATRIX.key }} rather than repeating the list per task.
Step 4 — Validate
Run local validation when orchestra-cli is available:
orchestra-cli validate <path/to/pipeline.yml>
If only Orchestra MCP is connected, use validate_pipeline with the YAML body instead.
Step 5 — Remediate errors
For each validation error, apply the fixes in the table in yaml-authoring.md. Re-validate
until clean.
Step 6 — Report
Summarise in a short paragraph or bullet list:
- File path created or modified
- Stages and tasks
- Connections or environment variables to configure in the Orchestra UI
- Placeholder values the user must replace
Keep the summary concise.
Notes
- Pipelines can represent both data workflows and AI agent workflows; use
PYTHON /
PYTHON_EXECUTE_SCRIPT with build_command and project_dir for in-repo agent entrypoints.
- After saving YAML in repos that use Cursor hooks,
orchestra-cli validate may run
automatically on *.yml / *.yaml — still run validation explicitly when unsure.
- For Git-backed pipelines, committing YAML does not deploy until the repo is connected in
Orchestra; call out any UI setup the user still needs.
- Task group and task IDs only need to be unique strings, not real UUIDs.
- Don't add
run_inputs to a trigger unless asked — that's for setting inputs dynamically per
trigger, not a default every pipeline needs.
