dagu

# DAG Authoring Load only the reference file that matches the task. ## Default Approach - Prefer `type: graph` for new DAGs. It supports both sequential flow via `depends:` and parallel flow. - Use an Agent DAG (`type: agent`) only when the step order cannot be written down in advance and an LLM must choose it. It requires `llm:` and a `tasks:` list stating when the run is finished. The model may select multiple distinct, independent actions in one turn; Dagu runs them concurrently up to `max_active_steps` (`0` is unlimited, `1` is serial). Same-batch actions cannot consume sibling outputs, while failures do not cancel siblings. `set_task_status` and `ask_user` must be called alone. - Use `type: build` only for local regular-file pipelines whose unchanged transformations should be reused across runs. - Prefer `id` on every step. Omit `name` unless the display label must differ from the step ID. - Prefer `dagu schema ...` and `dagu validate ...` over guessing field names or shapes. - Prefer `action: template.render` when generating text files, prompts, or artifacts instead of assembling them with shell `echo` or heredocs. - Prefer `file.*` actions for local file operations such as stat, read, write, copy, move, delete, mkdir, and list instead of shelling out to `cp`, `mv`, `rm`, or `mkdir`. - Prefer `git.worktree.add` and `git.worktree.remove` when steps need isolated branches inside an existing local Git repository. Add an explicit remove step when the workflow should delete the worktree. - Prefer `stdout.artifact` / `stderr.artifact` when a command stream should become a DAG-run artifact, especially for large reports, JSON, Markdown, logs, or generated files. - Prefer `artifact.*` actions for explicit artifact reads/writes/lists. Use `DAG_RUN_ARTIFACTS_DIR` only when a tool truly needs a filesystem path inside the step. - Prefer string-form `output: VAR_NAME` for capturing small stdout values into flat variables. - Prefer object-form `output:` when downstream steps need structured values via `${step_id.output.*}`. - Prefer declared step `outputs:` with `$DAGU_OUTPUT_FILE` when a step must publish explicit values for `${steps.<step_id>.outputs.<name>}`. - Use `action: human.task` when an operator must provide typed input before downstream steps continue. Human task form outputs use `${steps.<step_id>.outputs.<name>}` without an authored `outputs:` field. - Prefer `stdout.outputs` or `action: outputs.write` when a DAG or remote action needs to return caller-visible values via `${step_id.outputs.*}`. - Prefer `state.*` actions for small persistent JSON state across DAG runs, such as cursors, checkpoints, and previous-value comparisons. - Prefer temporary files in the artifacts dir only when downstream steps need file paths; otherwise let commands write large artifact content to stdout and attach it with `stdout.artifact`. - Prefer scoped Dagu references for named values: `${consts.NAME}`, `${params.NAME}`, and `${env.NAME}`. Avoid unscoped braced names in examples unless the example is intentionally showing shell syntax. - Declare portable external CLI dependencies in top-level `tools` using aqua shorthand when the binary version affects reproducibility, for example `tools: ["jqlang/jq@jq-1.7.1"]`. Append `#sha256:<64 hex>` to also pin the downloaded artifact content for the run platform, for example `jqlang/jq@jq-1.7.1#sha256:<hex>`. - For remote actions, put `tools` in the referenced action DAG file, not in `dagu-action.yaml`; caller DAG tools are not inherited across the action boundary. - Declare step-level `dependencies` when a run needs files from the DAG working directory. Use literal working-directory-relative paths or glob patterns; do not use value references. - Use remote action packages (`dagu-action.yaml`) when reusable logic needs helper files, its own DAG, versioning, or an input/output schema contract. ## High-Signal Rules - `output:` has two modes: - string form captures trimmed stdout into an env-scope variable such as `${env.VERSION}` - object form publishes structured step-scoped output for `${step_id.output.*}` access - Value declarations in step `outputs:` publish explicit values through `${steps.<step_id>.outputs.<name>}`. Write those values to `$DAGU_OUTPUT_FILE`; Dagu captures them only after the command succeeds. Build path declarations publish the final materialization path after commit or reuse. - `human.task` is a processless root-DAG step with an explicit `id`, a required `with.prompt`, and an optional flat scalar form. A root DAG containing one can run locally or on a distributed worker. Every declared form property is a step output, published when submitted or defaulted, and available as `${steps.<step_id>.outputs.<name>}`. Optional `with.push_back` (`rewind_to` plus an optional feedback `form`) lets the operator send the work back to an upstream step instead of completing it. - `stdout.artifact` / `stderr.artifact` store command stdout/stderr directly as relative artifact paths, for example `stdout: {artifact: reports/report.md}`. Artifact outputs auto-enable artifacts unless `artifacts.enabled: false` is explicitly set, which is invalid. - `${step_id.stdout}` is a log file path, not stdout content. - Step `stdin:` pipes a file to the command's standard input, so `stdin: ${step_id.stdout}` replaces `cat "${step_id.stdout}" | command`. Only command and shell steps accept it. - Use `${context.*}` for run metadata in DAG YAML, for example `${context.dag.name}`, `${context.run.id}`, or `${context.paths.artifacts_dir}`. Unavailable context values remain unresolved text instead of becoming empty strings. - In a build step, `${inputs.<name>}` is the final input path and `${outputs.<name>}` is a fresh attempt staging path. Write file results only to the staging path; dependencies read the committed path as `${steps.<step_id>.outputs.<name>}`. - Do not read attempt-only `${step_id.stdout}`, `${step_id.stderr}`, or `${step_id.exit_code}` from potentially reusable producers. This also applies to `${step_id.output.<name>}` and `${step_id.outputs.<name>}`, including their whole-value `${step_id.output}` and `${step_id.outputs}` forms. Path-output steps cannot use `continue_on.mark_success`. - Build workflows are local-only. Path declarations are supported only on host command or shell steps without containers, and stream redirects cannot target declared build inputs or outputs. - Use `${consts.NAME}`, `${params.NAME}`, and `${env.NAME}` for Dagu-side named values. Use shell `$NAME` or `printenv NAME` only when the target shell or process should read the variable at execution time. - `consts:` must use list form with one key per item, for example `consts: [{service: api}]`. Const values are resolved while loading the DAG and can reference inherited or earlier consts. - `env:` should use list-of-maps when values depend on earlier env vars. - `params:` values arrive as strings. The `params:` field supports JSON schema-like types and validation, check for schema to see how to specify types and validation rules. - Single-line `run:` values are command-form entries. Array-form `run:` entries run one by one. Multi-line `run:` values are scripts. Dagu does not split pipes, redirects, `&&`, or `;` into separate commands; those stay with the selected shell. - Do not assume `bash` for `run:` steps. If a script depends on a specific interpreter, add a shebang such as `#!/bin/sh` or `#!/usr/bin/env bash` only after checking that shell exists on the target host or container. Otherwise keep the script portable or set `with.shell:` explicitly. - `parallel:` currently requires `action: dag.run` to a child DAG. - Sub-DAGs do not inherit parent env vars; pass what you need via `params:`. - For arbitrary text inside shell steps, prefer `printenv VAR_NAME` or `action: template.render` over Dagu interpolation such as `${env.VAR_NAME}`. - `harness.run` supports built-in provider adapters (`aider`, `amp`, `claude`, `cline`, `codex`, `copilot`, `cursor`, `deepseek`, `droid`, `gemini`, `goose`, `kilo`, `kiro`, `opencode`, `pi`, `qwen`) and custom top-level `harnesses:` entries. Built-in OpenCode uses managed sessions on long-lived Dagu hosts by default; containers and unsupported options use the CLI path. It can use top-level `container:` or step-level `container:`. - Container runtime selection is service-level, not a DAG YAML field. Set `DAGU_CONTAINER_RUNTIME=podman` to use Podman, and set `DAGU_PODMAN_HOST` only when the Podman Docker-compatible socket is not the default. - DAG/action outputs are collected from string-form `output: VAR_NAME`, `stdout.outputs`, and `action: outputs.write`. Object-form `output:` stays step-scoped for `${step_id.output.*}` unless the workflow explicitly republishes values through `stdout.outputs` or `outputs.write`. - `state.get`, `state.set`, `state.delete`, `state.list`, and `state.diff` persist small JSON values across DAG runs. State scopes are `dag`, `root_dag`, `global`, and `custom`; use artifacts or external storage for large payloads. - Git worktree actions discover the repository from the step `working_dir`. Relative worktree paths resolve from the repository root, and the actions never fetch or push. - Remote action packages define `dagu-action.yaml` with `apiVersion: v1alpha1`, `name`, `dag`, and optional `inputs`/`outputs` JSON Schemas. `inputs` validates caller `with:` before the action DAG starts; `outputs` validates the final action output object after the action DAG returns. - Remote action manifests do not support `tools`. Declare external CLI tools in the action DAG itself so local and distributed workers prepare the right binaries for that action run. - `dependencies` snapshots files from the DAG working directory for both local and distributed execution. The materialized bundle is available through `DAG_RUN_WORK_DIR` and `${context.paths.work_dir}` and becomes the DAG process working directory. - In remote action examples, prefer `dag: workflow.yaml` for the action DAG filename. The `dag` field accepts any safe relative file path, but `workflow.yaml` avoids confusing the executable DAG with the `dagu-action.yaml` manifest. - Object-form `output:` with `decode: json` or `decode: yaml` can act as lightweight runtime validation. Malformed data or an unresolved `select:` path fails the step, so normal `retry_policy` applies. - Use DAG-level `shell` and `shell_args` only when every inherited `run:` step should use the same shell invocation. Use step-level `with.shell` and `with.shell_args` for a single step. - Use `dagu schema dag` to check the full list of available fields and their shapes. - Use `dagu example` to see different DAG patterns and how to express them in YAML. ## Example of Params, template step, and artifacts ```yaml params: type: object properties: name: type: string maxLength: 50 age: type: integer minimum: 0 maximum: 120 favorite_color: type: string required: [name, age] steps: - id: render action: template.render with: data: name: ${params.name} age: ${params.age} favorite_color: ${params.favorite_color} template: | Hello, {{ .name }}! You are {{ .age }} years old. {{- if .favorite_color }} Your favorite color is {{ .favorite_color }}. {{- end }} stdout: artifact: greeting.txt ``` ## Example of Large Command Output as Artifact ```yaml steps: - id: report run: ./generate-report --format markdown stdout: artifact: reports/report.md ``` ## Example of Reproducible External CLI ```yaml tools: - jqlang/jq@jq-1.7.1 steps: - id: inspect run: jq --version ``` ## Example of Object-Form Output ```yaml steps: - id: inspect_build run: echo '{"version":"v1.2.3","artifact":{"url":"https://example.test/app.tgz"}}' output: # decode + select act as a lightweight contract check: # malformed JSON or a missing selected field fails the step. version: from: stdout decode: json select: .version artifact: from: stdout decode: json select: .artifact - id: publish depends: [inspect_build] output: versionLabel: "ver - ${inspect_build.output.version}" artifactUrl: "${inspect_build.output.artifact.url}" ``` ## Example of Action Outputs ```yaml steps: - id: classify run: ./classify.sh "${params.INPUT}" stdout: outputs: fields: label: decode: json select: .label confidence: decode: json select: .confidence - id: publish depends: [classify] action: outputs.write with: values: label: ${classify.outputs.label} reviewed: false ``` ## Example of Human Input ```yaml steps: - id: review action: human.task with: prompt: Choose the deployment target form: type: object properties: environment: type: string enum: [staging, production] note: type: string default: "" required: [environment] - id: deploy depends: [review] run: ./deploy --environment '${steps.review.outputs.environment}' --note '${steps.review.outputs.note}' ``` Complete the task from a local CLI context with `dagu human-task complete --run-id=<run-id> --step=review --input environment=production <dag-name>`. A form is optional for acknowledgement-only tasks. Human tasks cannot be used in sub-DAGs; a distributed root run is re-queued through the scheduler after completion. ## Reference Guide Load only the file you need: - `references/steptypes.md` when choosing an action or checking action-specific behavior such as `human.task`, `dag.run`, `parallel`, `git.worktree.*`, `jq.filter`, `file.*`, `state.*`, `template.render`, `browser.*`, or `computer.*` - `references/dagu-action.md` when creating a reusable `dagu-action.yaml` package or checking action input/output schema behavior - `references/cli.md` when choosing or using Dagu CLI commands, including workflow inspection, execution, and cleanup operations - `references/context.md` when using `${context.*}` metadata references or declared step `outputs:` - `references/build.md` when creating or troubleshooting a `type: build` file workflow, path references, reuse decisions, or `--no-reuse` - `references/file-dependencies.md` when a DAG needs scripts, configuration, or other files from its working directory - `references/harnesses.md` only when the DAG invokes external CLI harnesses through `harness.run`