| name | migrate-job-flow |
| description | Guided full-job migration flow. Walks the user from "I want to migrate this Databricks job" through DAG → data check → migrate → status, asking before each phase. Composes aidp-build-dag, aidp-check-data, aidp-migrate-job, and migration-status. Use when the user wants the end-to-end guided experience instead of running individual phases manually. |
migrate-job-flow — guided full migration
Drive the migrator end-to-end with checkpoints between phases. Asks the user before running each long-running step so they can abort or adjust.
When to use
- User says "migrate this Databricks job", "port this workflow end-to-end", "run the full migration".
- User wants a single guided flow, not individual phase invocations.
- User wants checkpoints between phases (instead of fire-and-forget).
For individual phases, prefer:
Workflow
- Confirm prerequisites — invoke
aidp-migrator-bootstrap silently; surface any gaps.
- Ask for the source — either a Databricks workspace path OR a Databricks Job ID. Don't proceed without it.
- Build the DAG — invoke
aidp-build-dag. Show the resulting task count + dep count. Ask "looks right?" before next step.
- Data check — invoke
aidp-check-data. Show the OK / MISSING / EMPTY counts.
- Migrate — invoke
aidp-migrate-job. Surface live log tail.
- Status — when the run finishes, invoke
migration-status on the resulting JOB_REPORT.md.
Args
If the user supplied a Databricks Job ID or workspace path in their prompt, jump straight to step 3 (skip the question in step 2).
Checkpoints to surface to the user
Between phases, give the user a 1-line summary + the cost estimate of the next step:
[Phase 1/5] DAG built: 7 tasks, 18 dep notebooks, output reports/<MyJob>_manifest.json
[Phase 2/5] About to scan source data on cluster — ~2 min, no token cost. Proceed? (y/N)
...
[Phase 4/5] About to start Pass-2 migration — est. 30-90 min, token cost depends on the selected OpenAI model. Proceed? (y/N)
Do not auto-proceed past the migrate step without explicit user confirmation. Pass-2 is expensive.
When to stop
- User aborts at any checkpoint.
- Data check shows >50% MISSING — likely the catalog hasn't been migrated yet; route to
migrate-catalog-flow instead.
- Cluster is not Active — instruct the user to start it before continuing.
