| name | huawei-snowflake-to-dataarts-migration |
| version | 1.0.0 |
| description | Orchestrate or assist migration from Snowflake to Huawei Cloud DataArts (DLI + Factory) |
| category | migration |
| risk_level | medium |
| status | PARTIAL |
| requires_explicit_approval | true |
Purpose
Orchestrate or assist the migration of data pipelines and SQL workloads from Snowflake to Huawei Cloud DataArts (DataArts Factory + DLI), leveraging the dataarts-deploy-agent MCP for plan generation, execution, and equivalence validation.
Supported scenario
- Source: Snowflake (SQL tasks, task graphs, schemas, data)
- Target: Huawei Cloud DataArts Factory + DLI (Data Lake Insight)
- Mechanism: Artifact-based migration with SQL adaptation
- Scope: Currently supports demo/proof-of-concept flow (one-shot)
- Topology: Snowflake (external) → DataArts (Huawei Cloud)
When to use this skill
- Migrating Snowflake SQL tasks and task graphs to DataArts Factory jobs
- Validating equivalence between Snowflake and DataArts query results
- Proof-of-concept or demo migrations from Snowflake to DataArts
- Assessing migration feasibility for Snowflake workloads
When not to use this skill
- Full production migration (not yet supported end-to-end)
- Migrating non-Snowflake sources (use appropriate migration path)
- When real-time/CDC migration is required (DataArts is batch-oriented)
- When Snowflake features have no DataArts equivalent (e.g., Snowpipe, Streams, Tasks with CRON beyond Factory support)
Required inputs
- job_name: DataArts Factory job name for the migration
- artifact_dir: Path to migration artifacts directory (SQL, manifests)
- dli_queue: DLI queue name (default: "default")
Optional inputs
- run_id: Specific run ID for status check or report retrieval
- Snowflake connection details (for source extraction, not yet automated)
- DLI configuration overrides
Required MCPs
Optional MCPs
- huaweicloud-pricing (for DataArts/DLI cost estimation)
- huaweicloud-ticket (for support ticket creation)
- playwright (for console automation if needed)
Tool selection policy
- Use dataarts-deploy-agent tools for all migration operations
- Write operations (demo_run, demo_start) require confirm=true
- Use huaweicloud-pricing for cost estimation only (all tools read-only)
- Never execute demo_run or demo_start without explicit approval
Safety and approval gates
- demo_run requires confirm=true (executes real DataArts Factory jobs)
- demo_start requires confirm=true (starts async DataArts Factory jobs)
- Secret scrubbing is automatic in reports
- Stale result detection prevents acting on outdated results
- Plan-only operations (demo_plan) are safe and read-only
Workflow
Phase 1 — Discovery
Classification: AUTOMATED
- Run plan-only to discover migration scope:
snowflake_dataarts_demo_plan
- Review plan output for:
- SQL tasks to migrate
- Task graph dependencies
- Schema mappings
- Data volume estimates
- Check for existing runs:
snowflake_dataarts_demo_status
MCP tools used: snowflake_dataarts_demo_plan, snowflake_dataarts_demo_status
Phase 2 — Architecture validation
Classification: ASSISTED
- Review migration architecture:
- Source: Snowflake SQL tasks and task graphs
- Target: DataArts Factory jobs + DLI SQL
- Adapters: legacy-demo, native-dli, koocli, runtime-engine
- Validate artifact package structure
- Validate DLI queue availability
- Validate DataArts Factory workspace
- Document compatibility assessment
MCP tools used: snowflake_dataarts_demo_plan
Capability gaps:
- No automated Snowflake source extraction
- No automated schema mapping
- No automated SQL compatibility analysis
Phase 3 — Readiness and prechecks
Classification: ASSISTED
- Validate environment: DLI queue, DataArts Factory workspace
- Validate artifact package: SQL files, manifest, expected results
- Validate credentials: DataArts, DLI access
- Check for stale results from previous runs
- Review migration plan for warnings
MCP tools used: snowflake_dataarts_demo_status
Phase 4 — Plan generation
Classification: AUTOMATED
- Generate migration plan:
snowflake_dataarts_demo_plan
- Review plan for:
- SQL adaptation requirements
- DDL mappings
- Data transformation needs
- Expected equivalence validation queries
- Present plan for approval
MCP tools used: snowflake_dataarts_demo_plan
Phase 5 — Approval
Classification: MANUAL
- Review migration plan
- Review SQL adaptations
- Review expected costs
- Obtain explicit approval from stakeholder
- Document approval with timestamp and approver
MCP tools used: None
Phase 6 — Execution
Classification: ASSISTED
- Execute migration (synchronous):
snowflake_dataarts_demo_run with confirm=true
OR
Start migration (asynchronous): snowflake_dataarts_demo_start with confirm=true
- Monitor progress:
snowflake_dataarts_demo_status
- Wait for completion
MCP tools used: snowflake_dataarts_demo_run OR snowflake_dataarts_demo_start, snowflake_dataarts_demo_status
Phase 7 — Validation
Classification: AUTOMATED
- Check run status:
snowflake_dataarts_demo_status
- Generate equivalence summary:
snowflake_dataarts_demo_equivalence_summary
- Retrieve last report:
snowflake_dataarts_demo_last_report
- Review equivalence results:
- Row count match
- Value match
- Schema match
- Document validation results
MCP tools used: snowflake_dataarts_demo_status, snowflake_dataarts_demo_equivalence_summary, snowflake_dataarts_demo_last_report
Phase 8 — Cutover
Classification: MANUAL
Not applicable for demo/POC flow. For production migration (future):
- Validate production DataArts Factory jobs
- Switch downstream consumers to DataArts
- Decommission Snowflake tasks
Phase 9 — Rollback
Classification: MANUAL
- Review failed step from status
- Clean up DataArts Factory jobs created during migration
- Clean up DLI tables/data created during migration
- Revert to Snowflake as source of truth
- Document rollback reason
MCP tools used: snowflake_dataarts_demo_status (for diagnosis only)
Phase 10 — Closure and reporting
Classification: AUTOMATED
- Generate final report:
snowflake_dataarts_demo_last_report
- Generate equivalence summary:
snowflake_dataarts_demo_equivalence_summary
- Document:
- Migration scope (tasks migrated, SQL adapted)
- Equivalence results
- Issues encountered
- Recommendations for production migration
- Archive migration artifacts
MCP tools used: snowflake_dataarts_demo_last_report, snowflake_dataarts_demo_equivalence_summary
Capability gap handling
Known capability gaps:
- GAP-DA-001: No automated Snowflake source extraction (SQL, schemas, data)
- GAP-DA-002: No automated schema mapping between Snowflake and DataArts
- GAP-DA-003: No automated SQL compatibility analysis
- GAP-DA-004: No production migration flow (only demo/POC)
- GAP-DA-005: No automated rollback of DataArts resources
- GAP-DA-006: No incremental/delta migration support
Output artifacts
- migration-plan.md — Plan output from demo_plan
- execution-status.json — Run status and progress
- equivalence-summary.md — Source vs target equivalence results
- demo-report.md — Full migration report
- validation-results.json — Equivalence validation details
Failure handling
- Plan generation failure: Check artifact package structure, DLI queue, credentials
- Execution failure: Check DataArts Factory logs, DLI job logs, SQL errors
- Equivalence mismatch: Review SQL adaptation, data types, NULL handling
- Stale results: Clear previous run state before re-execution
Recovery procedure
- If plan fails: Validate artifact package and environment. Fix and retry.
- If execution fails: Review error in status. May need to fix SQL and re-run.
- If equivalence fails: Review mismatched results. May need SQL adaptation fixes.
- For rollback: Clean up DataArts/DLI resources. Snowflake remains untouched.
Evidence and traceability
- Migration plan preserved
- Execution status logged with timestamps
- Equivalence results preserved (row counts, value comparisons)
- Reports generated with secrets scrubbed
- Run IDs tracked for traceability
Known limitations
- Only demo/POC flow is supported (not production migration)
- Snowflake source extraction is manual
- Schema mapping is manual
- SQL adaptation is semi-automated (adapter-based)
- No incremental/delta migration
- No automated rollback of DataArts resources
- Equivalence validation is limited to query result comparison
- DLI queue must be pre-configured
- DataArts Factory workspace must be pre-configured
Status justification
Status: PARTIAL
Evidence:
- 6 dataarts-deploy-agent MCP tools available [VERIFIED_FROM_CODE]
- 2 write tools require confirm=true [VERIFIED_FROM_CODE]
- Demo/POC flow works end-to-end [VERIFIED_FROM_DOCUMENTATION]
- Golden packages validated (orders_pipeline_simple, customer_status_pipeline_simple) [VERIFIED_FROM_DOCUMENTATION]
- Secret scrubbing implemented [VERIFIED_FROM_CODE]
- Production migration flow NOT available [VERIFIED_FROM_DOCUMENTATION]
- Snowflake source extraction NOT automated [NOT_VERIFIED]
- Schema mapping NOT automated [NOT_VERIFIED]
