Functionally tests local Dataflow pipeline changes against the main branch using ephemeral GCP resources and gated approvals. Use ONLY when functionally testing one of these specific migration templates: gcs-spanner-dv, sourcedb-to-spanner,…
GoogleCloudPlatform/DataflowTemplates
SkillsMP has collected 11 skills from GoogleCloudPlatform/DataflowTemplates. Open a skill to review its source and details.
- Latest recorded source activity
- SkillsMP catalog refreshed
- skills collected
- 11
- GitHub stars
- 1,311
- GitHub forks
- 1,116
Skills in this repository
Showing 11 of 11 collected skills.
Guide for implementing a database source connector in the v2/datastream-to-spanner forward migration Dataflow template. Details scope boundaries, prerequisites, connector implementations, registry registrations, unit testing, and smoke testing guidelines.
Guide for implementing a database source connector in the v2/sourcedb-to-spanner forward migration Dataflow template. Details scope boundaries, prerequisites, connector implementations, registry registrations, unit testing, and smoke testing guidelines.
Guide for implementing a database source connector in the v2/spanner-to-sourcedb reverse migration Dataflow template. Details scope boundaries, prerequisites, connector implementations, registry registrations, unit testing, and smoke testing guidelines.
Specific runner skill that delegates to the Template-Agnostic Meta-Test Orchestrator for the datastream-to-spanner (CDC) template.
Specific runner skill that delegates to the Template-Agnostic Meta-Test Orchestrator for the sourcedb-to-spanner (Bulk) template.
Skill for adding data type integ tests for new source by provided datatype_mapping_matrix.
Skill for adding functional integ test for new source by taking reference from existing MySQL integ tests.
Template-agnostic Orchestrator Skill for generating and executing exhaustive testing suites for any migration template. It will generate the functional and datatype related test for the source.
Specific runner skill that delegates to the Template-Agnostic Meta-Test Orchestrator for the spanner-to-sourcedb (Reverse Migration) template.
Debugs logical errors and data discrepancies in Dataflow templates by launching jobs via Terraform and comparing source (e.g. Cloud SQL) vs destination (e.g. Spanner) data. Use ONLY when the pipeline launches and runs to completion (terminal state) but…