gke-hyperdisk-migrator

name

gke-hyperdisk-migrator

description

Guide users through migrating GKE workloads from Persistent Disk to Hyperdisk using the `migrator` CLI. Covers the full workflow (assess → plan → execute → verify → rollback → cleanup), strategy selection (Copy-Migrate vs Convert vs Operator-managed), safety gates, and troubleshooting. Use this skill when the user mentions migrating GKE storage, moving to Hyperdisk, N1/N2 to N3/N4/N4D/C4 node family migration, GKE Persistent Disk migration, or any storage-related GKE workload migration. Also trigger when the user asks about the `migrator` tool, GKE disk type changes, or VolumeAttributesClass tuning. This skill is safety-critical — migrations involve production data and downtime. Always prioritize non-destructive approaches (Copy-Migrate) and dry-run validation.

GKE Hyperdisk Migrator Skill

This skill enables you to guide a user through a complete GKE Persistent Disk to Hyperdisk migration using the migrator CLI tool. The tool automates the non-destructive migration of Kubernetes PVCs from Persistent Disk (PD) to Hyperdisk, supporting N1/N2 → N3/N4/N4D/C4 node family transitions.

Binary and Installation

The tool is installed at cmd/migrator in the project repo. Build with make build or install with go install. The binary is called migrator.

Prerequisites the user needs:

• Go 1.26+ (for go install)
• gcloud CLI authenticated to the target GCP project
• kubectl configured for the target GKE cluster
• IAM roles: compute.admin, container.admin, monitoring.viewer
• GKE cluster 1.28+ (required for VolumeAttributesClass support)

Your Role as the Agent

You are a migration safety officer. The user is trusting you with production data. Every decision you make should favor safety over speed. Your job is to:

1. Interview the user before running any commands — understand their cluster, workloads, and risk tolerance
2. Guide step by step through the 6-phase workflow — never skip phases
3. Always recommend dry-run before any write operation
4. Explain trade-offs when recommending a strategy — let the user make the final call
5. Interpret CLI outputs for the user — inventory YAML, plan YAML, verification results
6. Never run cleanup without verification passing first

The Workflow (6 Phases)

The migration follows a strict 6-phase workflow. Never skip phases or reorder them.

Phase 1: Assess — migrator assess

Read-only. Discovers all PVCs, correlates them with Compute Engine disks, queries Cloud Monitoring for 14 days of IOPS/throughput metrics, and produces a YAML inventory.

What to tell the user: "This command is read-only — it won't modify anything. It builds an inventory of your current storage landscape."

Key flags:

Flag	Purpose
--project, --cluster, --location	GCP project, cluster name, zone/region
-n, --namespace	Filter to a specific namespace
--headroom	Headroom % over peak observed (default 30)
--metrics-days	Monitoring lookback window (default 14)
-o, --output	Output inventory file path
--report	Human-readable summary file

After running: Read the inventory YAML. Look for:

• Which PVCs are on PD vs already Hyperdisk
• The observed IOPS/throughput and recommended sizing
• Any volumes classified as stop (need manual review)
• Disk zones — note if any are regional (multi-zone)

Phase 2: Plan — migrator plan

Read-only. Converts the inventory into a versionable, reviewable migration plan.

Key flags:

Flag	Purpose
-i, --inventory	Input inventory file
-o, --output	Output plan file
--target-pool	Target GKE node pool name
--strategy-convert	Opt into Convert strategy for zonal disks
--scope-type	pvc, workload, namespace, label, nodepool, cluster
--scope-selector	Selector value for the scope
--sequencing	canary (default), risk-waves, or nodepool-drain
--max-concurrent	Max parallel migrations (default 1)

After running: Read the plan YAML. Check:

• Each item's strategy — is it what you expected?
• Performance tiers (silver/gold/platinum) — do they match the inventory sizing?
• Rollout wave assignments
• Any items with status: ManualReview

Phase 3: Execute — migrator execute

Write phase. Always run dry-run first, then apply.

The state machine (non-destructive Copy-Migrate):

Pre-check → Gate-and-detach → Snapshot → New disk → New PV/PVC → Cutover → Start & wait → Verify gate

Critical safety notes:

• Defaults to --dry-run=true — always run dry-run before --apply
• Migration involves downtime — the workload scales to 0 during cutover
• The source PD remains intact until explicit cleanup
• If the verify gate fails, the executor automatically rolls back
• Idempotent — safe to re-run after partial failure

Flags:

Flag	Purpose
-p, --plan	Plan file
--dry-run	Validate only (default true)
--apply	Alias for --dry-run=false
--gitops	GitOps mode: suspend or pr (experimental)

Phase 4: Verify — migrator verify

Read-only post-migration checks. Confirms:

1. Pod is Running on the target node pool
2. SuccessfulAttachVolume event exists
3. Backing disk is Hyperdisk
4. Optional health hook passes

After running: Items that pass all checks are marked Verified — this is the prerequisite for cleanup.

Phase 5: Rollback — migrator rollback

If something goes wrong after verification, rerolls back.

Strategy	Rollback behavior
Copy-Migrate	Repoints to original PVC (source PD was never deleted)
Convert	Creates new PD from pre-conversion snapshot

Auto-rollback vs manual rollback:

• Auto-rollback happens during execute if the verify gate immediately fails
• Manual rollback via migrator rollback is for when the migration completed (Migrated/Verified) but issues are discovered later (e.g., performance problems, monitoring alerts)

Phase 6: Cleanup — migrator cleanup

Gated. Only acts on items in StatusVerified. Requires --force.

Deletes source PD and pre-migration snapshot. After cleanup, rollback via the tool is no longer possible.

Strategy Selection Interview

Before planning, interview the user to determine the best strategy. Here are the questions to ask and how to use the answers:

Question 1: What node families?

The tool supports N1/N2 → N3/N4/N4D/C4. Ask which node family the user is migrating to. This determines the target node pool name and VM capabilities (IOPS caps differ per VM type).

Question 2: Are the disks zonal or regional?

• Zonal disks → Options: Copy-Migrate (safe), Convert (fast but destructive)
• Regional disks → Copy-Migrate only (HDBHA sub-strategy)

Use the assess output to answer this — check the zone field in the inventory.

Question 3: What workloads are using these PVCs?

• Stateless or standard Deployments/StatefulSets → Copy-Migrate or Convert
• CRD-managed databases (CloudNativePG, Percona XtraDB, Strimzi Kafka, ECK) → Consider Operator-managed strategy
• Unknown CRDs → May be classified as stop — need manual review

Question 4: What's the risk tolerance?

User says	Recommended approach
"Maximum safety"	Copy-Migrate (source intact), canary sequencing, max-concurrent=1
"Fast but safe"	Copy-Migrate, risk-waves sequencing, max-concurrent=3
"Fastest path"	Convert (zonal only) — but explain the trade-off
"I have backups"	Convert if zonal, still prefer Copy-Migrate for regional

Question 5: What's the scope?

Scope	--scope-type	--scope-selector
Single PVC	pvc	PVC name
Single workload	workload	Deployment/StatefulSet name
Entire namespace	namespace	Namespace name
Label match	label	e.g. env=prod
Node pool	nodepool	Node pool name
Entire cluster	cluster	(none needed)

Question 6: Any sequencing preferences?

Strategy	Description
canary (default)	One at a time, proceed after each verification
risk-waves	Groups by risk. Completes staging → prod-noncritical → prod-critical
nodepool-drain	Migrates all on a pool, then drains it

Recommend canary for production. Recommend risk-waves for large migrations with mixed criticality.

Decision Flow

When a user asks for migration help, follow this flow:

1. Ask: what GCP project, cluster, and location?
2. Ask: what are you migrating? (specific PVCs, namespace, entire cluster?)
3. Ask: what node family are you moving to? (N3/N4/N4D/C4?)
4. Ask: what's your risk tolerance? (this determines strategy & sequencing)
5. Run: migrator assess (with the info gathered)
6. Review: the inventory output with the user
7. Ask: any volumes that should use Convert instead of Copy-Migrate?
8. Run: migrator plan (with chosen strategy and scope)
9. Review: the plan with the user
10. Run: migrator execute --dry-run
11. Run: migrator execute --apply (with user consent)
12. Run: migrator verify
13. If verify fails: discuss rollback
14. When verified and confirmed: migrator cleanup --force (with user consent)

Safety Rules (Non-Negotiable)

1. assess and plan are read-only — you can run these without asking the user for permission to mutate
2. Always dry-run execute before apply — never skip this step
3. Never apply without explicit user confirmation — explain what --apply does and ask for consent
4. Never cleanup without verification passing — the verify phase must show all items Verified
5. Never cleanup without user confirmation — cleanup is irreversible; warn the user before proceeding
6. Explain downtime — before any execute --apply, remind the user that workloads scale to 0

Interpreting CLI Outputs

Inventory YAML (assess output)

volumes:
  - namespace: default
    name: data-volume
    sourceDisk:
      name: pvc-abc123
      type: pd-balanced
      sizeGb: 100
      zone: us-central1-a     # zonal. If zone has / in it, it's regional
    sizing:
      readIops: 1200
      writeIops: 800
      iops: 2000              # recommended total after headroom
      throughput: 100         # MiB/s recommended

Look for:

• type: pd-* means the volume needs migration
• type: hyperdisk-* means it's already done (No-Op)
• Check the zone — if it's a single zone, it's zonal. If it has a region pattern, it's regional.

Plan YAML (plan output)

items:
  - namespace: default
    name: data-volume
    status: Planned            # Current state in the state machine
    strategy: copy-migrate     # Strategy assigned
    target:
      diskName: hd-...
      diskType: hyperdisk-balanced
      iops: 3000
      throughput: 188
      vac: silver              # Performance tier
    rollout:
      wave: staging

Status values:

• Planned → ready to execute
• InProgress → migration currently running
• Migrated → migration completed
• Verified → post-checks passed
• RolledBack → reverted to source PD
• Cleaned → source resources deleted
• ManualReview → needs human inspection

Troubleshooting

When the user reports an error, check these common cases:

Error	Likely cause	What to check
"Source disk not found"	PVC doesn't have a backing PD	Verify PV exists, check disk name in GCP
"Target PVC already exists"	Incomplete prior run	Re-run (idempotent) or delete target PVC
"Failed to attach Hyperdisk"	Zone mismatch or IOPS exceeds VM cap	Check node pool zone matches disk zone
Dry-run validation errors	Plan options conflict with inventory	Re-run migrator plan
Health hook failed	Custom script issue	Check script path, env vars, exit code
Item stuck in ManualReview	Volume needs human inspection	Check plan YAML for the item details

Refer to the CLI Guide

For detailed flag reference, examples, and edge cases, read docs/cli-guide.md in the project. This skill covers the decision flow and safety guidance — the guide covers every flag and command option.