migrate-glue-devendpoint-to-interactive-sessions

name	migrate-glue-devendpoint-to-interactive-sessions
description	Migrate a legacy AWS Glue development endpoint to a Glue interactive session, following the official AWS migration checklist. Inventories the dev endpoint config, maps it to an equivalent interactive session (runtime role, Glue version, workers, VPC connection, idle timeout), provisions the session, validates by running the developer's Spark/PySpark code, and deletes the dev endpoint after the user confirms. Use when: migrate glue dev endpoint, move off dev endpoints, dev endpoint to interactive sessions, replace glue dev endpoint, glue dev endpoint deprecated, modernize glue dev environment. Do NOT use for: upgrading a Glue ETL job version 0.9/1.0 to 4.0 (use glue-09-10-migration), creating a brand-new interactive session with no source dev endpoint, or migrating dev endpoints to Amazon EMR.
argument-hint	[dev-endpoint-name] [aws-region] [aws-cli-profile-name]
owner_team	AWS Analytics
owner_cti	AWS/Analytics/Agent Skills
stages	["preprod"]
version	1
metadata	{"service":["glue"],"task":["migrate","upgrade"],"persona":["developer","data-engineer"],"workload":["data-analytics"]}

Migrate Glue Dev Endpoint to Interactive Session

Move a developer off a legacy Glue development endpoint (Glue 0.9/1.0, no console since March 2023, billed continuously) onto a Glue interactive session (Glue 2.0+, sub-minute startup, idle timeout, console support). Follows the official checklist: https://docs.aws.amazon.com/glue/latest/dg/development-migration-checklist.html

The migration is additive and reversible until the final step: the session is created and validated alongside the still-running dev endpoint. The dev endpoint is deleted only after the user confirms the session works.

Reference Documentation

• references/migration-mapping.md -- DevEndpoint field → session magic/API mapping, the access-method decision table, IAM two-principal setup, and code-compatibility notes for the 0.9/1.0 → 3.0+ jump.

Workflow

1. Inventory the Dev Endpoint and Route

1.1. Confirm inputs: dev endpoint name, region, AWS CLI profile. MUST NOT proceed without a dev endpoint name.

1.2. Call glue:GetDevEndpoint(EndpointName=<name>). Store as source. Record: RoleArn, GlueVersion, WorkerType, NumberOfWorkers (or NumberOfNodes), SubnetId, SecurityGroupIds, Arguments, ExtraPythonLibsS3Path, ExtraJarsS3Path, PublicAddress/PrivateAddress, Status.

1.3. Route on access method — ask the user how they use the dev endpoint today (or infer from source). This determines the session interface, per the official checklist. See the decision table in references/migration-mapping.md:

• SageMaker / Jupyter / JupyterLab notebook → Glue Studio notebook (upload .ipynb)
• Zeppelin notebook → convert to Jupyter, then Glue Studio notebook
• IDE (PyCharm / VS Code) → IDE integration for interactive sessions
• REPL → local aws-glue-sessions package
• SSH → no direct equivalent; use the Glue Docker image for local dev

1.4. Route on code compatibility (this is the hard gate): interactive sessions run — they cannot run 0.9/1.0. If the dev endpoint's scripts depend on 0.9/1.0-specific behavior (HDFS, YARN configs, Spark 2.x semantics, Python 2 syntax), the code MUST be upgraded to Glue 3.0+ first.