| name | migrate-glue-devendpoint-to-interactive-sessions |
| description | Migrate a legacy AWS Glue development endpoint to a Glue interactive session, following the official AWS migration checklist. Inventories the dev endpoint config, maps it to an equivalent interactive session (runtime role, Glue version, workers, VPC connection, idle timeout), provisions the session, validates by running the developer's Spark/PySpark code, and deletes the dev endpoint after the user confirms. Use when: migrate glue dev endpoint, move off dev endpoints, dev endpoint to interactive sessions, replace glue dev endpoint, glue dev endpoint deprecated, modernize glue dev environment. Do NOT use for: upgrading a Glue ETL job version 0.9/1.0 to 4.0 (use glue-09-10-migration), creating a brand-new interactive session with no source dev endpoint, or migrating dev endpoints to Amazon EMR.
|
| argument-hint | [dev-endpoint-name] [aws-region] [aws-cli-profile-name] |
| owner_team | AWS Analytics |
| owner_cti | AWS/Analytics/Agent Skills |
| stages | ["preprod"] |
| version | 1 |
| metadata | {"service":["glue"],"task":["migrate","upgrade"],"persona":["developer","data-engineer"],"workload":["data-analytics"]} |
Migrate Glue Dev Endpoint to Interactive Session
Move a developer off a legacy Glue development endpoint (Glue 0.9/1.0, no console
since March 2023, billed continuously) onto a Glue interactive session (Glue 2.0+,
sub-minute startup, idle timeout, console support). Follows the official checklist:
https://docs.aws.amazon.com/glue/latest/dg/development-migration-checklist.html
The migration is additive and reversible until the final step: the session is
created and validated alongside the still-running dev endpoint. The dev endpoint is
deleted only after the user confirms the session works.
Reference Documentation
- references/migration-mapping.md -- DevEndpoint
field → session magic/API mapping, the access-method decision table, IAM
two-principal setup, and code-compatibility notes for the 0.9/1.0 → 3.0+ jump.
Workflow
1. Inventory the Dev Endpoint and Route
1.1. Confirm inputs: dev endpoint name, region, AWS CLI profile. MUST NOT proceed
without a dev endpoint name.
1.2. Call glue:GetDevEndpoint(EndpointName=<name>). Store as source. Record:
RoleArn, GlueVersion, WorkerType, NumberOfWorkers (or NumberOfNodes),
SubnetId, SecurityGroupIds, Arguments, ExtraPythonLibsS3Path,
ExtraJarsS3Path, PublicAddress/PrivateAddress, Status.
1.3. Route on access method — ask the user how they use the dev endpoint today
(or infer from source). This determines the session interface, per the official
checklist. See the decision table in references/migration-mapping.md:
- SageMaker / Jupyter / JupyterLab notebook → Glue Studio notebook (upload
.ipynb)
- Zeppelin notebook → convert to Jupyter, then Glue Studio notebook
- IDE (PyCharm / VS Code) → IDE integration for interactive sessions
- REPL → local
aws-glue-sessions package
- SSH → no direct equivalent; use the Glue Docker image for local dev
1.4. Route on code compatibility (this is the hard gate): interactive sessions
run — they cannot run 0.9/1.0. If the dev endpoint's scripts
depend on 0.9/1.0-specific behavior (HDFS, YARN configs, Spark 2.x semantics,
Python 2 syntax), the code MUST be upgraded to Glue 3.0+ first.
