aws-bootstrap

name	aws-bootstrap
description	Use when the user wants to provision AWS GPU instances, check GPU instance status, terminate instances, manage EBS data volumes, clean up cloud resources, check GPU vCPU quotas, or request quota increases. Wraps the aws-bootstrap CLI for EC2 GPU instance lifecycle management (launch, status, terminate, cleanup, list, quota).

aws-bootstrap -- AWS GPU Instance Management

You have access to the aws-bootstrap CLI tool for provisioning and managing AWS EC2 GPU instances. Use it via the Bash tool. Always pass -o json before the subcommand (e.g., aws-bootstrap -o json status) when you need to parse results programmatically. The --output/-o flag is a global option and must come before the command name — placing it after (e.g., aws-bootstrap status -o json) will fail.

Prerequisites

Before running any commands, verify:

1. The aws-bootstrap CLI is installed (pip install aws-bootstrap-g4dn or uv pip install aws-bootstrap-g4dn)
2. AWS credentials are configured (AWS_PROFILE env var or --profile flag)
3. An SSH key pair exists at ~/.ssh/id_ed25519 (or specify via --key-path)

You can check if the CLI is installed by running: aws-bootstrap --version

Quick Reference

Command	Purpose	Key Options
aws-bootstrap launch	Provision a GPU instance (spot by default)	--instance-type, --spot/--on-demand, --region (repeatable), --wait, --wait-timeout, --ebs-storage, --dry-run
aws-bootstrap status	List running instances (all enabled regions by default) with region, IPs, pricing; also reports orphaned EBS data volumes (detached, still billed) with cost estimates	--gpu (CUDA info), --region/-r (repeatable), --no-instructions
aws-bootstrap terminate	Terminate instances and clean up	[ID_OR_ALIAS...], --keep-ebs, --yes
aws-bootstrap cleanup	Sync ~/.ssh/config with live instances across ALL regions (remove stale; --sync adds missing / repairs drifted)	--sync, --include-ebs, --dry-run, --yes
aws-bootstrap list instance-types	Browse GPU instance types (+ suggested quota commands)	--prefix (default: g4dn), --region/-r (repeatable)
aws-bootstrap list amis	Browse Deep Learning AMIs (region-labelled; IDs differ per region)	--filter, --region/-r (repeatable)
aws-bootstrap quota show	Show GPU vCPU quotas (all families)	--family, --region/-r (repeatable)
aws-bootstrap quota request	Request a quota increase (per region)	--family, --type, --desired-value, --region/-r (repeatable), --yes
aws-bootstrap quota history	Show quota increase request history	--family, --type, --status, --region/-r (repeatable)
aws-bootstrap cluster launch	Launch/grow a multi-node training cluster (1 AZ + placement group)	--cluster-id (required), --nodes, --instance-type, --region, --spot/--on-demand, --wait, --wait-timeout
aws-bootstrap cluster status	Show cluster nodes (rank/state/AZ/IP); omit --cluster-id to list all clusters	--cluster-id, --region
aws-bootstrap cluster prepare	Verify nodes (reachable/GPU/consistent CUDA), write config, run a distributed canary	--cluster-id (required), --key-path, --no-canary
aws-bootstrap cluster test	Re-run the distributed canary across the cluster (heartbeat)	--cluster-id (required), --key-path
aws-bootstrap cluster run	Distribute a training script + run it across the cluster (torchrun)	--cluster-id (required), SCRIPT, -- ARGS, --data-script, --nproc-per-node, --log-dir
aws-bootstrap cluster terminate	Terminate all cluster nodes + delete placement group	--cluster-id (required), --yes

Global options (before the command): --output json|yaml|table|text, --profile, --region

Multi-node clusters (preview)

cluster launch provisions N GPU instances tagged with a shared --cluster-id, all in one AZ inside a cluster placement group, with a self-referencing security-group rule so they can run distributed torchrun jobs. Tags are the source of truth (no state file); re-run cluster launch with a higher --nodes to grow incrementally. Each node gets an aws-<cluster-id>-<rank> SSH alias; rank 0 is the rendezvous/master node. cluster prepare verifies the cluster (reachable, GPU, consistent CUDA — fails fast on skew) and runs a built-in distributed canary (DDP all-reduce + a few SGD steps via torchrun c10d rendezvous on all nodes); cluster test re-runs that canary. cluster run SCRIPT distributes your training script to every node and runs it across the cluster: a .py is launched with torchrun; a .sh runs as-is with the AWSB_* env contract exported (escape hatch). --data-script prep.sh runs once per node (idempotent, before training — recommended: pull data from S3 to /data). Per-node logs land under --log-dir.

aws-bootstrap cluster launch --cluster-id ml1 --nodes 4 --instance-type g5.xlarge --region us-east-1
aws-bootstrap -o json cluster status --cluster-id ml1
aws-bootstrap cluster prepare --cluster-id ml1                          # verify + canary
aws-bootstrap cluster test --cluster-id ml1                             # re-run canary
aws-bootstrap cluster run --cluster-id ml1 --data-script prep.sh train.py -- --epochs 5
aws-bootstrap cluster terminate --cluster-id ml1 --yes

Structured Output

Always use --output json (aliased as -o json) when you need to process results:

# Get instance status as JSON
aws-bootstrap -o json status

# Dry-run launch to see what would happen
aws-bootstrap -o json launch --dry-run

# Terminate with --yes (required in structured output modes)
aws-bootstrap -o json terminate --yes

# All cluster commands support -o json/yaml/table too. Run a job and read
# per-node exit codes programmatically (top-level "succeeded" is the gate):
aws-bootstrap -o json cluster run --cluster-id ml1 train.py -- --epochs 5
# -> {"cluster_id": "ml1", "succeeded": true, "log_dir": "...",
#     "results": [{"rank": 0, "instance_id": "i-0", "returncode": 0}, ...]}

# For spot clusters, use --wait so launch rides out capacity instead of
# failing (it never silently falls back to on-demand):
aws-bootstrap -o json cluster launch --cluster-id ml1 --nodes 2 --wait

Commands requiring confirmation (terminate, cleanup, cluster terminate) must include --yes when using --output json/yaml/table.

Common Workflows

Launch a GPU Instance

# Default: spot g4dn.xlarge (region: env/profile, else us-west-2)
aws-bootstrap launch

# Specify instance type and region
aws-bootstrap launch --instance-type g5.xlarge --region us-east-1

# Try multiple regions in order, then wait (backoff) for spot capacity
aws-bootstrap launch --region us-east-1 --region us-west-2 --wait --wait-timeout 30m

# On-demand pricing (no spot interruption risk)
aws-bootstrap launch --on-demand

# With persistent EBS data volume (survives termination)
aws-bootstrap launch --ebs-storage 96

# Dry run first to validate configuration
aws-bootstrap launch --dry-run

# Custom Python version in remote venv
aws-bootstrap launch --python-version 3.13

# Non-default SSH port
aws-bootstrap launch --ssh-port 2222

After launch, the CLI:

1. Creates the instance (spot with auto-fallback to on-demand)
2. Adds an SSH alias (e.g. aws-gpu1) to ~/.ssh/config
3. Runs remote setup (CUDA-matched PyTorch, Jupyter, GPU benchmark)
4. Mounts EBS volume at /data (if requested)

Check Instance Status

# Human-readable status
aws-bootstrap status

# With GPU info (CUDA toolkit, driver version, GPU name)
aws-bootstrap status --gpu

# Machine-readable
aws-bootstrap -o json status

Connect to an Instance

After launch, use the SSH alias printed in the output:

# Direct SSH (venv auto-activates)
ssh aws-gpu1

# Jupyter tunnel
ssh -NL 8888:localhost:8888 aws-gpu1
# Then open: http://localhost:8888

# VSCode Remote SSH
code --folder-uri vscode-remote://ssh-remote+aws-gpu1/home/ubuntu/workspace

# Run GPU benchmark
ssh aws-gpu1 '~/venv/bin/python ~/gpu_benchmark.py'

Terminate and Clean Up

# Terminate by alias
aws-bootstrap terminate aws-gpu1

# Terminate all instances (with confirmation)
aws-bootstrap terminate

# Terminate but keep EBS volumes for reuse
aws-bootstrap terminate --keep-ebs

# Sync ~/.ssh/config with live instances across ALL regions (remove stale aliases)
aws-bootstrap cleanup

# --sync also adds aliases for live instances missing one and repairs drifted IPs
aws-bootstrap -o json cleanup --sync --yes

# Preview (diagnostic) — reports stale/missing/drifted/orphaned, changes nothing
aws-bootstrap -o json cleanup --sync --include-ebs --dry-run

All four --output modes are supported. For programmatic use prefer -o json/yaml (the full result: regions_queried, regions_failed, cleaned/stale, added, updated, skipped, deleted_volumes/orphan_volumes). -o table renders a single unified action table (Action/Instance/Alias/IP-Volume).

Persistent Data with EBS

# Create a new volume on launch
aws-bootstrap launch --ebs-storage 96

# After terminating with --keep-ebs, reattach to a new instance
aws-bootstrap terminate --keep-ebs
# Note the volume ID from output, then:
aws-bootstrap launch --ebs-volume-id vol-0abc123def456

EBS volumes are mounted at /data, survive spot interruptions, and persist independently of instances. Use /data for large datasets, model checkpoints, and training outputs — it persists across instance lifecycles while the root volume does not.

When you reattach with --ebs-volume-id, the launch automatically pins the new instance to the volume's availability zone (EBS volumes are AZ-scoped, so the instance must be in the same AZ to attach). This means a --ebs-volume-id launch targets the volume's region, and spot capacity is constrained to that one AZ — if you hit InsufficientInstanceCapacity, add --wait to retry until capacity frees up. For example:

# Store training data on persistent volume
ssh aws-gpu1 'mkdir -p /data/datasets /data/checkpoints /data/outputs'

# Download a dataset to persistent storage
ssh aws-gpu1 'cd /data/datasets && wget https://example.com/dataset.tar.gz'

Remote Instance Environment

After launch and remote setup, each instance comes pre-configured with:

Python Virtual Environment (~/venv)

• Located at ~/venv, auto-activated on SSH login (via ~/.bashrc)
• PyTorch is pre-installed with the correct CUDA wheel matching the host's CUDA toolkit version (e.g. cu124, cu128) — torch.cuda.is_available() works out of the box
• torchvision is also pre-installed with matching CUDA support
• Additional numerical/ML libraries from requirements.txt: numpy, tqdm, and other common dependencies
• Use --python-version on launch to pin a specific Python version (e.g. 3.13)
• To install additional packages: ssh aws-gpu1 'pip install transformers datasets'

GPU and CUDA

• NVIDIA drivers and CUDA toolkit are pre-installed via the Deep Learning AMI
• nvidia-smi and nvcc are available on PATH
• A GPU benchmark is pre-installed at ~/gpu_benchmark.py (CNN + Transformer workloads)
• A Jupyter notebook for interactive GPU verification is at ~/gpu_smoke_test.ipynb

Jupyter

• JupyterLab runs as a systemd service on port 8888
• Access via SSH tunnel: ssh -NL 8888:localhost:8888 aws-gpu1, then open http://localhost:8888

EBS Data Volume (/data)

If launched with --ebs-storage or --ebs-volume-id, a persistent gp3 EBS volume is mounted at /data. Use this for:

• Large datasets — download and store training data here so it persists across spot interruptions
• Model checkpoints — save checkpoints to /data/checkpoints to avoid losing training progress
• Training outputs — write logs, metrics, and results to /data/outputs

The /data volume is not lost on spot interruption — when AWS reclaims the instance, the volume detaches automatically and can be reattached to a new instance with --ebs-volume-id.

Error Handling

• Spot capacity errors (InsufficientInstanceCapacity): prefer multiple --region values (tried in order) and/or --wait --wait-timeout over manual retry loops — the CLI handles bounded backoff internally. Without --wait, an exhausted spot pass auto-falls back to on-demand in structured modes. --wait hard-fails on timeout (it does not auto-buy on-demand).
• Quota limits (MaxSpotInstanceCountExceeded, VcpuLimitExceeded) and SpotMaxPriceTooLow: never retried by --wait, but in multi-region mode the launcher warns (with a region-pinned hint) and tries the next --region; it fails hard only when every region is blocked. Quotas are per-region — the suggested aws-bootstrap quota show / quota request commands are auto-pinned to --region <failed-region>. Run them as-is; other families: --family p (P2-P6), --family dl.
• SSH key handling: a missing local --key-path is auto-generated (ed25519); if an AWS key pair with the target --key-name exists but is a different key, the existing one is left untouched and the local key is imported under a derived name <key-name>-<fp8> so the instance is always reachable.
• SSH auth failure: launch fails fast (no long retry loop) and surfaces the real ssh error (e.g. Permission denied (publickey)) instead of a generic "SSH not ready" — indicates a key mismatch; relaunch with the correct --key-name/--key-path.
• SSH timeouts: Instance may still be initializing -- check aws-bootstrap status
• No public IP: Check VPC settings or assign an Elastic IP
• EBS mount failures: Non-fatal -- instance remains usable, may need manual mount

Detailed Command Reference

See commands.md for full option documentation and JSON output schemas.