aws-cli

name	aws-cli
description	Manage AWS services for ML workflows including S3, ECR, SageMaker, and EC2
version	1.0.0
author	kai-agent
metadata	{"kai":{"tags":["kai","compute","aws","s3","ecr","sagemaker","ec2","ml","inference"]}}

AWS CLI for ML Infrastructure

Skill for managing AWS services commonly used in ML and inference workflows: S3 for storage, ECR for container images, SageMaker for managed inference, and EC2 for GPU instances.

Authentication

This skill uses the following environment variables:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_DEFAULT_REGION (e.g., us-east-1)

If they are not set, ask your admin to add them in Agent Settings.

Optionally, AWS_SESSION_TOKEN is used for temporary credentials (STS/SSO).

Install

# AWS CLI v2
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip && sudo ./aws/install

# Python SDK
pip install boto3

S3 (Model and dataset storage)

Upload a model or dataset

# Upload a directory
aws s3 sync ./model-weights/ s3://your-bucket/models/llama-3.1-8b-awq/

# Upload a single file
aws s3 cp ./benchmark_results.json s3://your-bucket/results/benchmark_results.json

Download a model or dataset

# Download a directory
aws s3 sync s3://your-bucket/models/llama-3.1-8b-awq/ ./model-weights/

# Download a single file
aws s3 cp s3://your-bucket/datasets/calibration.jsonl ./calibration.jsonl

List objects

aws s3 ls s3://your-bucket/models/ --recursive --human-readable

Multipart upload for large files (Python)

import boto3

s3 = boto3.client("s3")

# Automatic multipart upload for large files
from boto3.s3.transfer import TransferConfig

config = TransferConfig(
    multipart_threshold=1024 * 1024 * 100,  # 100 MB
    max_concurrency=10,
    multipart_chunksize=1024 * 1024 * 100,
)

s3.upload_file(
    "./model.safetensors",
    "your-bucket",
    "models/llama-3.1-8b/model.safetensors",
    Config=config,
)

Presigned URLs (share without credentials)

aws s3 presign s3://your-bucket/models/llama-3.1-8b-awq/model.safetensors --expires-in 3600

url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "your-bucket", "Key": "models/model.safetensors"},
    ExpiresIn=3600,
)

Lifecycle rules (auto-cleanup)

# Delete objects older than 30 days in the tmp/ prefix
aws s3api put-bucket-lifecycle-configuration \
    --bucket your-bucket \
    --lifecycle-configuration '{
        "Rules": [{
            "ID": "cleanup-tmp",
            "Prefix": "tmp/",
            "Status": "Enabled",
            "Expiration": {"Days": 30}
        }]
    }'

ECR (Container Registry)

Create a repository

aws ecr create-repository --repository-name inference-server --region us-east-1

Build, tag, and push an image

# Authenticate Docker to ECR
aws ecr get-login-password --region us-east-1 | \
    docker login --username AWS --password-stdin <account-id>.dkr.ecr.us-east-1.amazonaws.com

# Build
docker build -t inference-server:latest .

# Tag
docker tag inference-server:latest \
    <account-id>.dkr.ecr.us-east-1.amazonaws.com/inference-server:latest

# Push
docker push <account-id>.dkr.ecr.us-east-1.amazonaws.com/inference-server:latest

List images

aws ecr list-images --repository-name inference-server

Pull an image

docker pull <account-id>.dkr.ecr.us-east-1.amazonaws.com/inference-server:latest

SageMaker (Managed inference)

Deploy a real-time endpoint

import boto3

sagemaker = boto3.client("sagemaker")

# Create model
sagemaker.create_model(
    ModelName="llama3-8b-vllm",
    PrimaryContainer={
        "Image": "<account-id>.dkr.ecr.us-east-1.amazonaws.com/inference-server:latest",
        "ModelDataUrl": "s3://your-bucket/models/llama-3.1-8b-awq/model.tar.gz",
        "Environment": {
            "MODEL_NAME": "meta-llama/Llama-3.1-8B-Instruct",
            "QUANTIZATION": "awq",
        },
    },
    ExecutionRoleArn="arn:aws:iam::<account-id>:role/SageMakerRole",
)

# Create endpoint config
sagemaker.create_endpoint_config(
    EndpointConfigName="llama3-8b-config",
    ProductionVariants=[{
        "VariantName": "primary",
        "ModelName": "llama3-8b-vllm",
        "InstanceType": "ml.g5.2xlarge",
        "InitialInstanceCount": 1,
    }],
)

# Create endpoint
sagemaker.create_endpoint(
    EndpointName="llama3-8b-endpoint",
    EndpointConfigName="llama3-8b-config",
)

Invoke an endpoint

runtime = boto3.client("sagemaker-runtime")

response = runtime.invoke_endpoint(
    EndpointName="llama3-8b-endpoint",
    ContentType="application/json",
    Body='{"prompt": "Explain batching:", "max_tokens": 256}',
)
print(response["Body"].read().decode())

List endpoints

aws sagemaker list-endpoints --status-equals InService

Delete an endpoint (stop billing)

aws sagemaker delete-endpoint --endpoint-name llama3-8b-endpoint
aws sagemaker delete-endpoint-config --endpoint-config-name llama3-8b-config
aws sagemaker delete-model --model-name llama3-8b-vllm

SageMaker instance types for inference

Instance	GPU	VRAM	Use case
ml.g5.xlarge	1x A10G	24 GB	Quantized models up to 13B
ml.g5.2xlarge	1x A10G	24 GB	Same GPU, more CPU/RAM
ml.g5.12xlarge	4x A10G	96 GB	Multi-GPU, 70B quantized
ml.p4d.24xlarge	8x A100 (40 GB)	320 GB	Large models, high throughput
ml.p5.48xlarge	8x H100 (80 GB)	640 GB	Maximum performance

EC2 (GPU instances)

Launch a GPU instance

aws ec2 run-instances \
    --image-id ami-0abcdef1234567890 \
    --instance-type g5.xlarge \
    --key-name your-key \
    --security-group-ids sg-xxxxx \
    --subnet-id subnet-xxxxx \
    --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":200}}]' \
    --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=inference-dev}]'

List running GPU instances

aws ec2 describe-instances \
    --filters "Name=instance-type,Values=g5.*,p4d.*,p5.*" \
              "Name=instance-state-name,Values=running" \
    --query 'Reservations[*].Instances[*].[InstanceId,InstanceType,LaunchTime,PublicIpAddress]' \
    --output table

Stop / start / terminate

aws ec2 stop-instances --instance-ids i-xxxxx
aws ec2 start-instances --instance-ids i-xxxxx
aws ec2 terminate-instances --instance-ids i-xxxxx

GPU instance types

Instance	GPU	VRAM	On-demand $/hr (approx)
g5.xlarge	1x A10G	24 GB	$1.01
g5.12xlarge	4x A10G	96 GB	$5.67
g6.xlarge	1x L4	24 GB	$0.80
p4d.24xlarge	8x A100 (40 GB)	320 GB	$32.77
p5.48xlarge	8x H100 (80 GB)	640 GB	$98.32

Use Spot Instances for cost savings

aws ec2 run-instances \
    --instance-type g5.xlarge \
    --instance-market-options '{"MarketType":"spot","SpotOptions":{"MaxPrice":"0.50"}}' \
    --image-id ami-0abcdef1234567890 \
    --key-name your-key

Spot instances cost 60-90% less but can be interrupted. Good for benchmarking and batch inference, not production serving.

Python (boto3) patterns

Check credentials are working

import boto3

sts = boto3.client("sts")
identity = sts.get_caller_identity()
print(f"Account: {identity['Account']}, User: {identity['Arn']}")

List S3 buckets

s3 = boto3.client("s3")
buckets = s3.list_buckets()
for b in buckets["Buckets"]:
    print(b["Name"])

Inference optimization patterns

1. S3 as model registry: Store model artifacts in S3 with versioned keys (e.g., models/llama3-8b/v2/model.safetensors).
2. ECR for reproducible serving: Package your inference server (vLLM, TGI) as a Docker image in ECR for consistent deployments.
3. Spot instances for benchmarking: Run inference benchmarks on spot g5/p4d instances at 60-90% discount.
4. SageMaker auto-scaling: Configure auto-scaling policies based on InvocationsPerInstance to handle traffic spikes.
5. S3 Transfer Acceleration: Enable for cross-region model uploads/downloads.
6. VPC endpoints for S3: Avoid data transfer charges and reduce latency by accessing S3 through a VPC endpoint.

Rules

1. ALWAYS delete SageMaker endpoints when done: They bill per hour even with zero traffic
2. ALWAYS terminate or stop EC2 GPU instances after use
3. Use lifecycle rules on S3 to auto-delete temporary artifacts
4. Never put credentials in code or Docker images: use environment variables or IAM roles
5. Tag all resources with project and owner for cost tracking
6. Prefer Spot for non-production workloads to reduce costs significantly