modal-knowledge

name

modal-knowledge

description

Modal.com serverless compute platform knowledge. PROACTIVELY activate for: (1) Modal app/function definitions (modal.App, @app.function, @app.cls), (2) running ML/AI workloads serverlessly, (3) GPU configuration (T4, L4, A10G, A100, H100, H200), (4) Modal volumes and shared state, (5) Modal Image builder (apt, pip, run_commands), (6) Modal secrets and environment management, (7) web endpoints (FastAPI, asgi_app, wsgi_app), (8) cron schedules and continuous deployments, (9) Modal sandboxes, (10) cost optimization on Modal (autoscaling, spot, idle timeout), (11) Modal CLI (modal deploy, modal run, modal serve). Provides: app templates, GPU sizing matrix, Image builder recipes, secret patterns, web endpoint examples, and cost-optimization guidance.

Modal Knowledge Skill

Comprehensive Modal.com platform knowledge covering all features, pricing, and best practices. Activate this skill when users need detailed information about Modal's serverless cloud platform.

Activation Triggers

Activate this skill when users ask about:

• Modal.com platform features and capabilities
• GPU-accelerated Python functions
• Serverless container configuration
• Modal pricing and billing
• Modal CLI commands
• Web endpoints and APIs on Modal
• Scheduled/cron jobs on Modal
• Modal volumes, secrets, and storage
• Parallel processing with Modal
• Modal deployment and CI/CD

---

Platform Overview

Modal is a serverless cloud platform for running Python code, optimized for AI/ML workloads with:

• Zero Configuration: Everything defined in Python code
• Fast GPU Startup: ~1 second container spin-up
• Automatic Scaling: Scale to zero, scale to thousands
• Per-Second Billing: Only pay for active compute
• Multi-Cloud: AWS, GCP, Oracle Cloud Infrastructure

---

Core Components Reference

Apps and Functions

import modal

app = modal.App("app-name")

@app.function()
def basic_function(arg: str) -> str:
    return f"Result: {arg}"

@app.local_entrypoint()
def main():
    result = basic_function.remote("test")
    print(result)

Function Decorator Parameters

Parameter	Type	Description
image	Image	Container image configuration
gpu	str/list	GPU type(s): "T4", "A100", ["H100", "A100"]
cpu	float	CPU cores (0.125 to 64)
memory	int	Memory in MB (128 to 262144)
timeout	int	Max execution seconds
retries	int	Retry attempts on failure
secrets	list	Secrets to inject
volumes	dict	Volume mount points
schedule	Cron/Period	Scheduled execution
concurrency_limit	int	Max concurrent executions
container_idle_timeout	int	Seconds to keep warm
include_source	bool	Auto-sync source code

---

GPU Reference

Available GPUs

GPU	Memory	Use Case	~Cost/hr
T4	16 GB	Small inference	$0.59
L4	24 GB	Medium inference	$0.80
A10G	24 GB	Inference/fine-tuning	$1.10
L40S	48 GB	Heavy inference	$1.50
A100-40GB	40 GB	Training	$2.00
A100-80GB	80 GB	Large models	$3.00
H100	80 GB	Cutting-edge	$5.00
H200	141 GB	Largest models	$5.00
B200	180+ GB	Latest gen	$6.25

GPU Configuration

# Single GPU
@app.function(gpu="A100")

# Specific memory variant
@app.function(gpu="A100-80GB")

# Multi-GPU
@app.function(gpu="H100:4")

# Fallbacks (tries in order)
@app.function(gpu=["H100", "A100", "any"])

# "any" = L4, A10G, or T4
@app.function(gpu="any")

---

Image Building

Base Images

# Debian slim (recommended)
modal.Image.debian_slim(python_version="3.11")

# From Dockerfile
modal.Image.from_dockerfile("./Dockerfile")

# From Docker registry
modal.Image.from_registry("nvidia/cuda:12.1.0-base-ubuntu22.04")

Package Installation

# pip (standard)
image.pip_install("torch", "transformers")

# uv (FASTER - 10-100x)
image.uv_pip_install("torch", "transformers")

# System packages
image.apt_install("ffmpeg", "libsm6")

# Shell commands
image.run_commands("apt-get update", "make install")

Adding Files

# Single file
image.add_local_file("./config.json", "/app/config.json")

# Directory
image.add_local_dir("./models", "/app/models")

# Python source
image.add_local_python_source("my_module")

# Environment variables
image.env({"VAR": "value"})

Build-Time Function

def download_model():
    from huggingface_hub import snapshot_download
    snapshot_download("model-name")

image.run_function(download_model, secrets=[...])

---

Storage

Volumes

# Create/reference volume
vol = modal.Volume.from_name("my-vol", create_if_missing=True)

# Mount in function
@app.function(volumes={"/data": vol})
def func():
    # Read/write to /data
    vol.commit()  # Persist changes

Secrets

# From dashboard (recommended)
modal.Secret.from_name("secret-name")

# From dictionary
modal.Secret.from_dict({"KEY": "value"})

# From local env
modal.Secret.from_local_environ(["KEY1", "KEY2"])

# From .env file
modal.Secret.from_dotenv()

# Usage
@app.function(secrets=[modal.Secret.from_name("api-keys")])
def func():
    import os
    key = os.environ["API_KEY"]

Dict and Queue

# Distributed dict
d = modal.Dict.from_name("cache", create_if_missing=True)
d["key"] = "value"
d.put("key", "value", ttl=3600)

# Distributed queue
q = modal.Queue.from_name("jobs", create_if_missing=True)
q.put("task")
item = q.get()

---

Web Endpoints

FastAPI Endpoint (Simple)

@app.function()
@modal.fastapi_endpoint()
def hello(name: str = "World"):
    return {"message": f"Hello, {name}!"}

ASGI App (Full FastAPI)

from fastapi import FastAPI
web_app = FastAPI()

@web_app.post("/predict")
def predict(text: str):
    return {"result": process(text)}

@app.function()
@modal.asgi_app()
def fastapi_app():
    return web_app

WSGI App (Flask)

from flask import Flask
flask_app = Flask(__name__)

@app.function()
@modal.wsgi_app()
def flask_endpoint():
    return flask_app

Custom Web Server

@app.function()
@modal.web_server(port=8000)
def custom_server():
    subprocess.run(["python", "-m", "http.server", "8000"])

Custom Domains

@modal.asgi_app(custom_domains=["api.example.com"])

---

Scheduling

Cron

# Daily at 8 AM UTC
@app.function(schedule=modal.Cron("0 8 * * *"))

# With timezone
@app.function(schedule=modal.Cron("0 6 * * *", timezone="America/New_York"))

Period

@app.function(schedule=modal.Period(hours=5))
@app.function(schedule=modal.Period(days=1))

Note: Scheduled functions only run with modal deploy, not modal run.

---

Parallel Processing

Map

# Parallel execution (up to 1000 concurrent)
results = list(func.map(items))

# Unordered (faster)
results = list(func.map(items, order_outputs=False))

Starmap

# Spread args
pairs = [(1, 2), (3, 4)]
results = list(add.starmap(pairs))

Spawn

# Async job (returns immediately)
call = func.spawn(data)
result = call.get()  # Get result later

# Spawn many
calls = [func.spawn(item) for item in items]
results = [call.get() for call in calls]

---

Container Lifecycle (Classes)

@app.cls(gpu="A100", container_idle_timeout=300)
class Server:

    @modal.enter()
    def load(self):
        self.model = load_model()

    @modal.method()
    def predict(self, text):
        return self.model(text)

    @modal.exit()
    def cleanup(self):
        del self.model

Concurrency

@modal.concurrent(max_inputs=100, target_inputs=80)
@modal.method()
def batched(self, item):
    pass

---

CLI Commands

Development

modal run app.py              # Run function
modal serve app.py            # Hot-reload dev server
modal shell app.py            # Interactive shell
modal shell app.py --gpu A100 # Shell with GPU

Deployment

modal deploy app.py           # Deploy
modal app list                # List apps
modal app logs app-name       # View logs
modal app stop app-name       # Stop app

Resources

# Volumes
modal volume create name
modal volume list
modal volume put name local remote
modal volume get name remote local

# Secrets
modal secret create name KEY=value
modal secret list

# Environments
modal environment create staging

---

Pricing (2025)

Plans

Plan	Price	Containers	GPU Concurrency
Starter	Free ($30 credits)	100	10
Team	$250/month	1000	50
Enterprise	Custom	Unlimited	Custom

Compute

• CPU: $0.0000131/core/sec
• Memory: $0.00000222/GiB/sec
• GPUs: See GPU table above

Special Programs

• Startups: Up to $25k credits
• Researchers: Up to $10k credits

---

Best Practices

1. Use @modal.enter() for model loading
2. Use uv_pip_install for faster builds
3. Use GPU fallbacks for availability
4. Set appropriate timeouts and retries
5. Use environments (dev/staging/prod)
6. Download models during build, not runtime
7. Use order_outputs=False when order doesn't matter
8. Set container_idle_timeout to balance cost/latency
9. Monitor costs in Modal dashboard
10. Test with modal run before modal deploy

---

Common Patterns

LLM Inference

@app.cls(gpu="A100", container_idle_timeout=300)
class LLM:
    @modal.enter()
    def load(self):
        from vllm import LLM
        self.llm = LLM(model="...")

    @modal.method()
    def generate(self, prompt):
        return self.llm.generate([prompt])

Batch Processing

@app.function(volumes={"/data": vol})
def process(file):
    # Process file
    vol.commit()

# Parallel
results = list(process.map(files))

Scheduled ETL

@app.function(
    schedule=modal.Cron("0 6 * * *"),
    secrets=[modal.Secret.from_name("db")]
)
def daily_etl():
    extract()
    transform()
    load()

---

Quick Reference

Task	Code
Create app	app = modal.App("name")
Basic function	@app.function()
With GPU	@app.function(gpu="A100")
With image	@app.function(image=img)
Web endpoint	@modal.asgi_app()
Scheduled	schedule=modal.Cron("...")
Mount volume	volumes={"/path": vol}
Use secret	secrets=[modal.Secret.from_name("x")]
Parallel map	func.map(items)
Async spawn	func.spawn(arg)
Class pattern	@app.cls() with @modal.enter()