lightning-cost-estimation

# Lightning AI cost estimation Every machine price on [lightning.ai/pricing](https://lightning.ai/pricing) comes from one public endpoint. **Always fetch prices, capacity and SKU slugs live; never quote them from memory.** They all change, and this file deliberately carries no price snapshot. **1 Lightning credit = $1 USD.** Machines bill **per second** while allocated; storage bills daily. Docs: [billing FAQ](https://lightning.ai/docs/platform/overview/faq/billing.md) · [manage costs](https://lightning.ai/docs/platform/team-management/organizations/manage-costs.md). ## Setup & auth **The accelerator catalog needs no auth**, so plain `curl` works before login: ```bash curl -s "https://lightning.ai/v1/core/accelerators?cloudProvider=MACHINE" | jq '.accelerator | length' ``` `lightning api "<path>"` returns the same JSON; use it when an agent's permission rules block `curl` but allow `lightning`. It always sends credentials, so it needs `lightning login` or an API key. Once signed in, every `curl` recipe below works with `lightning api "<path>"` in its place: ```bash export DEBUG=0 LIGHTNING_DEBUG=0 # an inherited DEBUG=1 logs the Authorization header; set this in every new shell # Use the Lightning AI CLI from the current env; install or upgrade it there if it's missing or older than 2026.9.18 v=$(lightning --version 2>/dev/null | sed -n 's/^Lightning CLI version //p') [ -n "$v" ] && [ "$(printf '%s\n' 2026.9.18 "$v" | sort -V | head -1)" = 2026.9.18 ] \ || uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning api "/v1/core/accelerators?cloudProvider=MACHINE" ``` Auth is only needed to check which clouds a teamspace can reach (`lightning login`, or `LIGHTNING_API_KEY`, optionally with `LIGHTNING_USER_ID`). Every recipe here needs `jq`. This uses, and if needed installs into, the agent's current environment (project venv, conda env, …); then call plain `lightning …` everywhere. If setup doesn't go cleanly: | Symptom | Fix | |---|---| | No active env, `externally-managed-environment`, or a conflict with the project's own pins | Install it on its own: `uv tool install lightning-sdk` (or `pipx install lightning-sdk`), and call `"$(uv tool dir --bin)/lightning"` if that isn't on `PATH` | | Still no `Lightning CLI version` line after installing | Another `lightning` (PyTorch Lightning's) or an inactive env comes first: check `command -v lightning`, then activate the env or call its `bin/lightning` | | `No such command`, `No such option` or `unexpected extra argument` | The CLI is too old: upgrade it where `command -v lightning` points (`uv pip install -U lightning-sdk`, or `uv tool upgrade lightning-sdk`) | | Installing is blocked (read-only env or home, agent sandbox) | Run each command as `UV_CACHE_DIR="${TMPDIR:-/tmp}/uv" UV_TOOL_DIR="${TMPDIR:-/tmp}/uv-tools" uvx lightning-sdk …` | | SSL/CA errors on every call, even `lightning --version`, only inside an agent sandbox | A `**/*.pem` read-deny rule hides certifi's `cacert.pem`: allow that one path; don't disable the sandbox | | `NameResolutionError`, only inside an agent sandbox | If `lightning api` fails too, ask the user to allow `lightning.ai` and `*.lightning.ai` (`/sandbox`). If only `studio`/`job` commands or the SDK fail, they bypass the sandbox proxy: run just those outside it. Background pollers fail the same way, silently | ## Resolving org and teamspace (only for availability checks) Pricing is global: you **don't** need a teamspace to quote. You need one to check which cloud accounts the user can launch on. **Never guess it**; resolve it, and ask if it's ambiguous: ```bash lightning api /v1/memberships | jq -r '.memberships[] | [.ownerType, .name, .projectId] | @tsv' PID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId' | head -1) lightning api "/v1/projects/$PID/clusters" | jq -r '.clusters[].id' ``` Map the cluster ids to catalog providers with the *Cloud providers* table. Don't quote a cloud the teamspace can't launch on as the answer; mention it only as "available if you enable that cloud account". ## The accelerator catalog `GET https://lightning.ai/v1/core/accelerators?cloudProvider=<PROVIDER>` → `{"accelerator": [ ... ]}`. The fields that matter for costing: | Field | Meaning | |---|---| | `cost` | **On-demand USD per hour for the whole instance, not per GPU.** `36` on an 8×H100 SKU is $4.50/GPU-hr; quoting it per GPU inflates an estimate 8×. | | `spotPrice` | Interruptible USD per hour. **`0` means the provider has no spot**, not free. Spot is **not always cheaper** than `cost`: take `min(cost, spotPrice)` over the non-zero values and say which you quoted. | | `family`, `resources.gpu`, `resources.gpuType` | The hardware: `H100` × `8` × `nvidia-h100-80gb`. **Compare clouds on these, not on slugs** (see below). `gpuType` sometimes omits VRAM (`nvidia-b200`); use the VRAM table. The SDK spells the RTX family `RTX PRO`, the catalog `RTXP`. | | `slugMultiCloud` / `slug` / `instanceId` | SKU names. They differ between clouds for the same hardware. | | `resources.memoryMb`, `.cpu`, `.storageGb` | Host RAM / vCPU / disk. Occasionally wrong (one AWS H200 SKU reported `141000000` MB); sanity-check RAM before quoting it. | | `outOfCapacity`, `availableInSeconds` (`…Spot`) | Whether you can get it now, and the expected wait. **A cheap SKU that is out of capacity, or waits 10⁴–10⁶ s, is not a real quote.** | | `dwsOnly` / `dwsCost` | GCP Dynamic Workload Scheduler: queued batch allocation, not on-demand, so not for an always-on endpoint. | | `reservable` / `capacityBlockPrice` | AWS capacity blocks: a much cheaper reserved rate for long runs. The discount varies widely by SKU, so read it per SKU and surface it for runs measured in weeks. | | `isTierRestricted` | Needs a paid plan tier; a free-tier account can't launch it at any price. | | `enabled`, `availableZones`, `clusterId` | Whether it's live, where, and which cloud account it belongs to. | ### Cloud providers | `cloudProvider=` | What it is | Cluster id (for `--cloud`) | Notes | |---|---|---|---| | `MACHINE` | **Lightning Cloud** (Lightning's own bare metal) | `lightning-baremetal` | The default cloud. Not the SDK's `LIGHTNING`, which is a placeholder here. | | `AWS` | AWS | `lightning-public-prod` | Widest SKU range; capacity blocks. | | `GCP` | GCP | `gcp-lightning-public-prod` | Only source of TPUs; some GPUs `dwsOnly`. | | `LAMBDA_LABS` | Lambda Labs | `lightning-lambda-prod` | | | `NEBIUS` | Nebius | `lightning-nebius-prod` | | | `VOLTAGE_PARK` | Voltage Park | `lightning-voltagepark-prod` | | | `VULTR` | Vultr | — | Niche SKUs (A16/A40/A100/L40S). | Quote Lightning Cloud first, then compare the others live, and recommend another cloud only when it wins on price **and** is in capacity for the user's teamspace. ### From a quote to a launch Launch with the `--machine` name (`Machine.<NAME>` in Python) used by `lightning-jobs`, `lightning-studios` and `lightning-deployments`, plus `--cloud <cluster-id>` for the cloud you priced; without `--cloud` the run lands on the teamspace default and may bill a different rate. | `--machine` | Catalog hardware | | `--machine` | Catalog hardware | |---|---|---|---|---| | `CPU_SMALL` / `CPU` / `CPU_X_8` / `CPU_X_16` | `cpu-2` / `cpu-4` / `cpu-8` / `cpu-16` | | `A100` … `A100_X_8` | `A100` × N, 40 or 80 GB by cloud | | `DATA_PREP` / `_MAX` / `_ULTRA` | `data-prep-mid` / `-max-large` / `-ultra-extra-large` | | `H100` … `H100_X_8` | `H100` × N | | `T4` / `L4` / `L40S` (`_X_N`) | `T4` / `L4` / `L40S` × N | | `H200` / `B200` (`_X_N`) | `H200` / `B200` × N | | `T4_SMALL` | small T4 | | `RTXP_6000` (`_X_N`) | `RTXP` × N | `A100_40GB*` / `A100_80GB*` pin the memory size but are hidden from CLI `--machine`; they work only as `Machine.A100_80GB_X_8` in Python or with `studio switch`. **Slugs differ between clouds for the same hardware, so never hardcode or join on one.** H200 is `lit-h200-1` on Nebius and `lit-h200-141gb-1` on Lightning Cloud, and `Machine.H100.slug` is `lit-h100-1` while the catalog says `lit-h100-80gb-1`. The SDK (2026.9.18+) maps all of these to the same machine. Match on `family` + `resources.gpu` (+ VRAM from `gpuType`), as the recipes below do; filtering on an assumed slug makes a machine that exists look unavailable. ## Price lookup recipes Load these into the shell once (bash or zsh), then use them for every quote. ```bash # _litfetch <PROVIDER> — catalog JSON, or an error and a non-zero exit _litfetch() { curl -sf --retry 3 --retry-delay 1 --max-time 20 \ "https://lightning.ai/v1/core/accelerators?cloudProvider=$1" \ || { echo "ERROR: could not fetch the $1 catalog; don't quote from a partial result" >&2; return 1; } } _LITJQ='def spot: if .spotPrice > 0 then (.spotPrice*100|round/100) else "n/a" end; def count: if .acceleratorType=="GPU" then .resources.gpu else .resources.cpu end; def status: [ (if .outOfCapacity then "OUT-OF-CAPACITY" else empty end), (if .dwsOnly then "DWS-ONLY" else empty end), (if .isTierRestricted then "TIER-RESTRICTED" else empty end), (if .reservable then "RESERVABLE" else empty end) ] | if length==0 then "ok" else join(",") end;' # litprice <PROVIDER> [family-regex] — one cloud's live catalog, one row per SKU litprice() { local json; json=$(_litfetch "${1:-MACHINE}") || return 1 printf '%s' "$json" | jq -r --arg re "${2:-.}" "$_LITJQ"' ["SKU","FAMILY","N","USD/HR","SPOT/HR","RAM_GB","WAIT_S","STATUS"], (.accelerator[] | select(.family|test($re;"i")) | [.slugMultiCloud, .family, count, .cost, spot, ((.resources.memoryMb|tonumber)/1024|round), .availableInSeconds, status]) | @tsv' | column -t -s $'\t' } # litcompare <family> <gpus> [gpuType-regex] — the same hardware on every cloud, cheapest first. # Stops if any cloud's catalog can't be fetched: a missing cloud could hide the real winner. litcompare() { local p json rows="" for p in MACHINE AWS GCP LAMBDA_LABS NEBIUS VOLTAGE_PARK VULTR; do json=$(_litfetch "$p") || return 1 rows+=$(printf '%s' "$json" | jq -r --arg p "$p" --arg f "$1" --argjson n "$2" --arg m "${3:-.}" "$_LITJQ"' .accelerator[] | select((.family|ascii_upcase)==($f|ascii_upcase) and .resources.gpu==$n and ((.resources.gpuType // "")|test($m;"i"))) | [$p, .slugMultiCloud, (.resources.gpuType // "?"), .cost, spot, .availableInSeconds, status] | @tsv')$'\n' || return 1 done { printf 'CLOUD\tSLUG\tGPU\tUSD/HR\tSPOT/HR\tWAIT_S\tSTATUS\n' printf '%s' "$rows" | grep . | sort -t $'\t' -k4,4 -g; } | column -t -s $'\t' } # litquote <usd_per_hour> <instances> <hours> [storage_gb] [storage_days] litquote() { jq -n --argjson r "$1" --argjson n "$2" --argjson h "$3" \ --argjson gb "${4:-0}" --argjson d "${5:-0}" ' ($r*$n*$h) as $c | (([$gb-10,0]|max) * 0.10 * ($d/30)) as $s | { rate_per_hour: $r, instances: $n, hours: $h, cluster_per_hour: ($r*$n*100|round/100), compute_usd: ($c*100|round/100), storage_usd: ($s*100|round/100), total_usd: (($c+$s)*100|round/100) }' } ``` ```bash litprice MACHINE # everything on Lightning Cloud litprice GCP 'B200|H200' # just those families on GCP litcompare H100 8 # every cloud's 8×H100, cheapest first litcompare A100 1 80gb # 1×A100, 80 GB variants only litquote 36 32 336 # 32 nodes × a $36/hr rate × 2 weeks ``` ## Cost formulas