| name | desi |
| description | Covers the DESI spectroscopic survey — data products, environment variables, production releases (mountain names), data layout under $DESI_SPECTRO_REDUX, Python I/O with desispec, and DESI-specific compute conventions. Most DESI runtime tools (desimodules, desispec, redrock, $SPECPROD, $DESI_ROOT) require NERSC; combine with the nersc skill for job submission and cluster setup. Use this skill whenever working with DESI data or code, importing desispec/redrock/fiberassign, setting $SPECPROD, reading DESI catalogs or spectra, or asking about DESI survey data products. |
DESI spectroscopic survey
DESI is a DOE spectroscopic survey instrument at KPNO; the data and software infrastructure lives at NERSC.
Runtime tools (desimodules, desispec, redrock, etc.) require NERSC — for job submission and cluster setup invoke /nersc.
Conceptual questions, data layout, and Python I/O patterns work anywhere.
Loading the software stack (NERSC only)
module load desimodules/master
module load desimodules/24.4
desimodules sets up desispec, desisim, redrock, fiberassign, desisurvey, specter, gpu_specter, simqso, speclite, prospect, desimeter, plus all DESI_* env vars.
Key environment variables
| Var | Value / meaning |
|---|
DESI_ROOT | /global/cfs/cdirs/desi |
DESI_SPECTRO_DATA | Raw nightly data, NIGHT/EXPID/... |
DESI_SPECTRO_REDUX | One subdir per SPECPROD |
DESI_TARGET | Targeting / fiberassign inputs |
DESI_SURVEYOPS | Survey ops trunk |
SPECPROD | Current production label — set per task |
export SPECPROD=matterhorn
export SPECPROD=loa
Productions ("mountain releases")
| Release | Public | Notes |
|---|
daily | — | Rolling pipeline output, never frozen |
fuji | EDR | Early Data Release |
guadalupe | DR1 | |
himalayas | (internal) | Frozen, never went public |
iron | DR1 | |
jura | (internal) | Frozen, never went public |
kibo | DR2 | |
loa | DR2 | |
matterhorn | (current internal) | Active as of 2026-04 |
Verify before citing — releases get promoted and the frontier moves.
At NERSC: scripts/desi_prods.sh (bundled with this skill) identifies the current frontier from live symlinks.
Data layout under $DESI_SPECTRO_REDUX/<SPECPROD>/
exposures/NIGHT/EXPID/ — frame, sframe, cframe, sky, fiberflat per exposure.tiles/cumulative/TILEID/LASTNIGHT/ — coadded spectra and redshifts per tile (most commonly used).tiles/pernight/TILEID/NIGHT/ — per-night summaries.healpix/<survey>/<program>/<hp//100>/<hp>/ — healpix-coadded spectra; better for sample-level work crossing tile boundaries.zcatalog/ — concatenated redshift catalogs across tiles/healpix.- Top-level indices:
tilepix.fits, exposures-<SPECPROD>.fits, tiles-<SPECPROD>.fits.
Authoritative HDU/column/bitmask reference: https://desidatamodel.readthedocs.io/en/latest/
Reading DESI files in Python
import desispec.io as dio
spectra = dio.read_spectra("coadd-TILEID-thru20240101.fits")
zbest = dio.read_zbest("redrock-TILEID-thru20240101.fits")
frame = dio.read_frame("frame-b0-00012345.fits")
desispec.io handles the multi-arm (B/R/Z) coadd structure, mask propagation, and resolution-matrix sparse storage.
For large zcatalog tables, prefer fitsio over astropy.io.fits — significantly faster on Lustre.
Output conventions
- Sims and intermediate I/O:
$PSCRATCH/desi/<project>/... — big budget, will be purged.
- Outputs to keep:
$DESI_ROOT/users/$USER/... — persistent, group-writable; don't park TBs without coordinating with ops.
- Collaboration catalogs: use the formal VAC submission process.
Compute repos and GPU tools (NERSC)
desi — CPU jobs. desi_g — GPU jobs (required for gpu_specter, redrock GPU mode).gpu_specter replaces CPU specter for spectral extraction; activated automatically in production scripts on -C gpu nodes.redrock has a CUDA backend (~5–10× speedup). Use desi_zproc --gpu.
Useful entry points
desi_proc_dashboard — pipeline status across recent nights.desi_zcatalog — build/inspect concatenated zcatalogs.desi_quicklook — fast on-the-fly QA.prospect_pages — HTML viewers for Redrock fits.
Deep reference
Full loading instructions, env vars, productions table, file layout details, Python I/O patterns, GPU pipeline, and output conventions: references/desi-at-nersc.md
Links
