desc

name

desc

description

Ecosystem-level guide to the DESC (Dark Energy Science Collaboration) Python stack. Covers which tools exist, how they fit together, general DESC coding conventions, and how to set up the shared DESC Python environment and Jupyter kernels at NERSC. Use this skill when the user is new to DESC or asks what DESC software exists; asks how DESC packages fit together; asks about cross-package data flow (e.g. TXPipe → sacc → Firecrown, or RAIL inside TXPipe); asks about general DESC coding conventions; needs to choose the right tool for a DESC analysis task; asks how to activate or install the DESC Python environment; asks about desc-python, desc-python-dev, desc-python-bleed, or other DESC Jupyter kernels; or asks a question spanning multiple DESC packages with no single obvious skill to invoke. For questions specific to a single package (ceci, TXPipe, RAIL, pyccl/CCL, pymaster/NaMaster, firecrown, photerr), prefer that package's dedicated skill instead.

LSST DESC Python ecosystem

You're helping a user write DESC science code. DESC (Dark Energy Science Collaboration) is the primary LSST weak-lensing / large-scale-structure science collaboration. Most work happens at NERSC (see the nersc skill for cluster conventions); this skill focuses on the DESC software stack itself.

For package-specific work, invoke the dedicated skill.

---

Package skills

Package	Purpose	Skill
ceci	Pipeline framework: stage definition, YAML pipelines, MPI, file I/O	/ceci
TXPipe	Main DESC 3×2pt pipeline: selection → maps → 2pt → covariance → theory	/txpipe
RAIL	Photo-z framework: creation (catalog simulation), estimation, evaluation	/rail
pyccl / CCL	Core Cosmology Library: theory predictions, tracers, angular/3D power spectra	/ccl
pymaster / NaMaster	Pseudo-Cl estimator: NmtField/Workspace, mode-coupling, Gaussian covariance	/namaster
firecrown	Likelihood framework: statistics, systematics, sacc I/O, CosmoSIS/NumCosmo connectors	/firecrown
photerr	Photometric error modeling for LSST, Euclid, and Roman; LsstErrorModel/EuclidErrorModel/RomanErrorModel	/photerr

---

How the packages fit together

TXPipe and RAIL are both built on top of ceci — all their stages are ceci PipelineStage subclasses. RAIL extends ceci with a DataStore/DataHandle pattern for managing in-memory data across stages. TXPipe uses RAIL stages for its photo-z steps (PZEstimatorLens/Source, PZRailSummarizeLens/Source).

TXPipe uses NaMaster (pymaster) for Fourier-space power spectrum estimation and CCL (pyccl) for theory predictions in its theory stages. TXPipe outputs sacc files that Firecrown consumes for cosmological parameter inference; some TXPipe theory stages call Firecrown's load_likelihood_from_script directly.

A typical DESC analysis pipeline flows: truth simulation → RAIL creation (degrade photometry with photerr) → RAIL estimation (photo-z) → TXPipe (2pt statistics) → Firecrown (parameter inference).

---

General DESC coding conventions

• DESC packages are Python 3; most require Python ≥ 3.9.
• Prefer the official DESC I/O helpers over rolling your own FITS/HDF5 reads — they handle unit conventions and multi-file layouts correctly.
• When in doubt about units, check the package docs: DESC packages are not always consistent with each other on angular units (radians vs arcmin vs degrees).
• For catalog-scale work (millions of objects), use numpy / fitsio / h5py over astropy table reads — they're dramatically faster on NERSC Lustre.
• DESC repos live on GitHub under the LSSTDESC organization: https://github.com/LSSTDESC.

---

DESC Python environment at NERSC

See desc-python-env.md for the full reference. Key facts:

• Activate (command line): source /global/common/software/lsst/common/miniconda/setup_current_python.sh
• Add Jupyter kernels (one-time): source /global/common/software/lsst/common/miniconda/kernels/setup.sh
• Requires membership in the NERSC lsst group; see desc-python-env.md for access instructions.
• Default kernel for notebooks: desc-python (current production release).
• Other kernels: desc-python-dev, desc-python-bleed, desc-python-old, desc-python-gpu, plus stack and specialized envs.