txpipe

name

txpipe

description

How to work with TXPipe, the DESC 3x2pt weak-lensing pipeline. Covers all pipeline phases (ingestion → photo-z → selection → calibration → masks → maps → noise maps → 2pt → covariance → theory → diagnostics), key stage names, file tags, pipeline YAML structure, and critical gotchas. Use this skill whenever the user imports txpipe; works with any TXPipe stage (TXTwoPoint, TXTwoPointFourier, TXSourceMaps, TXLensMaps, TXSourceSelector*, TXShearCalibration, TXTracerMetadata, TXSimpleMask, TXCustomMask, TXIngest*, TXBlinding, TXFourierNamasterCovariance, TXRealNamasterCovariance, TXJackknifeCenters, TXPSFDiagnostics, TXRoweStatistics, TXTauStatistics, etc.); writes a TXPipe pipeline YAML; asks about shape or shear catalogs, metacalibration or metadetect, source or lens selection, tomographic binning, 3x2pt two-point functions (shear-shear, galaxy-galaxy lensing, angular clustering), map generation, noise maps, jackknife covariance, PSF diagnostics, or theory predictions.

TXPipe — DESC 3×2pt weak-lensing pipeline

You're helping a user work with TXPipe, the main DESC analysis workhorse. TXPipe processes shear and photometric catalogs through sample selection, map generation, two-point measurements (real and Fourier space), covariance estimation, theory comparison, and diagnostics. All stages are ceci PipelineStage subclasses; for ceci conventions see the ceci skill. For NERSC job submission, see the nersc skill. Key library dependencies: NaMaster (namaster skill) for Fourier C_ℓ measurements and covariance; CCL (ccl skill) for theory predictions; Firecrown (firecrown skill) is used internally by theory/blinding stages and consumes TXPipe's sacc output for parameter inference.

Pipeline phases (in order)

1. Ingestion — TXIngestDataPreview02, TXCosmoDC2Mock, etc. — source-specific; using the wrong ingester silently reads wrong columns
2. Photo-z — RAIL stages (PZEstimatorLens/Source, PZRailSummarizeLens/Source); must produce QPNOfZFile before 2pt
3. Selection — TXSourceSelectorMetadetect/Metacal/Lensfit/HSC + TXSourceTomography; TXMeanLensSelector for lenses
4. Calibration — TXShearCalibration then TXTracerMetadata (both required; calibration must precede tomography)
5. Masks — TXSimpleMask or TXCustomMask; nside must match map nside everywhere
6. Maps — TXSourceMaps, TXLensMaps, TXAuxiliarySourceMaps/LensMaps, TXLSSWeights
7. Noise maps — TXSourceNoiseMaps (~30 realisations), TXLensNoiseMaps (~5); required for covariance
8. Two-point real — TXTwoPoint (TreeCorr; ξ±, γ_t, w(θ)); uses TXJackknifeCenters for jackknife cov
9. Two-point Fourier — TXTwoPointFourier (NaMaster; C_ℓ for shear-shear, shear-pos, pos-pos; see namaster skill)
10. Covariance — TXFourierNamasterCovariance / TXRealNamasterCovariance (expensive; use TXFourierGaussianCovariance for quick tests; see namaster skill)
11. Theory & blinding — TXTwoPointTheoryReal/Fourier (CCL; see ccl skill); TXBlinding (Firecrown internally; see firecrown skill)
12. Diagnostics — TXPSFDiagnostics, TXRoweStatistics, TXTauStatistics, TXSourceDiagnosticPlots, TXTwoPointPlots

The examples/metadetect/ pipeline is the canonical full-pipeline reference.

Reference

Topic	Reference
Pipeline phases and stage names	references/txpipe-stages.md
Key file types, pipeline YAML structure, example pipelines, gotchas	references/txpipe-pipeline.md