tcfuse

TC-Fuse hyperparameter search — Hydra Optuna sweeper + submitit launcher running parallel single-GPU SLURM trials (offline-safe, works on Jean-Zay). Search space in conf/hydra/sweeper/*.yaml; launcher SLURM spec in conf/hydra/launcher/jz_*_sweep.yaml; short-budget experiment in conf/experiment/pmw_gmi_sweep.yaml. With submitit=false, scripts/train/train.py returns the best val/loss as the objective Optuna minimizes. Respect the Perceiver's head-divisibility invariants when defining the space. Use when running a sweep, adding a search space, or changing the optimized metric.

TC-Fuse evaluation pipeline — computing metrics and building figures from saved predictions with `scripts/evaluation/evaluate.py` (Hydra, `conf/evaluation.yaml`). Plugin-based: each `Evaluation` (`src/tcfuse/evaluation/`) is enabled via the `conf/evaluation/` config group and writes into its own subfolder under `paths.results/<run_id>/<experiment_name>/`. The base contract imposes no data shape (flattening is plugin-dependent); the `quantitative_metrics` plugin computes RMSE/MAE/R2/MAPE per source/channel with numpy/scikit-learn, independent of torchmetrics. Use when evaluating a prediction run, adding a new evaluation plugin, or changing the results layout.

TC-Fuse inference and prediction pipeline — running a trained checkpoint over a split with `scripts/inference/infer.py` (Hydra, `conf/inference.yaml`), the single-GPU `trainer.predict` flow with task-specific target masking, the `PredictionWriter` callback, and the `SamplePrediction` / `PredictionRun` output format (per-window HDF5 + `index.parquet` + `manifest.yaml`) with its `compute_metrics` evaluation (bias, RMSE, MAE, R2 per channel). Use when running inference, evaluating a checkpoint, changing the prediction output format, or working in `src/tcfuse/data/predictions/` or `src/tcfuse/lightning/prediction_writer.py`.

Drive the CLEPS (Inria Paris) cluster from the local machine — rsync sync, preflight, SLURM submission via submitit, job monitoring, checkpoint resume, and storage quota. Use whenever a task involves `ssh cleps`, $SCRATCH, CLEPS SLURM partitions (cpu_devel / gpu / arches), pixi on the cluster, `submitit`, `setup=cleps_*`, `paths=cleps`, or running jobs on CLEPS. CLEPS uses pixi (no modules) and W&B online (no offline sync), and its scratch is persistent (no archive step).

TC-Fuse model backbone design — the architecture-agnostic embedding layer (TorchSource → EmbeddedSource via MultiSourceEncoder) and un-embedding layer (EmbeddedBatch → DecodedBatch via MultiSourceDecoder), the swappable encoder interface, candidate backbones to benchmark (Perceiver, cross-attention Transformer, hierarchical windowed attention), and the masked-source-reconstruction self-supervised pre-training task. Use when working in `src/tcfuse/models/`, adding or benchmarking a backbone, or changing the encoder/decoder interface.

Fix basedpyright diagnostics with clean, minimal Python typing changes. Use when the user mentions Pyright, basedpyright, type errors, typing fixes, type checking, or Python typing diagnostics.

Drive the Jean-Zay cluster from the local machine — rsync sync, preflight, SLURM submission via submitit, job monitoring, W&B offline sync, checkpoint resume, and storage quota. Use whenever a task involves `ssh jz`, $WORK, $SCRATCH, $STORE, SLURM partitions (gpu_p13 / gpu_p5 / gpu_p6 / cpu / prepost / archive), module loads, `submitit`, `setup=jz_*`, `paths=jz`, `requirements-jz.txt`, or running jobs on Jean-Zay.

TC-Fuse dataset preprocessing pipeline — Stage 0 IBTrACS prep + ATCF→SID translation table, Stage 1 per-source HDF5 snapshots, Stage 2 assembled per-storm HDF5 + concatenated index, Stage 3A season splits, Stage 3B window index building, normalization statistics, the I/O API in `src/tcfuse/data/sources/`. Use when preparing any dataset (TC-PRIMED, CyclObs, dropsondes, Argo), running `prepare_ibtracs.py` / `assemble.py` / `build_splits.py` / `build_windows.py` / `compute_normalization.py`, working with `Source` or `StormData`, or extending the per-source HDF5 / assembled formats.

Publication-quality TC-Fuse visualizations for journal articles and preprints — matplotlib + cartopy + cmocean, SVG output via `save_fig`, shared style constants (`COL1`, `COL2`, `AR_GOLDEN`, `INTENSITY_COLORS`, `SOURCE_COLORS`), thematic modules under `src/tcfuse/data/visualization/` (fields, tracks, profiles, timeline, storm_data_visu, training). Use when writing or modifying any plotting function, adding a new figure type, or touching `style.py`.