| name | fastmonai-upstream-guide |
| description | Consult MONAI, TorchIO, and nnU-Net documentation and source code for implementation guidance in fastMONAI. Use this skill proactively when implementing new transforms, loss functions, metrics, preprocessing pipelines, or patch-based workflows. Helps ensure fastMONAI implementations align with upstream library patterns and medical imaging best practices. |
fastMONAI Upstream Implementation Guide
This skill provides implementation guidance by consulting MONAI, TorchIO, and nnU-Net patterns via live documentation and source code fetching.
When to Use This Skill
Use this skill proactively when:
- Implementing new transforms or augmentations
- Adding loss functions or metrics
- Working on patch-based training/inference workflows
- Establishing preprocessing or postprocessing pipelines
- Ensuring consistency with upstream library conventions
Library-to-Module Mapping
| fastMONAI Module | Primary Upstream | What to Check |
|---|
| vision_augmentation | TorchIO | Transform classes, wrapper patterns |
| vision_metrics | MONAI + nnU-Net | Metric functions, accumulated patterns |
| vision_loss | MONAI | Loss class signatures, parameters |
| vision_patch | TorchIO | Queue, Sampler, GridAggregator |
| vision_core | TorchIO | ScalarImage, LabelMap, resampling |
Documentation URLs
TorchIO
MONAI
nnU-Net v2
Source Code URLs (for implementation details)
TorchIO GitHub
MONAI GitHub
nnU-Net v2 GitHub (source code)
Workflow Steps
-
Identify the implementation area
- Transform/augmentation → TorchIO
- Metric → MONAI (compute functions) + nnU-Net (accumulated patterns)
- Loss function → MONAI
- Patch workflow → TorchIO (Queue, GridSampler)
- Preprocessing/normalization → nnU-Net v2 conventions
- Sliding window inference → nnU-Net v2 (Gaussian weighting, tile step size)
- Evaluation metrics → nnU-Net v2 (TP/FP/FN accumulation, Dice, IoU)
-
Fetch relevant documentation
Use WebFetch to retrieve the appropriate documentation page from the URLs above.
-
Fetch source code if needed
For implementation details, fetch the raw GitHub file to see:
- Function signatures and parameters
- Default values and conventions
- Internal implementation logic
-
Identify the pattern
- What parameters does the upstream function accept?
- What tensor shapes does it expect?
- What are the default behaviors?
-
Apply to fastMONAI context
- Make changes in
nbs/*.ipynb notebooks (not .py files)
- Follow existing wrapper patterns
- Run
nbdev_prepare after changes
fastMONAI-Specific Conventions
Transform Wrappers
All transform wrappers must expose .tio_transform property:
class MyTransform(DictTransform):
def __init__(self, ...):
self._transform = tio.SomeTransform(...)
@property
def tio_transform(self):
return self._transform
Padding Mode
Always use padding_mode=0 (zero padding) for CropOrPad - this aligns with nnU-Net standards:
tio.CropOrPad(target_size, padding_mode=0)
tio.CropOrPad(target_size)
Tensor Shape Conventions
- MONAI metrics expect: [B, C, D, H, W]
- TorchIO uses: [C, D, H, W] for single images
- fastMONAI MedImage/MedMask: 4D tensors with affine tracking
Accumulated Metrics (nnU-Net v2 pattern)
For patch-based training, use accumulated TP/FP/FN rather than per-batch averaging:
class AccumulatedDice(Metric):
nnU-Net v2 Quick Reference
Architecture: The nnunetv2/ package is organized into: preprocessing, training, inference, evaluation, experiment_planning, postprocessing, utilities.
Normalization schemes (in default_normalization_schemes.py):
ZScoreNormalization - z-score with optional foreground masking (default for MR)CTNormalization - clip to 0.5th-99.5th percentiles then z-score (for CT)NoNormalization - passthrough (dtype conversion only)RescaleTo01Normalization - min-max to [0,1]RGBTo01Normalization - divide uint8 by 255
Loss functions (in training/loss/):
SoftDiceLoss(apply_nonlin, batch_dice, do_bg, smooth=1.0, ddp, clip_tp) - standard soft DiceMemoryEfficientSoftDiceLoss - same API, ~1.6GB less memory via no-grad one-hot encodingDC_and_CE_loss(soft_dice_kwargs, ce_kwargs, weight_ce=1, weight_dice=1, ignore_label) - Dice + Cross-Entropy (default nnU-Net loss)DC_and_BCE_loss(bce_kwargs, soft_dice_kwargs, weight_ce=1, weight_dice=1) - Dice + Binary CEDC_and_topk_loss(soft_dice_kwargs, ce_kwargs, weight_ce=1, weight_dice=1) - Dice + TopK
Evaluation metrics (in evaluation/evaluate_predictions.py):
- Computes per-class: Dice, IoU, TP, FP, FN, TN
compute_tp_fp_fn_tn(mask_ref, mask_pred, ignore_mask) - core confusion matrixcompute_metrics(reference_file, prediction_file, image_reader_writer, labels_or_regions, ignore_label) - per-file metricscompute_metrics_on_folder() / compute_metrics_on_folder_simple() - batch evaluation with multiprocessing
Sliding window inference (in inference/sliding_window_prediction.py):
compute_steps_for_sliding_window(image_size, tile_size, tile_step_size) - tile_step_size is 0-1 (0.5 = 50% overlap)compute_gaussian(tile_size) - Gaussian importance weighting for smooth tile aggregation (center-weighted, no zeros)- Supports test-time augmentation via mirror axes
Trainer variants (in training/nnUNetTrainer/variants/):
Subdirectories for: benchmarking, data_augmentation, loss, lr_schedule, network_architecture, optimizer, sampling, training_length
nbdev Workflow
- All code changes in
nbs/*.ipynb
- Run
nbdev_prepare to regenerate .py files and run tests
- Never edit
fastMONAI/*.py directly
Example Queries
When implementing, fetch relevant docs/code:
"I need to implement a new spatial transform"
→ Fetch TorchIO spatial transform docs and source
"What's the signature for MONAI's compute_dice?"
→ Fetch MONAI metrics documentation
"How does nnU-Net handle validation metrics?"
→ Fetch nnU-Net v2 evaluation code for TP/FP/FN accumulation patterns
"What loss does nnU-Net v2 use by default?"
→ Fetch compound_losses.py for DC_and_CE_loss (Dice + Cross-Entropy)
"How does nnU-Net v2 do sliding window inference?"
→ Fetch sliding_window_prediction.py for tile stepping and Gaussian weighting
"What normalization does nnU-Net v2 use for MR images?"
→ Fetch default_normalization_schemes.py for ZScoreNormalization
"I want to add a new loss function"
→ Fetch MONAI losses documentation and nnU-Net v2 compound_losses.py
Quick Reference Commands
To check upstream implementations during development:
# TorchIO transform documentation
WebFetch: https://torchio.readthedocs.io/transforms/augmentation.html
# MONAI metric implementation
WebFetch: https://raw.githubusercontent.com/Project-MONAI/MONAI/dev/monai/metrics/meandice.py
# nnU-Net v2 usage guide
WebFetch: https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/how_to_use_nnunet.md
# nnU-Net v2 normalization schemes (ZScore, CT, RescaleTo01, etc.)
WebFetch: https://raw.githubusercontent.com/MIC-DKFZ/nnUNet/master/nnunetv2/preprocessing/normalization/default_normalization_schemes.py
# nnU-Net v2 default loss (Dice + CE compound)
WebFetch: https://raw.githubusercontent.com/MIC-DKFZ/nnUNet/master/nnunetv2/training/loss/compound_losses.py
# nnU-Net v2 Dice loss implementation
WebFetch: https://raw.githubusercontent.com/MIC-DKFZ/nnUNet/master/nnunetv2/training/loss/dice.py
# nnU-Net v2 evaluation metrics (Dice, IoU, TP/FP/FN)
WebFetch: https://raw.githubusercontent.com/MIC-DKFZ/nnUNet/master/nnunetv2/evaluation/evaluate_predictions.py
# nnU-Net v2 sliding window inference (Gaussian weighting)
WebFetch: https://raw.githubusercontent.com/MIC-DKFZ/nnUNet/master/nnunetv2/inference/sliding_window_prediction.py
# nnU-Net v2 base trainer (training loop, validation, scheduling)
WebFetch: https://raw.githubusercontent.com/MIC-DKFZ/nnUNet/master/nnunetv2/training/nnUNetTrainer/nnUNetTrainer.py
# nnU-Net v2 extending guide (custom trainers, architectures)
WebFetch: https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/extending_nnunet.md
