| name | data-preprocessing |
| description | Load and preprocess biomedical data (DICOM, NIfTI, whole-slide images, tabular clinical data). Use when: (1) Loading DICOM/NIfTI/WSI files and extracting metadata, (2) Intensity normalization for CT/MRI, (3) Stain normalization for histopathology, (4) Building PyTorch Dataset classes with caching or HDF5, (5) Designing augmentation pipelines for medical imaging (2D or 3D).
|
Data Preprocessing
Workflow
Preprocessing biomedical data involves these steps:
- Identify data modality -- determine the file format and imaging type
- Load and validate -- read files, extract metadata, verify integrity
- Normalize -- apply modality-appropriate intensity normalization
- Build dataset -- create a PyTorch Dataset class for training
- Add augmentation -- apply augmentation (training only)
Decision Tree
Determine the data type first, then follow the appropriate path:
What is the data format?
What normalization is needed?
- CT imaging → CT windowing (soft tissue, lung, bone -- ask user which anatomy)
- MRI → Z-score normalization with foreground masking
- Histopathology → Stain normalization (Macenko or Vahadane)
- General → Percentile clipping + min-max
What dataset pattern fits?
- Dataset fits in RAM → CachedDataset
- Too large for RAM, need fast access → HDF5Dataset
- Whole-slide images → WSIPatchDataset (on-the-fly tile extraction)
- Tabular features → TabularDataset
ASK the user which data modality and format they are working with before selecting patterns. The choice of normalization, dataset class, and augmentation all depend on the modality.
References
