dataset-preprocessing

# Dataset Preprocessing Skill ## Triggers - "preprocess radiology data" - "DICOM preprocessing" - "image normalization" - "data augmentation" - "quality control pipeline" - "mask generation" - "multi-site harmonization" - "training data preparation" ## Parameters - `input_format` (required): Source data format - `dicom` - DICOM files - `nifti` - NIfTI volumes - `metadata` - Header/excel data - `mixed` - Multiple formats - `task_type` (required): Downstream ML task - `detection` - Object/bounding box detection - `segmentation` - Pixel-level segmentation - `classification` - Image classification - `regression` - Continuous value prediction - `modality` (optional): Imaging modality - `multi_vendor` (optional): Boolean for multi-site/multi-vendor data - `dataset_scale` (optional): Small (<1K), medium (1K-100K), large (>100K) ## Preprocessing Components ### Image Processing - Intensity normalization (z-score, min-max, percentile-based) - Windowing/leveling for CT/MRI - Resampling to isotropic voxel size - Brain extraction (skull stripping) - Bias field correction for MRI ### Quality Control - Automated quality scoring - Artifact detection - Contrast-to-noise ratio - Resolution verification - Human-in-the-loop review for edge cases ### Augmentation - Geometric: rotation, flip, scale, elastic deformation - Intensity: noise, contrast, brightness - Modality-specific: CT windowing variants, MRI sequence mixing - Generative: synthetic data augmentation ### Format Conversion - DICOM to NumPy/PyTorch/TensorFlow - DICOM to NIfTI for volumetric data - Annotation format conversion (CSV, COCO, YOLO, Pascal VOC) ## Output Format Returns structured JSON with: - Processing pipeline steps - Code snippets for each transformation - Validation checks and statistics - Expected output specifications - Common pitfalls and mitigations ## Usage Examples ``` input_format: dicom task_type: detection modality: CT multi_vendor: true input_format: nifti task_type: segmentation dataset_scale: large ```