data-scrubber

name	data-scrubber
description	Data cleaning automation expertise covering missing value strategies, outlier detection methods, duplicate detection and deduplication, data type correction, text normalization, date parsing across formats, encoding fixes, validation rules, pipeline design patterns, and data quality reporting. Use when the user asks about data scrubber, data scrubber best practices, or needs guidance on data scrubber implementation. Do NOT use when the user needs a different specialized skill or is asking about an unrelated technology domain.
license	Apache-2.0
metadata	{"author":"foundry-skills","version":"1.0.0","tags":"automation shell-scripting data-science","category":"software-engineering","subcategory":"developer-tools","depends":"","disclaimer":"none","difficulty":"intermediate"}

Data Scrubber

Core Philosophy

Data cleaning is the unglamorous but critical foundation of any data-driven system. Raw data is messy: missing values, inconsistent formats, duplicates, encoding errors, and outliers. A systematic data cleaning pipeline transforms raw chaos into reliable, analysis-ready data. The goal is not perfection -- it is fitness for purpose. Every cleaning decision should be documented, reversible, and auditable.

Data Cleaning Pipeline Design

Pipeline Architecture

from abc import ABC, abstractmethod
from dataclasses import dataclass, field
 typing  
 pandas  pd


 :
    
    total_rows_in:  = 
    total_rows_out:  = 
    steps: [] = field(default_factory=)

     ():
        .steps.append({
            : name,
            : rows_affected,
            : details,
        })

pipeline.add_step(NormalizeText(columns=[, ]))
pipeline.add_step(ParseDates(columns=[, ]))
pipeline.add_step(DetectOutliers(column=, method=))
pipeline.add_step(ValidateConstraints())

clean_df, report = pipeline.run(raw_df)
(report.summary())