numpy-masked

name	numpy-masked
description	Masked arrays for robust handling of missing or invalid data, ensuring they are excluded from statistical and mathematical computations. Triggers: masked array, numpy.ma, missing data, invalid values, hard mask.

Overview

The numpy.ma module provides masked arrays, which couple a data array with a boolean mask. Masked elements are ignored in operations like mean(), sum(), and log(), making them ideal for datasets where certain entries should be excluded without deleting them and losing shape information.

When to Use

• Handling sensor data with "no-data" values (e.g., -999).
• Performing statistics on arrays containing NaNs or Infs where you want the invalid values automatically excluded.
• Protecting specific data points from modification during processing using a "hard mask."
• Exporting data where missing values must be filled with a specific constant.

Decision Tree

1. Do you need to keep the original array shape while ignoring certain values?
   • Use ma.masked_array.
2. Are you performing math on risky values (e.g., negative numbers in log)?
   • Use ma.masked_invalid(arr) or ma.masked_less(arr, 0).
3. Want to extract only valid data for another tool?
   • Use the .compressed() method to get a 1D array of valid values.

Workflows

1. Calculating Stats on Tainted Data

   • Create a masked array from raw data using ma.masked_values(data, -999).
   • Perform a .mean() calculation.
   • Observe that the result only reflects valid, unmasked data points.