| name | python-pandas |
| description | Write Pandas code to this project's standards — vectorised operations, Pandas-native types, and Pandera schemas. Use when importing or using pandas — transforming DataFrames or Series, handling missing values, or type-hinting tabular data. |
Python Pandas
Standards for working with Pandas DataFrames and Series. Extends the "prefer library
idioms" rule in python-code-style with Pandas specifics.
Vectorise
- Operate on whole Series and DataFrames; don't loop with
.apply, .iterrows, or a
Python for over rows. Reach for Series methods — .where/.mask/.map/.clip/
.str.*, .between, .isin — and df.eval/df.query for arithmetic and filtering.
- Prefer
Series.map(mapping) to .apply(lambda x: mapping[x]), and use &/| only for
row-wise boolean masks — keep and/or for scalar conditionals.
- When you genuinely must iterate, use
.itertuples() (named, typed, fast), never
.iterrows().
- Stay in Pandas types all the way to the function boundary — don't drop to Python lists
or NumPy arrays mid-pipeline and convert back.
- Suffix DataFrame variables
_df so the type is visible at the call site; name Series
for their contents (e.g. unit_price, not s or data).
Don't mix NumPy into Pandas
- Use
pd.NA/pd.NaT for missing values, not np.nan, and prefer the nullable
extension dtypes (Int64, boolean, string) so missingness is first-class — NumPy
float columns silently coerce pd.NA to NaN.
- Prefer Series methods over
np.* functions on a Series (including via .apply), and
never use .values (the PD011 lint flags it) — reach for .to_numpy() only at a
boundary needing a raw array.
- Don't store NaN as a sentinel; model "missing" explicitly with a nullable dtype.
- Keeping everything Pandas-native also keeps a future move to Polars tractable.
Pandera schemas
- Lean on Pandera: type-hint every DataFrame parameter and return with
DataFrame[Model].
The payoff is readability — the schema becomes explicit at every reference — so use it
ubiquitously, not sparingly.
- Give each schema model its own module, and keep the raw (as-ingested) schema in a
separate module from the processed (validated or derived) one.
- Back categorical columns with a
Category dtype built from an Enum's values (iterate
the Enum to build the categories), and annotate timestamp columns as pd.Timestamp.
