feature-engineering

name	Feature Engineering
description	Create and transform features using encoding, scaling, polynomial features, and domain-specific transformations for improved model performance and interpretability

Feature Engineering

Overview

Feature engineering creates and transforms features to improve model performance, interpretability, and generalization through domain knowledge and mathematical transformations.

When to Use

• When you need to improve model performance beyond using raw features
• When dealing with categorical variables that need encoding for ML algorithms
• When features have different scales and require normalization
• When creating domain-specific features based on business knowledge
• When handling skewed distributions or non-linear relationships
• When preparing data for different types of ML algorithms with specific requirements

Engineering Techniques

• Encoding: Converting categorical to numerical
• Scaling: Normalizing feature ranges
• Polynomial Features: Higher-order terms
• Interactions: Combining features
• Domain-specific: Business-relevant transformations
• Temporal: Time-based features

Key Principles

• Create features based on domain knowledge
• Remove redundant features
• Scale features appropriately
• Handle categorical variables
• Create meaningful interactions

Implementation with Python

import pandas as pd
import numpy as np
import matplotlib.pyplot  plt
 sklearn.preprocessing  (
    StandardScaler, MinMaxScaler, RobustScaler, PolynomialFeatures,
    OneHotEncoder, OrdinalEncoder, LabelEncoder
)
 sklearn.pipeline  Pipeline
 sklearn.compose  ColumnTransformer
 seaborn  sns


np.random.seed()
df = pd.DataFrame({
    : np.random.uniform(, , ),
    : np.random.uniform(, , ),
    : np.random.uniform(, , ),
    : np.random.choice([, , ], ),
    : np.random.choice([, , ], ),
    : np.random.choice([, ], ),
})

()
(df.head())
(df.info())



()
df_ohe = pd.get_dummies(df, columns=[, ], drop_first=)
(df_ohe.head())


()
ordinal_encoder = OrdinalEncoder()
df[] = ordinal_encoder.fit_transform(df[[]])
(df[[, ]].head())


()
le = LabelEncoder()
df[] = le.fit_transform(df[])
(df[[, ]].head())


()
X = df[[, , ]].copy()


scaler = StandardScaler()
X_standard = scaler.fit_transform(X)


minmax_scaler = MinMaxScaler()
X_minmax = minmax_scaler.fit_transform(X)


robust_scaler = RobustScaler()
X_robust = robust_scaler.fit_transform(X)


fig, axes = plt.subplots(, , figsize=(, ))

axes[, ].hist(X[], bins=, edgecolor=)
axes[, ].set_title()

axes[, ].hist(X_standard[:, ], bins=, edgecolor=)
axes[, ].set_title()

axes[, ].hist(X_minmax[:, ], bins=, edgecolor=)
axes[, ].set_title()

axes[, ].hist(X_robust[:, ], bins=, edgecolor=)
axes[, ].set_title()

plt.tight_layout()
plt.show()


()
X_simple = df[[]].copy()
poly = PolynomialFeatures(degree=, include_bias=)
X_poly = poly.fit_transform(X_simple)
X_poly_df = pd.DataFrame(X_poly, columns=[, ])
(X_poly_df.head())


plt.figure(figsize=(, ))
plt.scatter(df[], df[], alpha=)
plt.xlabel()
plt.ylabel()
plt.title()
plt.grid(, alpha=)
plt.show()


()
df[] = df[] * df[] / 
df[] = df[] / (df[] + )
(df[[, , , ]].head())


()
df[] = pd.cut(df[], bins=[, , , , ],
                          labels=[, , , ])
df[] = pd.qcut(df[], q=, labels=[, , ])
df[] = np.log1p(df[])
df[] = np.sqrt(df[])

(df[[, , , , ]].head())


()
dates = pd.date_range(, periods=(df))
df[] = dates
df[] = df[].dt.year
df[] = df[].dt.month
df[] = df[].dt.dayofweek
df[] = df[].dt.quarter
df[] = df[].dt.dayofweek >= 

(df[[, , , , ]].head())


()


numerical_features = [, , ]
categorical_features = [, ]


preprocessor = ColumnTransformer(
    transformers=[
        (, StandardScaler(), numerical_features),
        (, OneHotEncoder(drop=), categorical_features),
    ]
)

X_processed = preprocessor.fit_transform(df[numerical_features + categorical_features])
()


()
X_for_stats = df[numerical_features].copy()
X_for_stats[] = (df[] == ).astype()
X_for_stats[] = (df[] == ).astype()

feature_stats = pd.DataFrame({
    : X_for_stats.columns,
    : X_for_stats.mean(),
    : X_for_stats.std(),
    : X_for_stats.(),
    : X_for_stats.(),
    : X_for_stats.skew(),
    : X_for_stats.kurtosis(),
})

(feature_stats)


fig, axes = plt.subplots(, , figsize=(, ))

X_numeric = df[numerical_features].copy()
X_numeric[] = df[]
corr_matrix = X_numeric.corr()

sns.heatmap(corr_matrix, annot=, cmap=, center=, ax=axes[])
axes[].set_title()


axes[].hist(df[], bins=, edgecolor=, alpha=)
axes[].set_title()
axes[].set_xlabel()
axes[].set_ylabel()

plt.tight_layout()
plt.show()


()
df[] = pd.cut(df[], bins=)
df[] = pd.qcut(df[], q=)
df[] = pd.cut(df[], bins=[, , , ])

()
(df[].value_counts().sort_index())

()
(df[].value_counts().sort_index())


()
df_with_missing = df.copy()
missing_indices = np.random.choice((df), , replace=)
df_with_missing.loc[missing_indices, ] = np.nan


age_mean = df_with_missing[].mean()
df_with_missing[] = df_with_missing[].fillna(age_mean)


age_median = df_with_missing[].median()
df_with_missing[] = df_with_missing[].fillna(age_median)


df_with_missing[] = df_with_missing[].fillna(method=)

(df_with_missing[[, , ]].head())

()
()
()