メニュー
Use when preparing raw data for modeling — handling missing values, duplicates, inconsistent types, outliers, and bad categorical values. Emphasizes fitting all imputation on train-only to avoid leakage.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Use when running ML experiments that need to be compared, reproduced, or shared. Covers MLflow/Weights & Biases logging, what to track, run organization, and model registry basics.
Use when starting on a new dataset, before modeling, or when asked to "explore", "profile", "understand", or "summarize" data. Covers structured EDA, distributions, correlations, target leakage checks, and visualization.
Use when creating, encoding, scaling, or selecting features for ML models. Covers categorical encoding, numeric transforms, datetime/text/aggregation features, and leakage-safe target encoding.
Use when optimizing model hyperparameters. Covers search strategy (random vs Bayesian/Optuna), leakage-safe tuning inside CV, search-space design, early stopping, and budget management.
Use when the target is rare (fraud, churn, disease, anomalies). Covers correct metrics, resampling (SMOTE/undersampling), class weights, threshold tuning, and avoiding the accuracy trap and resampling leakage.
Use when fine-tuning a large language model. Covers choosing full vs LoRA/QLoRA, dataset formatting, the transformers/PEFT/TRL stack, key hyperparameters, and evaluating fine-tunes without overfitting.
| name | data-cleaning |
| description | Use when preparing raw data for modeling — handling missing values, duplicates, inconsistent types, outliers, and bad categorical values. Emphasizes fitting all imputation on train-only to avoid leakage. |
Cleaning turns raw data into a consistent, model-ready table without leaking information from the future or the test set. The golden rule: every statistic used to clean (means, medians, modes, bounds, category maps) must be learned from the training split only, then applied to validation/test.
| Situation | Strategy |
|---|---|
| Numeric, MCAR, small % | Median impute (robust to skew) |
| Numeric, informative missingness | Impute + add was_missing indicator |
| Categorical | Impute with "Missing" as its own category |
| Time series | Forward/backward fill within group |
| >50% missing | Consider dropping the column |
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# Fit imputers on TRAIN ONLY
num_imputer = SimpleImputer(strategy="median").fit(X_train[num_cols])
X_train[num_cols] = num_imputer.transform(X_train[num_cols])
X_test[num_cols] = num_imputer.transform(X_test[num_cols]) # reuse train stats
Prefer doing this inside a Pipeline/ColumnTransformer (see the sklearn-pipelines skill) so leakage is impossible by construction.
# Winsorize numeric columns to train-derived 1st/99th percentiles
lo = X_train[col].quantile(0.01)
hi = X_train[col].quantile(0.99)
X_train[col] = X_train[col].clip(lo, hi)
X_test[col] = X_test[col].clip(lo, hi)
df.fillna(df.mean()) on the whole dataset — classic leakage. Compute stats on train only.dropna() on rows silently shrinks and biases the dataset; prefer imputation + indicators.Produce a clean, typed dataframe plus a documented list of decisions (what was imputed/capped/dropped and why) so feature-engineering and reproducible-ml can rely on it.