sklearn-explainability

# scikit-learn - Explainability & Interpretability In scientific research, a model's "why" is as important as its "what". This guide focuses on tools that reveal the decision-making process of machine learning models, ensuring they are scientifically valid and not just overfitting on artifacts. ## When to Use - Validating that a model uses physically meaningful features (e.g., in drug discovery). - Identifying biases or "shortcuts" the model has learned from the training data. - Explaining individual predictions to non-experts (Local explanations). - Ranking the global impact of variables on a complex system (Global explanations). - Scientific auditing and regulatory compliance. ## Core Principles ### 1. Model-Specific vs. Model-Agnostic - **Model-Specific**: Tools like `feature_importances_` in Random Forests. Fast but tied to one architecture. - **Model-Agnostic**: Tools like SHAP or Permutation Importance. Work on any model (SVM, MLP, etc.) but are more compute-intensive. ### 2. Global vs. Local Explanations - **Global**: How does the feature "Temperature" affect the model overall? - **Local**: Why did the model predict "Reaction Failed" for this specific sample? ### 3. Feature Importance vs. Feature Contribution Importance tells you if a feature is used; Contribution tells you how it changed the output (positive or negative). ## Quick Reference: Built-in Inspection ```python from sklearn.inspection import permutation_importance, PartialDependenceDisplay # 1. Permutation Importance (Better than default tree importance) result = permutation_importance(model, X_test, y_test, n_repeats=10) print(result.importances_mean) # 2. Partial Dependence Plots (How one feature affects prediction) PartialDependenceDisplay.from_estimator(model, X, features=['temp', 'pressure']) ``` ## Critical Rules ### ✅ DO - **Prefer Permutation Importance over default RandomForest.feature_importances_** - Default importance is biased toward high-cardinality features (like unique IDs). - **Use PartialDependenceDisplay** - To visualize the relationship between a feature and the target (Linear, Exponential, or Sigmoid). - **Scale Features before Interpretability** - Many models (like Logistic Regression) require scaling for their coefficients (β) to be comparable. - **Check Feature Correlations** - If two features are highly correlated, importance will be split between them, making both look "less important" than they are. ### ❌ DON'T - **Don't trust coefficients (β) of unregularized models** - High variance in coefficients can lead to false conclusions about feature importance. - **Don't use Feature Importance on Training Data** - Always calculate it on the Test Set to see what features actually help with generalization. - **Don't confuse Correlation with Causation** - ML models show which features are predictive, not necessarily which ones are causative. ## Interpretation Patterns ### 1. SHAP Integration (The Gold Standard) ```python import shap # Works for any scikit-learn model explainer = shap.Explainer(model.predict, X_test) shap_values = explainer(X_test) # Visualize global importance shap.plots.bar(shap_values) # Visualize local explanation for the first sample shap.plots.waterfall(shap_values[0]) ``` ### 2. Partial Dependence (PDP) for Science ```python from sklearn.inspection import PartialDependenceDisplay # Check if the model learned the correct physical law # (e.g., does the reaction rate increase with temperature?) fig, ax = plt.subplots(figsize=(8, 4)) PartialDependenceDisplay.from_estimator(model, X, [0, (0, 1)], ax=ax) # [0] is a 1D plot, [(0, 1)] is a 2D interaction plot ``` ### Advanced: Feature Contribution (ELI5 style) For a single prediction, see which features pushed it towards which class. ```python def explain_prediction(model, sample): # For linear models, this is: intercept + sum(coef * value) prediction = model.predict_proba(sample) # ... logic to map coefficients to feature names ... pass ``` ## Practical Workflows: Validating a Scientific Model ### Step 1: Detect "Leakage" Features If a feature has 99% importance and wasn't expected to, it's likely a data leak (e.g., a sample timestamp or ID). ### Step 2: Stability Analysis Run permutation importance with different random seeds. If the top features change significantly, the model is unstable and unreliable. ### Step 3: Interaction Check Use 2D PDP to see if the model captured the interaction between features (e.g., Pressure only matters if Temperature > 100°C). ## Common Pitfalls ### The "Default Importance" Bias In RandomForest, features with many categories (like Serial_Number) look very important because the tree can split on them many times. ```python # ✅ Solution: Use Permutation Importance on the test set instead. ``` ### Multicollinearity Ghosting If Feature_A and Feature_B are 100% correlated, the model might only use one. ```python # ✅ Solution: Use hierarchical clustering on features or check VIF # before interpreting importance. ``` Explainability turns Machine Learning into a true scientific tool. It allows researchers to move beyond the "Black Box" and extract new hypotheses directly from trained models.