sklearn

name	sklearn
description	Complete scikit-learn 1.9.0 code-writing reference. Use when writing sklearn code for classification, regression, clustering, dimensionality reduction, model selection, pipeline construction, or any sklearn estimator. Covers all model families, preprocessing, metrics, cross-validation, and common pitfalls from the official 1.9.0 documentation.

scikit-learn 1.9.0 — Complete Code-Writing Reference

Source: scikit-learn 1.9.0 official documentation (scikit-learn.cn mirror) | Chapters: 55 | Generated: 2026-07-10

How to Use This Skill

• Without arguments — loads this overview with estimator chooser and core patterns
• Estimator name — ask about SVC, RandomForest, HDBSCAN, etc.; I load the relevant chapter
• Task type — ask about classification, regression, clustering; I find and explain the best approach
• Chapter — ask for ch07, ch33, etc.; I load that specific chapter with code examples
• Pattern — ask about pipeline, grid search, cross validation; I find the technique
• Browse — ask "what chapters do you have?" to see the full index

Each chapter file contains preserved code from the official examples integrated into the relevant estimator documentation.

---

Core Frameworks & Mental Models

The Estimator API (universal)

Every sklearn object follows fit(X, y) → predict(X) or transform(X). This is the only pattern you need to memorize. Classifiers add predict_proba and decision_function; transformers add fit_transform. Hyperparameters are always __init__ arguments; learned parameters are always _-suffixed attributes (e.g., coef_, feature_importances_).

The Pipeline Rule: Never Preprocess Outside a Pipeline

If your preprocessing depends on data (scaling, imputation, feature selection, target encoding), wrap it in a Pipeline or ColumnTransformer before calling fit. This is the single most important rule to prevent data leakage. GridSearchCV works with Pipelines via 'stepname__paramname' syntax.

Estimator Selection Decision Tree

• Classification: < 100k samples → try LogisticRegression (linear) then SVC(kernel='rbf') then HistGradientBoostingClassifier. > 100k → SGDClassifier(loss='log_loss'). Text → SGDClassifier + TfidfVectorizer.
• Regression: Start with Ridge (linear), then HistGradientBoostingRegressor, then SVR(kernel='rbf'). Heavy-tailed targets → TransformedTargetRegressor. > 100k → SGDRegressor.
• Clustering: If you know n_clusters → KMeans. If not → HDBSCAN. Large data → MiniBatchKMeans or Birch. Hierarchical → AgglomerativeClustering.
• Dimensionality reduction: Interpretable → PCA. Sparse → TruncatedSVD. Visualization only → TSNE (not for preprocessing). Fast approximation → RandomProjection.

Regularization Direction (memorize this)

C (SVM, LogisticRegression): smaller = stronger regularization. alpha (Ridge, Lasso, ElasticNet, MLP, SGD, KernelRidge): larger = stronger regularization. Knowing which way to turn the knob is half of tuning.

Feature Engineering Stack (priority order)

1. Scale features: StandardScaler (default), RobustScaler (outliers), MinMaxScaler (bounded range needed)
2. Handle missing values: SimpleImputer → KNNImputer → IterativeImputer
3. Encode categoricals: OneHotEncoder (most cases), OrdinalEncoder (ordered), TargetEncoder (high cardinality)
4. Handle text: CountVectorizer/TfidfVectorizer → TruncatedSVD (for LSA)
5. Create interactions: PolynomialFeatures (small d), SplineTransformer (smooth nonlinear)
6. Reduce dimensionality: PCA (dense), TruncatedSVD (sparse), SelectKBest (univariate filter)

Model Selection Workflow (golden path)

1. Split data once into train/test. Lock away test data.
2. On training data: Pipeline → GridSearchCV/RandomizedSearchCV with cv=5 or StratifiedKFold.
3. Use cross_validate for reliable performance estimates (returns test_score, train_score, fit_time).
4. For unbiased evaluation of the tuned model: use nested CV (outer cross_val_score + inner GridSearchCV).
5. Check learning_curve for bias/variance diagnosis. Check validation_curve for hyperparameter sensitivity.
6. Only after all tuning: evaluate once on the held-out test set.

Tree Model Defaults (1.9 era)

Prefer HistGradientBoostingClassifier/Regressor (HGBT) over the older GradientBoostingClassifier/Regressor. HGBT is faster, handles missing values natively, supports categorical features without encoding, and includes early_stopping. Tree-tuning priority: n_estimators/max_iter → max_depth/max_leaf_nodes → min_samples_leaf → learning_rate.

When to Use What Cross-Validation Splitter

• Default: KFold(n_splits=5, shuffle=True, random_state=42) or StratifiedKFold for classification.
• Classification with class imbalance: StratifiedKFold (preserves class proportions per fold).
• Time series: TimeSeriesSplit (train always precedes test, no shuffling).
• Groups (patients, users): GroupKFold (same group never split across folds).
• Small data: RepeatedKFold or RepeatedStratifiedKFold (reduces variance of CV estimate).

Key Pitfalls (what every sklearn user should know)

1. Data leakage from preprocessing before split → always use Pipeline.
2. Using test set for hyperparameter tuning → cross-validate.
3. Evaluating on training data → always hold out a test set.
4. Ignoring class imbalance → use class_weight='balanced' or appropriate scoring (F1, not accuracy).
5. Interpreting coef_ on unscaled features → scale first, or use permutation_importance.
6. Using t-SNE as preprocessing for downstream models → t-SNE is visualization-only, not a general dimensionality reduction.
7. Forgetting random_state → set it everywhere for reproducibility.

---

Chapter Index

#	Title	Key Content
ch04	Linear Models	LinearRegression, Ridge, Lasso, ElasticNet, LogisticRegression, SGD, RANSAC, TheilSen, Huber, QuantileRegressor, PolynomialFeatures
ch05	LDA & QDA	LinearDiscriminantAnalysis, QuadraticDiscriminantAnalysis, shrinkage, dim reduction
ch06	Kernel Ridge Regression	KernelRidge, SVR comparison, kernel choices
ch07	Support Vector Machines	SVC, NuSVC, LinearSVC, SVR, OneClassSVM, kernels, C/gamma tuning
ch08	Stochastic Gradient Descent	SGDClassifier, SGDRegressor, partial_fit, loss functions, penalties
ch09	Nearest Neighbors	KNeighborsClassifier/Regressor, NCA, KernelDensity, LOF, NearestCentroid
ch10	Gaussian Processes	GaussianProcessRegressor/Classifier, RBF/Matern kernels, kernel arithmetic
ch11	Cross Decomposition	PLSRegression, PLSCanonical, CCA, PCR vs PLS
ch12	Naive Bayes	GaussianNB, MultinomialNB, ComplementNB, BernoulliNB, partial_fit
ch13	Decision Trees	DecisionTreeClassifier/Regressor, pruning, export, feature importance
ch14	Ensembles	RandomForest, HGBT, AdaBoost, GradientBoosting, Voting, Stacking, Bagging, IsolationForest
ch15	Multiclass	OneVsRest, OneVsOne, OutputCode, OvR vs OvO tradeoffs
ch16	Feature Selection	VarianceThreshold, SelectKBest, SelectFromModel, RFE, RFECV, SequentialFeatureSelector
ch17	Semi-Supervised	SelfTrainingClassifier, LabelPropagation, LabelSpreading
ch18	Isotonic Regression	IsotonicRegression
ch19	Calibration	CalibratedClassifierCV, calibration_curve, CalibrationDisplay
ch20	Neural Networks (Supervised)	MLPClassifier, MLPRegressor, activation, solver, early_stopping
ch22	Gaussian Mixtures	GaussianMixture, BayesianGaussianMixture, BIC selection
ch23	Manifold Learning	TSNE, MDS, Isomap, LLE, SpectralEmbedding
ch24	Clustering	KMeans, HDBSCAN, DBSCAN, OPTICS, Agglomerative, Spectral, Birch, AffinityPropagation
ch25	Biclustering	SpectralBiclustering, SpectralCoclustering
ch26	Decomposition	PCA, KernelPCA, TruncatedSVD, NMF, FastICA, FactorAnalysis, DictionaryLearning, LDA
ch27	Covariance Estimation	LedoitWolf, OAS, GraphicalLasso, MinCovDet, Mahalanobis
ch28	Outlier Detection	IsolationForest, LOF, OneClassSVM, EllipticEnvelope
ch29	Density Estimation	KernelDensity
ch30	Neural Networks (Unsup.)	BernoulliRBM
ch32	Cross-Validation	cross_val_score, cross_validate, KFold, StratifiedKFold, GroupKFold, TimeSeriesSplit
ch33	Hyperparameter Tuning	GridSearchCV, RandomizedSearchCV, HalvingGridSearchCV, pipeline param syntax
ch34	Classification Threshold	TunedThresholdClassifierCV, FixedThresholdClassifier
ch35	Metrics & Scoring	Classification/regression metrics, DummyClassifier, Display objects, make_scorer
ch36	Learning Curves	learning_curve, validation_curve, LearningCurveDisplay
ch37	Metadata Routing (1.9 exp.)	set_fit_request, groups/sample_weight routing
ch38	Inspection	Model inspection overview
ch39	Partial Dependence	PDP, ICE, PartialDependenceDisplay, 2D interactions
ch40	Permutation Importance	Model-agnostic importance, multicollinearity handling
ch41	Visualization	Display objects, pipeline diagram, plot_tree
ch42	Callbacks (1.9 exp.)	ProgressBar, ScoringMonitor, set_callbacks
ch44	Pipelines & Column Transformers	Pipeline, ColumnTransformer, FeatureUnion, TransformedTargetRegressor
ch45	Feature Extraction	CountVectorizer, TfidfVectorizer, HashingVectorizer, DictVectorizer
ch46	Preprocessing	StandardScaler, OneHotEncoder, OrdinalEncoder, TargetEncoder, KBinsDiscretizer, all scalers
ch47	Imputation	SimpleImputer, KNNImputer, IterativeImputer, MissingIndicator
ch48	Unsupervised Reduction	Overview of dim reduction methods
ch49	Random Projection	Gaussian/SparseRandomProjection, JL lemma
ch50	Kernel Approximation	RBFSampler, Nystroem, PolynomialCountSketch, AdditiveChi2Sampler
ch51	Pairwise Metrics	pairwise_distances, pairwise_kernels, all distance metrics and kernels
ch52	Target Preprocessing	LabelEncoder, LabelBinarizer, MultiLabelBinarizer
ch54	Toy Datasets	load_iris, load_digits, fetch_california_housing, make_*, fetch_openml
ch56	Sample Generators	make_classification, make_regression, make_blobs, make_moons, make_circles
ch58	Computing	config_context, threading, BLAS, parallelism notes
ch59	Scaling Strategies	partial_fit, out-of-core learning
ch60	Computational Performance	Prediction latency, caching, LinearSVC vs SVC
ch62	Model Persistence	joblib.dump/load, security, version compatibility
ch63	Common Pitfalls	Data leakage, preprocessing errors, wrong scoring
ch65	DataFrame Output	set_output(transform='pandas'), Polars support
ch67	Estimator Chooser	When to use which estimator — decision rules

Topic Index

• alpha (regularization) → ch04, ch07, ch08, ch20
• anomaly detection → ch28
• bagging → ch14
• biclustering → ch25
• C (regularization) → ch04, ch07
• calibration → ch19
• callbacks → ch42
• categorical features → ch14, ch46
• classification → ch04, ch05, ch07, ch08, ch09, ch10, ch12, ch13, ch14, ch15, ch19, ch20
• clustering → ch24, ch25
• ColumnTransformer → ch44
• cross-validation → ch32
• data leakage → ch63
• density → ch29
• dimensionality reduction → ch26, ch48, ch49
• ensembles → ch14
• feature extraction (text) → ch45
• feature selection → ch16
• Gaussian processes → ch10
• gradient boosting → ch14
• grid search → ch33
• imputation → ch47
• inspection → ch38
• kernel approximation → ch50
• kernel methods → ch06, ch07, ch50, ch51
• learning curves → ch36
• manifold learning → ch23
• metrics → ch35, ch51
• missing values → ch47
• model persistence → ch62
• multiclass → ch15
• multi-label → ch52
• outlier detection → ch28
• partial dependence → ch39
• partial_fit (online) → ch08, ch12, ch26, ch59
• permutation importance → ch40
• pipeline → ch44
• PLS/PCR/CCA → ch11
• preprocessing → ch46, ch52
• random forests → ch14
• regression → ch04, ch06, ch07, ch08, ch09, ch10, ch13, ch14, ch18, ch20
• scaling → ch46
• semi-supervised → ch17
• SGD → ch08
• sparse data → ch07, ch08, ch26, ch45, ch49
• SVM → ch07
• target encoding → ch46
• target transformation → ch52
• text processing → ch45
• threshold tuning → ch34
• tree models → ch13, ch14
• visualization → ch41

Traceability

Every code example in this skill is traceable back to its original source files in version_1_9_0/. Each chapter includes a ## Source Files table mapping topics to specific file numbers. Use these lookup tools to find any original example:

• To find original example code: Check the ## Source Files table at the end of each chapter, or use source-mapping.md for a complete chapter ↔ file mapping.
• To browse all examples by task: Use examples-index.md — lists every example file with its key APIs and which chapter it belongs to.
• To find source files by file number: ls version_1_9_0/<NNN>_*.md — all files are zero-padded prefixed.
• To find every file mentioning an estimator: grep -l "sklearn.svm.SVC" version_1_9_0/*.md

Supporting Files

• glossary.md — scikit-learn terminology with chapter references (~500 words)
• patterns.md — reusable code patterns (Pipeline+GridSearch, HGBT, CalibratedClassifierCV, etc.) (~660 words)
• cheatsheet.md — estimator chooser, parameter reference, scaling guide, tells & smells (~620 words)
• source-mapping.md — complete chapter ↔ source file number mapping for full traceability
• examples-index.md — all example files organized by category with key APIs and descriptions
• api-reference.md — flat listing of every sklearn class/function by module

---

Scope & Limits

This skill covers the scikit-learn 1.9.0 user guide and examples only. It does not cover: XGBoost/LightGBM/CatBoost (third-party), deep learning frameworks (PyTorch/TF/JAX), or domain-specific libraries. For hands-on code in your project, combine this skill with your project-specific tools and data exploration. For topics beyond sklearn 1.9.0, check the latest scikit-learn.org documentation.