machine-learning-foundations

Supervised and unsupervised learning, bias-variance tradeoff, cross-validation, decision trees, ensemble methods, neural network fundamentals, and the practitioner's workflow from problem framing through deployment. Covers classification, regression, clustering, dimensionality reduction, regularization, hyperparameter tuning, and evaluation metrics. Use when building predictive models, selecting algorithms, or understanding the machine learning pipeline.

Jump to install

Repository: Tibsfox/gsd-skill-creator
Last source activity: April 14, 2026 at 07:57
Detected SKILL.md language: English
Stars: 69
Forks: 9

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/Tibsfox/gsd-skill-creator --skill machine-learning-foundations

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

name	machine-learning-foundations
description	Supervised and unsupervised learning, bias-variance tradeoff, cross-validation, decision trees, ensemble methods, neural network fundamentals, and the practitioner's workflow from problem framing through deployment. Covers classification, regression, clustering, dimensionality reduction, regularization, hyperparameter tuning, and evaluation metrics. Use when building predictive models, selecting algorithms, or understanding the machine learning pipeline.
type	skill
category	data-science
status	stable
origin	tibsfox
modified	false
first_seen	"2026-04-12T00:00:00.000Z"
first_path	examples/skills/data-science/machine-learning-foundations/SKILL.md
superseded_by	null

Machine Learning Foundations

Machine learning is the practice of building systems that learn patterns from data and use those patterns to make predictions or decisions on new data. Where statistical modeling (the inference culture) asks "what is the relationship between X and Y?", machine learning (the prediction culture) asks "given X, what is the best prediction of Y?" This skill covers the foundational concepts, algorithms, and workflow of machine learning from the practitioner's perspective.

Agent affinity: breiman (algorithm selection, ensemble methods), tukey (feature engineering, EDA)

Concept IDs: data-correlation, data-distributions, data-measures-of-spread, data-hypothesis-testing

The ML Workflow

Stage	Goal	Key operations
1. Problem framing	Define the task precisely	Classification vs. regression vs. clustering; define target variable and success metric
2. Data collection	Assemble training data	Sources, sampling, labeling; ensure data represents the deployment population
3. Feature engineering	Create informative inputs	Domain-driven features, transformations, encoding categoricals
4. Train/test split	Prevent overfitting evaluation	Hold out 20-30% for testing; never touch test set during development
5. Model selection	Choose algorithm family	Based on data size, interpretability needs, problem structure
6. Training	Fit model parameters	Optimization (gradient descent, tree splitting, etc.)
7. Validation	Tune hyperparameters	k-fold cross-validation on training set only
8. Evaluation	Assess on held-out test set	Metrics appropriate to the problem (accuracy, F1, RMSE, etc.)
9. Interpretation	Understand what the model learned	Feature importance, partial dependence, SHAP values
10. Deployment	Put the model in production	Monitoring, drift detection, retraining schedule

Supervised Learning

Classification

The task: given features X, predict a categorical label y.

Key algorithms:

Logistic regression	Interpretable, fast, probabilistic	Linear decision boundary	Baseline; when interpretability matters
k-Nearest Neighbors	Non-parametric, no training phase	Slow at prediction, curse of dimensionality	Small datasets, low dimensionality
Decision tree	Interpretable, handles mixed types	Overfits easily, unstable	When interpretability is paramount; as building block for ensembles
Random forest	Robust, handles high dimensions	Less interpretable than single tree	Default for tabular data
Gradient boosting	State-of-the-art tabular performance	Prone to overfitting without tuning	Competition-grade tabular prediction
SVM	Effective in high dimensions	Slow on large datasets, kernel choice	Text classification, small-medium datasets
Neural network	Learns complex patterns, scales to huge data	Requires large data, expensive, black box	Images, text, sequences, very large datasets

Metric	Formula / Definition	When to use
Accuracy	Correct / Total	Balanced classes only
Precision	TP / (TP + FP)	Cost of false positives is high (spam detection)
Recall	TP / (TP + FN)	Cost of false negatives is high (cancer screening)
F1 score	2 * Precision * Recall / (Precision + Recall)	Need balance between precision and recall
ROC-AUC	Area under ROC curve	Ranking quality across thresholds
Log loss	Negative log-likelihood of predicted probabilities	When calibrated probabilities matter

Metric	Formula / Definition	When to use
MSE	Mean of (y - y_hat)^2	Default; penalizes large errors
RMSE	sqrt(MSE)	Same scale as y; more interpretable
MAE	Mean of	y - y_hat
R-squared	1 - (SS_res / SS_tot)	Proportion of variance explained
MAPE	Mean of	y - y_hat

Method	Penalty	Effect
Ridge (L2)	Sum of beta_j^2	Shrinks coefficients toward zero; keeps all predictors
Lasso (L1)	Sum of	beta_j
Elastic net	Alpha * L1 + (1 - Alpha) * L2	Combines ridge and lasso benefits
Tree depth limit	Max depth, min samples per leaf	Prevents tree from memorizing noise
Dropout	Randomly zero out neurons during training	Prevents neural network co-adaptation
Early stopping	Stop training when validation error increases	Universal; works for any iterative algorithm

Parameter	Effect
Max depth	Limits tree height; primary regularization lever
Min samples split	Minimum observations to attempt a split
Min samples leaf	Minimum observations in a terminal node
Max features	Number of features considered at each split (critical for random forests)

machine-learning-foundations

Install options

Review the source files

Machine Learning Foundations

The ML Workflow

Supervised Learning

Classification

Machine Learning Foundations

The ML Workflow

Supervised Learning

Classification

Regression

Evaluation Metrics

The Bias-Variance Tradeoff

Cross-Validation

k-Fold Cross-Validation

Critical Rules

Decision Trees

How They Work

Why Single Trees Overfit

Ensemble Methods

Bagging (Bootstrap Aggregating)

Boosting

Bagging vs. Boosting

Unsupervised Learning

Clustering

Dimensionality Reduction

Neural Networks Introduction

Architecture

Training

When to Use Neural Networks

Common Mistakes

Cross-References

References

Property	Bagging (Random Forest)	Boosting (Gradient Boosting)
Reduces	Variance	Bias (primarily) + variance
Training	Parallel (fast)	Sequential (slower)
Overfitting risk	Low	Higher without tuning
Tuning effort	Minimal	Significant (learning rate, depth, iterations)
Default choice	Yes, for most tabular problems	When you need maximum performance and will tune

Algorithm	Assumption	Strengths	Weaknesses
k-Means	Spherical, equal-size clusters	Fast, scalable	Must specify k; sensitive to initialization
DBSCAN	Density-based clusters	Finds arbitrary shapes, handles noise	Sensitive to epsilon and min_samples parameters
Hierarchical	Nested cluster structure	Dendrogram visualization, no k needed	O(n^2) or worse; not for large datasets
Gaussian Mixture	Elliptical clusters	Soft assignments (probabilities)	Must specify k; can converge to local optima

Method	Linear?	Preserves	Use when
PCA	Yes	Global variance	High-dimensional data, preprocessing for modeling
t-SNE	No	Local neighbor structure	2D/3D visualization of high-dimensional data
UMAP	No	Local + some global structure	Faster than t-SNE, better global structure

Mistake	Why it fails	Fix
Data leakage	Future information in training features	Audit every feature for temporal leakage
Not holding out a test set	Reported performance is overly optimistic	Split before any modeling decisions
Using accuracy on imbalanced data	95% accuracy is trivial when 95% of data is one class	Use F1, precision-recall, or balanced accuracy
Tuning on the test set	Test performance becomes optimistic	Tune on validation/CV only; test set opened once
Feature scaling mismatch	Fit scaler on full data, including test	Fit scaler on training data only, transform test with same scaler
Ignoring class imbalance	Model predicts majority class for everything	Oversampling (SMOTE), class weights, or threshold adjustment

machine-learning-foundations

Source facts

Install options

Review the source files

More from this repository

More from this repository

Machine Learning Foundations

The ML Workflow

Supervised Learning

Classification

Machine Learning Foundations

The ML Workflow

Supervised Learning

Classification

Regression

Evaluation Metrics

The Bias-Variance Tradeoff

Cross-Validation

k-Fold Cross-Validation

Critical Rules

Decision Trees

How They Work

Why Single Trees Overfit

Ensemble Methods

Bagging (Bootstrap Aggregating)

Boosting

Bagging vs. Boosting

Unsupervised Learning

Clustering

Dimensionality Reduction

Neural Networks Introduction

Architecture

Training

When to Use Neural Networks

Common Mistakes

Cross-References

References