| name | machine-learning-foundations |
| description | Supervised and unsupervised learning, bias-variance tradeoff, cross-validation, decision trees, ensemble methods, neural network fundamentals, and the practitioner's workflow from problem framing through deployment. Covers classification, regression, clustering, dimensionality reduction, regularization, hyperparameter tuning, and evaluation metrics. Use when building predictive models, selecting algorithms, or understanding the machine learning pipeline. |
| type | skill |
| category | data-science |
| status | stable |
| origin | tibsfox |
| modified | false |
| first_seen | "2026-04-12T00:00:00.000Z" |
| first_path | examples/skills/data-science/machine-learning-foundations/SKILL.md |
| superseded_by | null |
Machine Learning Foundations
Machine learning is the practice of building systems that learn patterns from data and use those patterns to make predictions or decisions on new data. Where statistical modeling (the inference culture) asks "what is the relationship between X and Y?", machine learning (the prediction culture) asks "given X, what is the best prediction of Y?" This skill covers the foundational concepts, algorithms, and workflow of machine learning from the practitioner's perspective.
Agent affinity: breiman (algorithm selection, ensemble methods), tukey (feature engineering, EDA)
Concept IDs: data-correlation, data-distributions, data-measures-of-spread, data-hypothesis-testing
The ML Workflow
| Stage | Goal | Key operations |
|---|
| 1. Problem framing | Define the task precisely | Classification vs. regression vs. clustering; define target variable and success metric |
| 2. Data collection | Assemble training data | Sources, sampling, labeling; ensure data represents the deployment population |
| 3. Feature engineering | Create informative inputs | Domain-driven features, transformations, encoding categoricals |
| 4. Train/test split | Prevent overfitting evaluation | Hold out 20-30% for testing; never touch test set during development |
| 5. Model selection | Choose algorithm family | Based on data size, interpretability needs, problem structure |
| 6. Training | Fit model parameters | Optimization (gradient descent, tree splitting, etc.) |
| 7. Validation | Tune hyperparameters | k-fold cross-validation on training set only |
| 8. Evaluation | Assess on held-out test set | Metrics appropriate to the problem (accuracy, F1, RMSE, etc.) |
| 9. Interpretation | Understand what the model learned | Feature importance, partial dependence, SHAP values |
| 10. Deployment | Put the model in production | Monitoring, drift detection, retraining schedule |
Supervised Learning
Classification
The task: given features X, predict a categorical label y.
Key algorithms:
