statistical-modeling

name	statistical-modeling
description	Regression analysis, ANOVA, generalized linear models, Bayesian methods, and model selection. Covers the full modeling workflow from problem formulation through diagnostics -- linear regression, logistic regression, Poisson regression, mixed-effects models, prior specification, posterior inference, AIC/BIC comparison, cross-validation for model selection, and assumption checking. Use when fitting models, testing hypotheses, or selecting among competing statistical explanations.
type	skill
category	data-science
status	stable
origin	tibsfox
modified	false
first_seen	"2026-04-12T00:00:00.000Z"
first_path	examples/skills/data-science/statistical-modeling/SKILL.md
superseded_by	null

Statistical Modeling

Statistical modeling is the practice of fitting mathematical structures to data in order to quantify relationships, test hypotheses, and make predictions. Unlike machine learning, which optimizes prediction, statistical modeling privileges interpretability and inference -- understanding why variables relate, not just that they do. Leo Breiman's "two cultures" paper (2001) crystallized this distinction. This skill covers the inferential tradition while acknowledging where the two cultures overlap.

Agent affinity: tukey (EDA and diagnostics), fisher (experimental design and ANOVA), breiman (model comparison)

Concept IDs: data-hypothesis-testing, data-confidence-intervals, data-correlation, data-normal-distribution

The Modeling Workflow

Stage	Goal	Key operations
1. Formulation	Define the question as a model	Specify response variable, predictors, functional form
2. Exploration	Understand data structure	Scatterplots, correlation matrices, distribution checks
3. Fitting	Estimate parameters	OLS, MLE, MCMC, IRLS depending on model class
4. Diagnostics	Check assumptions	Residual plots, Q-Q plots, leverage, VIF
5. Inference	Draw conclusions	Confidence intervals, hypothesis tests, effect sizes
6. Selection	Compare models	AIC, BIC, cross-validation, likelihood ratio tests
7. Communication	Report results	Effect estimates with uncertainty, not just p-values

Linear Regression

The Model

y = beta_0 + beta_1 * x_1 + beta_2 * x_2 + ... + beta_p * x_p + epsilon

where epsilon ~ N(0, sigma^2) independently. The betas are estimated by ordinary least squares (OLS), minimizing the sum of squared residuals.

Assumptions (LINE)

Assumption	Check	Violation consequence
Linearity	Residual vs. fitted plot -- no pattern	Biased estimates, meaningless coefficients
Independence	Study design, Durbin-Watson test	Underestimated standard errors, inflated significance