| name | time-series-ml |
| description | Deep time-series ML operational intuition — walk-forward CV with gap, temporal-leakage audit, probabilistic forecasting (pinball, CRPS, conformal), hierarchical reconciliation, foundation models (Chronos, TimesFM, MOIRAI), drift detection. Load when designing CV under temporal ordering, foundation-model vs tuned baseline, probabilistic intervals, reconciliation, or drift monitoring. Skip for tabular ML (use `classical-ml-pitfalls`), causal inference, or survival analysis. Triggers on: "walk-forward", "TimeSeriesSplit", "temporal leakage", "MASE", "pinball loss", "CRPS", "conformal prediction", "Chronos", "TimesFM", "MOIRAI", "N-BEATS", "PatchTST", "MinT", "ADWIN", "statsforecast", "neuralforecast".
|
Time-Series ML Operational Guide
Concise operational pointers for deep time-series and forecasting work where models tend to be shallow.
Assumes you already know what a time series is, basic ARIMA/ETS, and supervised ML. This skill covers the operational layer — walk-forward CV correctness, leakage vectors, stationarity tests, multi-seasonal decomposition, multi-step strategy, probabilistic + conformal intervals, hierarchical reconciliation, the 2024–2026 foundation-model landscape, neural architectures with DLinear as the bar — current as of late 2025/early 2026.
When to use
Load when the task is:
- Designing a CV scheme for autoregressive / lag-feature pipelines
- Deciding
gap in TimeSeriesSplit to avoid leakage between fold boundary and lag horizon
- Choosing among
statsforecast.cross_validation, mlforecast, sktime splitters, sklearn TimeSeriesSplit
- Picking direct vs recursive vs DirRec multi-step strategy for a horizon
h
- Picking a probabilistic head (quantile, distribution, conformal) for prediction intervals
- Deciding whether to fit a foundation model zero-shot, fine-tune, or train a global supervised model
- Auditing a feature pipeline for future-target / future-regressor / global-normalization leakage
- Choosing differencing order
d/D after ADF+KPSS
- Modelling multi-seasonality (intraday + weekly + yearly) via Fourier or Prophet
- Hierarchical reconciliation (BU/TD/MinT) over a product/store/region tree
- Selecting a metric — when MAPE is broken, when MASE is required
- Setting up online drift monitors (ADWIN, Page-Hinkley) and refit cadence
- Building anomaly detectors over residuals from STL or a forecast model
- Comparing a neural model against a
seasonal_naive skill-score benchmark before claiming a "win"
Do NOT load for: general supervised tabular ML (rows i.i.d.; use classical-ml-pitfalls), causal inference / DiD / synthetic control, LLM text generation, recommender / sessionised event sequences, survival analysis.
Walk-forward CV — corrections most pipelines miss
sklearn.model_selection.TimeSeriesSplit(n_splits=5, max_train_size=None, test_size=None, gap=0). The footgun: gap=0 by default. If features include lag_k of the target (or rolling stats with window w), the test fold uses targets the training fold's last rows already saw as features. Set gap ≥ max(lag_horizon, rolling_window) or, for direct-h forecasting, gap = h − 1. Without test_size, fold size = n_samples // (n_splits + 1).
sktime.split.ExpandingWindowSplitter(fh, initial_window, step_length) grows train each fold; SlidingWindowSplitter keeps it fixed (better when distribution drifts). fh is a ForecastingHorizon — relative or absolute. Both accept step_length < len(fh) to overlap test windows.
statsforecast.cross_validation(df, h, n_windows, step_size, refit) — cutoffs cutoff_max = T − h, cutoff_min = cutoff_max − (n_windows − 1)·step_size. refit=False trains once at earliest cutoff; refit=True per window; refit=k every k windows. For neural / global models with expensive fits use refit=False early, then a final refit=1 rerun for headline numbers. step_size = h non-overlapping; step_size = 1 true rolling-origin.
mlforecast uses the same cross_validation API; the recursive feature builder will silently use future rows for expanding_mean etc. unless you pass lag_transforms (lag-only).
Temporal leakage vectors (audit checklist)
(a) Future regressors at predict time — exogenous X must be known at the cutoff (calendar, promotions yes; weather forecasts only with the same lead time). statsforecast/neuralforecast call these futr_exog_list; if a column is unknown at horizon, declare hist_exog_list.
(b) Rolling stats on full series — compute pandas.Series.rolling(w).mean().shift(1) not .rolling(w).mean(). The shift(1) is the leakage fix everyone forgets.
(c) Target encoding using future labels — encode using expanding(min_periods=...).mean().shift(1).
(d) Normalization fit on full series — StandardScaler.fit(y) on the entire series leaks variance from test; fit per-train-fold. NeuralForecast/MOIRAI handle with per-window scaler_type='standard'/'robust'/'identity'.
(e) Imputation with bidirectional fill — bfill/interpolation across the cutoff. Use forward-only.
Stationarity — ADF+KPSS decision matrix, differencing
statsmodels.tsa.stattools.adfuller: H0 = unit root (non-stationary). Reject p<0.05 → "stationary".statsmodels.tsa.stattools.kpss(regression='c'|'ct'): H0 = stationary. Reject p<0.05 → "non-stationary". Opposite direction.- Combined matrix:
- ADF reject + KPSS not reject → stationary;
d=0.
- ADF not reject + KPSS reject → unit root; difference once, retest; ARIMA
d≥1.
- Both reject → trend-stationary; detrend (regress on
t), don't difference.
- Neither rejects → undetermined; usually difference and proceed.
- Seasonal differencing
D for season m: y_t − y_{t−m}. For monthly data m=12. ARIMA(p,d,q)(P,D,Q)_m. Use nsdiffs (Canova-Hansen / OCSB) for D, ndiffs for d — pmdarima.utils and statsforecast.arima.AutoARIMA both expose this.
- Box-Cox stabilises variance (positive series only); log is
λ=0. scipy.stats.boxcox returns optimal λ. Invert before scoring.
Seasonality, multi-seasonal, holidays
STL(period, seasonal=7, robust=True) (statsmodels) — robust to outliers; period must be odd ≥3.- MSTL (
statsforecast.MSTL with season_length=[24, 24*7]) for multi-seasonal data (daily-with-weekly hourly). Beats classical decompose.
- Fourier features:
sin(2πk·t/P), cos(2πk·t/P) for k=1..K. Prophet defaults: yearly K=10, weekly K=3, daily K=4. Higher K → more wiggle, more overfitting. For NeuralForecast/mlforecast use mlforecast.target_transforms.Differences and pass calendar Fourier columns as static_features or dynamic_features.
- Prophet —
changepoint_prior_scale (default 0.05) controls trend flexibility; reduce to 0.001 for stiff, raise to 0.5 for noisy. n_changepoints=25 over the first 80% of history. seasonality_mode='additive'|'multiplicative'. Holiday DataFrame with holiday, ds, optional lower_window, upper_window. Battle-tested but maintained at low velocity; NeuralProphet (PyTorch) adds AR + lagged covariates and is the contemporary alternative.
Direct vs recursive vs DirRec
- Recursive: one model, feed
ŷ_{t+1} back as input for t+2..t+h. Errors compound; unbiased only when f is linear and correctly specified (Hyndman 2014). Library default in mlforecast.MLForecast.
- Direct: train
h separate models, one per step. Avoids error compounding, no inter-step coherence, scales poorly in h. NeuralForecast does direct by default (output head sized h); statsforecast MFLES, Theta, AutoARIMA are recursive.
- DirRec / rectify: predict
ŷ_{t+1}, append, retrain, repeat. Best of both at training cost. darts.models.RegressionModel(multi_models=True, ...) exposes the choice.
- Multi-output (MIMO): single model emits the full vector. NeuralForecast (NHITS, PatchTST, iTransformer) is MIMO.
Probabilistic forecasting & conformal
- Quantile head: train with pinball loss
L_τ(y, ẑ_τ) = max(τ(y−ẑ_τ), (τ−1)(y−ẑ_τ)). NeuralForecast: loss=DistributionLoss('Normal'|'StudentT'|'NegativeBinomial') or loss=MQLoss(quantiles=[0.1,0.5,0.9]). Coverage of the 80% interval should be ≈80% on holdout — measure it (MIS / interval coverage). Calibration ≠ point accuracy; a model with worse RMSE may have well-calibrated 95% PIs and be more useful for inventory.
- CRPS =
∫(F̂(z) − 𝟙{y ≤ z})² dz; for an ensemble it's the average of |x_i − y| − 0.5·|x_i − x_j|. CRPS reduces to MAE at the median; pinball loss summed over a dense quantile grid approximates 2·CRPS.
- Conformal prediction (split / EnbPI): MAPIE 1.x exposes
MapieTimeSeriesRegressor with EnbPI (Xu & Xie 2021) — bootstrap residuals, build PIs with approximately marginal coverage 1−α without distributional assumptions. Exchangeability is violated in TS; use re-weighted / adaptive variants (ACI, AgACI). 2025: NeurIPS paper on conformal under change points; arXiv 2511.13608 is the current "gentle intro." TimeGPT uses split conformal under level=[80,95].
Local vs global, hierarchical reconciliation
- Local: one model per series (ARIMA, ETS, Theta). Strong on short, well-behaved series; collapses over thousands of items.
- Global: one model across all series with series-id embedding (mlforecast LightGBM, NeuralForecast). M5 finding: top 50 used LightGBM globally; ES benchmark beaten by ~7.5% of teams, only ML approaches won. M4 winner was ES-RNN hybrid (Smyl), 2nd was XGBoost stacking — global beats local at scale.
- Hierarchical reconciliation when series sum to higher levels.
Nixtla/hierarchicalforecast: BottomUp, TopDown(method='proportions'|'forecast_proportions'), MiddleOut, MinTrace(method='ols'|'wls_var'|'mint_shrink'), ERM. MinT (mint_shrink) uses residual covariance — minimum-variance unbiased reconciliation, generally beats BU/TD on real data (Wickramasuriya, Athanasopoulos, Hyndman 2019). Probabilistic coherent: BootstrapReconciler, Normality, PERMBU.
Foundation models 2024–2026
| Model | Architecture | Size | Niche |
|---|
| Chronos / Chronos-Bolt (Amazon) | T5 enc-dec, value tokenization via quantile binning | Tiny 9M → Large 710M; Bolt 250× faster, 20× less mem | Univariate zero-shot; Bolt-Base 205M beats original Chronos-Large |
| Chronos-2 (Oct 2025, arXiv 2510.15821) | 120M encoder-only, alternating time/group attention | 120M | Multivariate joint forecasting + covariates; tops GIFT-Eval late 2025 |
| TimeGPT (Nixtla) | Closed API; direct multi-step, conformal PIs | — | level= PIs; zero-shot + fine-tune (finetune_steps, finetune_loss, finetune_depth); anomaly endpoint |
| TimesFM (Google) | Decoder-only | 200M; v2.5 (Sep 2025) 16K context, native probabilistic head | Held #1 GIFT-Eval until Chronos-2 |
| MOIRAI (Salesforce) | Masked encoder, multi-patch, any-variate attention, mixture distribution head | LOTSA 27B obs | MoE adds sparse experts; MOIRAI 2.0 (Nov 2025) decoder-only, multi-token prediction |
| Lag-Llama | Decoder-only on lag features (Feb 2024) | open | No exogenous support — limits practical use |
| Toto (Datadog) | 151M, Student-t mixture head, factorised attention | 151M | SOTA on Datadog BOOM (350M obs, 2807 series); competitive on GIFT-Eval/LSF; Apache 2.0 |
| Moment (CMU, 2024) | T5-style multi-task | — | Forecast/classify/impute/anomaly |
Practical: zero-shot foundation models beat seasonal-naive on heterogeneous, low-volume, no-history series. They typically lose to a tuned global LightGBM/NHITS on a single rich domain with adequate history (M5-style). Always benchmark against SeasonalNaive and AutoETS before celebrating.
Neural architectures — what each is for
- N-BEATS (Oreshkin 2020) — pure MLP residual stacks, "interpretable" trend + seasonality basis. Strong univariate baseline; no covariates.
- N-HiTS (Challu 2022) — multi-rate via MaxPool on input + hierarchical interpolation on output → cheaper long horizon, ~25% lower MSE than Informer at 50× speed. Default for long-h univariate in NeuralForecast.
- TFT (Lim 2019) — variable selection per input type (static, past-known, future-known), LSTM seq2seq, multi-head interpretable attention, quantile head. Use when you have rich heterogeneous covariates and need feature attributions.
- PatchTST (Nie 2022, ICLR 2023) — channel-independent + 16-token patches → ~21% MSE reduction vs prior transformers on ETT/Traffic/Weather long-horizon.
- iTransformer (Liu 2024) — inverts the axis: each variate is a token, attention is across variates. Strong on multivariate where cross-series structure matters.
- DLinear (Zeng 2022, AAAI 2023) — series decomposition + two linear layers. Beat SOTA transformers (FEDformer, Autoformer, Informer) by 25–40% MSE on long-horizon multivariate. Still the bar: if your transformer doesn't beat DLinear, you have a paper, not a model. PatchTST and iTransformer are the two architectures that consistently do.
Evaluation metrics — what's broken
- MAPE: undefined at
y=0, asymmetric (over-forecast penalised more in percentage terms), incentivises low forecasts. Avoid for intermittent demand.
- sMAPE: bounded [0, 200%] but still asymmetric; two definitions in the wild (M3 vs M4) — quote which.
- MASE (Hyndman & Koehler 2006):
mean(|e_t|) / mean(|y_t − y_{t−m}|) over training, where m is seasonality (m=1 non-seasonal). Scale-free, defined unless series constant, MASE<1 ⇔ beats seasonal-naive. Default reporting metric for forecasting comps (M4, M5, GIFT-Eval).
- RMSE: scale-dependent, sensitive to outliers; pair with MAE.
- WAPE / WMAPE:
Σ|e_t| / Σ|y_t| — preferred over MAPE for retail.
- Skill score:
1 − metric_model / metric_baseline; baseline is SeasonalNaive. Positive = win, negative = lost to naive.
- Pinball loss / CRPS for probabilistic; MIS (Mean Interval Score, Gneiting-Raftery) for prediction intervals — reports both width and coverage.
Concept drift & refit cadence
river.drift.ADWIN(delta=0.002) — adaptive window, statistical guarantees, slowly adapts.river.drift.PageHinkley(min_instances=30, delta=0.005, threshold=50, alpha=1−1e-4) — cheaper, lowest RAM, often the most reliable in 2025 comparative studies.- KSWIN as a non-parametric alternative.
Operational pattern: monitor MAPE_t over a sliding 7-day window of 1-step residuals; trigger retrain if window MAPE > 1.5× training MAPE for 3 windows in a row, or any drift detector fires. Refit cadence guidance: daily forecasts → weekly refit; hourly → daily incremental fit; minute-level → online incremental (river, statsforecast with refit=1).
Anomaly detection in TS
- STL residuals + 3σ bands — fast, interpretable, weak on heteroscedastic noise; use IQR/MAD on residuals.
- Isolation Forest / Extended IForest on residuals — handles multivariate, struggles with seasonality unless deseasonalised first.
- LSTM-AE — reconstruction loss as score; needs labelled anomalies for threshold calibration; brittle to drift.
- Prophet residuals — built-in PIs become anomaly bounds; trend changes register as anomalies. Use
add_seasonality(prior_scale=...) and increase changepoint_prior_scale to absorb regime shifts.
- TimeGPT
detect_anomalies — conformal residuals, zero-shot. Toto/Chronos-2 likewise expose forecast-residual anomaly scoring.
Footgun: a true anomaly during a seasonal peak gets masked by the seasonal component; always detrend+deseasonalise before scoring, and inspect the seasonal component separately for changepoints.
Recent changes (2024–2026)
- Foundation models hit production: Chronos-Bolt (Aug 2024), TimesFM 1.0 (Feb 2024) → 2.5 (Sep 2025), MOIRAI 1.0 (Mar 2024) → MoE → 2.0 (Nov 2025), Toto + BOOM (May 2025), Chronos-2 multivariate (Oct 2025). Chronos-2 / TimesFM-2.5 trade #1 on GIFT-Eval roughly quarterly.
- Conformal prediction matured for TS — MAPIE 1.x, EnbPI mainstream, NeurIPS 2025 conformal-under-changepoints.
- Nixtla ecosystem consolidation:
statsforecast + mlforecast + neuralforecast + hierarchicalforecast + nixtla (TimeGPT) under one cross-compatible cross_validation API; coreforecast shared C kernels.
- DLinear is still a hard baseline three years on. Treat it as the entry tariff for a forecasting paper.
Authoritative references
Guardrails
Before recommending a non-trivial time-series change (CV gap, multi-step strategy, foundation-model adoption, conformal layer, reconciliation method):
- Quote the parameter / function and any default
- Cite the originating paper / library doc
- Make the recommendation conditional on observed evidence (MASE vs SeasonalNaive, interval coverage, drift-detector signal) — never blanket-tune
- Verify the library version. Many features gate (Chronos-2 multivariate Oct 2025, TimesFM 2.5 16K ctx Sep 2025, MAPIE 1.x conformal API)
Tuning without measurement is worse than defaults.
