| name | r-tidymodels |
| description | Expert R data science using tidymodels for machine learning. Use when working with tidymodels, recipes, parsnip, tune, workflows, user mentions "machine learning in R", "ML em R", "predictive modeling", "modelagem preditiva", "modelo de classificação", "classification model", "modelo de regressão", "regression model", "feature engineering", "engenharia de features", "model tuning", "tunear modelo", "tuning", "tunar", "tune boosting", "tune gradient boosting", "xgboost tuning", "random forest tuning", "hyperparameters", "hiperparâmetros", "cross-validation", "validação cruzada", "cv", "recipe steps", "data preprocessing", discusses ML workflows, or model development in R. |
| version | 1.4.0 |
| user-invocable | false |
| allowed-tools | Read, Write, Edit, Bash(Rscript *), Bash(R *), WebFetch |
R Tidymodels - Expert Machine Learning in R
Expert-level tidymodels framework knowledge for building, tuning, and deploying production-ready machine learning models in R.
Overview
This skill provides comprehensive expertise in the tidymodels ecosystem, following the principled three-phase workflow from "Tidy Modeling with R":
Phase 1: Foundation - Data splitting, model specification, preprocessing, basic evaluation
Phase 2: Optimization - Resampling, hyperparameter tuning, model comparison
Phase 3: Production-Ready - Ensembles, explainability, deployment, trustworthiness assessment
Core Principles
- Data Budgeting First - Split data before any analysis to prevent leakage
- Honest Estimation - All preprocessing must be included in validation
- Composable Workflows - Bundle recipes + models for consistency
- Prevention Over Correction - Framework prevents common pitfalls by design
- Tidy Philosophy - All outputs work seamlessly with dplyr and ggplot2
- Parallel by Default - Use parallel processing for tuning and resampling to maximize efficiency
Proactive Parallelization Guidance
When building ML pipelines, ALWAYS suggest setting up parallel processing for operations involving:
tune_grid() with 10+ grid points
tune_bayes() with iterative optimization
fit_resamples() with 5+ folds
workflow_map() comparing multiple models
- Any workflow expected to take >30 seconds
Default recommendation: Use doFuture backend (modern, cross-platform, clean API)
Essential Package Ecosystem
library(tidymodels)
library(tidyverse)
Core Packages (loaded by tidymodels):
rsample - Data splitting and resampling infrastructure
recipes - Feature engineering and preprocessing
parsnip - Unified model interface
workflows - Bundle preprocessing + modeling
tune - Hyperparameter optimization
yardstick - Performance metrics
broom - Tidy model outputs
dials - Tuning parameter management
Specialized Extensions:
themis - Class imbalance (SMOTE, upsampling)
embed - Advanced encoding (target encoding, embeddings)
textrecipes - Text preprocessing
stacks - Model ensembling
finetune - Advanced tuning strategies
probably - Probability calibration
applicable - Applicability domain assessment
vip - Variable importance plots
Dynamic Reference Lookup
This skill combines curated local knowledge (~100 recipe steps, ~50 models) with the ability to search the complete tidymodels reference (300+ steps, 160+ models) when needed.
When to Use Dynamic Lookup
Use WebFetch to search online when:
- User asks about a specific recipe step not in local knowledge
- User needs to find models with specific capabilities
- User asks about prediction type support for model/engine combinations
- User needs sparse data compatibility information
- Searching for recently added tidymodels functionality
Use local knowledge when:
- Providing general guidance on workflow and best practices
- Explaining common patterns and step ordering
- Teaching concepts and principles
- Most common recipe steps and models (covered comprehensively)
Available Search Tools
1. Recipe Steps Search (300+ steps)
- URL: https://www.tidymodels.org/find/recipes/
- Columns: Title (description), Topic (function name), Package
- Use for: Finding specific preprocessing steps, exploring packages like themis, textrecipes, embed
Example queries:
WebFetch: "Search for recipe steps related to 'holiday' or 'date features'"
WebFetch: "Find all SMOTE-related recipe steps from themis package"
2. Parsnip Models Search (160+ models)
Example queries:
WebFetch: "Find all gradient boosting models available in parsnip"
WebFetch: "What engines are available for neural networks?"
3. Prediction Types Matrix
Example queries:
WebFetch: "Does svm_rbf with liquidSVM engine support confidence intervals?"
WebFetch: "Which random forest engines support prediction intervals?"
4. Sparse Data Compatibility
Example queries:
WebFetch: "Which recipe steps work with sparse matrices?"
WebFetch: "Can I use step_pca with sparse data?"
5. Complete Tidymodels Search
Search Pattern
When user asks about specific functionality:
- Check local knowledge first - Most common steps/models are documented
- If not found or user asks for comprehensive list - Use WebFetch to search appropriate tool
- Extract relevant information - Function names, descriptions, packages
- Provide context - How it fits in workflow, when to use, example code
Example workflow:
User: "Is there a recipe step for handling holidays in date data?"
1. Check local knowledge → step_holiday() is documented ✓
2. Provide answer with example from local knowledge
User: "What are ALL the date-related recipe steps available?"
1. Local knowledge has main ones (step_date, step_holiday, step_time)
2. Use WebFetch to search https://www.tidymodels.org/find/recipes/ for "date"
3. Return comprehensive list with descriptions
Phase 1: Foundation Workflow
Step 1: Data Splitting Strategy
library(tidymodels)
library(tidyverse)
data(ames, package = "modeldata")
set.seed(123)
ames_split <- initial_split(ames, prop = 0.80, strata = Sale_Price)
ames_train <- training(ames_split)
ames_test <- testing(ames_split)
set.seed(234)
ames_val <- initial_validation_split(ames, prop = c(0.6, 0.2), strata = Sale_Price)
Splitting Functions:
initial_split() - Simple train/test split
initial_validation_split() - Train/validation/test split
initial_time_split() - Time series split
group_initial_split() - Split by groups
- Always use
strata for classification and skewed outcomes
Step 2: Feature Engineering with Recipes
ames_rec <- recipe(Sale_Price ~ ., data = ames_train) |>
update_role(Id, new_role = "ID") |>
step_impute_median(all_numeric_predictors()) |>
step_impute_mode(all_nominal_predictors()) |>
step_mutate(
House_Age = Year_Sold - Year_Built,
Remod_Age = Year_Sold - Year_Remod_Add,
Total_SF = Gr_Liv_Area + Total_Bsmt_SF
) |>
step_log(Sale_Price, base = 10) |>
step_novel(all_nominal_predictors()) |>
step_unknown(all_nominal_predictors()) |>
step_other(all_nominal_predictors(), threshold = 0.01) |>
step_dummy(all_nominal_predictors(), one_hot = FALSE) |>
step_zv(all_predictors()) |>
step_nzv(all_predictors()) |>
step_normalize(all_numeric_predictors()) |>
step_corr(all_numeric_predictors(), threshold = 0.9)
Recipe Step Order (Critical):
- Update roles (ID variables, case weights)
- Handle missing data
- Create new features
- Transform outcomes
- Handle novel/unknown factor levels
- Pool infrequent categories
- Create dummy variables
- Remove zero/near-zero variance
- Normalize/scale numeric predictors
- Remove correlations or apply dimensionality reduction
Role Selectors:
all_predictors() / all_outcomes() - By role
all_numeric_predictors() / all_nominal_predictors() - By type
has_role("ID") / has_type("date") - Specific criteria
- Never hard-code column names if avoidable
See references/recipe-steps-guide.md for complete step catalog.
Step 3: Model Specification with Parsnip
rf_spec <- rand_forest(
mtry = tune(),
trees = 1000,
min_n = tune()
) |>
set_engine("ranger", importance = "impurity") |>
set_mode("regression")
xgb_spec <- boost_tree(
trees = tune(),
tree_depth = tune(),
min_n = tune(),
learn_rate = tune(),
loss_reduction = tune()
) |>
set_engine("xgboost") |>
set_mode("regression")
glmnet_spec <- linear_reg(
penalty = tune(),
mixture = tune()
) |>
set_engine("glmnet")
Key Model Functions:
| Model Type | Function | Modes | Common Engines |
|---|
| Linear/Logistic Reg | linear_reg() / logistic_reg() | regression / classification | glm, glmnet, stan |
| Decision Trees | decision_tree() | both | rpart, C5.0 |
| Random Forest | rand_forest() | both | ranger, randomForest |
| Boosted Trees | boost_tree() | both | xgboost, lightgbm |
| SVM | svm_rbf(), svm_poly() | both | kernlab |
| Neural Networks | mlp() | both | nnet, keras, brulee |
| Nearest Neighbors | nearest_neighbor() | both | kknn |
| Naive Bayes | naive_Bayes() | classification | klaR, naivebayes |
Tuning Parameters:
- Mark with
tune() for hyperparameter optimization
- Use
set_engine() for implementation-specific options
- Always set
mode explicitly
Step 4: Create Workflow
rf_wflow <- workflow() |>
add_recipe(ames_rec) |>
add_model(rf_spec)
rf_wflow_formula <- workflow() |>
add_formula(Sale_Price ~ Lot_Area + Neighborhood) |>
add_model(rf_spec)
rf_wflow_weighted <- workflow() |>
add_recipe(ames_rec) |>
add_model(rf_spec) |>
add_case_weights(weight_column)
Why Workflows:
- Ensures preprocessing consistency across train/test/predict
- Simplifies tuning (tunes both recipe and model parameters)
- Bundles everything for deployment
- Prevents preprocessing from being excluded from validation
Step 5: Basic Evaluation
ames_fit <- rf_wflow |>
fit(data = ames_train)
ames_pred <- augment(ames_fit, new_data = ames_test)
ames_pred |>
metrics(truth = Sale_Price, estimate = .pred)
Phase 2: Optimization Workflow
Step 1: Create Resampling Strategy
set.seed(345)
ames_folds <- vfold_cv(ames_train, v = 10, strata = Sale_Price)
ames_folds_rep <- vfold_cv(ames_train, v = 10, repeats = 3, strata = Sale_Price)
ames_boots <- bootstraps(ames_train, times = 25, strata = Sale_Price)
ames_mc <- mc_cv(ames_train, prop = 0.9, times = 20, strata = Sale_Price)
time_folds <- rolling_origin(
time_data,
initial = 365,
assess = 30,
skip = 29,
cumulative = TRUE
)
Resampling Strategy Guide:
- 10-fold CV: Default choice, good balance
- Repeated CV: When you need more robust estimates
- Bootstrap: For small datasets or confidence intervals
- Monte Carlo: For very large datasets
- Rolling origin: For time series only
Step 2: Evaluate Without Tuning
rf_res <- rf_wflow |>
fit_resamples(
resamples = ames_folds,
metrics = metric_set(rmse, rsq, mae),
control = control_resamples(save_pred = TRUE)
)
collect_metrics(rf_res)
collect_predictions(rf_res) |>
ggplot(aes(x = Sale_Price, y = .pred)) +
geom_abline(lty = 2) +
geom_point(alpha = 0.3) +
coord_obs_pred()
Step 3: Hyperparameter Tuning - Grid Search
library(doFuture)
registerDoFuture()
plan(multisession, workers = parallel::detectCores() - 1)
rf_grid <- grid_latin_hypercube(
mtry(range = c(10, 30)),
min_n(range = c(2, 10)),
size = 20
)
rf_tuned <- rf_wflow |>
tune_grid(
resamples = ames_folds,
grid = rf_grid,
metrics = metric_set(rmse, rsq, mae),
control = control_grid(
save_pred = TRUE,
verbose = TRUE,
parallel_over = "everything"
)
)
show_best(rf_tuned, metric = "rmse", n = 5)
autoplot(rf_tuned, metric = "rmse")
best_rmse <- select_best(rf_tuned, metric = "rmse")
Grid Strategies:
grid_regular() - Full factorial grid (can be huge)
grid_random() - Random search
grid_latin_hypercube() - Recommended: space-filling design
- Start with 20-30 points for initial exploration
Step 4: Iterative Tuning (Bayesian Optimization)
ctrl_bayes <- control_bayes(
no_improve = 10,
verbose = TRUE,
save_pred = TRUE,
parallel_over = "everything"
)
xgb_params <- extract_parameter_set_dials(xgb_wflow) |>
update(
trees = trees(range = c(100, 2000)),
learn_rate = learn_rate(range = c(-3, -0.5))
)
set.seed(456)
xgb_bayes <- xgb_wflow |>
tune_bayes(
resamples = ames_folds,
param_info = xgb_params,
initial = 10,
iter = 50,
metrics = metric_set(rmse, rsq),
control = ctrl_bayes
)
autoplot(xgb_bayes, type = "performance")
autoplot(xgb_bayes, type = "parameters")
Iterative Strategies:
tune_bayes() - Bayesian optimization (best for expensive models)
tune_sim_anneal() - Simulated annealing
tune_race_anova() - Racing with ANOVA (from finetune package)
Step 5: Compare Multiple Models
wf_set <- workflow_set(
preproc = list(basic = ames_rec),
models = list(
rf = rf_spec,
xgb = xgb_spec,
glmnet = glmnet_spec
)
)
wf_results <- wf_set |>
workflow_map(
fn = "tune_grid",
resamples = ames_folds,
grid = 20,
metrics = metric_set(rmse, rsq),
verbose = TRUE
)
rank_results(wf_results, rank_metric = "rmse", select_best = TRUE)
autoplot(wf_results, metric = "rmse")
Step 6: Finalize and Test
final_wflow <- rf_wflow |>
finalize_workflow(best_rmse)
final_fit <- final_wflow |>
last_fit(ames_split, metrics = metric_set(rmse, rsq, mae))
collect_metrics(final_fit)
collect_predictions(final_fit) |>
ggplot(aes(x = Sale_Price, y = .pred)) +
geom_abline(lty = 2) +
geom_point(alpha = 0.5) +
coord_obs_pred()
final_model <- extract_workflow(final_fit)
Phase 3: Production-Ready Models
Variable Importance & Interpretability
library(vip)
final_fit |>
extract_fit_parsnip() |>
vip(num_features = 20, geom = "point")
glmnet_fit |>
extract_fit_parsnip() |>
tidy() |>
filter(term != "(Intercept)") |>
ggplot(aes(x = estimate, y = reorder(term, estimate))) +
geom_col()
Model Stacking (Ensembles)
library(stacks)
model_st <- stacks() |>
add_candidates(rf_tuned) |>
add_candidates(xgb_tuned) |>
add_candidates(glmnet_tuned)
ensemble_fit <- model_st |>
blend_predictions(
penalty = 10^(-6:-1),
mixture = c(0, 0.5, 1)
) |>
fit_members()
autoplot(ensemble_fit, type = "weights")
predict(ensemble_fit, new_data = ames_test)
Class Imbalance Handling
library(themis)
balanced_rec <- recipe(class ~ ., data = train_data) |>
step_upsample(class, over_ratio = 0.8) |>
step_normalize(all_numeric_predictors())
Probability Calibration
library(probably)
cal_obj <- rf_res |>
collect_predictions() |>
cal_estimate_beta(truth = class, estimate = dplyr::starts_with(".pred_"))
calibrated_preds <- augment(rf_fit, new_data = test_data) |>
cal_apply(cal_obj)
cal_plot_breaks(cal_obj)
Model Deployment
saveRDS(final_model, "models/ames_rf_model.rds")
model <- readRDS("models/ames_rf_model.rds")
predictions <- predict(model, new_data = new_houses)
predict_sale_price <- function(new_data) {
model <- readRDS("models/ames_rf_model.rds")
pred <- predict(model, new_data = new_data) |>
bind_cols(
predict(model, new_data = new_data, type = "conf_int")
)
return(pred)
}
Best Practices & Common Pitfalls
✅ DO:
- Split data first - Before any exploration or analysis
- Stratify splits - Use
strata for classification and skewed outcomes
- Use workflows - Bundle recipe + model for consistency
- Set seeds - For reproducibility:
set.seed(123)
- Setup parallel processing - Use
doFuture or doParallel for tuning/resampling
- Include preprocessing in validation - Always use recipes within workflows
- Use role selectors -
all_numeric_predictors() instead of hard-coded names
- Handle novel levels -
step_novel() before step_dummy()
- Normalize after dummies - Create indicators first, then scale
- Multiple metrics - Use
metric_set(rmse, rsq, mae) for comprehensive view
- Visualize tuning - Use
autoplot() to understand parameter effects
❌ DON'T:
- Preprocess before splitting - Causes data leakage
- Tune on test set - Only use for final evaluation
- Skip validation - Always use resampling, never just training metrics
- Forget
step_zv() - Dummy coding can create zero-variance predictors
- Normalize before dummies - Order matters!
- Ignore novel factor levels - Will error in production
- Use
step_dummy(all_predictors()) - Excludes numeric predictors, use all_nominal_predictors()
- Evaluate only accuracy - Consider business costs and trade-offs
- Skip test set evaluation - Training/CV metrics can be optimistic
- Trust all predictions equally - Assess applicability domain
Performance Metrics Guide
Regression Metrics
metric_set(rmse, rsq, mae, mape, huber_loss)
rmse - Root mean squared error (penalizes large errors)
rsq - R-squared (proportion of variance explained)
mae - Mean absolute error (robust to outliers)
mape - Mean absolute percentage error
huber_loss - Combination of MSE and MAE
Classification Metrics
metric_set(accuracy, roc_auc, pr_auc, f_meas, sensitivity, specificity)
metric_set(accuracy, roc_auc, mn_log_loss, bal_accuracy)
accuracy - Overall correct predictions
roc_auc - Area under ROC curve
pr_auc - Area under precision-recall curve
f_meas - F1 score (harmonic mean of precision and recall)
sensitivity - True positive rate (recall)
specificity - True negative rate
bal_accuracy - Balanced accuracy for imbalanced data
mn_log_loss - Multinomial log loss
Parallel Processing (Always Recommend for Tuning)
When to Use Parallelization
ALWAYS suggest parallel processing when the user is:
- Running
tune_grid() with 10+ grid configurations
- Running
tune_bayes() with iterative optimization
- Running
fit_resamples() with 5+ CV folds
- Using
workflow_map() to compare multiple models
- Building any ML pipeline expected to take >30 seconds
Ask proactively: "Would you like me to set up parallel processing? This can speed up tuning by 3-7x depending on available cores."
Recommended Setup (doFuture - Modern & Cross-Platform)
library(doFuture)
registerDoFuture()
plan(multisession, workers = parallel::detectCores() - 1)
tuned_results <- tune_grid(
workflow,
resamples = folds,
grid = param_grid,
control = control_grid(parallel_over = "everything")
)
plan(sequential)
Alternative Setup (doParallel - Traditional)
library(doParallel)
cl <- makePSOCKcluster(parallel::detectCores() - 1)
registerDoParallel(cl)
tuned_results <- tune_grid(
workflow,
resamples = folds,
grid = param_grid,
control = control_grid(parallel_over = "everything")
)
stopCluster(cl)
registerDoSEQ()
Backend Comparison
| Backend | Pros | Cons | Best For |
|---|
| doFuture | Modern, cross-platform, auto-cleanup, flexible | Slightly more setup | Recommended default |
| doParallel (PSOCK) | Stable, well-tested, cross-platform | Manual cleanup needed | Production stability |
| doParallel (fork) | Lowest overhead, fast | Unix/Mac only | Mac/Linux power users |
Control Options
control_grid(parallel_over = "everything")
control_grid(parallel_over = "resamples")
control_grid(parallel_over = NULL)
Expected Performance Gains
- 10-fold CV with 4 cores → ~3.5x speedup
- Grid search (50 points) with 8 cores → ~7x speedup
- Bayesian tuning (50 iterations) with 8 cores → ~6x speedup
- workflow_map (5 models) with 4 cores → ~3.8x speedup
Quick Reference
Essential Workflow Pattern (With Parallel Processing)
library(tidymodels)
library(tidyverse)
library(doFuture)
registerDoFuture()
plan(multisession, workers = parallel::detectCores() - 1)
split <- initial_split(data, prop = 0.8, strata = outcome)
train <- training(split)
test <- testing(split)
rec <- recipe(outcome ~ ., data = train) |>
step_impute_median(all_numeric_predictors()) |>
step_novel(all_nominal_predictors()) |>
step_dummy(all_nominal_predictors()) |>
step_zv(all_predictors()) |>
step_normalize(all_numeric_predictors())
spec <- rand_forest(mtry = tune(), min_n = tune()) |>
set_engine("ranger") |>
set_mode("classification")
wflow <- workflow() |>
add_recipe(rec) |>
add_model(spec)
folds <- vfold_cv(train, v = 10, strata = outcome)
results <- wflow |>
tune_grid(
resamples = folds,
grid = grid_latin_hypercube(mtry(), min_n(), size = 20),
control = control_grid(parallel_over = "everything")
)
best <- select_best(results, metric = "roc_auc")
final_wflow <- finalize_workflow(wflow, best)
final_fit <- last_fit(final_wflow, split)
collect_metrics(final_fit)
plan(sequential)
Supporting Resources
External Resources