Skip to main content
optuna-hyperparameter-tuner Optuna integration skill for automated hyperparameter optimization with advanced search strategies, pruning, multi-objective optimization, and visualization capabilities.
Jump to install Skills Marketplace Discover and explore AI skills built by the community.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Show prompt details A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/a5c-ai/babysitter --skill optuna-hyperparameter-tunerThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Download Zip Downloading... More from this repository Reference for querying the Atlas knowledge graph through its MCP tools — the SECONDARY enrichment/comparison layer that adds best-practice context to systems you have ALREADY scanned from your real sources (`az`, repos, dirs). Use when you need to look up nodes, edges, kinds, clusters, stats, or wiki pages in Atlas to compare against your real inventory. (atlas graph, query atlas, atlas mcp, search the graph, graph neighbors, atlas record, atlas kinds, enrichment layer)
Atlas turns your STATED NEED into a real systems atlas by SCANNING your actual sources (Azure via `az`, git repos, local dirs) and process/data mining them, THEN enriching against the Atlas knowledge graph. Use this skill when asked to inventory/map your real systems, scan your cloud + repos + directories, mine the real processes or data they contain, or collect their real constraints/gotchas. (atlas, scan my systems, inventory our azure account, map my repos, real systems atlas, process mining, data mining, collect nuances, system discovery)
assimilate-popular-workflows This skill should be used when the user asks to "find skills in the wild", "assimilate popular workflows", "discover SKILL.md files in repos", "research external skills", "find workflow patterns", "survey the skill landscape", "what skills exist out there", or wants to investigate public repositories for extractable processes, babysitter plugins, and reusable procedural insights. Searches GitHub for SKILL.md files, classifies repos by archetype, and maintains structured research under docs/reference-repos/.
Related occupations SOC
Based on SOC occupation classification
name optuna-hyperparameter-tuner description Optuna integration skill for automated hyperparameter optimization with advanced search strategies, pruning, multi-objective optimization, and visualization capabilities. allowed-tools Read, Grep, Write, Bash, Edit, Glob, WebFetch graph {"domains":["domain:data-science"],"specializations":["specialization:data-science-ml"],"skillAreas":["skill-area:hyperparameter-tuning-experiment-management","skill-area:data-science-experimentation"],"roles":["role:ml-engineer","role:data-scientist"],"workflows":["workflow:ml-model-lifecycle","workflow:experiment-design"]}
Optuna Hyperparameter Tuner
Optimize hyperparameters using Optuna with advanced search strategies, pruning, and visualization.
Overview
This skill provides comprehensive capabilities for hyperparameter optimization using Optuna, the state-of-the-art hyperparameter optimization framework. It supports various samplers, pruners, multi-objective optimization, and integration with popular ML frameworks.
Capabilities
Search Strategies
Tree-structured Parzen Estimator (TPE) - default, efficient
CMA-ES - for continuous parameters
Grid search - exhaustive
Random search - baseline
NSGAII - multi-objective optimization
QMC (Quasi-Monte Carlo) - low-discrepancy sampling
Pruning Strategies
Median pruning - early stop underperformers
Hyperband (ASHA) - aggressive resource allocation
Percentile pruning - threshold-based
Successive Halving - efficient resource use
Wilcoxon pruning - statistical comparison
Multi-Objective Optimization
Pareto front optimization
Multiple objective functions
Constraint handling
Trade-off visualization
Study Management
Study persistence (SQLite, PostgreSQL, MySQL)
Study resumption
Parallel/distributed optimization
Trial importance analysis
Parameter relationship analysis
Visualization
Optimization history
Parameter importance
Parallel coordinate plots
Slice plots
Contour plots
Prerequisites
Installation pip install optuna>=3.0.0
Optional Dependencies
pip install optuna[mysql]
pip install optuna[postgresql]
pip install optuna-dashboard
pip install plotly
pip install optuna-integration[sklearn]
pip install optuna-integration[pytorch]
pip install optuna-integration[tensorflow]
Usage Patterns
Basic Optimization import optuna
def objective (trial ):
learning_rate = trial.suggest_float('learning_rate' , 1e-5 , 1e-1 , log=True )
n_estimators = trial.suggest_int('n_estimators' , 50 , 500 )
max_depth = trial.suggest_int('max_depth' , 3 , 15 )
subsample = trial.suggest_float('subsample' , 0.5 , 1.0 )
model = XGBClassifier(
learning_rate=learning_rate,
n_estimators=n_estimators,
max_depth=max_depth,
subsample=subsample,
random_state=42
)
score = cross_val_score(model, X_train, y_train, cv=5 , scoring='accuracy' ).mean()
return score
study = optuna.create_study(
direction='maximize' ,
study_name='xgboost-tuning' ,
storage='sqlite:///optuna.db' ,
load_if_exists=True
)
study.optimize(objective, n_trials=100 , timeout=3600 )
print (f"Best trial: {study.best_trial.number} " )
print (f"Best value: {study.best_value:.4 f} " )
print (f"Best params: {study.best_params} " )
With Pruning import optuna
from optuna.pruners import MedianPruner
def objective_with_pruning (trial ):
learning_rate = trial.suggest_float('learning_rate' , 1e-5 , 1e-1 , log=True )
n_epochs = trial.suggest_int('n_epochs' , 10 , 100 )
model = create_model(learning_rate)
for epoch in range (n_epochs):
train_loss = train_one_epoch(model)
val_accuracy = evaluate(model)
trial.report(val_accuracy, epoch)
if trial.should_prune():
raise optuna.TrialPruned()
return val_accuracy
study = optuna.create_study(
direction='maximize' ,
pruner=MedianPruner(n_startup_trials=5 , n_warmup_steps=10 )
)
study.optimize(objective_with_pruning, n_trials=100 )
Multi-Objective Optimization import optuna
def multi_objective (trial ):
learning_rate = trial.suggest_float('learning_rate' , 1e-5 , 1e-1 , log=True )
model_size = trial.suggest_categorical('model_size' , ['small' , 'medium' , 'large' ])
model = create_model(learning_rate, model_size)
train(model)
accuracy = evaluate_accuracy(model)
inference_time = measure_inference_time(model)
return accuracy, inference_time
study = optuna.create_study(
directions=['maximize' , 'minimize' ],
study_name='pareto-optimization'
)
study.optimize(multi_objective, n_trials=100 )
pareto_front = study.best_trials
for trial in pareto_front:
print (f"Accuracy: {trial.values[0 ]:.4 f} , Time: {trial.values[1 ]:.4 f} " )
Scikit-learn Integration import optuna
from optuna.integration import OptunaSearchCV
param_distributions = {
'n_estimators' : optuna.distributions.IntDistribution(50 , 500 ),
'max_depth' : optuna.distributions.IntDistribution(3 , 15 ),
'learning_rate' : optuna.distributions.FloatDistribution(1e-5 , 1e-1 , log=True ),
'subsample' : optuna.distributions.FloatDistribution(0.5 , 1.0 )
}
search = OptunaSearchCV(
XGBClassifier(random_state=42 ),
param_distributions,
n_trials=100 ,
cv=5 ,
scoring='accuracy' ,
study=study,
n_jobs=-1
)
search.fit(X_train, y_train)
print (f"Best score: {search.best_score_:.4 f} " )
print (f"Best params: {search.best_params_} " )
PyTorch Integration import optuna
from optuna.integration import PyTorchLightningPruningCallback
def objective (trial ):
lr = trial.suggest_float('lr' , 1e-5 , 1e-1 , log=True )
hidden_size = trial.suggest_int('hidden_size' , 32 , 256 )
dropout = trial.suggest_float('dropout' , 0.1 , 0.5 )
model = LightningModel(
hidden_size=hidden_size,
dropout=dropout,
lr=lr
)
trainer = pl.Trainer(
max_epochs=100 ,
callbacks=[
PyTorchLightningPruningCallback(trial, monitor='val_accuracy' )
]
)
trainer.fit(model, train_loader, val_loader)
return trainer.callback_metrics['val_accuracy' ].item()
Distributed Optimization import optuna
study = optuna.create_study(
study_name='distributed-study' ,
storage='postgresql://user:pass@host:5432/optuna' ,
direction='maximize' ,
load_if_exists=True
)
study.optimize(objective, n_trials=25 )
print (f"Total trials: {len (study.trials)} " )
Integration with Babysitter SDK
Task Definition Example const hyperparameterTuningTask = defineTask ({
name : 'optuna-hyperparameter-tuning' ,
description : 'Optimize hyperparameters using Optuna' ,
inputs : {
studyName : { type : 'string' , required : true },
direction : { type : 'string' , default : 'maximize' },
nTrials : { type : 'number' , default : 100 },
timeout : { type : 'number' },
parameterSpace : { type : 'object' , required : true },
objectiveScript : { type : 'string' , required : true },
sampler : { type : 'string' , default : 'tpe' },
pruner : { type : 'string' , default : 'median' }
},
outputs : {
bestValue : { type : 'number' },
bestParams : { type : 'object' },
nTrialsCompleted : { type : 'number' },
studyPath : { type : 'string' }
},
async run (inputs, taskCtx ) {
return {
kind : 'skill' ,
title : `Optimize: ${inputs.studyName} ` ,
skill : {
name : 'optuna-hyperparameter-tuner' ,
context : {
operation : 'optimize' ,
studyName : inputs.studyName ,
direction : inputs.direction ,
nTrials : inputs.nTrials ,
timeout : inputs.timeout ,
parameterSpace : inputs.parameterSpace ,
objectiveScript : inputs.objectiveScript ,
sampler : inputs.sampler ,
pruner : inputs.pruner
}
},
io : {
inputJsonPath : `tasks/${taskCtx.effectId} /input.json` ,
outputJsonPath : `tasks/${taskCtx.effectId} /result.json`
}
};
}
});
MCP Server Integration
Using optuna-mcp (Official) {
"mcpServers" : {
"optuna" : {
"command" : "uvx" ,
"args" : [ "optuna-mcp" ] ,
"env" : {
"OPTUNA_STORAGE" : "sqlite:///optuna.db"
}
}
}
}
Available MCP Tools
optuna_create_study - Create new optimization studyoptuna_get_study - Retrieve study informationoptuna_list_studies - List all studiesoptuna_get_best_trial - Get best trial from studyoptuna_get_trials - List trials in studyoptuna_visualize - Generate visualizationoptuna_suggest_params - Get parameter suggestions
Sampler Selection Guide Sampler Use Case Pros Cons TPESamplerDefault, most cases Efficient, handles conditionals May miss global optimum CmaEsSamplerContinuous parameters Good for correlated params Only continuous GridSamplerSmall discrete spaces Exhaustive Exponential complexity RandomSamplerBaseline, parallel Simple, embarrassingly parallel Inefficient NSGAIISamplerMulti-objective Pareto optimization Slower convergence QMCSamplerSpace exploration Low discrepancy Not adaptive
Pruner Selection Guide Pruner Use Case Aggressiveness MedianPrunerDefault, safe Moderate HyperbandPrunerDeep learning Aggressive SuccessiveHalvingPrunerResource-efficient High PercentilePrunerConfigurable threshold Variable NopPrunerNo pruning needed None
Visualization
Generate Visualizations import optuna.visualization as vis
fig = vis.plot_optimization_history(study)
fig.write_html('optimization_history.html' )
fig = vis.plot_param_importances(study)
fig.write_html('param_importance.html' )
fig = vis.plot_parallel_coordinate(study)
fig.write_html('parallel_coordinate.html' )
fig = vis.plot_contour(study, params=['learning_rate' , 'max_depth' ])
fig.write_html('contour.html' )
fig = vis.plot_slice(study)
fig.write_html('slice.html' )
Optuna Dashboard
optuna-dashboard sqlite:///optuna.db
Best Practices
Start with TPE : Use default sampler unless you have specific needsUse Pruning : Enable early stopping for iterative algorithmsPersist Studies : Use database storage for resumabilityLog Intermediate Values : Enable pruning and progress trackingSet Timeouts : Prevent runaway optimizationAnalyze Importance : Focus on high-impact parametersUse Conditional Parameters : Model dependencies between params
References 