메뉴
Use when optimizing model hyperparameters. Covers search strategy (random vs Bayesian/Optuna), leakage-safe tuning inside CV, search-space design, early stopping, and budget management.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | hyperparameter-tuning |
| description | Use when optimizing model hyperparameters. Covers search strategy (random vs Bayesian/Optuna), leakage-safe tuning inside CV, search-space design, early stopping, and budget management. |
Tuning squeezes the last 5-15% out of a model — but done carelessly it overfits the validation set and leaks preprocessing. The rules: tune the whole pipeline inside cross-validation, search smart (not grid), and keep a final untouched test set.
| Situation | Method |
|---|---|
| Few params, cheap model | GridSearchCV |
| Many params / continuous | RandomizedSearchCV (often beats grid per compute) |
| Expensive model, want efficiency | Bayesian / Optuna (TPE) |
| Neural nets | Optuna + early stopping + pruning |
import optuna
from sklearn.model_selection import cross_val_score, StratifiedKFold
cv = StratifiedKFold(5, shuffle=True, random_state=42)
def objective(trial):
params = {
"clf__learning_rate": trial.suggest_float("lr", 1e-3, 0.3, log=True),
"clf__max_depth": trial.suggest_int("max_depth", 3, 12),
"clf__l2_regularization": trial.suggest_float("l2", 1e-3, 10, log=True),
}
model.set_params(**params)
scores = cross_val_score(model, X_train, y_train, cv=cv, scoring="roc_auc")
return scores.mean()
study = optuna.create_study(direction="maximize",
sampler=optuna.samplers.TPESampler(seed=42))
study.optimize(objective, n_trials=50, timeout=1800)
print(study.best_params, study.best_value)
Note the clf__ prefix — you're tuning the estimator inside the pipeline, so preprocessing re-fits per fold.
n_estimators to early stopping rather than tuning it directly.timeout and n_trials ceilings; use Optuna pruning to kill hopeless trials early.Best params + a refit pipeline, logged via experiment-tracking and scored once on the holdout via model-evaluation.
Use when preparing raw data for modeling — handling missing values, duplicates, inconsistent types, outliers, and bad categorical values. Emphasizes fitting all imputation on train-only to avoid leakage.
Use when running ML experiments that need to be compared, reproduced, or shared. Covers MLflow/Weights & Biases logging, what to track, run organization, and model registry basics.
Use when starting on a new dataset, before modeling, or when asked to "explore", "profile", "understand", or "summarize" data. Covers structured EDA, distributions, correlations, target leakage checks, and visualization.
Use when creating, encoding, scaling, or selecting features for ML models. Covers categorical encoding, numeric transforms, datetime/text/aggregation features, and leakage-safe target encoding.
Use when the target is rare (fraud, churn, disease, anomalies). Covers correct metrics, resampling (SMOTE/undersampling), class weights, threshold tuning, and avoiding the accuracy trap and resampling leakage.
Use when fine-tuning a large language model. Covers choosing full vs LoRA/QLoRA, dataset formatting, the transformers/PEFT/TRL stack, key hyperparameters, and evaluating fine-tunes without overfitting.