nevergrad-skill

name

nevergrad-skill

description

Gradient-free optimization using Facebook Research's nevergrad library. Use when performing black-box optimization, hyperparameter tuning, evolutionary optimization, derivative-free optimization, or any task requiring minimization of a function without gradients. Triggers include mentions of "nevergrad", "gradient-free optimization", "black-box optimization", "hyperparameter tuning", "evolutionary algorithm", "CMA-ES", "differential evolution", "particle swarm", or requests to optimize parameters of a simulation, model, or external code without access to derivatives.

Nevergrad Skill

Gradient-free optimization platform (Facebook Research, MIT license). Provides 100+ optimizers for continuous, discrete, and mixed parameter spaces with parallel evaluation support.

Environment Setup

Run the setup script to create a virtual environment with nevergrad:

# Core installation (optimization + parametrization)
bash scripts/setup_env.sh [env-name] [parent-dir]
# Default: creates ./nevergrad-env

# With benchmark suite (adds torch, tensorflow, gym, etc.)
bash scripts/setup_env_benchmark.sh [env-name] [parent-dir]

Or install manually:

pip install "numpy<2" && pip install nevergrad   # core (numpy<2 avoids metamodel bug)
pip install "numpy<2" && pip install nevergrad[benchmark]  # + benchmarks

Important: nevergrad 1.0.12 has a compatibility issue with numpy 2.x in the metamodel module (NGOpt fails with TypeError: only 0-dimensional arrays can be converted to Python scalars). Always pin numpy<2.

Quick Start

Simplest case: minimize f(x) over a continuous array

import nevergrad as ng

def objective(x):
    return sum((x - 0.5) ** 2)

optimizer = ng.optimizers.NGOpt(parametrization=2, budget=100)
recommendation = optimizer.minimize(objective)
print(recommendation.value)  # ~[0.5, 0.5]

ML hyperparameter tuning

import nevergrad as ng

def train_and_eval(learning_rate: float, batch_size: int, architecture: str) -> float:
    # ... your training code ...
    return validation_loss

parametrization = ng.p.Instrumentation(
    learning_rate=ng.p.Log(lower=0.001, upper=1.0),
    batch_size=ng.p.Scalar(lower=1, upper=128).set_integer_casting(),
    architecture=ng.p.Choice(["conv", "fc", "transformer"]),
)

optimizer = ng.optimizers.NGOpt(parametrization=parametrization, budget=200)
recommendation = optimizer.minimize(train_and_eval)
print(recommendation.kwargs)

Ask/Tell interface (full control)

import nevergrad as ng

instrum = ng.p.Instrumentation(ng.p.Array(shape=(3,)), y=ng.p.Scalar())
optimizer = ng.optimizers.NGOpt(parametrization=instrum, budget=100, num_workers=4)

for _ in range(optimizer.budget):
    candidate = optimizer.ask()
    loss = my_function(*candidate.args, **candidate.kwargs)
    optimizer.tell(candidate, loss)

best = optimizer.provide_recommendation()

Optimizer Selection

Default: NGOpt (adaptive, auto-selects strategy based on budget/workers/params).

Scenario	Optimizer
General / unsure	NGOpt
Continuous, budget > 1000*dim	CMA
High parallelism	TwoPointsDE or PSO
Discrete / mixed	PortfolioDiscreteOnePlusOne
Noisy objective	TBPSA
One-shot (workers=budget)	ScrHammersleySearchPlusMiddlePoint
Multi-objective	TwoPointsDE

For detailed optimizer selection guidance, see references/optimizer_guide.md.

Parameter Types

Type	Purpose	Example
Array(shape)	Continuous array	ng.p.Array(shape=(5,)).set_bounds(0, 1)
Scalar()	Single value	ng.p.Scalar(lower=0, upper=10)
Log(lower, upper)	Log-scale scalar	ng.p.Log(lower=1e-4, upper=1.0)
Choice(items)	Unordered categorical	ng.p.Choice(["a", "b", "c"])
TransitionChoice(items)	Ordered categorical	ng.p.TransitionChoice(range(10))
Instrumentation(*a, **kw)	Function args	Multi-arg parametrization
Dict(**kw) / Tuple(*a)	Containers	Named/positional groups

Integer casting: .set_integer_casting() on any Array/Scalar. Bounds: .set_bounds(lower, upper) on Array/Scalar. Mutation sigma: .set_mutation(sigma=0.1) on Array.

For full parametrization reference, see references/parametrization_guide.md.

Constraints

# Float-valued cheap constraint (preferred, >= 0 means OK)
optimizer.parametrization.register_cheap_constraint(lambda x: x[0] - 1.0)

# Constraint violations via tell
optimizer.tell(candidate, loss, [constraint_violation1, constraint_violation2])

# Constraints via minimize
optimizer.minimize(loss_fn, constraint_violations=constraint_fn)

Parallel Execution

from concurrent import futures

optimizer = ng.optimizers.NGOpt(parametrization=10, budget=100, num_workers=4)
with futures.ProcessPoolExecutor(max_workers=4) as executor:
    recommendation = optimizer.minimize(objective, executor=executor, batch_mode=False)

Advanced Topics

For multi-objective optimization, chaining optimizers, callbacks, warm starting, reproducibility, and inoculation, see references/advanced_patterns.md.