Evolution Strategies

An optimizer can search for better model parameters even when it cannot differentiate the performance measure. Lilian Weng's 2019 evolution-strategies tutorial describes maintaining a distribution of candidate parameter settings, sampling candidates, evaluating their scores and shifting the distribution toward stronger results.

In the simple Gaussian version, that distribution has a center and a spread. Higher-scoring candidates guide where the next population is sampled. The required feedback is an evaluation of each candidate, which makes the approach useful when the objective is accessible only through trials. The search still needs those evaluations; avoiding derivatives does not remove the work of testing possible solutions.

“Stochastic gradient descent is a universal choice for optimizing deep learning models.”

“With black-box optimization algorithms, you can evaluate a target function $f(x): \mathbb{R}^n \to \mathbb{R}$, even when you don’t know the precise analytic form of $f(x)$ and thus cannot compute gradients or the Hessian matrix.”