Evolution Strategies
lilianweng.github.io- Category
- Other
- Type
- ARTICLE
- Builder
- @lilianweng
- Added
- Jul 21, 2026
About
Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option. With black-box optimization algorithms, you can evaluate a target function $f(x): \mathbb{R}^n \to \mathbb{R}$, even when you don’t know the precise analytic form of $f(x)$ and thus cannot compute gradients or the Hessian matrix. Examples of black-box optimization methods include Simulated Annealing , Hill Climbing and Nelder-Mead method .
Why it made the leaderboard
Explains how evolution strategies can optimize objectives where gradients are unavailable or unreliable, giving engineers a practical alternative to SGD for black-box or non-differentiable problems like RL reward shaping and hyperparameter tuning.
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.