Go Explore

Go-Explore makes progress on difficult exploration tasks by remembering places worth returning to. In Uber Engineering's 2018 research account, the algorithm archives game states and the paths leading to them, returns to a selected state, then explores further. This preserves progress when useful rewards are too infrequent to guide random exploration from the beginning.

The Atari experiments restored saved game states during exploration. A later learning phase used the discovered paths to train a player that could tolerate randomness. The article and its updates distinguish finding a solution in a resettable environment from making its execution reliable, a condition that matters when interpreting the results.