Bridges the gap between deep RL theory and working code, walking through actual TensorFlow + OpenAI Gym implementations of models most tutorials only describe abstractly. Useful for engineers who want reference implementations rather than yet another conceptual overview.
Key quotes
“The OpenAI Gym toolkit provides a set of physical simulation environments, games, and robot simulators that we can play with and design reinforcement learning agents for.”
Lilian Weng
“Deep Q-network is a seminal piece of work to make the training of Q-learning more stable and more data-efficient, when the Q value is approximated with a nonlinear function. Two key ingredients are experience replay and a separately updated target network.”
Lilian Weng
“This two-step reinforcing procedure could potentially lead to overestimation of an (already) overestimated value, further leading to training instability.”
Lilian Weng
“The dueling Q-network ( Wang et al., 2016 ) is equipped with an enhanced network architecture: the output layer branches out into two heads, one for predicting state value, V, and the other for advantage , A.”
Lilian Weng
“The actor-critic algorithm learns two models at the same time, the actor for learning the best policy and the critic for estimating the state value.”
