If you're training RL policies in simulation and struggling to deploy them on physical robots, this breaks down how domain randomization over physical parameters (friction, mass, damping) closes the reality gap and where naive sim modeling fails.
Key quotes
“Due to the sample inefficiency of deep RL algorithms and the cost of data collection on real robots, we often need to train models in a simulator which theoretically provides an infinite amount of data.”
“The gap is triggered by an inconsistency between physical parameters (i.e. friction, kp, damping, mass, density) and, more fatally, the incorrect physical modeling (i.e. collision between soft surfaces).”
