Dissertation Talk: Real-World Robotic Perception and Control Using Synthetic Data

Lecture: Dissertation Talk: CS | April 29 | 1-2 p.m. | 405 Soda Hall

 Josh Tobin


Modern deep learning techniques are data-hungry, which presents a problem in robotics because real-world robotic data is difficult and expensive to collect. In contrast, simulated data is cheap and scalable, but jumping the "reality gap" to use simulated data for real-world tasks is challenging. In this talk, we discuss applications of Domain Randomization: a technique for bridging the reality gap by massively randomizing the simulator. We first show how to use domain randomization to perform object pose estimation. Then we discuss several of the limitations of the technique, including the need to build models of the environment, dealing with ambiguities arising from object symmetry, and the need for calibrated camera setups, and present methods to alleviate these limitations. We conclude with some ideas for future research in sim-to-real transfer for robotics.

 CA, jtobin@berkeley.edu, n/a