Using games to train machine learning models has proven to be increasingly successful in recent years, and has helped to spread public understanding of algorithms and deep learning methods through mainstream media coverage, such as Google DeepMind's use of Atari, Montezuma's Revenge, Space Invaders and more. But why are video games a particularly effective element in training models? Many games allow the AI to experiment, and that is helping computers learn how to do things that no programmer could teach them. Junhyuk Oh, a PhD student in Computer Science & Engineering at the University of Michigan, is working at the intersection between deep learning and reinforcement learning with the game Minecraft, using a set of reinforcement learning tasks to systematically compare existing deep reinforcement learning (DRL) architectures with new memory-based architectures. These tasks are designed to emphasize issues that pose challenges including partial observability, delayed rewards, high-dimensional visual observations, and the need to use active perception so as to perform well in the tasks. I asked him a few questions ahead of his presentation at the Machine Intelligence Summit in San Fran later this month to learn more about this research.Tell us more about your research at University of Michigan.My current research focuses on deep reinforcement learning especially for handling partial observability, improving exploration, and better generalization. Other than deep reinforcement learning, I have a broad interest in deep generative models and modeling sequential data with application to video and text. What do you feel are the leading factors enabling recent advancements in deep reinforcement learning?I think the recent advancements mainly come from the advances in deep learning (e.g., GPU, better optimization methods, etc).Researchers in the deep reinforcement learning area have also developed many techniques for training highly non-linear functions from noisy error signals in reinforcement learning.In addition, good benchmarks like Arcade Learning Environment (Atari games) are also playing an important role by allowing us to systematically compare different algorithms.I believe that more challenging and interesting benchmarks (e.g., 3D world, language-based environment) will push the boundaries further. What present or potential future applications of Machine Intelligence excite you most?Autonomous driving is one of the exciting applications because this may change our lives.I think healthcare (e.g., diagnosis) would be also one interesting future application. Which industries will be most disrupted by Machine Intelligence?In the short-term future, AI will be useful for some tasks that require fast reactive time with low-level decisions and discovering complex patterns from massive data.Car industry and medical industry are both examples that could be highly affected by AI. What developments can we expect to see in Machine Intelligence in the next 5 years?I think interesting things are happening in generative modeling and unsupervised/semi-supervised learning. This may affect the deep reinforcement learning area for learning better representation without reward signal.People have also started to apply deep reinforcement learning approaches to natural language processing problems such as dialog system.Program induction and synthesis have been receiving good attention in the recent years, which can be very cool if they work very well. Junhyuk Oh will be speaking at the Machine Intelligence Summit, taking place alongside the Machine Intelligence in Autonomous Vehicles Summit in San Francisco on 23-24 March. Newly confirmed speakers include Robinson Piramuthu, Head of AI, New Product Development at eBay; Stuart Feffer, Co-Founder & CEO of Reality AI; Hossein Taghavi, Senior Research Engineer at Netflix; and Melody Guan, Deep Learning Resident at Google Brain. View more speakers and topics here.
Tickets are limited for this event. Register your place now.