Q&A, Junhyuk Oh, ‘Task Generalisation with Deep Reinforcement Learning’

The ability to generalise over new tasks is incredibly important in research when aiming to build scalable reinforcement learning agents. At the Deep Learning Summit in San Francisco this January 25 & 26, Junhyuk Oh, PhD Researcher at University of Michigan will discuss the challenges and progressions on reaching task generalisation with deep reinforcement learning. He will explore how to easily train agents to categorise and generalise previously unseen tasks in a zero-shot fashion.

In the run up to the event, we spoke with Junhyuk to hear more about his current work in the field.

How did you begin your work in deep learning, and more specifically in deep reinforcement learning?

I was very lucky to start my PhD by collaborating with Prof. Honglak and Prof. Satinder who are experts in deep learning and reinforcement learning respectively. I was originally interested in deep generative models before I started my PhD. However, after finishing my first project on video prediction in Atari games, I became much more interested in reinforcement learning problems because I have been curious how and why humans learn and act in the world and whether we can mathematically formulate it.

What are the main challenges in building scalable reinforcement learning and why do they exist?

I think sample complexity (data efficiency) is the main challenge for building scalable reinforcement learning agents. This is mainly because it is usually hard to get labeled data (e.g., optimal demonstrations) in many reinforcement learning domains. We have been relying on distributed computing and fast simulators to handle this issue. But, I think we need more algorithmic advances such as few-shot learning and improved generalization to overcome this challenge.

How are you using AI for a positive impact?

Although I am mostly working on reinforcement learning problems with simulators, which may not have an immediate positive impact, I believe that advanced techniques in RL will have a big impact on many future applications and products.

What challenges have you had with implementing AI software in your work and how have you overcome these?

I often find that there is no proper domain/environment to evaluate different aspects of reinforcement learning agents such as generalization and planning. I sometimes had to spend a huge amount of time implementing a new environment (e.g., Minecraft) to explore new problems and challenges. Besides, such a new environment should be very fast so that we can run large-scale experiments, which requires a lot of engineering.

What developments of AI are you most excited for, and which industries do you think will be most impacted?

I am personally excited about recent advances in generative adversarial networks (GANs), because they have been shown to be able to generate extremely realistic examples. This line of work seems very useful in fashion, music, graphics, and design industry. I think machine learning techniques in general can have immediate impacts on healthcare, smart homes, and self-driving cars for the next few years.

There’s so much discussion around AI ‘stealing’ jobs - do you think this is the case, or will we actually see more jobs being created as a result of AI?

I would say AI will make our lives easier rather than 'steal' our jobs. I agree that AI can possibly replace many existing jobs, so we should be careful and prepared for such a sudden change in the industry and the job market. Nevertheless, I believe that AI will have a huge positive impact in the long run (e.g., reduced working hours), as long as we manage such risks well.

To hear more about Junhyuk's current research and most recent progressions, join us at the Deep Learning Summit in San Francisco. In the spirit of Black Friday and Cyber Monday we're offering 25% of passes to all summits with the code CYBER25. Register before Wednesday Nov 29th to receive the discount.