Recap: Videos From RE•WORK Deep Learning Summit, San Francisco 2016

by Nikita Johnson

At this year’s Deep Learning Summit in San Francisco, over four hundred world experts, data scientists and entrepreneurs gathered for two days to explore the opportunities of advancing trends in deep learning and their impact on business and society. Over the two days of the summit, we had a mix panel discussions, presentations and fireside chats, as well as investor meetings, media briefings and new partnerships being formed. Topics and sessions included natural language processing, recurrent networks, unsupervised learning, computer vision and applications in e-commerce, healthcare, robotics and more.

Visualizing and Understanding Recurrent Networks
Andrej Karpathy, Research Scientist at OpenAI

Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LSTMs provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood. Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing a comprehensive analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, an extensive analysis with finite horizon n-gram models suggest that these dependencies are actively discovered and utilized by the networks. Finally, we provide detailed error analysis that suggests areas for further study.

Deep Reinforcement Learning for Robotics
Pieter Abbeel, Associate Professor, UC Berkeley

Deep learning has enabled significant advances in supervised learning problems such as speech recognition and visual recognition. Reinforcement learning provides only a weaker supervisory signal, posing additional challenges in the form of temporal credit assignment and exploration. Nevertheless, deep reinforcement learning has already enabled learning to play Atari games from raw pixels (without access to the underlying game state) and learning certain types of visuomotor manipulation primitives. In this talk, Pieter discusses major challenges for, as well as some preliminary promising results towards, making deep reinforcement learning applicable to real robotic problems. View more videos from the summit on our playlist here. Join us at future Deep Learning Summit:

View all upcoming RE•WORK summits here.