At this year’s Deep Learning Summit in San Francisco, over four hundred world experts, data scientists and entrepreneurs gathered for two days to explore the opportunities of advancing trends in deep learning and their impact on business and society. Over the two days of the summit, we had a mix panel discussions, presentations and fireside chats, as well as investor meetings, media briefings and new partnerships being formed. Topics and sessions included natural language processing, recurrent networks, unsupervised learning, computer vision and applications in e-commerce, healthcare, robotics and more.
Visualizing and Understanding Recurrent Networks
Andrej Karpathy, Research Scientist at OpenAI
Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LSTMs provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood. Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing a comprehensive analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, an extensive analysis with finite horizon n-gram models suggest that these dependencies are actively discovered and utilized by the networks. Finally, we provide detailed error analysis that suggests areas for further study.
Deep Reinforcement Learning for Robotics
Pieter Abbeel, Associate Professor, UC Berkeley
Deep learning has enabled significant advances in supervised learning problems such as speech recognition and visual recognition. Reinforcement learning provides only a weaker supervisory signal, posing additional challenges in the form of temporal credit assignment and exploration. Nevertheless, deep reinforcement learning has already enabled learning to play Atari games from raw pixels (without access to the underlying game state) and learning certain types of visuomotor manipulation primitives. In this talk, Pieter discusses major challenges for, as well as some preliminary promising results towards, making deep reinforcement learning applicable to real robotic problems. View more videos from the summit on our playlist here. Join us at future Deep Learning Summit:
- Deep Learning in Healthcare Summit, London, 7-8 April 2016
- Deep Learning Summit, Boston, 12-13 May 2016
- Deep Learning in Finance Summit, London, 23 September 2016
- Deep Learning Summit, London, 22-23 September 2016
- Deep Learning Summit, Singapore, 20-21 October 2016
- Deep Learning Summit, San Francisco, 26-27 January 2017
- Deep Learning Summit, New York, 12-13 October 2017
View all upcoming RE•WORK summits here.