Samir Kumar is Senior Director of Business Development for Qualcomm, partners of the Deep Learning Summit. In this role Samir is responsible for technology transfer and driving projects to successful commercialization on behalf of the Qualcomm Research group, as well as collaborating with potential start-ups and key partners to build interest in a variety of research initiatives.
We caught up with Samir before the Deep Learning Summit this week to hear his thoughts on the current and future advancements of deep learning.
What do you feel are the leading factors enabling recent advancements in deep learning?
Leading factors leading to advancements in deep learning can for the most part be attributed to successes in the effective training of deep networks.
1) Data- The explosion in the amount of labeled data available via the web and other sources (especially in the imaging domain).
2) Compute hardware- The availability of highly data parallel architectures like GPUs for training deep networks but also distributed computing techniques that enable training on large clusters of CPUs.
3) Algorithms- In addition to back propagation, adding “dropout” to train networks works as an effective technique to deal with the problem of over-fitting during training
Which industries do you think will be disrupted by deep learning in the future?
Mobile is ripe for deep learning disruption already! Our phones have a variety of sensors (including the camera), that are collecting data that would be well served from better pattern matching techniques. Using deep networks to infer these patterns is already possible today on Qualcomm Snapdragon. In the future mobile devices participating in large distributed learning networks is an area we are quite excited about.
In terms of other industries, we are already starting to see some of this being used however, other highly data driven domains such as healthcare and financial services are candidates for disruption. With healthcare, we are already seeing exciting potential in the area of medical imaging.
What is currently being developed in your field that will be essential to future progress?
In the mobile and embedded domain there are a couple of key areas of R&D that are essential
1) Efficient training and inference run-time of deep networks- Eliminating redundant pieces of information to have the most compact yet accurate representation of the data. This has benefits of smaller models but also can reduce the amount of computation required to process the network, making them more suitable for embedded applications.
2) Heterogeneous computing- Mobile & embedded environments have much greater power and thermal constraints than large GPU clusters running in the cloud. Enabling deep networks to run on the right core for the right scenario will be essential for successful commercial deployments on smartphones, tablets, cars, robots/drones. Etc.
Which areas do you feel could benefit from cross-industry collaboration?
As deep learning is very much a data driven approach, there is always a need for better and bigger data sets for validation & testing purposes. Joint initiatives between cross-industry entities and academia to drive curation of more of these standard datasets but based on real world data would benefit everyone.
What developments can we expect to see in deep learning in the next 5 years?
1) Semi-supervised/Unsupervised learning- Implementations that are commercially viable. These would open the door for personalization and on-target learning/adaptation once a network is deployed.
2) Multimodal learning- Fusing data from multiple sources or sensors to create joint intermediate or higher level feature representations
3) Temporal learning & classification- Recurrent neural networks are showing state of the art results in temporal classification problems like speech & video.
What advancements excite you most in this field?
There are a couple things that excite me most in this field from both a personal and business interest perspective.
1) Deep networks on your phone classifying device data from camera, audio, other sensors to better understand your world while respecting your privacy.
2) Sensing and perception applications in robotics and especially in UAVs and autonomous driving
Personally, from a computing fabric/hardware standpoint, quantum computing is a very exciting area to enable further breakthroughs in deep learning. A quantum computer could explore optimization strategies for training deep networks exponentially faster than our fastest CPU/GPU clusters
The Deep Learning Summit is taking place in San Francisco on 29-30 January. You can get more information here.