5 Steps to Implement Deep Learning

by Hollie Jaques

AI allows researchers to focus on the problems at hand instead of spending time creating new tools to approach new problems.  (Ian Goodfellow, Google Brain)

Whilst AI is now readily available for implementation cross-industry, there are still challenges in the commercial applications of deep learning. Despite the availability of multiple research oriented open-source deep learning frameworks, they are not efficient enough for application in many domains (e.g., cybersecurity); their incorporation in the final product is complicated; and even where applicable, highly skilled and scarce deep learning experts are required to employ them. As a result, today most of the real-world applications of deep learning in industry are in computer vision, because they mainly rely on convolutional neural networks only, for which the barrier of entry is lower.

Eli David from Deep Instinct shared his expertise on overcoming this obstacle: "I expect that in the next few years we will see deep learning being employed effectively in additional industries in real-world solutions. We will have more deep learning frameworks which focus on efficiency in real-world rather than being geared towards research, as is the case with currently available deep learning frameworks, and they will lower the barrier of entry for adoption in the industry."

Not sure where to start on taking your AI to the next level? Here are 5 Steps to implement Deep Learning:

What is deep learning?

Deep learning is a branch of machine learning, employing numerous similar, yet distinct, deep neural network architectures to solve various problems in natural language processing, computer vision, and bioinformatics, among other fields. Deep learning has experienced a tremendous recent research resurgence and has been shown to deliver state of the art results in numerous applications. In essence, deep learning is the implementation of neural networks with more than a single hidden layer of neurons. (Matthew Mayo, KDnuggets)

How to get started, the basics:
1. Identify Your Problems

Before delving straight into applying deep learning, pinpoint the problems you’re trying to solve. Creating a model with vague goals will not produce the outcomes you desire, and furthermore, not every business or organisation is ready for deep learning. Find out how to prepare your business for implementation from Coursera.

2. Pick a tool & build a strategy

How are you going to build your model and train your neural networks? Ensuring you have the right strategy is the make or break of your model. Here are a couple of suggestions of the best place to start:

3. Assemble Your Data Sets

AI requires very large datasets, and due to the physical recordings of data that were popular until recently, these high-quality datasets can be hard to come by in certain industries. Data found from a single source can be biased to internal practices, so it’s important to ensure that models you are training your system on have a varied and extensive data sources. When a model is trained with limited data the network has memorised the training examples but is then unable to generalise in new situations. Find out how to improve neural network generalisation and avoid overfitting from MathWorks.

4. Build Your Model

You’ve now identified what problem you’re trying to solve, you’ve selected your tools, and you’ve got reliable and robust datasets - time to build your model. Do you have the expertise in your team to build the model yourself or are you going to need to collaborate with an external party?  In recent years, the Deep Learning community has developed much better open source libraries and tools for deep learning. For example, Theano was a breakthrough in how we can implement DL models, which inspired the now widely popular TensorFlow.ArXiv and social media have become the prime strategy for publicising research. This has made it possible to iterate and build on other researchers' work at a much higher pace. (Hugo Larochelle, Google Brain)

5. Optimise, Test & Deploy Your Models

Are you going to be deploying your model in the cloud? How are you going to ensure users data is secure? What platform are you optimising for - mobile, web, in a car? Think about the limitations such as battery life, location data and so on. NVIDIA suggest using TensorRT, a "high-performance deep learning inference optimiser and runtime that delivers low latency and high throughput for deep learning inference applications."

ML is always held back by limitations in the amount of computation we can use. These advances are able to have very apparent impacts on real-world problems, and Ian Goodfellow explained:

“The May 2017 announcement of the new generation of Google TPUs is huge. The new Google TPU helps bridge the gap between the amount of computation we can leverage in DL experiments and the amount of computation used in a biological nervous system. The previous generation was available only to Google engineers, but the new one will be available to Cloud customers, and researchers can apply to get access for free.”

“Whilst we have seen a lot of progress in unsupervised learning, especially in generative modelling, we have yet to see it’s full potential. It’s impossible to attribute progress to a specific technique, but it’s fair to say that progress has been driven by new ideas and insight rather than bigger datasets and faster computers.” (Jörg Bornschien, DeepMind)

Keen to learn more about applying AI, ML and DL in your business? At the RE•WORK Applied AI Summit in Houston this November 29 - 30, we will explore real-life AI applications, case studies, business insights & results from leading companies leveraging AI to solve problems in enterprise including experts from Tractica, Reddit, Netflix and more.