Baidu Releases Open Source Deep Learning Code Warp-CTC

by Nikita Johnson

Today, Baidu Research's Silicon Valley AI Lab (SVAIL) is releasing an open source code called Warp-CTC. This tool can plug into existing machine learning frameworks to significantly speed up AI development - up to 400 times faster than previous implementations.    This is the first time SVAIL has offered open source code to the machine learning community, and the lab plans to release AI additional tools in future. Warp-CTC's release comes with the aim to make end-to-end deep learning easier and faster, so researchers can make quicker progress, particularly in areas where Baidu researchers have found previous code for training end-to-end networks for sequences has been too slow.

What is Warp-CTC?

Warp-CTC is an open source implementation of the CTC algorithm for CPUs and NVIDIA GPUs.

What is CTC?

CTC is an objective function that can be used while doing supervised training for sequence prediction, without knowing an alignment between the input and output. The CTC algorithm was developed by Alex Graves, Santiago Fernandez, Faustino Gomez and Juergen Schmidhuber at IDSIA.

Why was Warp-CTC developed?

SVAIL engineers developed Warp-CTC while we were building our Deep Speech end-to-end speech recognition system to improve the scalability of models trained using CTC. We found that currently available implementations of CTC generally required significantly more memory and/or were tens to hundreds of times slower.

Why is SVAIL releasing open source software?

We want to make end-to-end deep learning easier and faster so researchers can make more rapid progress. A lot of open source software for deep learning exists, but previous code for training end-to-end networks for sequences (like our Deep Speech engine) has been too slow. We want to start contributing to the machine learning community by sharing an important piece of code that we created.

Which machine learning frameworks does Warp-CTC support?

We are releasing Warp-CTC as a C library along with integration for Torch, a scientific computing framework. Additionally, Nervana Systems is integrating Warp-CTC into Neon, their machine learning framework.

How can you access Warp-CTC?

Warp-CTC will be available on GitHub here.    SVAIL was founded in 2014 when Andrew Ng and Adam Coates joined Baidu, as Chief Scientist of Baidu and Director of SVAIL, respectively. The lab's research is built on a combination of deep learning, large datasets and high performance computing, with it's deep learning and systems teams working closely together to explore the latest in deep learning algorithms and discover innovative ways to accelerate AI research with new hardware and software technologies. View more information about the lab in the video below:To learn more about deep learning at Baidu, join us at the RE•WORK Deep Learning Summit, in San Francisco on 28-29 January. Andrew Ng, Chief Scientist at Baidu, will be speaking at summit alongside experts from Google, Twitter, MIT, OpenAI, Enlitic and more. The Deep Learning Summit will be taking place alongside the Virtual Assistant Summit. Tickets are now limited, for more information and to register please visit the event page here.