The RE •WORK Community’s Top Deep Learning Trends for 2022
After a couple of troublesome years, many businesses are looking forward to 2022 with cautious optimism.
The worldwide global pandemic has accelerated the digital ambitions of many enterprises, creating a fertile ground for the development of AI and related technologies.
Ahead of RE•WORK’s upcoming Deep Learning Hybrid Summit on February 17-18, we asked some of our expert community what they predict for AI and Deep Learning in 2022.
Wenting Sun, Senior Data Science Manager, Ericsson
Which trend associated with Deep Learning and AI are you most interested in/passionate about? Why do you think this is so relevant today?
I am most interested in connections between graph-based ML and probabilistic graphical theories, and how they can be effectively leveraged to address causality using Deep Learning.
It is relevant because causality is a much deeper level of understanding of events, and it will benefit hugely for high-impact use cases such as prevention, and proactive actions to mitigate risks. It also holds the keys to model explainability, especially for Deep Learning models.
2. Deeper integration and end-to-end optimization between hardware and software design when applying AI to enable faster industry adoptions.
3. More prominent advances in data-driven approaches when applying AI to real-world applications.
Stephan Zheng, Lead Research Scientist, Senior Manager, Salesforce
Which trend associated with Deep Learning and AI are you most interested in/passionate about? Why do you think this is so relevant today?
AI is growing up. Its scale and capabilities present exciting new opportunities to address critical socio-economic and scientific challenges.
Researchers and developers are now applying AI to more diverse real-world problems. Exciting new areas include AI for scientific discovery, economics, biology, medicine, math, robotics, sustainability, amongst others.
In our work, we’ve shown that Reinforcement Learning and AI economic simulations can improve equality, productivity, and social welfare, compared to traditional policy design methods in economics.
What are the top three Deep Learning trends you think will take AI to the next level in 2022, and why?
Using simulations and generative modeling to go beyond learning from noisy and biased real-world data. Simulations can generate infinite data and focus on creating a bottom-up understanding of the world.
Building hybrid models that combine Deep Learning with logical programs, formal verification, symbolic reasoning, physical intuition, causal reasoning, and domain-specific models.
Systems engineering, integration with existing systems, distributed computing, and custom hardware accelerators (GPUs, FPGAs, TPUs, etc).
Leonardo Neves, Principal Research Scientist, Manager, Snap Inc.
Which trend associated with Deep Learning and AI are you most interested in/passionate about? Why do you think this is so relevant today?
I am currently fascinated by the effect of time in machine learning models, especially in language.
In the past, it would take several years or decades to see language change. With the popularization of social platforms, we now see language changing weekly, if not daily.
As we create methods that want to model and understand language, adapting to these changes without significant drops in performance is critical.
What are the top three Deep Learning trends you think will take AI to the next level in 2022, and why?
Self-supervised learning, active learning, and graph neural networks are three things that I believe will continue to drive immense value in the next couple of years.
Leveraging unlabeled data unlocks tremendous potential, as we have seen in multiple areas like NLP and computer vision, and I believe there is room for even better results from self-supervised methods. On the other hand, we will still need high-quality annotated data for a while, and making training more data-efficient is a critical research area where active learning can help.
Lastly, graph neural networks have gained popularity recently, but I also see much room for growth as networks get deeper and more complex.
Heather Nolis, Principal Engineer, Software, Machine Learning, T-Mobile
Which trend associated with Deep Learning and AI are you most interested in/passionate about? Why do you think this is so relevant today?
Explainability and interpretability. With Deep Learning powering so many production systems, stakeholders want to understand how these predictions are made – and why the model made them.
What are the top three Deep Learning trends you think will take AI to the next level in 2022, and why?
1. Lower compute costs for training and deployment. Many companies aren’t seeing ROI on their AI projects, but if compute and training costs reduce then the real value may still be had in those same models.
2. Low resource transformer architectures. “Zero-shot” ML has all sorts of pitfalls, but “few-shot” ML – like a few rounds of tuning on a hugging face transformer – has the potential to deliver high accuracy models with little training data. We just replaced a giant legacy neural net on my team with few-shot transformers
3. Auto MLOps. AutoML has known pitfalls, but auto MLOps – the ability to automatically monitor model performance, data drift, and feature importance out of the box will get teams a pane of glass into their models that may not have been there previously.
Interested in hearing more about the latest trends and challenges in Deep Learning? Join our expert speakers this February at the Deep Learning Summit, online or in-person. Register here or find out more by accessing the brochure!