What did you miss from Netflix, Facebook, X, the moonshot factory at the Women in AI Dinner?
Talk of autonomous cars driving around cities and computers being able to have conversations with humans is no longer solely the subject of science fiction films. These technologies - whilst not perfect - are very much a part of our reality. As more and more companies begin to integrate AI into their services it’s essential to ensure that the goal of these systems has an overall intention of creating positive social outcomes for users and society more generally. One of the biggest challenges in achieving this is the presence of negative bias being ingrained in the AI. Currently, 71% of applicants for AI jobs in the USA are male, and 80% of AI Professors at top Universities are male. However well-intentioned these experts are, the lack of gender diversity in AI means that the scope of perspective in the teams is limited.
This evening at the Women in AI Dinner in San Francisco, we have been joined by both men and women keen to support diversity in AI. The evening saw keynote presentations from leading female minds working in AI share their latest work, and discuss the urgency for more diversity in top tech companies.
When speaking about this challenge, Sarah Laszlo, Senior Neuroscientist at X, the moonshot factory shared how she thinks we can encourage more women from a diverse range of backgrounds into AI:
“I’ve met women when they are sophomores or juniors in college, or even earlier, and then kept them in my life. That can be as simple as staying connected on LinkedIn, or making sure we catch up at a conference. It's important that women are mentored early on in their career, because you can't expect diverse candidates with exactly the training you need to suddenly appear exactly at the moment you need them. Better to have those relationships and nurture them over the long term.“
The evening began with a champagne reception and plenty of networking at in the One Market Restaurant, and attendees voiced their enthusiasm to be surrounded by such influential experts working in the field:
“I have a PhD, but I don’t see that as a barrier to entry to the field. What do you really lose if you don’t have that? In my work it’s very broad, so I don’t think it’s necessary which is why networking and events like this are essential.” - Vilas Veeraraghavan, Walmart Labs
“The fact that you cover topics like ethics and diversity as well as the real technical elements makes it really inclusive.” - Dr. Ariane David, Independent
Once the guests were seated, introducing the evening’s speakers was our compere, Sachi Paul, Machine Learning Scientist at Amazon Lab126. Sachi has been at Amazon since 2016 and is currently working on natural language understanding, deep learning and deep dialogue. She began by introducing Julie Pitt, Director of Machine Learning Infrastructure at Netflix.
Julie explained how her team’s goal is to scale Data Science while still increasing innovation. She previously built streaming infrastructure behind the "play" button while Netflix was transitioning from domestic DVD-by-mail service to international streaming service. Julie told us that applied ML still has a long way to go before it reaches maturity and for Data Scientists it’s hard to collaborate, hard to be productive and hard to deploy to production. They’re currently facing challenges such as scaling Data Science innovation by making collaboration effortless and enabling Data Science to single-handedly and reliably introduce their models to production. ‘How do we make it easy to develop machine learning models that humans trust?’ asked Julie.
How can I use the full data set? How do I stop training models from taking a really long time? There are so many questions! She explained that ‘this is the boat Netflix was in when I joined back in 2015 - this is exactly why we are spending all this time in ML infrastructure. Our users had specific needs that we could satisfy.' Julie went on to explain that ‘there’s a lot of diversity in the machine learning problems at Netflix, and the power of infrastructure can help unlock the next wave of innovation in machine learning.’
During the Q&A after Julie’s talk, we learned that the ML infrastructure team are relatively small with just 6 engineers plus Julie, ‘so it’s a dynamic and collaborative team. We have a tension where we want to support innovation but we also want there to be real applications, so we developed our platform with both of these things in mind.’
At each of our women in AI dinners, we encourage guests to expand their network and learn about other experts’ work in the field. To maximize new conversations and these opportunities, we invite attendees to move seats between each course.
Next up to share her work was Anusha Balakrishnan from Facebook. As a research engineer, Anusha spoke about Conversational AI at Facebook where she works specifically on the problem of converting structured input to unstructured text. “This is useful in conversational systems, where a dialogue manager may decide "what" to say, and the NLG system then needs to decide "how" to say it.” In her keynote presentation, Anusha explained how templates are used in conversational systems fairly often as a standard practice in industry. The advantage of templates is that they're easy to write, and there's a very high degree of controllability (particularly for quality and brand/personality). There are challenges in this however, and “ templates can be really hard to scale and maintain, particularly as you want your system to talk about even wider sets of ideas and entities.” Anush gave us an example, “let's say you have a template like "<restaurant> serves <cuisine> food"; that works well for some values, but if you substitute "fast food" for cuisine, you get "<restaurant> serves fast food food". This is why we should move towards using machine learning for natural language generation. It has been demonstrated in some recent papers that DL can be used here - “the idea is that you show annotators (Mechanical Turk raters for example) structured input, and ask them to write responses that express it.” A big advantage of this is that these models can dynamically generate text that fits a given set of arguments, e.g. "<restaurant> serves fast food". These models also provide straightforward ways to incorporate context from preceding turns in the dialogue (or even just the user query). Of course there are practical challenges here, and on the flipside, you no longer have a good way to ensure grammaticality, or to ensure that the content that's generated is "safe".
Anusha explained that “as a related problem, it's really hard to evaluate NLG systems. There's no clear metric that's easy to define - at a high level an ideal NLG system would improve "naturalness" or "engagement", but it's unclear what some good proxy measures are to evaluate that.” Anusha wrapped up her presentation by explaining that “Human evaluation has been used fairly often (both offline, as well as in a live setting like the Alexa prize) - but it's extremely subjective and often expensive.”
Between presentations, I was able to chat to our speakers, and ask them about their work in AI as well as their experiences as women in the field?
What's your favourite thing about working in AI?
Sarah Laszlo: I love feeling like I am working on the future. That's a great thing about working at X in particular — we very much feel like we are creating the future (sometimes we’re actually doing it, too!). I'm a futurist, so I love that. I love turning on Star Trek, or some other optimistic science fiction, and feeling like I'm making it happen.
What can be done to encourage more women from a diverse range of backgrounds into AI?
Julie Pitt: The single most valuable tool for me in my career has been my network. If you are experienced, forming personal connections to members of underrepresented groups will lower the barrier to entry for people of diverse backgrounds. I've started volunteering as an advisor with BuiltByGirls, which helps high school and college age girls and young women begin forming professional networks that will launch their careers. For those of us who have the privilege of being established in the field, we can use that privilege for good by letting our employers and prospective employers know that inclusion and diversity matter to us.
Anusha Balakrishnan: I think creating visibility for the kind of impact that AI can have is really important, particularly at the school and college levels. Anything from talks on AI and its applications, to diversity-focused summer internships or bootcamp programs at companies working on AI (which there are a lot of). In addition, I think it's really important to spread awareness about just how much work there is to do in AI right now - a background in AI is one of the most highly demanded skillsets right now, and jobs in AI are incredibly fulfilling!
After the main course, Sarah Laszlo, Senior Neuroscientist at X, the moonshot factory spoke about her work in CEREBRO: A Neuroscience-led Effort for Stable, High-Accuracy, Brain Biometrics. Sarah leads research for an early stage biosignals project and is working with biometric credentials in cybersecurity. Sarah explained that one of the challenges about working at a moonshot factory is that “we’re trying to do things that have never been done before. That means we need people with a combination of skills that might not be “traditional”. So one of the biggest (and most exciting) challenges I have at work is finding the right people with just the right expertise to come and take moonshots with us.” In the spirit of networking at the event, Sarah mentioned that they’re always looking to hire great people — especially ML experts!
She shared that in September of 2015, the New York Times reported that Chinese cyberespionage agents had stolen the fingerprint records of 5.6 million U.S. federal employees from the Office of Personnel Management (OPM). This wasn’t like a normal security breach - because it involved biometrics, the data can’t just be cancelled - those users cannot grow new fingerprints. “This breach demonstrates two challenging facts about the current cybersecurity landscape. First, biometric credentials are vulnerable to compromise. And, second, biometrics that cannot be replaced if stolen are even more vulnerable to theft.” Sarah discussed a novel, brain-based biometric that avoids both of these problems, and first steps towards how it can be implemented in a brain-computer-interface that identifies users of sensitive information.