At the beginning of September at the AI in Finance Summit, we spoke to Andrew Clark, Principal Machine Learning Auditor at Capital One who is working on reinventing auditing with machine learning. At the summit, we were fortunate enough to chat with Andrew about both his current work, and his career in AI and the financial industry. Here's what Andrew had to say:
We started off by discussing Andrew's background in machine learning and Finance, and we were interested to hear what came first. With an undergraduate degree in accounting, he explained that 'my way into machine learning and finance has been a roundabout path. I have been intrigued with finance since high school used small amounts of money to trade in the stock market, with some success. I was on the path towards a traditional CPA until I realized accounting was rather boring, and I started moving more into the statistics/economics route. I finished my accounting degree while concurrently studying programming and math. I went on to get a master’s degree in data science and began building machine learning-powered applications for financial auditing at a manufacturing company before moving to Capital One.'
Now working to reinvent auditing, Andrew explained how Internal Audit is responsible for providing the 3rd line of defense assurance over the effectiveness of controls in mitigating enterprise risks. This is primarily a judgment-based operation, relying on "humanness" to ascertain if risks are sufficiently being mitigated. This sort of environment makes it difficult to employ machine learning, so we were keen to hear more about the challenges in the process. Andrew explained that many manual processes change slightly from audit to audit, so automation is difficult. "When analyzing processes in depth, however, there are certain elements of procedures where standardized tools can provide drastic efficiencies, and in some instances, improve the quality of the work product. For example, in reporting of audit issues, categorizing issues between compliance, security, and other areas is a very time-consuming task and is based on judgement. However, this process can be made more consistent and almost instantaneous by using a natural language processing (NLP) classifier. For ease of use and updates, all applications I’ve built I have deployed on AWS as web applications."
Here are some more questions we discussed at the Summit:
1. What new approaches to machine learning are helping you in your work?
Advances in NLP, specifically NLP transfer learning, have the potential to provide significant lift in the quality of auditing models. Audits tend to have a fairly limited amount of data for training models, which provides difficulties for the implementation of machine learning. I think the potential to use pre-trained models and customize one’s needs could provide leverage over legacy processes, such as attempting to build a convolutional neural net (CNN) from scratch with limited data, for instance.
2. What challenges are you currently facing?
Challenges for machine learning auditors are often lack of data (both quantity and quality), resource support, practitioner adoption (automated tools have yet to catch on universally in the auditing space), as well as the dearth of quality machine learning use cases in audit (there are many “good” uses, but few “great” ones with significant volume, such as fraud detection).
3. How are you using AI for social good?
As the field of auditing leverages AI and machine learning more, it will increasingly be able to provide higher-levels of assurance.
4. What other areas of AI interest you and why?
I’ve always been fascinated by image recognition, specifically as it applies to finance. For instance, it’s been reported that some hedge funds are using deep learning image recognition systems on satellite images to predict quarterly sales for retailers. The same technology can easily be applied to industries like oil and agriculture, and I find it intriguing that this technique could serve as a financial indicator in the future. As an economist at heart, the implications for the intersection of AI and macroeconomic analysis are very exciting to me.
Given my focus on auditing, security of AI systems is also a passion point for me. The work on adversarial machine learning that has been done by experts like Ian Goodfellow has been very intriguing and is a clear indicator that before we can deploy AI-driven technologies such as self-driving cars, we have a lot of security work to do; I think the role of a technology auditor will only become more important over time.
5. What's next for you?
The million-dollar question! I plan to keep doing what I’m doing for the foreseeable future as well as starting a part-time PhD in Economics this Fall. My proposed dissertation is: “International Reserve Currency Shifts: a historical, data-driven approach,” which will focus on the rise and fall of reserve currencies, from the Roman Denarius to the potential of the IMF SDR using ML, accounting, history, as well as traditional economic perspectives. International monetary economics has always fascinated me (I’m a history buff)—in reading monetary economics works, I notice a lot of great theory but a gap in the research that utilizes modern data science and machine learning techniques. Hopefully, I can make some meaningful contributions to the field using my interdisciplinary skills.
Interested in learning more from Andrew? Sign up to watch video presentations from the AI in Finance Summit in New York.