The A.I. Gold-Mine: Predicting Stock Market Success

The ability to successfully and consistently predict the stock market is, obviously, a gold mine which technologists have been working towards for many years. Thanks to recent rapid developments in deep learning algorithms, more individuals and companies are able rely on stock market forecasting from artificial intelligence, as the technology has begins to predict better than the pros.I Know First is a fintech company that brings science and math to the financial world by providing daily investment forecasts, based on an advanced self-learning algorithm. These algorithmic forecasts are used to identify the best investment opportunities, to develop systematic trading and allocation strategies, as well as monitor current portfolio holdings. Yaron Golgher, CEO and Co-founder of I Know First, will join us at the AI in Finance Summit this September 6 - 7 in New York to present his most recent work in AI and Finance. Last year in London, we spoke with Yaron in an interview about the applications of AI in the financial sector, so here's an idea of what to expect from his presentation this year.Can you tell us more about your role, and what you're currently working on?Before becoming Co-founder and CEO of I Know First, I was working for 10+ years with a leading Israeli consulting firm as a senior consultant and division manager, focusing on forecasting, budgeting and IT projects in the financial industry. Together with my Co-founder Dr. Lipa Roitman we identified the growing need of artificial intelligence application in the capital markets and founded I Know First back in 2011. Our algorithm was developed by Lipa, who as CTO also heads our R&D team, after extensively researching the nature of chaotic systems. He holds a PhD in organic and physical organic chemistry from the Weizmann Institute of Science and was previously obsessed with searching for and finding order in highly complex and even seemingly random chemical processes. He succeeded in it by applying a wide range of machine learning techniques, including deep neural networks and genetic algorithms. And this is how he got into deep learning. His unique R&D team consists of PhD’s and AI and Machine Learning experts, including IDF intelligence veterans. We also consult with Prof. Yakov Yakubov, a mathematician from Tel Aviv University.Capital market is a very complex system, continuously evolving beyond established theories. Market participants are overwhelmed by huge and growing amounts of data that need to be digested and understood to be able to navigate through all kinds of market environments successfully. I Know First’s forecasting algorithm utilizes artificial intelligence and deep learning techniques to find relationships and patterns in large sets of historical stock market data in order to analyze and predict its behavior.We're focusing on the advantages of the deep learning technology applied to capital markets in comparison to more traditional approaches and explain the challenges and key issues. So far only the largest players in the industry with enough resources and know-how could develop and maintain this type of models internally. With I Know First’s algorithmic AI-based forecasting solutions it became possible also for smaller or non-specialized institutions in this area as well as for individual investors to benefit from this technology. Which industries or areas do you feel deep learning will have the most beneficial impact?Obviously, deep learning will be most beneficially impacting any industry where a high level of abstraction is required in order to analyze large and very complex sets of data. In today’s world, through digitalization, it actually applies to most of the industries. But specifically, it holds for automotive (incl. self-driving cars), the image and speech recognition fields, biotech/biomed when searching for patterns in genes and developing drugs, customer analysis & business intelligence in general, and of course, in the investment management industry. In the latter, e.g. forecasting and decision algorithms using recurrent neural networks and reinforcement learning. In these areas, AI in general will definitely have an enduring impact.What advancements in deep learning would hope to see in the next 3 years?The field is advancing rapidly, fuelled by demand in AI applications in many fields, and availability of cheap high powered GPUs and cloud computing services. “Learning to learn” computing algorithms will soon decide which flavor self-learning algorithms are suitable for a given task. Furthermore, a general AI (Artificial general intelligence) is also on the horizon.What are the challenges your company faced when introducing deep learning techniques within the financial industry?Technology-wise, the first challenge we were facing with clients was to explain to them what exactly separates the machine learning technology from other quantitative but more traditional models. Once that is done, the main challenge is to make the customers feel comfortable with the outputs of the system and to make it clear to them that it’s not going to be the case as e.g. with traditional econometric models that they’ll be able to follow or comprehend the reasoning behind those outputs. Because this is precisely the idea here behind the application of deep learning: it’s just too complex, too dynamic and too many parameters/features play a role making it impossible for a human brain to digest and analyze it. When it comes to the execution based on the insights given by the model, it’s not the question of “why” the system recommends this and that, but rather whether you might want to add an additional “human” filter on top of it before you follow the recommendations.On the business side, we’ve been frequently asked why we share our forecasts and not just use them for trading. The answer is rather simple. When we started the company, Lipa and I were both not coming from the asset management industry and in order to operate profitably as soon as possible (what we’ve been doing since 2012) the distribution of the forecasts was our first income source. Later we added the money management side, partnering with diverse financial institutions, sometimes based on certain exclusivity rights. And so, we’ve been able unify the two business strategies under one roof by structuring the solutions in a way those or the respective customers are not competing with each other.How disruptive will the popularity of deep learning be for the trading industry?Nowadays, many of the traditional momentum, mean-reversion or arbitrage based trading strategies as well as fundamentally reasoned or factor based investment and allocation tend to become less profitable as many investors are informed about and utilizing the very same methods. Additionally, it’s becoming increasingly difficult to find new ways to further diversify.With deep learning technology becoming more popular in the trading industry when it comes to finding the most promising trade ideas and gaining the edge, we expect a much broader range of uncorrelated strategies to be created as well as the respective funds in the future applying it. With growing amounts of all kinds of data that can have predictive value, deep learning is literally opening so many doors to discover new things and we’re very excited to contribute to and to see what is going to happen in the investment industry in the next few years.Can you tell us more about special features of the I Know First algorithm, with an implementation example from a client?The self-learning algorithm combines generality and adaptability. Trained on over 10 years of data, it knows when the rules are changing, and adapts accordingly. An interesting feature of our algorithm is that it assigns to each asset a predictability indicator, a correlation based quality measure of the respective forecast. Thus allows filtering to focus on the most predictable assets.Our product selection includes various industry groups. We customize the forecasts using multiple criteria such as market cap and liquidity, PE ratios, book value, geographic location, and more.An example of an interesting implementation project is a current cooperation with a large European bank. Based on our predictive algorithm, we are developing an AI based trading ideas generator for the private banking division, focusing on the EURONEXT listed equities.