John Spindler, Co-Founder of AI Seed gives an insight into some of the initial hurdles when investing in AI first startups. John will be joining us in London for the Deep Learning Summit on September 19-20.  

AI Seed started investing in the pre-seed and seed rounds of AI First startups as they emerged from universities and accelerators in September 2017. Since then they have invested small cheques in 35+ startups in London and as far away as Cambridge, Belfast, Stockholm, Singapore, New York and SF. For those startups raising pre-seed rounds up to $500K AI Seed have in most cases led the round and now sit on their boards. For the other approximately 50% of the deals, who were raising larger Seed rounds, AI Seed have participated by investing in the rounds alongside some of Europes leading and specialist seed stage investors.

John has certainly learned a lot over the past two years about AI Seed startups and about investing in them and although one of the co-founders of their fund Thomas Stone is a PhD qualified data scientist and an exited startup founder, it has been a particular steep learning curve for John. This is due to John not being a trained data scientist and although he has now been investing money in tech startups since 2012, he has had to go back to school, crash ML lectures, quiz world leading experts, read 100's of papers to learn and re-learn data science concepts, processes and models and mostly try to keep up with amazing technical teams AI Seed are investing in.

Below are the top 5 lessons John learn't in his own words.

Lesson 1. The number 1 question - can it be built?

One of the most annoying things I hear (especially from investors) is that Machine Learning is or will be a commodity. Training and then deploying a model so that it can significantly outperform humans or other software applications is fundamentally F****** hard. Whilst the talent to perform these tasks is in short supply building Machine Learning powered solutions will remaining out of the reach of most companies and most startup founders.

Since the publications of Steve Blank, Eric Reis, Paul Graham etc at the end of the last decade, the recommended pathway to start a potentially world class tech business has been first to work out if there is a problem in the market worth solving, a group of customers with an itch they can't scratch and whether you can design a solution that can "scratch that itch". Secondly to find out whether you can sell your newly designed solution to actual customers (and preferably whilst you are at it, discover the best route to market and potential financial value of that market) and then, and only then, tackle the 3rd challenge which is can you build the solution, and through a process of iterations, build the best in class solution.

Now if you are going to be an AI First startup you cannot ignore step 1 or 2 but if you do not first have the skill, resources and plan to give confidence to yourself and others that the AI powered product you envisage as the solution to the "itch that can't be scratched" can be built then the rest is immaterial. Those who follow the classic "Lean Startup" approach of often end up with unrealistic expectation that Machine Learning is the "Miracle Cure" to the market problem they have identified. But in truth a Machine Learning Model is super difficult to build, super expensive to deploy and maintain and for the most part only to be attempted by an expensively trained specialist.

When investing at pre-seed or seed the key concept leading your decision to invest is called "founders fit". Does this startup team have the credibility, capability and unfair advantages over the next 18-24 months to discover and then achieve a super competitive solution to a big market problem and and the means to create a repeatable and scalable business model.

If you are going to need ML to build this super competitive solution then you better have a ML expert in the team that has more than a good idea of how the ML model can be built and deployed. This is your category 1 challenge, not the after thought.

For this reason we at AI Seed have found that we should only invest in founding teams that include ML specialist (usually educated to PhD level). You need a scientist-entrepreneur.

Lesson 2. Does it matter?

When we launched AI Seed we were inundated with 100"s of founders claiming to be AI startups. We have always done 2-3 stages of technical due diligence in startups we invest in. The process starts with asking the startup to show and tell us about the tech (ML model) that is "under the hood" of the product they are building. If they pass that then either my partner Thomas Stone or one of the small army of AI expert associates of AI Seed, will carry out a 1-2 day investigation and produce a Tech DD report. Finally we will work with the startup who survive out Tech DD process to further develop and refine their product/ tech development roadmap. As this process has been become known about in the London startup eco-system, the "fake AI startups" have kept away. I still recall fondly that when I explained our process at a so called "Invest in AI startups" event in London about 18 months ago none of the 50+ AI startups who attended were brave enough to approach us for investment.

So now we tend to only see AI First startups led by deeply technical teams but the filter we now apply is less about can you build the model to reach the threshold sensitivity and accuracy level but whether it is worth it. Will the AI powered product really enable the users to do something important for the first time, achieve some business goal, which to date has been out of reach, make and/ or save a lot of money. The payoff in terms of value to both the startup and to their customers must be worth the increased technical risk of using ML models. Evidence that this will be or is the case is now our key investment litmus test.

Lesson 3. Is the world really ready for ML powered products?

There is seemingly ( at least in London) no shortage of business customers interested in trying out the latest ML powered products. We have found the standard process for our AI First portfolio startups is to start off by attempting to sell a Proof of Concept (POC) pilot, usually to the head of innovation in a large corporate who is keen to keep up with the latest innovations. ( Only 1 out of 35 AI First startups we invested in is B2C - which probably says something.)

This is no doubt the best route to market ,for in most cases it will take between 6-12 months for a machine learning model to achieve the sensitivity to real world data, the specifity/ accuracy of classification and prediction required to beat the present gold standard and the speed and ease of use for the ML product to be deployable. Determining what that performance threshold is, that once crossed, allows the ML powered product to be super competitive/ super valuable to the users / customers is one of the reasons for an AI startup to initially sell POCs. But there are traps. You may find there is just insufficient amount of labelled and correctly annotated data to build a NN model that works. Most customers (and in most cases investors also), expect ML powered software to work, just like rule based code, straight out of the box and often failed to understand how initially how experimental ML models are. To address the issue that customers often want a solution that works out of the box, we have invested in startups that initially use an heuristic model to base their products on. This allows them to launch a production quality product on to the market and acquire sufficient labeled customer/ user data to eventually build out an NN model. As long as the product still delivers value to the client. then this is a fine strategy but quiet often it is not the case. But the expectations of clients do vary and it pays to find out if it really maters to the client to try to fully automate a process/ task or whether aiming for say 80-90% accuracy is ok. Not only is a lower threshold performance easier for the AI First startup to achieve but it will also leave sufficient space to allow employers to keep some of the existing "Humans in the Loop" in their jobs overseeing the ML model and dealing with hard cases. It is surprising how many organisations find it politically and practically more expedient to buy ML products whose less than optimum performance means they still can keep their present employees in jobs. Go figure.

As we said POC's can be a trap. For one reason they can lead to excessive customisation meaning that the AI First startup ends up delivering projects as opposed to building a truly scalable product and finally there is the trap that individuals in the organisation that commission the POC have no clout (and some times no intention) to role out the ML solution to the wider organisation.

So in short sales cycles for any enterprise software is long but for the reasons above they can be longer for AI First startups and there is also a greater risk that the POC's ( even if successful) are never put into full production by the client. It turns out that getting sign off inside a corporation for a game changing innovative product that has a chance to change that organisations future for the better, is still incredibly difficult especially if the source of this innovation is a startup. No wonder many "wiser" investors than us wait until an AI First startup has overcome these challenges before investing. Is the market ready for the real power and capabilities of ML powered products? - The jury is still out.

Lesson 4. We are not alone in investing in AI startups but sometimes if feels that way.

I have mentioned before when we started AI Seed in 2017 we expected that we would be one of many such pre-seed funds investing exclusively in AI First startups as they emerged from universities, accelerators and ML developer networks. We were wrong.

For many reasons ( some of them mentioned above) I think we were right to believe that AI First startups are special, have a different more challenging product development roadmap and route to product-market fit than standard software startups and therefore require a specialist investor especially at pre-seed/ seed stage. We were wrong that others would join us in specialising not just in the selection but also in providing added value post investment technical support and assistance. That is not to say that there is not strong interest from investors in AI First startups.

At this years AI Seed Portfolio day on the 4th September we had over 100 investors from 58 funds in attendance. We have invested in our 36 AI First startups alongside amazing super angels, similar size SEIS/EIS funds (such as Ascension and SFC), University and Accelerator funds (such as UCL Tech Fund, Cambridge Enterprise, Seedcamp and EF), UK VC stalwarts (such as Balderton, Local Globe, ADV and Octopus), European VC's (such as Speedinvest, Open Ocean, Fly VC) and specialist Deep Tech investors ( such as IQ Capital, Luminous and Pentech).

So we definitely not a lone investor in AI first startups but we are probably unique in our approach and almost alone in encouraging and enabling our AI first startup founders at pre-seed stage to build out their technology so that it respects their unique talent and product vision and not spend all their time chasing revenue opportunities where they can find it. Sometimes these judgements make us out of step with other investors and because we do not have the funds to continue to back our judgements or advice, this can be a lonely place to be.

Lesson 5. It is a privilege managing AI Seed but we might not make any money.

I am extremely confident that the investors in AI Seed will make a very healthy return - since they all get S/EIS tax relief it would be hard for us not to give them a return. This issue is that I am not sure myself and the general partners of AI Seed will. Our management fee from investors for our size of pre-seed fund hardly covers our legal and regulation costs and to be honest the management time we need to donate to our portfolio companies is more than forecast. This and alongside the fact that in Europe there is presently very little M&A activity for deep tech AI startups means (where are the early exits?) that we must now expect (like most investors) to wait for 8-10 years for significant exits. That is a long time to have to continue to provide portfolio management without a fee and it is highly debatable whether any return will cover our lost opportunity costs.

If ourselves with all the advantages we have through our Capital Enterprise ties in of amazing deal-flow and the ability to help our portfolio startups to access public funding support, struggle to make running a pre-seed S/EIS fund pay, then who can make it pay.

I am obviously partial but if the UK government wants to encouraging investment in early stage deep tech startups then they should look to increase the SESIS threshold to at least £300K. This single measure would double the size of the investments AI Seed can make, would allow us to at least double the size of our fund and thereby generate sufficient management fee to cover our costs. As far as I can see one of the few dividends from Brexit would be the freedom from state aid rules that would allow the doubling of tax allowances for investors to happen. If this does not become a possibility then AI Seed will probably need to raise from institutions and family offices a larger fund and inconsequence significantly reduce our investments in pre-seed rounds. Whether we carry on as we are or succeed in raising a larger fund, the main take away I have banked from being the GP of AI Seed is the experience of working with the founders of AI first startups.

It has been over 7 years now since I started investing my "winnings" from a previous career as an entrepreneur in early stage startups. I also had the pleasure over 4 years ago to co-found the super successful London Co-Investment Fund. Yet I would say investing in scientist-entrepreneurs who seek to use the amazing break-throughs in Machine Learning to produce products with the capability to radically change our world for the better has been the most inspiring and satisfying thing I have done in my career to date.

It is an honour to invest in smart people doing smart things, and lest I forget this privilege is the main lesson I have learned from investing AI First startup.

See more on the work of AI Seed and John here!