The market opportunity for selling hardware AI servers

AI is a buzz. AI is also expensive, and a headache.

And a black box.

We’ve seen a few breakthroughs in large language models the last couple of years. At an astonishing pace it has become very ‘normal’ to chat with a bot. We’ve come to develop some intuition around that interaction. And we’ve come to expect it.

This fuelled one of the biggest tech opportunities; cloud AI.

All cloud providers introduced ‘developer friendly’ ways of incorporating AI and large language models into applications. Thus fue;ling a new wave of AI startups and tools.

But these businesses are also learning that AI is quite expensive. And a black box.

Luckily there is an alternative. Open source models have gotten a lot better, and would (theoretically) offer much more control over your data. But they are tedious to setup and easily require a couple hundred gigs of your precious laptop storage.

Which brings me to 2 opportunities I see:

• Opportunity 1: AI servers. Servers with dedicated hardware for AI processing, similar to what is running at the big cloud providers. But then made available as a private server. You can buy them to run them yourself, or they would fit in with more ’traditional’ private server models like Hetzner. Businesses can then combine the scalability of cloud infrastructure for things like storage and network and combine that with a more centrally managed private server for sensitive data and business critical machine learning models.
• Opportunity 2: Intuitive AI tweaking. Software that lowers the bar of integrating various data sources, but more importantly, makes it a lot easier and user-friendly to setup, run and tweak large language models. This can be done combined with the AI servers, inside a ‘managed AI’ service proposition. Or it can be done as a licensed software model.

Lastly - why I think this is a good trend. The demand for large language models won’t fade.

Cloud providers will continue to make it easy to incorporate their models. But they would quickly need to ‘Chinese Wall’ data inside those models, to assure enterprises of data security. And that’s where the black box reputation of these models is a business killer.

Because Chinese Wall ultimately comes down to trust. And how much will enterprises with sensitive data trust Azure, Amazon and Google to truly have setup effective Chinese Walls within their software?