Waiting for AI's iPhone Moment

About fifteen years ago I was at Arkena, working with streaming, an emerging technology hardly anyone had heard of. We built our whole platform on Flash, which at the time powered almost every streaming solution, including YouTube. There was just one problem. Flash was a power hog. It routinely consumed up to 100% of a CPU. It was built for the desktop era, and hardware and PC manufacturers loved it because it drove the sales of new computers. But then along came the iPhone, and Steve Jobs refused to even try to support it. That was the beginning of the end. Today Flash is legacy technology, and the smartphone defined a new era of mobility.

As I watch Nvidia ship computers the size of containers and data centers tile the earth to meet the demand for AI, I can't help but think about that iPhone moment. AI is a different and much more transformative technology than streaming, but back then one innovative device changed the trajectory from brute scale to something nimbler, smarter and far more energy-efficient. I wonder if we will see something similar happen with AI someday. That story was in the back of my head when I sat down with Hans Salomonsson from Embedl to discuss the state of edge AI in Europe.

Hans is CEO and co-founder of Embedl, a Swedish deep-tech company that started as a research project at Chalmers University of Technology in Gothenburg in 2018. Today Embedl optimises AI models for cars, robots and defence systems across almost every chip platform on the market. One of its customers is Bosch, the world's largest automotive supplier, where Embedl made the in-car system that checks drivers for drowsiness run faster on both Qualcomm and Arm chips.

Last week Embedl made that work available to everyone. At NVIDIA's GTC conference in Berlin, the company opened a free library of more than 1,000 AI models that are already optimised to run on NVIDIA's small computers for robots and machines. Normally, a company that wants to put an AI model inside a product has to buy the hardware and test, again and again, to find out whether the model will run, how fast, and how much it loses along the way. Embedl publishes those answers up front. One example: a model that lets machines understand images and text runs almost six times faster than the original on a small computer that costs less than a phone, and gives practically the same answers

That is edge AI: AI small enough and efficient enough to run inside the head of a robot, or reliable enough to run in a car, without any data center in the cloud. Inside a car or a robot, scaling up is not an option. That makes the edge the perfect place to look if you want a peek into a possible future of decentralised AI. I started by asking Hans what Embedl actually does.

Hans: What we do as a company is basically brain surgery for AI models. An AI model is a computational graph. With our methods and algorithms we go in, analyse a particular model and make changes to it. We can cut things out, we can add things, we can exchange things, and we can also change the precision, which is called quantisation and is the first thing you do when you optimise. So that's what we do. We're brain surgeons for AI models.

Lars: Are there any requirements for those models?

Hans: They need to be open, of course. It can be done to any model, but we are focused on physical AI applications, so we tend not to go to the very largest LLMs. For generative AI models below 30 billion parameters, that is where we have our sweet spot, and around 1 to 3 billion we have the world's most efficient.

Lars: And what can you actually achieve?

Hans: It depends very much on the application. Our customers have hand-optimised their model. They know the application, they know the model, they designed it and built it. And we can still come in as an outsider and make it faster every time. That could be anything between 20 percent and 500 times faster.

When people talk about European AI, the conversation usually ends with Mistral and the question of whether we can build a frontier model of our own. Physical AI changes that conversation, because it plays out on home ground. Bosch is the world's largest automotive supplier. Siemens and ABB automate factories across the globe. NXP and Infineon make the chips that sit in cars and industrial equipment. We have the factories, the engineers and decades of experience building things that must never fail.

The market is not small either. Nvidia's Jensen Huang puts physical AI at 50 trillion dollars, and Hans believes it will eventually be bigger than data center AI, even if that takes ten to twenty years. So I asked him whether optimisation could become a European strategy, given that we are not building data centers at the scale of the US and China, and our grids would not keep up if we tried.

Hans: Absolutely. It has a very crucial role in what I believe. Europe could actually compete and win. Where we have an advantage is that we have a long history of very good industries. If those are complemented in the right way with physical AI and edge AI, where optimisation and the most intelligence per dollar and per watt can be combined, then we are in a really, really nice situation. We sort of lost the foundation model race. We have Mistral, and they're doing great things, but they have tough competition.

If Europe has the industry, the engineers and the technology, why is it moving so slowly? Hans was frank about where he sees it getting stuck.

Hans: It's quite painful to see, because I strongly believe in what I just said. You have people on the boards of really good industrial companies with centuries of developing great technology, and they want to embrace AI. The engineers want help and tools, and they realise they need them. But then you have the management. They take all the risk, they sit with a budget, and they have deadlines next quarter or next half year for the current products. It gets stuck there, with a management that doesn't take that risk. That is painful to see, because everyone wants to invest.

In the full episode of the Nowable podcast, Hans explains why a faster model halves the hardware bill in a data center while the same speed-up in a car leaves the computer unchanged, and why he would rather publish models on Hugging Face than quote benchmark numbers. I also explain FlashHead, Embedl's trick for speeding up the very last step where a language model picks its next word. And Hans shares where he thinks everyday AI is heading: your phone or laptop will first check whether it can answer locally, and only pass the question on to a server when it can't.

That last point brings me back to Flash. If AI moves in that direction, its iPhone moment could look like thousands of small, efficient models running in cars, robots and phones. Some of them might well come out of Gothenburg.

Listen to the episode at the top of this page or in the Nowable podcast section.