Hot French startup ZML releases free product to speed inference across lots of AI chips

ZML Unlocks Multi-Chip LLM Inference

French AI startup ZML has released ZML/LLMD, a new inference server designed to run open-source large language models across a wide array of hardware. The move directly addresses the growing importance of inference optimization and the market's concern over vendor lock-in and rising operational costs. Founder Steeve Morin aims to give enterprises the flexibility to mix and match chips for AI workloads, a significant departure from the current siloed hardware ecosystem dominated by a few key players.

Technical Capabilities and Go-to-Market

Backed by $20 million in funding from firms including 20VC and Kima Ventures, the Paris-based company has developed software to achieve peak performance on diverse silicon. The goal is to make different chips available for AI use cases at their maximum available speed. Unlike the company's first framework, ZML/LLMD is not open source, but it is launching as a free product to gather usage data before monetization.

• Supported Hardware: NVIDIA, AMD, Google TPU, Apple Metal, Intel Arc
• Product: ZML/LLMD Inference Server
• Funding: $20 million
• Team: 20 people, led by Zenly's former VP of Engineering, Steeve Morin

This software-first approach could alter market dynamics by enabling cost and energy efficiencies, potentially benefiting novel AI chipmakers, many of which are based in Europe. While ZML is not positioned against NVIDIA—Morin notes a good relationship with the chip giant—it enters a competitive space alongside players like Inferact (from the vLLM creators) and RadixArk (SGLang). ZML's broader ambition, however, extends to co-designing silicon, suggesting a deeper play in the hardware stack.

ZML's strategy signals a critical shift in the AI infrastructure battleground, where software abstraction layers, not just raw chip performance, are becoming the key to unlocking efficiency, controlling costs, and dismantling hardware monopolies.