Accelerating vision-language models with LFM2.5-VL-DSpark

LiquidAI Releases DSpark Drafter to Accelerate Vision-Language Model Inference

LiquidAI has released an experimental DSpark draft model for its LFM2.5-VL-3B vision-language model (VLM), introducing a speculative decoding path to significantly increase inference speed. The company reports that this method provides substantial performance gains while only minimally increasing the memory footprint and leaving the model's output quality unchanged. This development is notable for its immediate applicability in on-device and server-side deployments where VLM latency is a critical bottleneck.

The technical implementation involves a 280 million parameter drafter model that adds just 8.9% to the base model's size. The drafter itself is a simplified 4-layer, attention-only architecture that predicts blocks of candidate tokens by conditioning on the target model's internal hidden states. Key performance metrics include:

• Decode Speedup: Up to 3.13x faster on an Apple M5 Max and up to 2.66x on an NVIDIA H100 GPU.
• End-to-End Gains: Overall latency improvements of up to 2.62x on-device and 2.27x on the H100.
• Drafter Model Size: 279.5M parameters, comprising a decoder stack, hidden-state projection, and a Markov head.
• Integration: Day-one support is available for popular inference frameworks, including llama.cpp, MLX-VLM, and SGLang.

While the decode speedups are significant, the overall impact on end-to-end latency is governed by Amdahl's law. Speculative decoding does not accelerate the vision encoding or prompt prefill stages of VLM inference. These compute-bound stages can represent a large portion of the total processing time, especially on edge devices, thereby capping the ceiling for total possible performance gains. LiquidAI's release of this open-weight model provides developers with a tool to mitigate decoding latency, but it also highlights the industry's ongoing challenge to optimize the entire vision-language inference pipeline.

Strategic Takeaway: LiquidAI's application of speculative decoding to vision-language models demonstrates a practical path to performance gains on existing hardware, but it also underscores that Amdahl's Law remains a fundamental barrier, as un-accelerated vision prefill increasingly dominates end-to-end latency.