How does Quantization-Aware Distillation (QAD) differ from standard Post-Training Quantization (PTQ)?

The primary difference is timing. PTQ is a process applied to an already trained, full-precision model to reduce its size. In contrast, QAD integrates the quantization process directly into the model's training loop. A quantized 'student' model learns by distilling knowledge from a full-precision 'teacher' model, allowing it to adapt and compensate for the loss of precision from the very beginning, which generally results in higher final accuracy.

LFM2.5 Q4\_0 Checkpoints from Quantization-Aware Distillation

LiquidAI has released new 4-bit quantized checkpoints for its LFM2.5 model family, addressing a persistent challenge in edge AI: the performance degradation associated with model compression. The new QAD Q4_0 GGUF files for the 230M, 350M, 1.2B, and 2.6B models allow developers to deploy these systems on consumer hardware with the memory and speed benefits of 4-bit quantization, but without the typical drop in reasoning and instruction-following capabilities.

Technical Details: Quantization-Aware Distillation

Unlike conventional post-training quantization (PTQ) which compresses a fully-trained model, LiquidAI's approach uses Quantization-Aware Distillation (QAD). This technique involves training the low-precision 'student' model from the start, using a high-precision 'teacher' model to guide it. This allows the quantized model to learn how to operate effectively within its precision constraints, rather than having them imposed after the fact. According to the company's benchmarks, this method is highly effective.

Performance Recovery: The QAD models recover an average of 97% of the accuracy lost from their original BF16 versions during quantization.
Benchmark Suite: Testing was conducted across a range of benchmarks, including GPQA Diamond, MMLU-Pro, and IFEval, to evaluate reasoning, instruction-following, and agentic skills.
Hardware Targets: The models were benchmarked on edge devices like the MacBook Pro, Samsung Galaxy S26 Ultra, and Raspberry Pi 5.
Format: The checkpoints are released in the GGUF format and are available on Hugging Face for use with runtimes like `llama.cpp`.

Impact on the Edge AI Ecosystem

This release provides a tangible path for deploying more capable local AI agents on resource-constrained devices. LiquidAI's performance data shows their QAD Q4_0 checkpoints can match the quality of higher-bitrate formats like Q4_K_M and Q5_K_M while delivering 3-33% higher decode throughput. This improved efficiency could reduce reliance on cloud-based APIs for certain tasks and accelerate the development of applications with on-device intelligence, from advanced smartphone assistants to autonomous robotics.

LiquidAI's implementation of Quantization-Aware Distillation shifts quantization from a post-hoc optimization step to an integral component of the model training process. This fundamentally improves the performance-per-watt profile for edge deployments, establishing a new baseline for how to effectively shrink large models for consumer hardware without making unacceptable quality compromises.

>> Verify Original Transmission at Hugging Face