MPS in PyTorch 2.15

PyTorch 2.15 Boosts AI Performance on Apple Silicon

PyTorch version 2.15 has been released, bringing substantial performance and memory efficiency upgrades for developers running AI models on Apple silicon Macs. This update, with significant contributions from the Hugging Face team, targets the Metal Performance Shaders (MPS) backend to address previous bottlenecks. The enhancements make local model inference and light training more practical on consumer hardware, improving the viability of Macs as a platform for AI development.

Key Technical Enhancements

The update moves beyond the generic MPSGraph framework by implementing dedicated Metal kernels for critical operations, resulting in direct performance gains across language, audio, and video models. This addresses long-standing issues with unsupported operations, performance cliffs, and unexpected memory growth that developers previously faced on the platform.

• Faster LLM Decoding: Mixture-of-Experts (MoE) models see a 5–6.5x speedup in token generation. For example, Qwen3.6-35B-A3B accelerates from 9.4 to 61.3 tokens/s on an M3 Ultra. Dense models also benefit from faster single-row linear layers and an 8x faster softmax operation for sampling.
• FP8 Support: The release introduces `torch.float8_e4m3fn` tensors, halving the memory footprint compared to BF16 and enabling scaled matrix multiplication directly on the GPU.
• Drastic Memory Reduction: A fix in constant padding reduces peak driver memory for Wan2.2's video VAE decode from an impractical 270 GiB to a manageable 8 GiB, making complex video generation feasible.
• Expanded Operations: New Metal kernels accelerate convolutions, padding, dropout, and complex linear algebra, including LU factorization and triangular solves.

Impact on the Developer Ecosystem

These improvements directly affect developers using popular libraries like `transformers` and `diffusers` or applications such as ComfyUI. The changes translate to faster iteration times and a more responsive user experience for local AI tasks. By solidifying the Mac's role as a capable platform within the PyTorch ecosystem, the update encourages more development and experimentation outside of cloud-based GPU environments. However, developers should note that the update introduces stricter requirements, now demanding macOS 15 and Python 3.11 or later.

The focused effort by Hugging Face and the community to optimize PyTorch for Apple's Metal API highlights a key industry trend: high-performance consumer hardware is becoming a primary target for sophisticated AI workloads, shifting development from the data center to the desktop.