Native-speed vLLM transformers modeling backend

Transformers Backend Reaches Native vLLM Speed

The transformers library's modeling backend for the vLLM inference engine has been updated to match, and in some cases exceed, the performance of native, hand-written vLLM model implementations. This development removes a critical optimization step for developers, allowing them to use their existing transformers models for high-throughput serving without needing to port the code. The new capability, demonstrated across a range of Qwen3 models from 4B to 235B parameters, is activated with a single command-line flag: --model-impl transformers.

Under the Hood: Dynamic Code Optimization

The performance gains are achieved through a new runtime optimization process that goes beyond previous attention-kernel replacements. The backend now uses torch.fx to perform static analysis on a model's computational graph, identifying patterns that can be fused into more efficient operations. It then leverages Python's abstract syntax tree (ast) to rewrite the model's source code on the fly, applying optimizations that were previously only available in custom implementations.

• Fused Kernels: Automatically maps operations to optimized vLLM kernels, particularly for complex structures like Mixture-of-Experts (MoE) models requiring Expert Parallelization.
• Automated Parallelism: Infers optimal plans for tensor parallelism (TP) by fusing linear layers into blocks like MergedColumnParallelLinear and QKVParallelLinear.
• Full Compatibility: The dynamically modified models remain fully compatible with torch.compile and CUDA Graphs, ensuring they benefit from the entire stack of modern deep learning optimizations.

Unifying Development and Deployment

This update significantly streamlines the model lifecycle. Previously, engineering teams often maintained two separate codebases: one in transformers for training and experimentation, and a second, custom-built version for high-performance vLLM deployment. By eliminating the need for this specialized inference port, the ecosystem can move faster from research to production. This allows novel architectures to be deployed with state-of-the-art speed as soon as they are integrated into the transformers library, reducing redundant engineering work across the industry.

By collapsing the path from model training to high-performance inference, the transformers-vLLM integration commoditizes a layer of specialized optimization engineering. This allows teams to focus resources on model architecture and capabilities rather than on deployment-specific code porting.