Big unlock for open-source AI inference: Hugging Face Transformers models can now run in vLLM at native speed, often matching or beating hand-written implementations.

Until now, every new architecture often needed to be built twice:

• Once in Transformers for training and research
• Again in vLLM for fast production inference

That duplication slowed down new models, added maintenance, and created room for implementations to diverge. Now, model authors can implement an architecture once in Transformers and immediately benefit from vLLM’s optimized inference stack.

In our benchmarks, the Transformers backend matched or beat native vLLM throughput across models from 4B to 235B parameters, including tensor parallel and MoE setups. One readable model implementation can now power training, fine-tuning, evaluation, RL rollouts, and production inference.

The conventional wisdom is that abstractions make systems slower. The best abstractions make the whole ecosystem faster.

Write the model once. Deploy it everywhere.

https://t.co/nTXcwAV0Bf