Baidu Releases vLLM-Kunlun — a Plugin for Running LLMs on Kunlun XPU Accelerators

Baidu has open-sourced vLLM-Kunlun — a community-maintained hardware plugin that makes it possible to run the vLLM inference engine for large language models on Kunlun XPU accelerators. The repository appeared on GitHub on December 8, 2025; by now it has gathered several hundred stars and dozens of forks.

A plugin, not a fork

The project's central idea is integration without modifying vLLM's code. vLLM-Kunlun works as a standard platform plugin through Python's entry points mechanism and follows the principles laid out in the RFC on pluggable hardware (Hardware Pluggable). This approach decouples support for specific hardware from vLLM's main codebase, which simplifies maintenance on both sides.

Through this layer, popular open models of a wide variety of architectures run on the Kunlun XPU: classic transformers, models with a sparse mixture of experts (MoE), embedding models, and multimodal LLMs.

What's supported

Support is claimed for more than two dozen common models, including the Qwen, Llama, DeepSeek, GLM, Gemma4, and Kimi-K2 families, as well as multimodal variants (Qwen-VL, InternVL, and others). Beyond that, the developers list:

• W8A8 (INT8), AWQ, GPTQ, and compressed-tensors W4A16 quantization for dense and MoE models;
• fine-tuning via LoRA and Multi-LoRA for the Qwen series of models;
• computation-graph optimization with Piecewise Kunlun Graph;
• optimized FlashMLA attention for DeepSeek MLA architectures;
• speculative decoding — multi-token prediction (MTP) for DeepSeek-V3.2 and DFlash/EAGLE-style predictors;
• tensor parallelism for distributed inference across multiple devices;
• a server with an OpenAI-compatible API.

Requirements and status

Running it requires a Kunlun3 P800 accelerator, Ubuntu 20.04, Python 3.10 or higher, and a build of PyTorch made specifically for KL3 (xpytorch). The latest features from the Qwen3.5 family require version 5.x of the transformers library.

The most recent stable release is v0.11.0, put out in December 2025; version v0.25.1 is in development, adding support for Qwen3.5 and Qwen3.5-MoE, text and multimodal Gemma4, GLM MoE DSA, and DFlash speculative decoding. The project is distributed under the Apache 2.0 license.

The arrival of such a plugin continues a trend: accelerator makers are working to ensure their hardware is compatible with the de facto standard inference tools, so that developers don't have to rewrite their infrastructure for each new platform.

Source: Lobsters