vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

70,529stars

Forks

13,511

Open issues

3,388

Watchers

70,529

Size

160.8 MB

PythonApache License 2.0

amdblackwellcudadeepseekdeepseek-v3gptgpt-ossinferencekimillamallmllm-servingmodel-servingmoeopenaipytorchqwenqwen3tputransformer

Created: Feb 9, 2023

Updated: Feb 18, 2026

Last push: Feb 18, 2026