vLLM

A high-throughput engine for serving language models on GPU servers, with PagedAttention, continuous batching and an OpenAI-compatible API.

96k stars, 24.2k forks Python On GitHub since February 2023

Visit website View on GitHub How to run it

What is vLLM?

vLLM serves language models to many users at once. PagedAttention manages the memory that models use for context, continuous batching groups incoming requests, and prefix caching reuses shared prompts, which together raise the number of requests a GPU can handle. It supports many model families, quantization formats such as FP8, INT4, GPTQ, AWQ and GGUF, and distributed inference across several GPUs. Started at UC Berkeley's Sky Computing Lab, it now has contributors from many companies and universities.

Good fit if

  • You run an internal or public model API and care about throughput and cost per request.
  • You have NVIDIA or AMD GPUs and want an OpenAI-compatible endpoint on them.
  • You deploy on Kubernetes: a Helm chart is in the repository.

Look elsewhere if

  • You run models on one laptop: Ollama or llama.cpp are easier.
  • You want a chat interface rather than a serving engine: pair it with Open WebUI.

Before you adopt it

vLLM is Apache-2.0 and community-run, with no paid parts. It is a serving engine, not an app, so plan for GPU drivers, model storage and monitoring yourself.

Drafted with AI from the project's README and repository data, then checked against that data on 30 October 2026. Report a correction

vLLM health: is it still maintained?

84 of 100, actively developed

  • Activity
    4,839 commits in the last 90 days, the latest on 1 November 2026.
    30 / 30
  • Contributors
    76 people wrote the latest 100 commits; the most active wrote 9% of them.
    20 / 20
  • Releases
    15 releases in the last 12 months; the latest, v0.32.0, on 18 October 2026.
    19 / 20
  • Replies to issues
    Too few recent issues from outside the team to measure, so this part gets a neutral score.
    10 / 20
  • Security practices
    No OpenSSF Scorecard for this repository yet, so this part gets a neutral score.
    5 / 10

Read from GitHub on 1 November 2026. How the score works

vLLM license: is it open source?

vLLM uses Apache-2.0, an open-source license approved by the OSI.

  1. February 2023First published on GitHub.
  2. TodayApache-2.0.

How to run vLLM yourself

Docker
Dockerfile or compose file in the repository
Compose file
None in the repository
Setup effort
Not rated: no compose file to judge from
Kubernetes
Helm chart in the repository
Desktop and mobile
Builds for Linux, macOS, including ARM in GitHub releases
Interface languages
English only

Install instructions: vllm.ai or the README on GitHub.

Alternatives to vLLM

  • OllamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. 89 health · MIT · 186k stars
  • FastGPTFastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box… 82 health · FastGPT Open Source License · 30k stars
  • ChatboxPowerful AI Client 79 health · GPL-3.0 · 42.2k stars
  • LobeHubLobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and… 73 health · LobeHub Community License · 83.7k stars
  • LocalAILocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware.… 75 health · MIT · 50.1k stars

All 36 projects in AI chat & local LLMs

Questions about vLLM

Is vLLM open source?

Yes. vLLM is published under Apache-2.0, an OSI-approved open-source license.

Is vLLM still maintained?

Its health score is 84 of 100 (actively developed). It had 4,839 commits in the last 90 days, the latest on 1 November 2026, and its latest release came out on 18 October 2026.

How do I run vLLM?

You can run it with Docker, on Kubernetes with the Helm chart in the repository, as an app for Linux, macOS, including ARM. The README on GitHub has the exact steps.

What are the alternatives to vLLM?

Close alternatives in the AI chat & local LLMs category include Ollama, FastGPT, Chatbox.