Reading the repo

Opening the box…

How to install vllm on Linux (vllm-project/vllm) · gitmanual

How to install on Linux

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

PythonC/C++ (CMake)#amd#blackwell#cuda#deepseek

Difficulty: Moderate
Time: about 15 min
Stars: ★ 96,462
Activity: Updated this month
License: Apache-2.0
Open issues: 9,953
Got it running: Be the first
Looked up: 5 h ago

Saved setup from 5 h ago

Start here

Website ↗vllm.aiThe project's own site.Docs ↗docs.vllm.aiLinked from the README.

Setup script in the repo

The maintainers included tools/install_gdrcopy.sh. Run it from the cloned folder after reading it.

bash tools/install_gdrcopy.sh

Also: tools/install_snapshot_runtime.sh, tools/install_torchcodec_rocm.sh, tools/install_triton_from_source.sh, tools/setup_deepgemm_pythons.sh

Will it work on my device?

Yes, it runs on Linux

You'll build it yourself with Python, C/C++ compiler, CMake.

Checking for

macOS Windows Linux

4 steps on Linux

Download setup.sh

Then run bash setup.sh. It stops at the first error and is safe to run again. Servers and tests aren't started; it lists them at the end. Read it first.

Clone the repository

git clone https://github.com/vllm-project/vllm.git
cd vllm

Install the required runtimes

Python >=3.11,<3.15, C/C++ compiler, CMake 3.26

curl -LsSf https://astral.sh/uv/install.sh | sh
sudo apt install build-essential  # Debian/Ubuntu
sudo apt install cmake

Install dependencies

uv sync
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

4
Run the tests
```
uv run pytest
```

From the README: Getting Started

Install vLLM with uv (recommended) or pip:

uv pip install vllm
vllm download-kernels  # precompiled kernels for faster startup

Or build from source for development.

Visit our documentation to learn more.

Did you get it running?

No reports for Linux yet.

Still stuck? Ask an AI

Opens a chat with this manual already filled in, for Linux. It uses your own account; gitmanual sends nothing.

Ask Claude Ask ChatGPT

Parts list

1×Python >=3.11,<3.15runtime
1×C/C++ compiler runtime
1×CMake 3.26runtime
1×uv package manager

Share this manual

Is this your repo?

Add a badge to your README so newcomers find these steps.

[![Setup manual](https://gitmanual.com/vllm-project/vllm/badge.svg)](https://gitmanual.com/vllm-project/vllm)

Good to know

Native projects often need extra system libraries. If the build stops with a missing package, check the maintainers' build guide (see Start here).
The README mentions NVIDIA CUDA. Check its hardware section: some features (or all of them) may need an NVIDIA GPU.
Has AGENTS.md — instructions for coding agents. You can open Claude Code (or another agent) in the cloned folder and ask it to do the setup for you.
Nested projects: benchmarks/kernels/requirements.txt, benchmarks/multi_turn/requirements.txt, examples/features/structured_outputs/pyproject.toml, rust/Cargo.toml and 11 more

Files we read (12)

.github/CONTRIBUTING.md

.github/workflows/macos-smoke-test.yml

.github/workflows/notify-ci-authorized.yml

.github/workflows/recipes-preview.yml

.github/workflows/recipes.yml

.github/workflows/record-ci-approval.yml

.github/workflows/run-ci-command.yml

CMakeLists.txt

README.md

pyproject.toml

rust-toolchain.toml

setup.py

Branch main · 8,981 files in the repo