Reading the repo

Opening the box…

How to set up vllm (vllm-project/vllm) · gitmanual

Assembly instructions

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

PythonC/C++ (CMake)#amd#blackwell#cuda#deepseek

Difficulty: Moderate
Time: about 15 min
Stars: ★ 96,339
Activity: Updated this month
License: Apache-2.0
Open issues: 9,872
Got it running: Be the first
Looked up: 2 h ago

Saved setup from 2 h ago

Start here

Website ↗vllm.aiThe project's own site.Docs ↗docs.vllm.aiLinked from the README.

Setup script in the repo

The maintainers included tools/install_gdrcopy.sh. Run it from the cloned folder after reading it.

bash tools/install_gdrcopy.sh

Also: tools/install_snapshot_runtime.sh, tools/install_torchcodec_rocm.sh, tools/install_triton_from_source.sh, tools/setup_deepgemm_pythons.sh

Will it work on my device?

Yes, it runs on macOS

You'll build it yourself with Python, C/C++ compiler, CMake.

Checking for

4 steps on macOS

Clone the repository

Needs Git: xcode-select --install

git clone https://github.com/vllm-project/vllm.git
cd vllm

2
Install the required runtimes
Python >=3.11,<3.15, C/C++ compiler, CMake 3.26
```
brew install uv
xcode-select --install
brew install cmake
```

Install dependencies

uv sync
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

4
Run the tests
```
uv run pytest
```

From the README: Getting Started

Install vLLM with uv (recommended) or pip:

uv pip install vllm
vllm download-kernels  # precompiled kernels for faster startup

Or build from source for development.

Visit our documentation to learn more.

Did you get it running?

No reports for macOS yet.

Still stuck? Get a written guide

Claude reads the 12 key files and writes a step-by-step guide for macOS. Once written, it's saved for everyone.

Parts list

1×Python >=3.11,<3.15runtime
1×C/C++ compiler runtime
1×CMake 3.26runtime
1×uv package manager

Share this manual

Good to know

Native projects often need extra system libraries. If the build stops with a missing package, check the maintainers' build guide (see Start here).
The README mentions NVIDIA CUDA. Check its hardware section: some features (or all of them) may need an NVIDIA GPU.
Has AGENTS.md — instructions for coding agents. You can open Claude Code (or another agent) in the cloned folder and ask it to do the setup for you.
Nested projects: benchmarks/kernels/requirements.txt, benchmarks/multi_turn/requirements.txt, examples/features/structured_outputs/pyproject.toml, rust/Cargo.toml and 11 more

Files we read (12)

.github/CONTRIBUTING.md

.github/workflows/macos-smoke-test.yml

.github/workflows/notify-ci-authorized.yml

.github/workflows/recipes-preview.yml

.github/workflows/recipes.yml

.github/workflows/record-ci-approval.yml

.github/workflows/run-ci-command.yml

CMakeLists.txt

README.md

pyproject.toml

rust-toolchain.toml

setup.py

Branch main · 8,932 files in the repo