Reading the repo
Reading the repo
How to install on Linux
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
Saved setup from 4 h ago
Will it work on my device?
Then run bash setup.sh. It stops at the first error and is safe to run again. Servers and tests aren't started; it lists them at the end. Read it first.
The repo uses git submodules, so the clone fetches them too.
git clone --recurse-submodules https://github.com/deepseek-ai/DeepInfer.git cd DeepInfer
Python >=3.12
sudo apt install python3 python3.12 python3-venv # Debian/Ubuntu
python -m venv .venv source .venv/bin/activate pip install -r requirements.txt pip install -r requirements-dev.txt
pytest
Before building DeepInfer, please create and activate an empty Python 3.12 environment.
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh
The default checkpoint, deepseek-ai/DeepSeek-V4.1-Flash-1017, is
downloaded into the Hugging Face cache on the first run; set DEEPINFER_HF_MODEL_DIR=/path/to/DeepSeek-V4.1-Flash-1017 to
run from a local copy instead.
python examples/offline_inference.py
See examples for details.
No reports for Linux yet.
Opens a chat with this manual already filled in, for Linux. It uses your own account; gitmanual sends nothing.