lyogavin/airllm

AirLLM works out of the box with virtually every popular open LLM — just pass its Hugging Face ID to AutoModel.from_pretrained(...). That covers all the major families:

Llama (2 / 3 / 3.1 / 3.3 / 4) · Qwen (1 / 2 / 2.5 / 3 / 3.5 / 3.8, including MoE, Flash-Next, FP8, and native VL) · DeepSeek (V2 / V3 / R1) · Mistral & Mixtral · Phi · Gemma · ChatGLM · Baichuan · InternLM · Yi · Kimi K3 — and most new models the day they're released.

Tiny GPU, huge models

The trick: AirLLM only ever keeps one layer on the GPU at a time, so the VRAM you need depends on the model's layer size — not its total size. That's how a 671B model fits on a hobbyist card:

| Model | Size | GPU VRAM | |---|---|---| | Qwen3 / Mistral / Phi (≈8B) | 8B | ~1–2 GB | | Qwen3-30B / Mixtral (MoE) | 30–47B | ~1–3 GB | | Qwen3.8-27B (dense VL) | 27B | 3.33 GB | | Qwen3.8-Flash-Next (MoE + PLE) | ~180B | 5.95 GB | | Qwen3-235B (MoE) | 235B | ~3 GB | | Llama 3.x 70B (full precision) | 70B | ~4 GB | | Llama 3.1 405B | 405B | ~8 GB | | DeepSeek-V3 | 671B | ~12 GB |

Same one line of code for all of them — no special setup.