Lists · Linux

Run LLMs locally on Linux

Open-source apps and runtimes for running language models on your own Linux computer, offline, with setup steps for each.

22 repos, most stars first

ollama/★ 186kollamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. Easy~5 minGo✓ 1 got it running
open-webui/★ 158kopen-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...) Involved30+ minSvelte
vllm-project/★ 97kvllmA high-throughput and memory-efficient inference and serving engine for LLMs Moderate~15 minPython
unslothai/★ 79kunslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more. Easy~5 minPython
Mintplex-Labs/★ 68kanything-llmStop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience Easy~5 minJavaScript
CherryHQ/★ 53kcherry-studioYour AI workspace for models, agents, and productivity. Moderate~15 minTypeScript
Aider-AI/★ 50kaideraider is AI pair programming in your terminal Moderate~15 minPython
HKUDS/★ 50knanobotYour lightweight open-source personal agent. Easy~5 minPython
chatboxai/★ 42kchatboxPowerful AI Client Easy~5 minTypeScript
khoj-ai/★ 38kkhojYour AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free. Moderate~15 minPython
dyad-sh/★ 22kdyadLocal, open-source AI app builder for power users ✨ v0 / Lovable / Replit / Bolt alternative 🌟 Star if you like it! Moderate~15 minTypeScript
MakazhanAlpamys/★ 10kSoupFine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. Easy~5 minPython
Osmantic/★ 7.8kODSODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server. Moderate~15 minPython
ARahim3/★ 6.3kDigUpSearch your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2. Easy~5 minSwift
higgsfield-ai/★ 6.3khiggsfieldFault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters Moderate~15 minJupyter Notebook
umlx5h/★ 4.6kLLPlayerThe media player for language learning, with dual subtitles, AI-generated subtitles, real-time translation, and more! Easy~5 minC#
gensecaihq/★ 63Wazuh-AutopilotAgentic SOC for Wazuh built on Strands Agents: a 13-agent swarm with 37 Wazuh-verified skills triages, investigates and plans response through the Wazuh MCP Server. Policy-gated autonomy, approvals, RBAC, audit, tracing and evals in one Docker stack. Bedrock, Anthropic, NVIDIA NIM, vLLM, LiteLLM or air-gapped Ollama. Moderate~15 minPython
sajanv88/★ 51full_stack_fastapi_react_templateAbout Full stack, modern multi-tenancy web template. Using FastAPI, React, MongoDB, Beanie ODM, Docker, Helm and more. Involved30+ minTypeScript
ariyankhalid97-creator/★ 1lead-intelligence-workbenchLocal-first Windows lead research and outreach workspace with multi-source discovery, evidence-backed verification, optional Ollama AI, and approval-gated Zoho campaigns. Easy~5 minPython
EdgeAgent/★ 1robot-orchestraRobot Orchestra: local-first Windows multi-agent desktop copilot with Ollama, Hermes, browser scanning and a draggable robot head. Easy~5 minTypeScript
Lumax-Solutions/★ 0netconfig-skillAgent Skill + desktop app for safe router, switch & firewall changes: state check, minimal diff, armed rollback, allow/deny tests. Cisco, FortiGate, Palo Alto, Juniper, MikroTik + more. Works with Claude, Codex, Cursor, Copilot, Gemini, or local AI (Ollama, llama.cpp, LM Studio). Easy~5 minPython
rahmankhanjohn/★ 0airgap-ai-stackPack, verify, deploy, and attest open-weight AI models inside enclaves with no internet egress. Go, zero dependencies. Easy~5 minGo

Run LLMs locally on Linux

More for Linux

Contact