Check its license. It has its own license: read it before using it for work.

Does litellm use paid AI services?

Can use OpenAI, Cohere, Replicate or Anthropic. Its settings have keys for these paid APIs. Its README says whether one is required.

Is litellm still maintained?

Yes, active. Last change today. Latest release v1.106.8, today.

Widely used. 63k stars, 3 years old, by an organization. As with any download, get it from the official repo.

Assembly instructions

BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Node.jsPython#ai-gateway#anthropic#azure-openai#bedrock

Before you start

Is it free?: Check its licenseIt has its own license: read it before using it for work.
Paid AI services?: Can use OpenAI, Cohere, Replicate or AnthropicIts settings have keys for these paid APIs. Its README says whether one is required.
Still maintained?: Yes, activeLast change today. Latest release v1.106.8, today.
Can I trust it?: Widely used63k stars, 3 years old, by an organization. As with any download, get it from the official repo.

Difficulty: Involved
Time: 30 min or more
Stars: ★ 63,101
Activity: Updated this month

Saved setup from 6 h ago

Will it work on macOS?

Yes, it installs on your macOS

The README gives a pipx / uv install for macOS.

Your system

macOS Windows Linux

Start here

Website ↗docs.litellm.aiThe project's own site.

Also: Docs ↗ · In the browser ↗

Heads-upBuilding it yourself? It also needs PostgreSQL and 18 settings in .env (11 secret).

Install with a package manager

pipx / uv

uv tool install 'litellm[proxy]'

From the README, exactly as the maintainers wrote it.

Use it in your own project

This is a library: you add it to your code rather than opening an app.

Add to your Python project

uv add litellm

From the README, exactly as the maintainers wrote it.

Did you get it running?

No reports for macOS yet.

Still stuck? Get a written guide

GitManual AI Agent reads the 17 key files and writes a step-by-step guide for macOS. Once written, it's saved for everyone.

Parts list

1×Node.js LTS (no version pinned)runtime
1×Python >=3.10, <3.16runtime
1×make tool
1×npm package manager
1×uv package manager
1×PostgreSQL service

18× settings in .env

OPENAI_API_KEYsecret
OPENAI_BASE_URL
COHERE_API_KEYsecret
OR_SITE_URL
OR_APP_NAME
OR_API_KEYsecret
AZURE_API_BASE
AZURE_API_VERSION
AZURE_API_KEYsecret
REPLICATE_API_KEYsecret
REPLICATE_API_TOKENsecret
ANTHROPIC_API_KEYsecret
INFISICAL_TOKENsecret
NOVITA_API_KEYsecret
INFINITY_API_KEYsecret
LITELLM_MASTER_KEYsecret
DATABASE_URL
STORE_MODEL_IN_DB

Share this manual

Is this your repo?

Add a badge to your README so newcomers find these steps.

[![Setup manual](https://gitmanual.com/BerriAI/litellm/badge.svg)](https://gitmanual.com/BerriAI/litellm)

Good to know

Dockerfile available (base image: $UV_IMAGE).
Dev Container config found — you can open it in VS Code Dev Containers or GitHub Codespaces for a zero-setup environment.
Makefile has a make bootstrap target — it may automate most of these steps.
Has AGENTS.md — instructions for coding agents. You can open Claude Code (or another agent) in the cloned folder and ask it to do the setup for you.
Nested projects: cookbook/anthropic_agent_sdk/requirements.txt, cookbook/gollem_go_agent_framework/go.mod, cookbook/livekit_agent_sdk/requirements.txt, enterprise/pyproject.toml and 11 more

Details

License: Not detected
Open issues: 5,144
Branch: main

Files we read (17)

.devcontainer/devcontainer.json

.env.example

.github/workflows/check-schema-sync.yml

.github/workflows/cost-map-pins-recheck.yml

.github/workflows/duplicate_issue_check.yml

.github/workflows/guard-fork-dependencies.yml

.github/workflows/mutation-test.yml

.github/workflows/publish-lint-base-counts.yml

CONTRIBUTING.md

Dockerfile

Makefile

README.md

docker-compose.yml

docker/.env.example

package.json

pyproject.toml

rust-toolchain.toml

15,770 files in the repo