LiteLLM
FreemiumOne OpenAI-compatible gateway to every model provider, run on your own infrastructure.
Published 27 September 2026Last updated 6 October 2026
Scores
Popularity4/5
Widely used
56K+ GitHub stars and production adoption at Stripe, Google ADK, Greptile, and OpenHands make it the clear self-hosted leader in this genre.
Learning Curve2/5
Easy
Requires standing up and maintaining real infrastructure, PostgreSQL, Redis, the proxy server itself, meaningfully more setup than a zero-ops hosted aggregator like OpenRouter.
Flexibility5/5
Unopinionated
Full control over routing logic, guardrails, virtual keys, and spend limits across 100+ providers, the most configurable option in the aggregator genre precisely because you own the deployment.
Performance4/5
Fast
8ms P95 latency at 1,000 RPS and a Rust core deliver genuinely fast proxy performance, though real-world throughput depends on the self-hosted infrastructure backing it.
Portability5/5
No lock-in
MIT licensed, genuinely self-hostable anywhere with Docker or the provided Terraform modules, no vendor lock-in to a hosted service.
About LiteLLM
LiteLLM is an open-source AI gateway, available as a Python SDK and as a proxy server you run yourself. Applications call one OpenAI-compatible interface, and LiteLLM translates each request for the provider behind it, across more than 100 providers including OpenAI, Anthropic, Azure, Bedrock, and Gemini, so there is no per-vendor SDK, auth scheme, or request format to juggle. It does the same job OpenRouter does as a hosted service, but on infrastructure you operate, and the vendor offers no managed version of its own.
Beyond routing, the proxy adds virtual API keys with budgets and rate limits per key, team, and user, spend tracking, load balancing and fallbacks across deployments, guardrails, an admin interface on the same port, and Prometheus metrics. PostgreSQL is required for keys and spend tracking, and Redis becomes necessary once more than one proxy instance runs, to share rate limits, router state, and cache. The official Docker Compose file starts the proxy with PostgreSQL and Prometheus, Helm charts cover Kubernetes, and official Terraform modules cover AWS and GCP. The vendor cites about 8 milliseconds of added latency at the 95th percentile at 1,000 requests per second.
The core is MIT-licensed and free to self-host, so you pay only for your servers and what the model providers charge. An Enterprise license, priced by quote and never per token, adds single sign-on beyond five users, SCIM, OIDC and JWT authentication, audit logs, secret-manager integration, and multi-region deployment. Companies the vendor names as users include Netflix, NVIDIA, Okta, and IBM.
Key Features
- OpenAI-compatible endpoint in front of more than 100 LLM providers
- A Python SDK and a self-hosted proxy server from the same project
- Virtual API keys with budgets, rate limits, and spend tracking per key, team, and user
- Load balancing, retries, and fallbacks across deployments of the same model
- An admin interface on the proxy's own port for keys, models, and spend
- Prometheus metrics, plus callbacks to tracing tools such as Langfuse and LangSmith
- An official Docker Compose file, Helm charts, and Terraform modules for AWS and GCP
- MIT-licensed core, with an Enterprise license for SSO beyond five users and audit logs
Pros
- MIT-licensed core with keys, budgets, load balancing, and metrics included, and no per-token fee
- One OpenAI-style client works for every provider, so switching models is a configuration change
- Prompts and provider credentials stay on infrastructure you operate
- Official Helm charts and Terraform modules make the production route a documented one
- Callbacks send traces and costs to the observability tool you already use
Cons
- You run it: all model traffic depends on its uptime, and the vendor's production guidance is two or more instances behind a load balancer
- PostgreSQL is required and Redis joins it once there is more than one instance, which is more to operate than a hosted aggregator like OpenRouter
- SSO beyond five users, SCIM, JWT and OIDC authentication, and audit logs need an Enterprise license that is priced by quote
- The official Compose file publishes the database and metrics ports with a default password, so it needs hardening before it faces the internet
- Provider outages and price changes still reach you unless fallbacks are configured, since the gateway routes requests and adds no capacity of its own
LiteLLM Pricing
Freemium- MIT-licensed core, free to self-host with no license fee
- Virtual keys, budgets, load balancing, and Prometheus metrics included
- SSO free for up to five users
- You pay only for your own infrastructure and the model providers
- SSO beyond five users, SCIM, OIDC and JWT authentication
- Audit logs, secret-manager integration, IP allowlists, and multi-region deployment
- Priced by quote and never per token, with a 30-day trial
Tech Stacks with LiteLLM
LiteLLM Self-Hosted
Infrastructure
InfrastructureRun LiteLLM, the open-source AI gateway, in Docker on your own server. One OpenAI-compatible endpoint sits in front of every model provider, PostgreSQL stores the virtual keys, budgets, and spend, and Prometheus watches the traffic, so applications and agents call a single address while the provider keys and the costs stay under your control.
OpenClaw Self-Hosted
InfrastructureRun OpenClaw, the open-source personal AI agent, in Docker on your own server or home hardware. It answers in the messaging apps you already use, thinks with a model such as Claude or OpenAI, and stays closed to the internet, with its dashboard reached over an SSH tunnel.
Hermes Agent Self-Hosted
InfrastructureRun Hermes Agent, Nous Research's open-source agent that writes its own skills as it works, in Docker on your own server or home hardware. It answers in your messaging apps, runs on a low-cost model such as DeepSeek or a frontier one such as Claude, and keeps its dashboard private behind an SSH tunnel.
Tools Related to LiteLLM
Works well with LiteLLM(1)
Groq is one of LiteLLM's 100+ documented supported providers — a self-hosted LiteLLM gateway can route requests to Groq's fast LPU-based inference alongside any other provider.
Integrates with LiteLLM(6)
LiteLLM has a built-in Langfuse callback, so a gateway can log every routed request, with tokens and cost, to Langfuse by setting it as a success callback.
The OpenAI Agents SDK ships an optional LiteLLM extension (the LitellmModel class) that routes agents to non-OpenAI models through LiteLLM's provider layer.
LiteLLM has a dedicated vLLM provider (the hosted_vllm/ model prefix), so a LiteLLM gateway can route requests to self-hosted vLLM servers alongside hosted APIs.
LiteLLM has an LM Studio provider, so a LiteLLM gateway can route requests to models running in LM Studio's local server alongside hosted APIs.
LiteLLM ships a Fireworks AI provider under the fireworks_ai/ model prefix, which lets one gateway send traffic to Fireworks AI's low-latency endpoints and fall back to other providers.
LiteLLM has a Together AI provider (the together_ai/ model prefix), so a self-hosted LiteLLM gateway can route requests to Together AI's hosted open models alongside other providers.
Alternatives to LiteLLM(1)
Both are OpenAI-compatible gateways to 100+ LLM providers. LiteLLM is self-hosted (MIT licensed, run it yourself), while OpenRouter is a zero-infrastructure hosted service — the same job, opposite deployment trade-off.