LiteLLM
Acts as a unified proxy across 100+ LLMs, normalizing them to the OpenAI format while handling virtual keys, budgets, rate limits, fallbacks, and cost tracking.
Technical Profile & Specs
verified 2026-11-07
- Cost
- $0 / forever (Open Source)
- License
- Open Source
- Hosting Model
- Self-host (your server)
- Platforms
- webself-hosteddocker to self-host
- Installation
- Clone https://github.com/berriai/litellm or deploy with Docker Compose
- Operational Setup
- Deploy via Docker or binary release from GitHub and configure environment variables.
- Where Your Data Lives
- All application data and configuration live on your own server or local database.
- The Catch / Trade-off
- Requires self-hosting setup, updates, and server maintenance compared to managed Cloudflare AI Gateway.
Replaces 8 paid apps
up to $200/mo ($2,400/yr) in subscriptions
LiteLLM FAQ
Common questions
Is LiteLLM free and open source?+
LiteLLM is an open-source tool licensed under Open Source that runs on self-host. Acts as a unified proxy across 100+ LLMs, normalizing them to the OpenAI format while handling virtual keys, budgets, rate limits, fallbacks, and cost tracking.
What paid software does LiteLLM replace?+
LiteLLM serves as a free replacement for 8 paid SaaS applications: Cloudflare AI Gateway, Eden AI, Kong AI Gateway, Msty Nexus, OpenRouter, Requesty, Router, Vercel AI Gateway, saving up to $200/month ($2,400/year) in subscription bills.
How do you install and set up LiteLLM?+
Installation: Clone https://github.com/berriai/litellm or deploy with Docker Compose. Setup notes: Deploy via Docker or binary release from GitHub and configure environment variables.
What is the catch or trade-off with LiteLLM?+
Requires self-hosting setup, updates, and server maintenance compared to managed Cloudflare AI Gateway.