Connect applications to multiple model providers through a self-hosted AI gateway or Python SDK. Manage virtual keys, routing, usage, and team budgets, with optional Enterprise governance.
Used this tool? Rate it
Used this tool? Rate it
LiteLLM is an open-source AI gateway and Python SDK for connecting applications to multiple large language model providers. Its LLM proxy gives applications a common OpenAI-compatible endpoint, while the SDK puts the integration directly inside a Python application. Enterprise adds governance, security features, and support to the same core code.
The gateway sits between an application and its selected model provider: it receives the request, forwards it upstream, and returns the response in the OpenAI format. That distinction defines the product. LiteLLM supplies the access and management layer; the model behind a configured route supplies the inference. A compatible API endpoint is an integration capability, rather than a bundled allocation of every model it can reach.
The SDK puts provider adaptation inside one application. A shared gateway centralizes access, usage attribution, and routing for multiple clients.
The current commercial offering centers on self-hosting. The free gateway and paid Enterprise license address different administration and support requirements. They are infrastructure offerings, rather than consumer chat subscriptions.
The SDK exposes completion, embeddings, and image generation through a shared interface, alongside a Router, OpenAI-compatible exceptions, and observability callbacks.
stream=True returns chunks as they are generated, allowing a client to display an answer incrementally.Model routing includes cooldowns, fallbacks, timeouts, and retries across deployments and providers. Retry behavior can use fixed or exponential backoff. Redis can hold shared cooldown and usage state in production. These mechanisms react to unavailable or rate-limited deployments; fallback success still depends on upstream availability.
The Router offers weighted selection, rate-limit-aware routing, least-busy routing, latency-based routing, and cost-based routing. A weighted pool, a latency-selected pool, and a price-selected pool have different behavior. Selecting one strategy does not optimize every criterion at once.
The MCP Gateway provides one fixed endpoint for MCP tools and controls access by key and team. Its documented operations include listing and calling tools, prompts, and resources. Supported transports include Streamable HTTP, SSE, and standard input/output. Several tool servers can share the access endpoint.
The AI Gateway also handles A2A agents. Model routes, tool calls, and agent endpoints remain different upstream resources even when they share a gateway address.
Virtual-key management requires a Postgres database and a master key that acts as the proxy administrator key. The documented setup uses DATABASE_URL for the database connection and LITELLM_MASTER_KEY for administration; keys can then be generated through /key/generate.
model_list entries, the Postgres connection, and the master key under general_settings.litellm --config /path/to/config.yaml, using the saved configuration file./key/generate with the master key in the Bearer authorization header, JSON content type, and a models list defining the generated key's model access.base_url to the gateway URL and its api_key to the generated virtual key.The master key administers the proxy; a virtual key authenticates an application to it. Provider credentials authorize the configured upstream connection. These credentials have different scopes.
Spend tracking attributes model costs to keys, users, and teams across providers. LiteLLM automatically prices known models and applies provider-specific tier information when the response includes that metadata. This separates application or department consumption even when upstream accounts are shared.
The calculation depends on the model cost map and current pricing data. Gateway spend is a calculated operational record, rather than the upstream invoice.
Load balancing can distribute calls across Azure, Bedrock, and other provider deployments, including retrying in another region when a call fails. A shared model alias can therefore represent several configured deployments rather than exposing each endpoint as a separate choice to every application.
This maintains a request path across deployment failures. The selected model and provider still determine the response; a shared alias does not make their outputs identical.
SSO is documented as free for up to five users, with an Enterprise license required beyond that. That identity-feature threshold is separate from free gateway availability.
As captured on October 23, 2026, a third-party launch platform displayed a 5.0 rating based on 24 LiteLLM reviews. This small product-level sample does not measure a deployment's latency, security, or operating cost.
OpenAI-compatible clients connect using the gateway base URL and a virtual key. Each compatible client can call that API without embedding the Python SDK.
LiteLLM also supports bringing internal, fine-tuned, and self-hosted models behind the same key. Internal and commercial endpoints can share an access pattern while retaining different hosting, credentials, and behavior.
Production deployment paths include Helm on Amazon EKS, Google GKE, and Azure AKS. Official Terraform modules are documented for AWS and GCP; the deployment guide specifies AKS with Helm for Azure and says there is no official Azure Terraform module.
Two service layouts are documented. A monolithic image serves model traffic, management APIs, and the UI together. A microservices layout separates the gateway, backend, and UI. PostgreSQL stores keys, teams, users, spend logs, and configuration and is required for proxy authentication and tracking features. Redis shares rate-limit, routing, and cache state and is documented as required once more than one instance is running.
The operator runs the state stores and management service as well as the request-forwarding component.
The pricing captured on October 23, 2026 separates a free self-hosted core from an annual Enterprise license. Enterprise pricing is sized to annual gateway request capacity, deployment architecture, and support needs, and is described as never per token. The checked pricing page does not publish a fixed Enterprise subscription amount.
| Offering | Published access model | Included scope |
|---|---|---|
| Open Source | $0; free forever; self-hosted | Virtual keys, users and teams, spend tracking, budgets, rate limits, fallbacks, request/response logging, and Prometheus metrics |
| Enterprise | Annual sales quote based on deployment and request capacity | The core gateway plus additional identity, audit, secret-management, administration, multi-region, and support features |
Enterprise uses a license key on the same gateway image, rather than requiring migration to a separate hosted product. Its listed additions include SSO and SCIM, audit logs, organization and team administrators, secret managers, IP allowlists, per-team logging, multi-region features, and support SLAs.
A 30-day Enterprise trial key is available by email. Procurement is offered directly, through AWS Marketplace, or through authorized resellers. The trial grants time-limited Enterprise access.
Standard Enterprise support includes a dedicated Slack or Teams channel with LiteLLM engineers, with published hours of 9am to 9pm PST on weekdays and no guaranteed response time. Guaranteed 24/7 response SLAs are an additional paid option.
The upstream license puts code outside the enterprise directory and stated restrictions under MIT; enterprise content has a separate license. The MIT grant permits use, modification, distribution, sublicensing, and sale, subject to conditions including retention of the copyright and permission notice.
The $0 listing covers the self-hosted software license. Infrastructure and upstream inference are separate parts of the architecture; provider compatibility does not establish an included token balance or model quota.
Portkey positions itself as a unified interface for generative AI applications, with tools for control, visibility, and security. Its stated scope combines model access with monitoring and governance.
The comparison concerns the operational layer as well as API access. LiteLLM supplies the self-hosted gateway and Python SDK with optional Enterprise activation; the capability overlap does not establish equivalent licensing or support.
OpenRouter offers access to hundreds of models through one API endpoint, with automatic fallbacks and selection of a cost-effective option for a request. LiteLLM addresses common model access through software running in the organization's deployment, with its own providers and access controls.
The choices overlap in API aggregation. Their model catalogs alone do not describe the difference between a Python library, an operated gateway, and a service endpoint.
drop_params is enabled. That setting drops unsupported OpenAI parameters; other parameters are treated as provider-specific request fields. A common request format does not guarantee that every provider implements every model feature.LITELLM_SALT_KEY encrypts provider credentials stored in the database. The deployment guide states that changing it after models are added makes those credentials unreadable. This differs from rotating a virtual key or upstream provider key.hide_secrets and openai_moderations, require Enterprise.Turning off message logging prevents messages and responses from being sent to the configured logging provider, while request metadata such as spend remains tracked. Content redaction retains operational metadata. Inference providers and configured logging destinations remain separate parts of the data path.
Usage telemetry is documented from version v1.107.0 as off unless enabled, with no recording until a destination is selected. It has no default HTTP endpoint. Enabled groups can keep aggregate counts in the deployment's database or send them to the configured HTTP endpoint; telemetry does not automatically mean a report is sent to LiteLLM.
The declared telemetry schema excludes prompts, completions, API keys, user and team IDs, model names, IP addresses, header values, and request bodies. Token counts are summed per row rather than retained per request. These exclusions concern telemetry, not every logging or inference path.
Enterprise connections have another boundary. The compliance documentation says billable request metering sends a count of successful requests to a LiteLLM collector, while license checks can use the license server or validate an offline signed payload. It states that prompts, responses, virtual keys, and the license key do not leave the deployment in that metering context. This metering is separate from optional usage telemetry.
The security, telemetry, logging, and compliance pages checked in this capture do not specify a general model-training policy for all upstream providers. Their self-hosting and data-flow statements do not establish how every selected model service handles submissions.
The maintainer describes a supply-chain incident on March 24, 2026 affecting PyPI packages v1.82.7 and v1.82.8. Its update says those packages were quarantined and subsequently deleted. The statement concerns those historical release packages, rather than every installation or current release.
Separately, independent security researchers assessed v1.77.7 and reported configuration-access and provider-credential issues tracked as CVE-2026-35029 and CVE-2026-59823. Their report states that the issues were resolved as of v1.83.9. The described impacts included low-privilege configuration takeover and redirecting provider calls to exfiltrate credentials. These are version-scoped findings, not a claim that the same flaws persist in all later releases.
LiteLLM supplies the gateway and SDK. Configured upstream providers supply inference; a catalog integration does not establish an included model allowance.
Enterprise enables features with a key on the same self-hosted gateway image. The license adds governance and support.
No. Message logging can suppress content while spend metadata remains. Usage telemetry and Enterprise request metering have separate controls.