BerriAI-litellm logo

BerriAI-litellm

BerriAI-litellm routes API calls to over 100 LLM providers through a self-hosted gateway or Python SDK using the OpenAI format.

Social Media:
BerriAI-litellm

BerriAI Product Information

What is BerriAI-litellm?

BerriAI-litellm is an open-source AI gateway that provides a unified interface to call over 100 large language model providers using the OpenAI format or native APIs. It serves ML platform teams and developers building LLM applications who need centralized access, cost tracking, and load balancing. The product is self-hosted via Docker or Terraform, or used directly as a Python SDK.

What are the features of BerriAI-litellm?

  • Unified API interface
    Routes requests to over 100 LLM providers across endpoints including chat completions, embeddings, image generation, audio transcription, and reranking.
  • Python SDK integration
    Provides direct library access for calling models, with application-level load balancing, retry logic, fallback routing, and observability callbacks.
  • Auto routing
    Classifies prompt complexity using heuristic tiers to automatically select between configured fast or capable models.
  • MCP tool bridge
    Connects Model Context Protocol servers to any supported LLM by loading tools in OpenAI format for use in completion calls.
  • A2A protocol support
    Invokes Agent-to-Agent protocols through the gateway or SDK, supporting providers like LangGraph, Vertex AI Agent Engine, and Bedrock AgentCore.
  • Agent harnesses
    Runs coding agents such as Claude Code, Codex, OpenCode, pi, or Deep Agents on any configured model through the Python SDK.

What are the use cases of BerriAI-litellm?

  • Multi-tenant LLM platform management
    ML platform teams deploy the proxy server to centralize authentication, virtual keys, spend tracking per project or user, and guardrails.
  • Infrastructure deployment on AWS or GCP
    Teams provision a componentized stack with managed Postgres, Redis, and object storage using published Terraform modules.
  • IDE tool integration
    Developers configure Cursor IDE to route MCP tool requests through the local gateway instance using a master key header.

How to use BerriAI-litellm?

  1. Install the proxy tool
    Run uv tool install 'litellm[proxy]' in the terminal to install the AI gateway.
  2. Start the gateway
    Execute litellm --model gpt-4o to launch the proxy server locally.
  3. Send a request via SDK
    Initialize the OpenAI Python client with base_url="http://0.0.0.0:4000" and any API key, then call client.chat.completions.create with the target model and messages.

BerriAI-litellm Pricing Information:

  • Open source
    Self-hosted software available under an open-source license at no stated cost; stable Docker images are published after 12-hour load tests.
  • Enterprise License
    Custom pricing obtained by contacting the founders; includes features under the LiteLLM Commercial License, feature prioritization, custom integrations, professional support via dedicated Discord or Slack, custom SLAs, and single sign-on.

BerriAI-litellm Contact Email:

ishaan@berri.ai

Do you like this tool?

Upvote to help others discover it!

BerriAI Alternatives

Eden AI

Eden AI

Eden AI routes API requests to approximately 1,000 models across 80+ providers through a single OpenAI-compatible endpoint with built-in cost tracking.

FR
12.59%
173.8K
0
OpenRouter

OpenRouter

OpenRouter provides a single OpenAI-compatible API to route inference requests across hundreds of AI models and providers.

US
16.13%
19.9M
0
Portkey.ai - Control Panel for AI Apps

Portkey.ai - Control Panel for AI Apps

Portkey.ai - Control Panel for AI Apps provides a unified API gateway to route, observe, and govern requests across over 1,600 language models.

US
34.91%
195.7K
0
AI/ML API

AI/ML API

AI/ML API consolidates access to 500+ models and 1,000+ agent tools under one bill with zero markup, supporting SDK compatibility and automated cost-routing.

US
14.37%
232.9K
0
ApiFlux

ApiFlux

ApiFluxAI is an AI router that gives developers one API key for 100+ frontier models with transparent per-token pricing at 85% of list price, automatic failover, and a live dashboard.

US
100.00%
2.3K
0
CometAPI

CometAPI

CometAPI routes requests to over 500 AI models through a single OpenAI-compatible endpoint with pay-as-you-go billing.

IN
23.42%
603.0K
0
Scalekit

Scalekit

Scalekit AgentKit routes AI agent tool calls through a managed MCP gateway that handles per-user credentials, access scoping, and audit logging.

US
33.64%
108.2K
0
Higress.AI

Higress.AI

Higress.AI is an AI gateway that simplifies managing AI agents and LLM APIs with features like multi-model switching, content security, and token management.

CN
72.00%
13.3K
0

BerriAI Related Other Categories

BerriAI Traffic Analysis

This product is hosted on a shared platform (github.com). Platform-wide traffic is not shown, and the listing is excluded from traffic rankings.

BerriAI FAQ

Can I use BerriAI-litellm without deploying the proxy server?

Yes, you can install the Python SDK directly into your codebase using uv add litellm or use the independent litellm-core distribution for library integration without the dashboard or CLI.

How are provider API keys secured when deploying via Terraform?

Provider API keys are stored in AWS Secrets Manager or GCP Secret Manager, and their ARNs or resource IDs are referenced via the gateway_extra_secrets configuration input.

BerriAI Reviews

0
0 Reviews
Sign Into leave a review

Recent Reviews

No reviews yet

BerriAI Embed

Use website badges to drive community support for SeekTool.ai. They are easy to embed in your homepage or footer.

Light
Dark
BerriAI-litellm - Featured on SeekTool.aiBerriAI-litellm - Featured on SeekTool.ai
How to install?

Category Rankings

Ranked by monthly traffic