Contributing

Prerequisites

Rust toolchain — rustup.rs
Docker Desktop, or another reachable Postgres database
An Anthropic API key (for end-to-end testing)

Run the server locally

1. Clone and build

git clone <repo>
cd litellm-agent-platform
cargo build

The Rust crate is named litellm-rust, and the binary is lite. 2. Create a config

cp config.yaml.example config.yaml

config.yaml.example ships with local provider entries. The api_key, master_key, and database_url fields read from environment variables by default:

model_list:
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

3. Start Postgres The compose Postgres service is reachable from other containers as postgres:5432 and from your host as 127.0.0.1:${POSTGRES_PORT:-15432}. Start only the database when you want to run the Rust gateway directly:

export POSTGRES_PORT=${POSTGRES_PORT:-15432}
docker compose up -d --wait postgres
export DATABASE_URL=postgres://lap:lap@127.0.0.1:${POSTGRES_PORT}/litellm_agents

If that host port is already in use, pick another before starting compose:

export POSTGRES_PORT=15433
docker compose up -d --wait postgres
export DATABASE_URL=postgres://lap:lap@127.0.0.1:${POSTGRES_PORT}/litellm_agents

The gateway runs database migrations on startup. 4. Start the server

export ANTHROPIC_API_KEY=sk-ant-...
export LITELLM_MASTER_KEY=sk-mykey

cargo run -- --config config.yaml

Server listens on http://localhost:4000 by default. 5. Verify

# Health check
curl http://localhost:4000/health

# Chat request
curl -X POST http://localhost:4000/v1/messages \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hello"}], "max_tokens": 10}'

The proxy exposes /v1/messages (Anthropic-native protocol). There is no /v1/chat/completions (OpenAI-compat) route yet.

Run tests

cargo test

Most integration tests in tests/ spin up a local wiremock server — no real API calls needed. Postgres-backed tests are skipped unless TEST_DATABASE_URL is set. To run them against the compose database:

export TEST_DATABASE_URL=$DATABASE_URL
cargo test --test managed_agents_api

Add a provider

Drop a new folder under src/sdk/providers/:

src/sdk/providers/openai/
├── mod.rs
└── openai_responses/
    ├── mod.rs              # pub fn init(registry) { registry.register("openai", ...) }
    └── transformation.rs   # impl Transformation

build.rs auto-discovers the folder and wires it in. No other files need editing. See src/sdk/providers/anthropic/anthropic_messages/ for a reference implementation.

Project layout

src/
  sdk/
    routing.rs # request/model routing above provider endpoint transformation
    providers/
      base/ # endpoint-family base traits + runtime adapter base trait
      <provider>/<endpoint>/ # provider-owned endpoint/ and runtime/ modules
    agents/    # Agent Runtime SDK client resources + types
  proxy/       # config, master-key auth, AppState
  http/        # axum endpoints + outbound HTTP (http/llm.rs)
  cli/         # CLI wizard
  errors.rs    # shared GatewayError

See Internal SDK contract for runtime adapter types and event normalization.

Get started

Runtimes

Channels

Learn

Engineering

Contributing

Contributing

Prerequisites

Run the server locally

Run tests

Add a provider

Project layout

​Contributing

​Prerequisites

​Run the server locally

​Run tests

​Add a provider

​Project layout

Contributing

Prerequisites

Run the server locally

Run tests

Add a provider

Project layout