Skip to main content

​
Contributing

​
Prerequisites

  • Rust toolchain — rustup.rs
  • Docker Desktop, or another reachable Postgres database
  • An Anthropic API key (for end-to-end testing)

​
Run the server locally

1. Clone and build 
git clone <repo>
cd litellm-agent-platform
cargo build
The Rust crate is named litellm-rust, and the binary is lite. 2. Create a config 
cp config.yaml.example config.yaml
config.yaml.example ships with local provider entries. The api_key, master_key, and database_url fields read from environment variables by default: 
model_list:
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
3. Start Postgres The compose Postgres service is reachable from other containers as postgres:5432 and from your host as 127.0.0.1:${POSTGRES_PORT:-15432}. Start only the database when you want to run the Rust gateway directly: 
export POSTGRES_PORT=${POSTGRES_PORT:-15432}
docker compose up -d --wait postgres
export DATABASE_URL=postgres://lap:lap@127.0.0.1:${POSTGRES_PORT}/litellm_agents
If that host port is already in use, pick another before starting compose: 
export POSTGRES_PORT=15433
docker compose up -d --wait postgres
export DATABASE_URL=postgres://lap:lap@127.0.0.1:${POSTGRES_PORT}/litellm_agents
The gateway runs database migrations on startup. 4. Start the server 
export ANTHROPIC_API_KEY=sk-ant-...
export LITELLM_MASTER_KEY=sk-mykey

cargo run -- --config config.yaml
Server listens on http://localhost:4000 by default. 5. Verify 
# Health check
curl http://localhost:4000/health

# Chat request
curl -X POST http://localhost:4000/v1/messages \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hello"}], "max_tokens": 10}'
The proxy exposes /v1/messages (Anthropic-native protocol). There is no /v1/chat/completions (OpenAI-compat) route yet.

​
Run tests

cargo test
Most integration tests in tests/ spin up a local wiremock server — no real API calls needed. Postgres-backed tests are skipped unless TEST_DATABASE_URL is set. To run them against the compose database: 
export TEST_DATABASE_URL=$DATABASE_URL
cargo test --test managed_agents_api

​
Add a provider

Drop a new folder under src/sdk/providers/: 
src/sdk/providers/openai/
├── mod.rs
└── openai_responses/
    ├── mod.rs              # pub fn init(registry) { registry.register("openai", ...) }
    └── transformation.rs   # impl Transformation
build.rs auto-discovers the folder and wires it in. No other files need editing. See src/sdk/providers/anthropic/anthropic_messages/ for a reference implementation.

​
Project layout

src/
  sdk/
    routing.rs # request/model routing above provider endpoint transformation
    providers/
      base/ # endpoint-family base traits + runtime adapter base trait
      <provider>/<endpoint>/ # provider-owned endpoint/ and runtime/ modules
    agents/    # Agent Runtime SDK client resources + types
  proxy/       # config, master-key auth, AppState
  http/        # axum endpoints + outbound HTTP (http/llm.rs)
  cli/         # CLI wizard
  errors.rs    # shared GatewayError
See Internal SDK contract for runtime adapter types and event normalization.