# Create an agent Source: https://docs.litellm-agent-platform.ai/api-reference/agents/create POST /api/agents # Delete an agent Source: https://docs.litellm-agent-platform.ai/api-reference/agents/delete DELETE /api/agents/{agent_id} # Get an agent Source: https://docs.litellm-agent-platform.ai/api-reference/agents/get GET /api/agents/{agent_id} # List agents Source: https://docs.litellm-agent-platform.ai/api-reference/agents/list GET /api/agents # Update an agent Source: https://docs.litellm-agent-platform.ai/api-reference/agents/update PATCH /api/agents/{agent_id} # Create API key Source: https://docs.litellm-agent-platform.ai/api-reference/keys/create POST /api/keys # Delete API key Source: https://docs.litellm-agent-platform.ai/api-reference/keys/delete DELETE /api/keys/{id} # List API keys Source: https://docs.litellm-agent-platform.ai/api-reference/keys/list GET /api/keys # MCP proxy Source: https://docs.litellm-agent-platform.ai/api-reference/mcp POST /mcp/{server_id} # Create a message Source: https://docs.litellm-agent-platform.ai/api-reference/messages POST /v1/messages # List models Source: https://docs.litellm-agent-platform.ai/api-reference/models GET /v1/models # API Overview Source: https://docs.litellm-agent-platform.ai/api-reference/overview The LiteLLM Agent Platform REST API. The LiteLLM Agent Platform exposes a REST API for managing runtimes, agents, and sessions. All endpoints require a `Authorization: Bearer ` header (or `?key=` for SSE streams). ## Base URL | Environment | URL | | -------------------------- | ----------------------------------- | | **Production** | `https://litellm-rust.onrender.com` | | **Local (Docker Compose)** | `http://localhost:4000` | ## Authentication All endpoints require your master key: ```bash theme={null} curl https://litellm-rust.onrender.com/api/agents \ -H "Authorization: Bearer $MASTER_KEY" ``` The default master key for local Docker Compose is `sk-local`. ## Live Swagger UI The full interactive Swagger UI is available at: * **Production:** [litellm-rust.onrender.com/docs](https://litellm-rust.onrender.com/docs) * **Local:** [localhost:4000/docs](http://localhost:4000/docs) The raw spec is at `/openapi.json` — import it into Postman, Insomnia, or any OpenAPI-compatible client. ## Endpoint groups | Group | Description | | ------------ | -------------------------------------------------------------------------- | | **System** | Health check and capabilities | | **Models** | List available model aliases | | **Messages** | Anthropic-compatible chat completions | | **Agents** | Create and manage agents | | **Sessions** | Create sessions and stream events | | **Runtimes** | Configure built-in runtime credentials and manage custom runtime harnesses | | **API Keys** | Issue and revoke gateway keys | | **MCP** | Proxy requests to MCP tool servers | # Register a runtime harness Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/create POST /api/runtime-harnesses # Delete a runtime harness Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/delete DELETE /api/runtime-harnesses/{alias} # Delete runtime credentials Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/delete-credentials DELETE /api/agent-runtimes/{runtime}/credentials # List runtime harnesses Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/list GET /api/runtime-harnesses # List built-in runtimes Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/list-built-in GET /api/agent-runtimes # Save runtime credentials Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/save-credentials PUT /api/agent-runtimes/{runtime}/credentials # Update a runtime harness Source: https://docs.litellm-agent-platform.ai/api-reference/runtimes/update PUT /api/runtime-harnesses/{alias} # Create a session Source: https://docs.litellm-agent-platform.ai/api-reference/sessions/create POST /session # Send a prompt Source: https://docs.litellm-agent-platform.ai/api-reference/sessions/send-event POST /session/{session_id}/prompt_async # Stream events (SSE) Source: https://docs.litellm-agent-platform.ai/api-reference/sessions/stream GET /v1/sessions/{session_id}/events/stream # Capabilities Source: https://docs.litellm-agent-platform.ai/api-reference/system/capabilities GET /api/capabilities # Health check Source: https://docs.litellm-agent-platform.ai/api-reference/system/health GET /health # API Source: https://docs.litellm-agent-platform.ai/channels/api Talk to your agents programmatically using the LAP REST API. Every action available in the UI is also available over the REST API. Authenticate with `Authorization: Bearer `. ## Base URL | Environment | URL | | ----------- | ----------------------------------- | | Production | `https://litellm-rust.onrender.com` | | Local | `http://localhost:4000` | ## Create an agent ```bash theme={null} curl -X POST $LAP_URL/api/agents \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-agent", "owner_id": "local-user", "runtime": "cursor", "model": "claude-opus-4-5", "system": "You are a helpful assistant." }' ``` ## Start and run a session ```bash theme={null} SESSION=$(curl -s -X POST $LAP_URL/session \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "runtime": "cursor", "agent_id": "", "prompt": "Summarize the README." }' | jq -r .id) ``` ## Send another prompt ```bash theme={null} curl -X POST $LAP_URL/session/$SESSION/prompt_async \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": { "modelID": "claude-opus-4-5" }, "parts": [{ "type": "text", "text": "Summarize the README." }] }' ``` ## Stream the response ```bash theme={null} curl -N "$LAP_URL/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer $MASTER_KEY" ``` Events are server-sent (SSE), newline-delimited: ``` data: {"type":"session.status_running"} data: {"type":"agent.message","content":"Here is the summary..."} data: {"type":"session.status_idle"} ``` ## SDK examples ```python Python theme={null} import requests session = requests.post( f"{LAP_URL}/session", headers={"Authorization": f"Bearer {MASTER_KEY}"}, json={ "runtime": "cursor", "agent_id": agent_id, "prompt": "Hello!", }, ).json() requests.post( f"{LAP_URL}/session/{session['id']}/prompt_async", headers={"Authorization": f"Bearer {MASTER_KEY}"}, json={ "model": {"modelID": "claude-opus-4-5"}, "parts": [{"type": "text", "text": "Hello!"}], }, ) ``` ```javascript JavaScript theme={null} const session = await fetch(`${LAP_URL}/session`, { method: "POST", headers: { Authorization: `Bearer ${MASTER_KEY}`, "Content-Type": "application/json" }, body: JSON.stringify({ runtime: "cursor", agent_id: agentId, prompt: "Hello!", }), }).then(r => r.json()); const stream = await fetch(`${LAP_URL}/v1/sessions/${session.id}/events/stream`, { headers: { Authorization: `Bearer ${MASTER_KEY}` }, }); ``` See the [API Reference](/api-reference/overview) for the full endpoint list. # Google Chat Source: https://docs.litellm-agent-platform.ai/channels/google-chat Add an agent as a Google Chat app. The Google Chat channel lets people talk to a LAP agent from Google Chat. Each connected LAP agent uses one Google Chat app endpoint and one service account for replies. ## Prerequisites * LiteLLM Agent Platform deployed at a public HTTPS URL * A LAP agent with a working runtime and model * A Google Cloud project with the Google Chat API enabled * Permission to create or edit a Google Chat app * A service account JSON key for sending Google Chat replies For local testing, expose LAP with an HTTPS tunnel and use that public URL when you configure the Google Chat app. ## 1. Open the agent Google Chat flow In the LAP UI, open **Agents**, choose an agent, then click **Add to Google Chat**. The flow shows the agent-specific event endpoint: ```text theme={null} $LAP_URL/api/agents//google-chat/events ``` Use the same URL as the Google Chat app URL and auth audience. ## 2. Configure the Google Chat app In Google Cloud Console, open the Google Chat API configuration for your project and create or edit a Chat app. Set the app endpoint to the LAP event endpoint: ```text theme={null} $LAP_URL/api/agents//google-chat/events ``` Set the auth audience to the same endpoint URL. Configure app visibility for the workspace or test users that should be able to install and message the app. ## 3. Save LAP credentials Paste these values into the LAP Google Chat flow: * App name * Service account JSON key Click **Save Configuration**. LAP stores the service account JSON in the vault and saves only the vault key name on the agent config. The service account JSON is used to request the `chat.bot` OAuth scope and create or update Google Chat replies. The saved agent config looks like this: ```json theme={null} { "google_chat": { "app_name": "Lite Agent", "status": "connected", "auth_audience": "$LAP_URL/api/agents//google-chat/events", "service_account_json_key": "GOOGLE_CHAT__SERVICE_ACCOUNT_JSON" } } ``` ## 4. Install and test the app Start a direct message with the Google Chat app: ```text theme={null} Can you summarize the latest incident notes? ``` Or mention it in a space: ```text theme={null} @YourAgent check the deployment status and summarize the result ``` The app starts or reuses an agent session, streams the response, and replies in the same Google Chat space or thread. ## Message behavior * Direct messages create or reuse a session for that DM space. * Mentions in a space create or reuse a session for that thread. * Replies inside an existing thread continue the session for that thread. * Unmentioned space messages do not start a new session. * Non-message events, such as card clicks, are ignored by the agent runner. ## Manual API setup The UI handles this setup automatically. If you need to configure an agent through the API, store the service account JSON in the personal vault first: ```bash theme={null} export AGENT_ID="" export SERVICE_ACCOUNT_KEY="GOOGLE_CHAT_${AGENT_ID}_SERVICE_ACCOUNT_JSON" export SERVICE_ACCOUNT_JSON="$(cat google-chat-service-account.json)" curl -X POST "$LAP_URL/api/vault/default" \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d "$(jq -n \ --arg key "$SERVICE_ACCOUNT_KEY" \ --arg value "$SERVICE_ACCOUNT_JSON" \ '{key: $key, value: $value, scope: "personal"}')" ``` Then patch the agent config while preserving any existing config values: ```bash theme={null} export GOOGLE_CHAT_ENDPOINT="$LAP_URL/api/agents/$AGENT_ID/google-chat/events" AGENT_CONFIG="$(curl -s "$LAP_URL/api/agents/$AGENT_ID" \ -H "Authorization: Bearer $MASTER_KEY" | jq '.config')" curl -X PATCH "$LAP_URL/api/agents/$AGENT_ID" \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d "$(jq -n \ --argjson config "$AGENT_CONFIG" \ --arg endpoint "$GOOGLE_CHAT_ENDPOINT" \ --arg key "$SERVICE_ACCOUNT_KEY" \ '{ config: ($config + { google_chat: ({ app_name: "Lite Agent", status: "connected", auth_audience: $endpoint, service_account_json_key: $key }) }) }')" ``` ## Troubleshooting * **No callback reaches LAP:** Confirm the Google Chat app URL uses the public HTTPS LAP URL and ends with `/api/agents//google-chat/events`. * **Unauthorized callback:** Confirm the auth audience saved in LAP matches the app URL exactly. * **Bot cannot reply:** Confirm the service account JSON is valid and belongs to the Google Cloud project that owns the Chat app. * **Bot replies with an agent error:** Confirm the agent runtime and model provider credentials work from the LAP UI before testing Google Chat. ## Routing to different agents Create a separate Google Chat app for each LAP agent you want to expose. Each app points to its own `/api/agents//google-chat/events` endpoint. # Slack Source: https://docs.litellm-agent-platform.ai/channels/slack Talk to your agents directly from Slack. The Slack channel lets your team trigger agent sessions and receive responses without leaving Slack. Each connected LAP agent gets its own dedicated Slack app. ## Prerequisites * LiteLLM Agent Platform deployed and accessible * A LAP agent * A Slack workspace where you can create or install apps ## 1. Open the agent Slack flow In the LAP UI, open **Agents**, choose an agent, then click **Connect to Slack**. The flow opens Slack with a pre-filled manifest. The manifest uses agent-specific callback URLs: ```text theme={null} $LAP_URL/api/agents//slack/events $LAP_URL/api/agents//slack/interactivity $LAP_URL/host-oauth-callback/ ``` ## 2. Create the Slack app Review the pre-filled manifest in Slack, create the app, then return to LAP. The app includes the bot scopes needed for mentions, direct messages, channel messages, and replies. ## 3. Save app credentials Paste these values from Slack's **Basic Information** page back into the LAP flow: * App ID * Client ID * Client Secret * Signing Secret LAP stores the secrets in the vault and saves only key names on the agent config. ## 4. Connect OAuth Click **Connect OAuth**. LAP creates a one-time state value through: ```bash theme={null} curl -X POST $LAP_URL/api/agents//slack/oauth-state \ -H "Authorization: Bearer $MASTER_KEY" ``` Slack redirects back to `$LAP_URL/host-oauth-callback/`. On success, LAP stores the bot token in the vault and marks the agent as connected. ## 5. Talk to your agent **Mention the bot in any channel:** ``` @YourBot summarize the latest PRs in the repo ``` **Or send a direct message:** ``` Hey, can you check the error logs and tell me what's failing? ``` The bot starts a new agent session, streams the response, and posts it back in the thread. ## Routing to different agents Create a separate Slack app for each LAP agent you want to expose. Each app points to its own `/api/agents//slack/...` endpoints. # Microsoft Teams Source: https://docs.litellm-agent-platform.ai/channels/teams Add an agent as a bot in Microsoft Teams. The Teams channel lets people talk to a LAP agent from Microsoft Teams. Each connected LAP agent uses one Bot Framework registration and one Teams app package. ## Prerequisites * LiteLLM Agent Platform deployed at a public HTTPS URL * A LAP agent with a working runtime and model * A Microsoft 365 tenant where you can upload custom Teams apps, or an admin who can publish the app * Permission to create or edit an Azure Bot registration, or an existing Bot Framework bot For local testing, expose LAP with an HTTPS tunnel and use that public URL when you configure the bot endpoint. ## 1. Register the bot Create an Azure Bot or reuse an existing Bot Framework bot. Set the bot messaging endpoint to the agent-specific Teams endpoint: ```text theme={null} $LAP_URL/api/agents//teams/messages ``` For example: ```text theme={null} https://agents.example.com/api/agents/agent_123/teams/messages ``` Enable the **Microsoft Teams** channel for the bot. Keep these values handy: * Microsoft App ID * Tenant ID * App password or client secret value ## 2. Open the agent Teams flow In the LAP UI, open **Agents**, choose an agent, then click **Add to Teams**. The flow shows the same messaging endpoint that the Azure Bot must use: ```text theme={null} $LAP_URL/api/agents//teams/messages ``` ## 3. Save bot credentials Paste these values into the LAP flow: * Microsoft App ID * Tenant ID * App password Click **Save Bot Credentials**. LAP stores the app password in the vault and saves only the vault key name on the agent config. ## 4. Download the Teams package Click **Download Teams Package**. The package contains the Teams app manifest and icons for this agent. The manifest points Teams at the Microsoft App ID you saved in LAP and enables personal, group chat, and team scopes. ## 5. Install the app in Teams Upload the downloaded `.zip` package through one of these paths: * **Teams** -> **Apps** -> **Manage your apps** -> **Upload an app** * Teams Developer Portal * Microsoft Teams admin center, if your organization requires admin publishing If custom app upload is disabled, ask a Teams admin to publish or allow the app for your tenant. ## 6. Talk to your agent Start a direct message with the bot: ```text theme={null} Can you summarize the latest incident notes? ``` Or mention it in a group chat or team channel: ```text theme={null} @YourAgent check the deployment status and summarize the result ``` The bot starts or reuses an agent session for that Teams conversation, streams the response, and replies in the same conversation. Here is an end-to-end direct message test where Teams delivered the prompt to the agent and the bot replied in the same chat: Teams bot replying to an agent prompt ## Troubleshooting * **No callback reaches LAP:** Confirm the Azure Bot messaging endpoint uses the public HTTPS LAP URL and ends with `/api/agents//teams/messages`. * **Teams upload is blocked:** Custom app upload may be disabled. Use Teams admin center publishing. * **Unauthorized callback:** Confirm the Microsoft App ID, tenant ID, and app password saved in LAP match the Azure Bot registration. * **Bot replies with an agent error:** Confirm the agent runtime and model provider credentials work from the LAP UI before testing Teams. ## Routing to different agents Create a separate Teams app package for each LAP agent you want to expose. Each package points to its own `/api/agents//teams/messages` endpoint. # UI Source: https://docs.litellm-agent-platform.ai/channels/ui Talk to your agents through the LiteLLM Agent Platform web interface. The LAP web UI is the fastest way to create, run, and chat with agents. No API calls or CLI required. ## Access Open [http://localhost:4000](http://localhost:4000) (local) or your deployed LAP URL and sign in with your master key. ## Create and run an agent 1. Click **New Agent** in the sidebar 2. Choose a runtime (OpenCode, Cursor, Claude Managed Agents, etc.) 3. Select a model and optionally set a system prompt 4. Click **Run** to start a session The chat panel opens inline — type a message and the agent responds in real time with streamed output. ## Session history All past sessions are listed under your agent. Click any session to replay the conversation or resume it if it's still active. ## CRON schedules Open an agent → **Schedules** → **Add schedule** to run the agent automatically on a cron expression. Each scheduled run creates a new session and appears in the session history. ## Multi-agent view The dashboard lists all agents across all runtimes. Filter by runtime or model using the search bar. # Webhook Source: https://docs.litellm-agent-platform.ai/channels/webhook Trigger agents from systems that can send outbound webhooks. The Webhook channel exposes an agent-specific endpoint that accepts a JSON payload, sends the full payload to the agent as a user message, and queues the run through the normal CMA runtime path. ## Configure in the UI Open **Agents**, find the agent, then click the webhook icon in the row actions. The dialog shows the endpoint and stores the webhook token. ## Endpoint ```text theme={null} POST $LAP_URL/api/agents//webhook ``` Webhook calls do not use the gateway master key. Configure the sender to use bearer token auth: ```text theme={null} Authorization: Bearer ``` By default LAP loads that token from vault key: ```text theme={null} WEBHOOK__SECRET ``` API users can also configure the webhook by updating the agent's `config.webhook` object with `PATCH /api/agents/`: ```json theme={null} { "webhook": { "secret_key": "ZENDESK_WEBHOOK_SECRET" } } ``` ## What the agent receives The webhook body is not sent to the model as an HTTP request. LAP creates a managed-agent session, then sends a normal user message to that session. For example, this payload: ```json theme={null} { "ticket": { "id": "123", "description": "Customer cannot log in", "priority": "high" } } ``` becomes this user message to the agent: ```json theme={null} { "ticket": { "id": "123", "description": "Customer cannot log in", "priority": "high" } } ``` ## Zendesk example Configure the Zendesk webhook to send JSON and use bearer token auth: ```json theme={null} { "ticket": { "id": "{{ticket.id}}", "subject": "{{ticket.title}}", "description": "{{ticket.description}}", "priority": "{{ticket.priority}}" }, "requester": { "email": "{{ticket.requester.email}}" } } ``` After LAP creates the session and sends the payload into it, the endpoint responds with `202 Accepted`: ```json theme={null} { "status": "accepted", "agent_id": "agent_123", "session_id": "session_456", "request_id": "webhook_..." } ``` If the same delivery id is received again, LAP returns `202 Accepted` with `status` set to `duplicate` and does not enqueue another prompt. If a retry arrives while the first delivery is still processing, LAP returns `503 Service Unavailable` with `status` set to `processing` so the sender can retry instead of creating a duplicate run. Use the returned `session_id` with the sessions API or UI to inspect the run. # Contributing Source: https://docs.litellm-agent-platform.ai/engineering/contributing Build, run, test, and extend the gateway locally. # Contributing ## Prerequisites * Rust toolchain — [rustup.rs](https://rustup.rs) * Docker Desktop, or another reachable Postgres database * An Anthropic API key (for end-to-end testing) ## Run the server locally **1. Clone and build** ```bash theme={null} git clone cd litellm-agent-platform cargo build ``` The Rust crate is named `litellm-rust`, and the binary is `lite`. **2. Create a config** ```bash theme={null} cp config.yaml.example config.yaml ``` `config.yaml.example` ships with local provider entries. The `api_key`, `master_key`, and `database_url` fields read from environment variables by default: ```yaml theme={null} model_list: - model_name: claude-sonnet litellm_params: model: anthropic/claude-sonnet-4-5 api_key: os.environ/ANTHROPIC_API_KEY general_settings: master_key: os.environ/LITELLM_MASTER_KEY database_url: os.environ/DATABASE_URL ``` **3. Start Postgres** The compose Postgres service is reachable from other containers as `postgres:5432` and from your host as `127.0.0.1:${POSTGRES_PORT:-15432}`. Start only the database when you want to run the Rust gateway directly: ```bash theme={null} export POSTGRES_PORT=${POSTGRES_PORT:-15432} docker compose up -d --wait postgres export DATABASE_URL=postgres://lap:lap@127.0.0.1:${POSTGRES_PORT}/litellm_agents ``` If that host port is already in use, pick another before starting compose: ```bash theme={null} export POSTGRES_PORT=15433 docker compose up -d --wait postgres export DATABASE_URL=postgres://lap:lap@127.0.0.1:${POSTGRES_PORT}/litellm_agents ``` The gateway runs database migrations on startup. **4. Start the server** ```bash theme={null} export ANTHROPIC_API_KEY=sk-ant-... export LITELLM_MASTER_KEY=sk-mykey cargo run -- --config config.yaml ``` Server listens on `http://localhost:4000` by default. **5. Verify** ```bash theme={null} # Health check curl http://localhost:4000/health # Chat request curl -X POST http://localhost:4000/v1/messages \ -H "Authorization: Bearer $LITELLM_MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hello"}], "max_tokens": 10}' ``` The proxy exposes `/v1/messages` (Anthropic-native protocol). There is no `/v1/chat/completions` (OpenAI-compat) route yet. ## Run tests ```bash theme={null} cargo test ``` Most integration tests in `tests/` spin up a local wiremock server — no real API calls needed. Postgres-backed tests are skipped unless `TEST_DATABASE_URL` is set. To run them against the compose database: ```bash theme={null} export TEST_DATABASE_URL=$DATABASE_URL cargo test --test managed_agents_api ``` ## Add a provider Drop a new folder under `src/sdk/providers/`: ``` src/sdk/providers/openai/ ├── mod.rs └── openai_responses/ ├── mod.rs # pub fn init(registry) { registry.register("openai", ...) } └── transformation.rs # impl Transformation ``` `build.rs` auto-discovers the folder and wires it in. No other files need editing. See `src/sdk/providers/anthropic/anthropic_messages/` for a reference implementation. ## Project layout ``` src/ sdk/ routing.rs # request/model routing above provider endpoint transformation providers/ base/ # endpoint-family base traits + runtime adapter base trait // # provider-owned endpoint/ and runtime/ modules agents/ # Agent Runtime SDK client resources + types proxy/ # config, master-key auth, AppState http/ # axum endpoints + outbound HTTP (http/llm.rs) cli/ # CLI wizard errors.rs # shared GatewayError ``` See [Internal SDK contract](/engineering/sdk-api-contract) for runtime adapter types and event normalization. # Debugging Source: https://docs.litellm-agent-platform.ai/engineering/debugging Debug routing, pricing, and configuration issues. # Debugging ## Enable debug logs Set `RUST_LOG` before starting the proxy: ```bash theme={null} RUST_LOG=litellm_rust=debug ./target/debug/lite --config config.yaml ``` For trace-level output (very verbose): ```bash theme={null} RUST_LOG=litellm_rust=trace ./target/debug/lite --config config.yaml ``` ## Router resolution Every request logs how the incoming model name was resolved to an upstream deployment. **Exact match** — model name matched a named route in `model_list`: ``` router: exact match model="claude" upstream_model="claude-sonnet-4-5" provider="anthropic" ``` **Wildcard match** — no exact match; fell through to the `/*` wildcard route. Provider prefix stripped if present: ``` router: wildcard match — stripped provider prefix model="anthropic/claude-opus-4-8" upstream_model="claude-opus-4-8" provider="anthropic" ``` **No route** — no exact match and no wildcard configured: ``` router: no exact match and no wildcard route model="gpt-4o" ``` Resolution is a single O(1) HashMap lookup — there is no retrying or brute-forcing. ## Model cost map On startup the proxy fetches the LiteLLM model pricing sheet. If the fetch fails it falls back to an embedded backup: ``` WARN Failed to fetch model cost map from https://... — using backup ``` If both fail (parse error, network issue), the proxy continues with an empty cost map and logs: ``` ERROR Failed to parse embedded backup model cost map: ... ``` Pricing data is used for cost tracking only — the proxy still routes requests normally with an empty map. ## Config validation errors Bad config exits immediately with a descriptive message: | Error | Cause | | -------------------------------------------- | ------------------------------------------------- | | `model must include provider prefix` | `litellm_params.model` missing `provider/` prefix | | `model missing name after provider prefix` | `litellm_params.model` set to just `anthropic/` | | `unsupported provider: ` | Provider not registered (e.g. typo) | | `missing litellm_params.api_key` | No API key for that model entry | | `only one wildcard model route is supported` | More than one `/*` entry in `model_list` | # Internal SDK contract Source: https://docs.litellm-agent-platform.ai/engineering/sdk-api-contract The normalized API contract for managed agent runtimes. # Managed Agent Runtime SDK — API Contract The SDK exposes a single Anthropic-shaped surface for every runtime (Claude Managed Agents, Cursor, Gemini Antigravity, and any future runtime). Callers use the same four-step flow regardless of which runtime is active. Provider differences are fully encapsulated inside `sdk/providers//runtime/`. Reference shape: [Anthropic Managed Agents API](https://platform.claude.com/docs/en/api/beta/agents/create) *** ## Public flow ```rust theme={null} let client = Lap::new(LapConfig::anthropic(api_key)); // or ::cursor / ::gemini_antigravity let agent = client.beta().agents().create(params).await?; let agents = client.beta().agents().list(params).await?; let models = client.beta().models().list(params).await?; let env = client.beta().environments().create(params).await?; let session = client.beta().sessions().create(params).await?; // send a prompt client.beta().sessions().events().send(session_id, params).await?; // stream the reply let stream = client.beta().sessions().events().stream(session_id).await?; ``` The flow is identical for every runtime. No `match runtime` or `if runtime == X` blocks appear in calling code. *** ## Input types ### `CreateAgentParams` | Field | Type | Notes | | ------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | `lap_agent_runtime` | `AgentRuntime` | Required. Selects the runtime adapter. | | `name` | `String` | Agent display name. | | `model` | `AgentModel` | `AgentModel::Id("claude-opus-4-8")` or `AgentModel::Config { id, speed }`. | | `system` | `String` | System prompt. Cursor uses this as `prompt.text`; Gemini sends it as `system_instruction`. | | `description` | `Option` | Optional description. | | `tools` | `Vec` | Tool definitions. Cursor ignores this field. Gemini accepts `code_execution`, `google_search`, and `url_context`. | | `mcp_servers` | `Vec` | MCP server configs. Cursor maps these to `mcpServers`. | | `workspace` | `Option` | Repository + PR intent. Cursor maps to `repos`/`autoCreatePR`; Gemini maps the repository to `base_environment.sources`. | | `env_vars` | `Option>` | Runtime environment variables. Reserved for future vault support. | | `metadata` | `Option>` | Stored in provider metadata. Anthropic includes it; Cursor ignores. | ### `AgentWorkspace` | Field | Type | Notes | | ---------------- | ---------------- | -------------------------------------------- | | `repository` | `String` | Repository URL. | | `ref_name` | `Option` | Branch or commit ref. Defaults to `"main"`. | | `auto_create_pr` | `bool` | Whether to open a PR when the run completes. | ### `CreateEnvironmentParams` | Field | Type | Notes | | ------------------- | ---------------- | ------------------------------------------------------------------------------------------ | | `lap_agent_runtime` | `AgentRuntime` | Required. | | `name` | `String` | Environment name. Cursor returns this as the synthetic environment ID. | | `config` | `Value` | Anthropic environment config (`{ "type": "cloud", "networking": {...} }`). Cursor ignores. | | `description` | `Option` | | | `scope` | `Option` | | ### `CreateSessionParams` | Field | Type | Notes | | ------------------- | --------------------------------- | -------------------------------------------------------------- | | `agent` | `String` | Agent ID from `agents.create`. | | `environment_id` | `String` | Environment ID from `environments.create`. | | `title` | `String` | Session title. | | `lap_agent_runtime` | `Option` | Inferred from client config if only one runtime is configured. | | `metadata` | `Option>` | Anthropic stores; Cursor ignores. | | `resources` | `Option` | Reserved. | ### `SendEventsParams` ```json theme={null} { "events": [ { "type": "user.message", "content": [{ "type": "text", "text": "..." }] } ] } ``` ### Agent CRUD Gemini Antigravity additionally supports saved-agent management through the normalized `agents` resource: ```rust theme={null} client.beta().agents().list(ListAgentsParams { ... }).await?; client.beta().agents().get(GetAgentParams { ... }).await?; client.beta().agents().delete(DeleteAgentParams { ... }).await?; ``` *** ## Output types ### `ManagedAgent` Anthropic-shaped. All fields extracted from the raw response where available. | Field | Type | Notes | | ------------- | ---------------- | ------------------------------------------------ | | `id` | `String` | Provider agent ID. Always present. | | `version` | `Option` | Anthropic only. | | `name` | `Option` | | | `description` | `Option` | | | `model` | `Option` | Model ID string. | | `system` | `Option` | | | `tools` | `Vec` | | | `mcp_servers` | `Vec` | | | `metadata` | `Option` | | | `created_at` | `Option` | Unix timestamp. | | `updated_at` | `Option` | Unix timestamp. | | `raw` | `Value` | Full unmodified provider response. Escape hatch. | ### `Environment` | Field | Type | Notes | | ----- | -------- | ------------------------------------------------------------ | | `id` | `String` | Provider environment ID. Always present. | | `raw` | `Value` | Full provider response. Cursor returns `{ "id": "" }`. | ### `Session` Anthropic-shaped. | Field | Type | Notes | | ---------------- | ---------------- | ------------------------------------ | | `id` | `String` | Provider session ID. Always present. | | `agent` | `Option` | Agent ID. | | `environment_id` | `Option` | Environment ID. | | `status` | `Option` | Session status string. | | `metadata` | `Option` | | | `created_at` | `Option` | Unix timestamp. | | `updated_at` | `Option` | Unix timestamp. | | `raw` | `Value` | Full provider response. | ### `AgentEvent` Anthropic-shaped. Cursor events are normalized to this shape by the cursor adapter before the stream is returned to the caller. | Field | Type | Notes | | ------------ | -------------------- | ------------------------------------------------------------------------------ | | `event_type` | `String` | Anthropic event type string (e.g. `"agent.message"`, `"session.status_idle"`). | | `data` | `Map` | Event payload. | Use `event.kind()` → `AgentEventKind` and `event.payload()` → `AgentEventPayload` for typed access. Key event types: | Event | Meaning | | ------------------------ | -------------------------------------------------------------- | | `session.status_running` | Session is processing. | | `agent.message` | Assistant reply. `data.content` is an array of content blocks. | | `agent.tool_use` | Agent called a tool. | | `agent.tool_result` | Tool returned a result. | | `session.status_idle` | Session is done. Terminal event — stop reading the stream. | *** ## Provider normalization Each runtime maps its native response shape to the Anthropic-shaped outputs above. Callers never see provider-specific fields unless they inspect `.raw`. | Concern | Anthropic | Cursor | Gemini Antigravity | | ------------------------- | ---------------------------------------- | ---------------------------------------------- | ---------------------------------------- | | `agents.create` HTTP path | `POST /v1/agents` | `POST /v1/agents` | `POST /v1beta/agents` | | Agent CRUD | Create only | Create only | Create, list, get, delete | | `environments.create` | `POST /v1/environments` | Synthetic (`{ id: name }`) | Synthetic (`remote` or supplied ID) | | `sessions.create` | `POST /v1/sessions` | Synthetic (uses agent ID) | Synthetic conversation handle | | `send_events` | `POST /v1/sessions/{id}/events` | `POST /v1/agents/{id}/runs` | `POST /v1beta/interactions` | | `stream_events` | `GET /v1/sessions/{id}/events/stream` | `GET /v1/agents/{id}/runs/{run_id}/stream` | `GET /v1beta/interactions/{id}` | | Event normalization | Native Anthropic SSE | Cursor events -> Anthropic shape | Interaction steps -> Anthropic shape | | Initial run on create | No — send first prompt via `send_events` | Yes — `agents.create` starts a run immediately | No — send first prompt via `send_events` | *** ## Adapter contract Each runtime implements `RuntimeAdapter` in `sdk/providers//runtime/mod.rs`. Required methods: ```rust theme={null} fn configure_request(&self, request: RequestBuilder, api_key: &str) -> RequestBuilder; fn create_agent<'a>(&'a self, client: &'a Lap, params: CreateAgentParams) -> AdapterFuture<'a, ManagedAgent>; fn create_environment<'a>(...) -> AdapterFuture<'a, Environment>; fn create_session<'a>(...) -> AdapterFuture<'a, Session>; fn send_events<'a>(...) -> AdapterFuture<'a, SendEventsResponse>; fn stream_events<'a>(...) -> AdapterFuture<'a, AgentEventStream>; ``` Optional overrides (default: `None`): ```rust theme={null} fn normalize_stream(&self, stream: AgentEventStream) -> AgentEventStream; fn provider_run_id_from_agent_raw(&self, raw: &Value) -> Option; fn provider_url_from_agent_raw(&self, raw: &Value) -> Option; fn provider_agent_id_from_session_id(&self, session_id: &str) -> Option; ``` To add a new runtime: create `sdk/providers//runtime/mod.rs`, implement `RuntimeAdapter`, add `RUNTIME_ID`/`RUNTIME_NAME`/`DEFAULT_API_BASE` constants, and call `registry.register(...)` in the provider's `mod.rs`. No other files change. # Introduction Source: https://docs.litellm-agent-platform.ai/introduction 1 place to call all your agents - OpenCode, OpenClaw, Hermes, Claude Managed Agents, Cursor Agents API, Deep Agents. [![Discord](https://img.shields.io/badge/Discord-Chat-5865F2?logo=discord\&logoColor=white)](https://discord.gg/Nkxw3rm3EE) ![LiteLLM Agent Platform dashboard](https://github.com/user-attachments/assets/04333758-829c-4b19-bde3-23ade37bb9f1) LiteLLM Agent Platform sits on top of any runtime. Pick a runtime, create an agent, give your team one UI. It manages: * **Unified API across runtimes** - one API to create and run agents, regardless of the runtime underneath * **Access** - developers create and run agents here, no Bedrock or Anthropic console access required * **Session management** - persistent agent sessions across runs * **CRON schedules** - run agents on a schedule * **Memory** - agents remember context across sessions Run LAP locally with Docker in under a minute. Claude Managed Agents, Cursor, OpenCode, OpenClaw, Deep Agents, Elastic. Talk to agents via UI, API, Slack, Teams, or Google Chat. Full REST API reference with live try-it panel. # Architecture Source: https://docs.litellm-agent-platform.ai/learn/architecture LiteLLM Agent Platform is one service plus Postgres. LiteLLM Agent Platform is intentionally small: * One Docker image runs the `lite` service. * One Postgres database stores persistent state. * Everything else is either an upstream API call or an optional runtime profile. The `lite` service serves the dashboard, REST API, LiteLLM-compatible gateway routes, MCP proxy routes, Slack callbacks, and runtime event streams from the same process. ## What Runs | Process | Required | What it does | | ----------------- | -------: | ---------------------------------------------------------------------------------------- | | `lite` | Yes | Serves the UI, API, gateway, MCP proxy, Slack callbacks, and session streams | | Postgres | Yes | Stores agents, sessions, credentials, runtime harnesses, schedules, memory, and settings | | Template runtimes | Optional | Local OpenCode, OpenClaw, Deep Agents, or Hermes services for development and demos | The LiteLLM gateway is not a separate deployment. It is part of the same `lite` service. ## Docker Shape The repository has one main `Dockerfile`. It builds the static UI, builds the Rust `lite` binary, and packages both into one runtime image: ```text theme={null} browser/API/Slack -> lite container -> Postgres | +-> model providers, MCP servers, or agent runtimes ``` For local development, `compose.yaml` starts: * `lap`, built from the root `Dockerfile` * `postgres` Optional Compose profiles add local runtime services: ```bash theme={null} docker compose --profile opencode up docker compose --profile deepagents up docker compose --profile hermes up docker compose --profile openclaw up ``` Those profiles are conveniences. They are not required to run LAP. ## Request Flow 1. A user opens the dashboard, calls the REST API, or talks to a connected Slack app. 2. The request reaches the `lite` service. 3. LAP reads or writes state in Postgres. 4. If the request needs a model, MCP server, or agent runtime, LAP calls that upstream service. 5. Runtime events stream back through the same `lite` service to the UI or API client. ## Local stack The smallest local stack is: ```bash theme={null} docker compose up ``` That starts LAP and Postgres. Add a runtime profile only when you want a local template runtime registered automatically. ## Implementation details For the normalized runtime SDK contract, see [Internal SDK contract](/engineering/sdk-api-contract). # Authentication Source: https://docs.litellm-agent-platform.ai/learn/auth How the LiteLLM gateway authenticates incoming requests. Every request to the LiteLLM gateway is authenticated against a single **master key** before routing. Auth runs as the first step of every API endpoint. ## Configure the master key Set `master_key` under `general_settings` in `config.yaml`: ```yaml theme={null} general_settings: master_key: os.environ/LITELLM_MASTER_KEY ``` The value is read from the `LITELLM_MASTER_KEY` environment variable at boot. For local Docker Compose, the default master key is `sk-local`. Change it by setting `LITELLM_MASTER_KEY` in your `.env` file. ## Supported header formats The gateway accepts the master key in two header styles: | Priority | Header | Format | | -------- | --------------- | ------------------ | | 1 | `Authorization` | `Bearer ` | | 2 | `x-api-key` | raw key, no prefix | ## Response codes | Case | Status | | ---------------------------------------- | -------------------------------- | | Key matches | Request proceeds | | Key wrong or missing (master key is set) | `401 Unauthorized` | | No master key configured | Request proceeds (auth disabled) | A `401` returns: ```json theme={null} { "error": { "type": "gateway_error", "message": "unauthorized" } } ``` ## Key separation The master key authenticates **callers to the gateway**. It is separate from the provider API keys the gateway uses to call upstream LLMs. Provider keys are stored encrypted in the credentials vault and never exposed to callers. ## Per-user keys (teams) For team access, issue per-user virtual keys instead of sharing the master key. In **Settings → Keys**, click **New Key** and set budget limits, model restrictions, and expiry. ```bash theme={null} curl -X POST $LAP_URL/api/keys \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "label": "alice" }' ``` Team members use their virtual key as `Authorization: Bearer ` — they never see the master key or any provider credential. # MCP Support Source: https://docs.litellm-agent-platform.ai/learn/mcp Use the LiteLLM gateway as an MCP proxy to inject upstream auth into tool calls. The LiteLLM gateway acts as an **MCP (Model Context Protocol) proxy** — it forwards requests from AI clients to MCP servers and injects upstream credentials at the wire level. Agents never hold real MCP server credentials. ## Configuration MCP servers are declared in `config.yaml` using the same dict-keyed format as LiteLLM: ```yaml theme={null} mcp_servers: github: url: https://api.github.com/mcp auth_type: bearer_token auth_value: os.environ/GITHUB_TOKEN postgres: url: http://localhost:5432/mcp auth_type: api_key auth_value: os.environ/POSTGRES_MCP_KEY public_tool: url: https://example.com/mcp auth_type: none ``` ## Endpoints ### Route to a specific server ``` /mcp/{server_id} ``` ```bash theme={null} curl -X POST $LAP_URL/mcp/github \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{"method":"tools/list"}' ``` ### Auto-select server ``` /mcp ``` The gateway selects the server using (in priority order): 1. `x-litellm-mcp-server: ` header 2. `?server=` query parameter 3. First configured server (if only one) ```bash theme={null} curl -X POST "$LAP_URL/mcp?server=github" \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{"method":"tools/call","params":{"name":"search_repos","arguments":{"q":"litellm"}}}' ``` All MCP endpoints require `Authorization: Bearer `. ## Supported auth types | Type | Description | | --------------- | ------------------------------------------- | | `none` | No auth injected | | `api_key` | Adds `x-api-key: ` header | | `bearer_token` | Adds `Authorization: Bearer ` header | | `basic` | Adds `Authorization: Basic ` header | | `authorization` | Adds a raw `Authorization: ` header | | `token` | Adds `Token ` header | ## Referencing secrets Auth values starting with `os.environ/` are read from environment variables at boot: ```yaml theme={null} auth_value: os.environ/MY_MCP_TOKEN ``` This keeps credentials out of `config.yaml` and version control. ## Current limitations These features from LiteLLM's hosted MCP are not yet supported: * Multi-server aggregation behind a single `/mcp` endpoint * Per-request upstream credentials via `x-mcp-{server}-{header}` * `oauth2` and `aws_sigv4` auth types * `sse` and `stdio` transports (HTTP only) # Quick Start Source: https://docs.litellm-agent-platform.ai/quickstart Run LiteLLM Agent Platform locally in under a minute. ## Prerequisites Docker Desktop installed and running. ## Start the stack ```bash theme={null} docker compose --profile opencode up ``` Open [http://localhost:4000](http://localhost:4000) and sign in with the master key (`sk-local` by default). Compose starts the LiteLLM Agent Platform web/API service, a Postgres database, the OpenCode template runtime, and registers `local-opencode` in the UI automatically. To start only the base LAP stack (no runtime): ```bash theme={null} docker compose up ``` To start other runtimes and register them automatically: ```bash theme={null} docker compose --profile deepagents up docker compose --profile hermes up docker compose --profile openclaw up docker compose --profile opencode --profile deepagents up ``` Add provider credentials in **Settings** before running agents against a hosted model provider. ## Create an agent ### 1. Make an agent in the UI ![Create agent screen](https://github.com/user-attachments/assets/d2083454-b7c1-4337-b2c2-4c4ba99991b6) ### 2. Select tools and skills ![Select tools and skills](https://github.com/user-attachments/assets/efd59a4e-dcc7-487a-923b-005ac44b44b0) ### 3. Run your agent Select your agent and the runtime you want to run it on. ![Run agent on a runtime](https://github.com/user-attachments/assets/be9cfd8c-4475-4309-bed0-4edcd7dd1de1) ## Next steps Connect Claude Managed Agents, Gemini Antigravity, Cursor, OpenCode, Deep Agents, Pydantic Deep Agents, OpenClaw, or Elastic. Register the Pydantic Deep Agents bridge as an Anthropic Managed Agents-compatible runtime. Run OpenClaw locally with Docker Compose or register a hosted bridge. Talk to your agents via UI, API, Slack, Teams, or Google Chat. Use the REST API to create and run agents programmatically. Issue API keys and manage access for your team. # Claude Managed Agents Source: https://docs.litellm-agent-platform.ai/runtimes/claude-managed-agents Run agents using Anthropic's Claude Managed Agents runtime. Claude Managed Agents is Anthropic's hosted agent runtime — the platform proxies agent creation, session management, and event streaming through your LiteLLM gateway so your team never touches Anthropic credentials directly. ## Prerequisites * LiteLLM Agent Platform running (`docker compose up` or deployed) * Anthropic API key ## 1. Add your Anthropic API key Open **Settings → Credentials** in the UI and add your Anthropic API key. The platform stores it encrypted and injects it at the wire level. Or via the API: ```bash theme={null} curl -X PUT $LAP_URL/api/agent-runtimes/claude_managed_agents/credentials \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "api_key": "", "api_base": "https://api.anthropic.com" }' ``` ## 2. Select the built-in runtime `claude_managed_agents` is a built-in runtime. You do not need to register a custom harness for it. ## 3. Create an agent In the UI, click **New Agent**, choose `claude_managed_agents` as the runtime, select a model, and save. Or via the API: ```bash theme={null} curl -X POST $LAP_URL/api/agents \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-claude-agent", "owner_id": "local-user", "runtime": "claude_managed_agents", "model": "claude-opus-4-5", "system": "You are a helpful assistant." }' ``` ## 4. Start a session and stream events ```bash theme={null} SESSION=$(curl -s -X POST $LAP_URL/session \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "runtime": "claude_managed_agents", "agent_id": "", "prompt": "Summarize the latest changes in this repo." }' | jq -r .id) curl -N "$LAP_URL/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer $MASTER_KEY" ``` Events follow the Anthropic Managed Agents SSE shape: ``` data: {"type":"session.status_running"} data: {"type":"agent.message","content":"Here are the latest changes..."} data: {"type":"session.status_idle"} ``` See [Internal SDK contract](/engineering/sdk-api-contract) for the Rust SDK types. ## Event kinds | Event kind | Meaning | | ------------------------ | -------------------------- | | `session.status_running` | Agent is processing | | `agent.message` | Text output from the agent | | `agent.tool_use` | Agent is calling a tool | | `agent.tool_result` | Tool result returned | | `session.status_idle` | Turn complete | | `session.error` | Runtime error | # Cursor Agents API Source: https://docs.litellm-agent-platform.ai/runtimes/cursor Run agents using the Cursor Agents runtime. The Cursor runtime lets you create and run Cursor background agents through the LiteLLM Agent Platform. Sessions support repository and pull-request context. The platform normalizes Cursor's event stream to the standard Anthropic Managed Agents shape, so calling code is runtime-agnostic. ## Prerequisites * LiteLLM Agent Platform running * Cursor API key (from your Cursor account settings) ## 1. Add your Cursor API key Open **Settings → Credentials** in the UI and add your Cursor API key. Or via the API: ```bash theme={null} curl -X PUT $LAP_URL/api/agent-runtimes/cursor/credentials \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "api_key": "", "api_base": "https://api.cursor.com" }' ``` ## 2. Select the built-in runtime `cursor` is a built-in runtime. You do not need to register a custom runtime harness unless you are proxying through your own Cursor-compatible service. ## 3. Create an agent In the UI, click **New Agent**, choose `cursor` as the runtime, and save. Or via the API: ```bash theme={null} curl -X POST $LAP_URL/api/agents \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-cursor-agent", "owner_id": "local-user", "runtime": "cursor", "model": "claude-opus-4-5", "system": "You are a coding assistant." }' ``` ## 4. Start a session with repo context Cursor sessions accept `environment` fields for repository and PR settings: ```bash theme={null} SESSION=$(curl -s -X POST $LAP_URL/session \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "runtime": "cursor", "agent_id": "", "prompt": "Fix the failing tests in src/utils.ts", "environment": { "repository": "https://github.com/your-org/your-repo", "ref": "main", "auto_create_pr": false } }' | jq -r .id) ``` ## 5. Stream events ```bash theme={null} curl -N "$LAP_URL/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer $MASTER_KEY" ``` Cursor's native events (`assistant`, `tool_call`, `status`, `result`) are normalized to the Anthropic event surface: | Cursor event | Normalized to | | ------------------ | ------------------------ | | `status` (running) | `session.status_running` | | `assistant` | `agent.message` | | `tool_call` | `agent.tool_use` | | `result` | `agent.tool_result` | | `status` (idle) | `session.status_idle` | See [Internal SDK contract](/engineering/sdk-api-contract) for the Rust SDK types. Cursor sessions can start immediately when you include `prompt` in the `POST /session` body. # Deep Agents Source: https://docs.litellm-agent-platform.ai/runtimes/deepagents Run agents using the Deep Agents runtime via Docker Compose. [Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview) is a multi-step agent runtime with deep reasoning and planning capabilities. The Docker Compose profile starts Deep Agents alongside the LiteLLM Agent Platform and registers `local-deepagents` automatically. This page covers the LangChain Deep Agents Compose runtime. For the Pydantic Deep Agents bridge template, see [Pydantic Deep Agents](/runtimes/pydantic-deepagents). ## Prerequisites * Docker Desktop installed and running * LiteLLM Agent Platform repo cloned ## 1. Start the stack ```bash theme={null} docker compose --profile deepagents up ``` This starts: * The LiteLLM Agent Platform web/API service * A Postgres database * The Deep Agents runtime harness * Registers `local-deepagents` in the UI automatically Open [http://localhost:4000](http://localhost:4000) and sign in with your master key (`sk-local` by default). ## 2. Add model provider credentials In **Settings → Credentials**, add a model provider API key. Deep Agents routes all model calls through your LiteLLM gateway. ## 3. Create an agent In the UI, click **New Agent**, choose `local-deepagents` as the runtime, select a model, and set a system prompt describing the agent's role. Or via the API: ```bash theme={null} curl -X POST http://localhost:4000/api/agents \ -H "Authorization: Bearer sk-local" \ -H "Content-Type: application/json" \ -d '{ "name": "my-deepagent", "owner_id": "local-user", "runtime": "local-deepagents", "model": "claude-opus-4-5", "system": "You are an autonomous research agent. Plan carefully before acting." }' ``` ## 4. Run your agent Select your agent in the UI and start a session. Or via the API: ```bash theme={null} # Start and run a session SESSION=$(curl -s -X POST http://localhost:4000/session \ -H "Authorization: Bearer sk-local" \ -H "Content-Type: application/json" \ -d '{ "runtime": "local-deepagents", "agent_id": "", "prompt": "Research the top 5 open-source vector databases and compare them on latency and scalability." }' | jq -r .id) # Stream the response curl -N "http://localhost:4000/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer sk-local" ``` ## Stop the stack ```bash theme={null} docker compose --profile deepagents down ``` ## Run alongside other runtimes ```bash theme={null} docker compose --profile deepagents --profile opencode up ``` ## CRON schedules Deep Agents works well with scheduled runs. In the UI, open your agent, go to **Schedules**, and add a CRON expression: ``` 0 9 * * 1-5 ``` This runs the agent every weekday at 9 AM and creates a new session automatically. # Elastic Agent Builder Source: https://docs.litellm-agent-platform.ai/runtimes/elastic-runtime Run LAP sessions against an existing Elastic Agent Builder agent. The Elastic Agent Builder runtime connects LAP to an existing Elastic Agent Builder agent. LAP stores a local agent record, binds it to an Elastic agent ID, and sends prompts through the Kibana Agent Builder API. ## How it works Unlike the Docker Compose runtimes (`opencode`, `deepagents`, `hermes`), the Elastic runtime does not start local containers. It: * Uses the built-in `elastic_agent_builder` runtime * Requires an existing Elastic Agent Builder agent ID * Sends turns to Elastic's `/api/agent_builder/converse/async` endpoint * Can optionally target an Elastic space or connector ## Prerequisites * LiteLLM Agent Platform running * Elastic API key * Kibana base URL * Existing Elastic Agent Builder agent ID ## 1. Add Elastic runtime credentials Open **Agent Runtimes**, expand **Elastic Agent Builder**, paste your Elastic API key and Kibana base URL, then click **Connect**. Elastic Agent Builder runtime credential form Or via the API: ```bash theme={null} curl -X PUT $LAP_URL/api/agent-runtimes/elastic_agent_builder/credentials \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "api_key": "", "api_base": "https://your-kibana.example.com" }' ``` ## 2. Create an agent In the UI, create a new agent and choose the **Elastic Agent Builder** runtime once the runtime credentials are connected. The current UI handles the LAP agent fields. The Elastic-specific binding still lives in the agent `config`, so use the API when you need to bind an existing Elastic Agent Builder agent ID: ```bash theme={null} curl -X POST $LAP_URL/api/agents \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-elastic-agent", "owner_id": "local-user", "runtime": "elastic_agent_builder", "model": "claude-opus-4-5", "system": "You are a helpful assistant.", "config": { "elastic_agent_id": "", "elastic_space_id": "default" } }' ``` ## 3. Start a session ```bash theme={null} SESSION=$(curl -s -X POST $LAP_URL/session \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "runtime": "elastic_agent_builder", "agent_id": "", "prompt": "Analyze the error logs and suggest fixes." }' | jq -r .id) ``` ## 4. Stream events ```bash theme={null} curl -N "$LAP_URL/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer $MASTER_KEY" ``` ## Memory across sessions Store provider-specific binding options in the agent `config`. You can patch them later: ```bash theme={null} curl -X PATCH $LAP_URL/api/agents/ \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "config": { "elastic_agent_id": "", "elastic_space_id": "default", "elastic_connector_id": "" } }' ``` ## Comparison: Elastic vs Docker Compose runtimes | | Elastic | OpenCode / Deep Agents / Hermes | | -------------- | ---------------------------------------------------- | ---------------------------------------------------- | | **Runtime ID** | `elastic_agent_builder` | `local-opencode`, `local-deepagents`, `local-hermes` | | **Backend** | Existing Elastic Agent Builder agent | Local template service | | **Setup** | Save Elastic credentials and bind `elastic_agent_id` | `docker compose --profile up` | | **Best for** | Existing Elastic Agent Builder workflows | Local dev, demos | # Gemini Antigravity Source: https://docs.litellm-agent-platform.ai/runtimes/gemini-antigravity Run agents using the Gemini Antigravity runtime. The Gemini Antigravity runtime lets LAP create and run Gemini-backed managed agents through the same session API used by the other runtimes. LAP normalizes Gemini interaction events to the shared runtime event stream. ## Prerequisites * LiteLLM Agent Platform running * Gemini API key ## 1. Add your Gemini API key Open **Settings → Credentials** in the UI and add your Gemini API key. Or via the API: ```bash theme={null} curl -X PUT $LAP_URL/api/agent-runtimes/gemini_antigravity/credentials \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "api_key": "", "api_base": "https://generativelanguage.googleapis.com" }' ``` ## 2. Select the built-in runtime `gemini_antigravity` is a built-in runtime. You do not need to register a custom runtime harness unless you are proxying through your own Gemini-compatible service. ## 3. Create an agent In the UI, click **New Agent**, choose `gemini_antigravity` as the runtime, and save. Or via the API: ```bash theme={null} curl -X POST $LAP_URL/api/agents \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-gemini-agent", "owner_id": "local-user", "runtime": "gemini_antigravity", "model": "antigravity-preview-05-2026", "system": "You are a coding assistant.", "tools": [ { "type": "code_execution" }, { "type": "google_search" }, { "type": "url_context" } ] }' ``` ## 4. Start a session with repo context Gemini sessions accept `environment` fields for repository context: ```bash theme={null} SESSION=$(curl -s -X POST $LAP_URL/session \ -H "Authorization: Bearer $MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "runtime": "gemini_antigravity", "agent_id": "", "prompt": "Review this repository and summarize the main risks.", "environment": { "repository": "https://github.com/your-org/your-repo", "ref": "main" } }' | jq -r .id) ``` ## 5. Stream events ```bash theme={null} curl -N "$LAP_URL/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer $MASTER_KEY" ``` Gemini interaction events are normalized to the shared runtime event surface: | Gemini event | Normalized to | | -------------- | ------------------------ | | Running status | `session.status_running` | | Assistant text | `agent.message` | | Tool use | `agent.tool_use` | | Tool result | `agent.tool_result` | | Completion | `session.status_idle` | See [Internal SDK contract](/engineering/sdk-api-contract) for the Rust SDK types. # OpenClaw Source: https://docs.litellm-agent-platform.ai/runtimes/openclaw Run agents using OpenClaw locally or as a hosted runtime. OpenClaw is an open-source agent runtime with gateway, browser, and memory tooling. The Docker Compose profile starts OpenClaw alongside the LiteLLM Agent Platform and registers `local-openclaw` automatically. ## Prerequisites * Docker Desktop installed and running * LiteLLM Agent Platform repo cloned ## 1. Start the stack ```bash theme={null} docker compose --profile openclaw up ``` This starts: * The LiteLLM Agent Platform web/API service * A Postgres database * The OpenClaw runtime bridge * An OpenClaw Gateway configured to call the LAP gateway for model inference * Registers `local-openclaw` in the UI automatically Open [http://localhost:4000](http://localhost:4000) and sign in with your master key (`sk-local` by default). ## 2. Add model provider credentials In **Settings → Credentials**, add a model provider API key. OpenClaw routes model calls through your LiteLLM gateway, so provider keys stay in LAP. ## 3. Create an agent In the UI, click **New Agent**, choose `local-openclaw` as the runtime, select a model, and optionally set a system prompt. Or via the API: ```bash theme={null} curl -X POST http://localhost:4000/api/agents \ -H "Authorization: Bearer sk-local" \ -H "Content-Type: application/json" \ -d '{ "name": "my-openclaw-agent", "owner_id": "local-user", "runtime": "local-openclaw", "model": "claude-sonnet-4-6", "system": "You are concise. Reply plainly." }' ``` ## 4. Run your agent Select your agent in the UI and start a session. The bridge exposes OpenClaw through the same Anthropic Managed Agents-compatible event stream as the other template runtimes. Or via the API: ```bash theme={null} # Start and run a session SESSION=$(curl -s -X POST http://localhost:4000/session \ -H "Authorization: Bearer sk-local" \ -H "Content-Type: application/json" \ -d '{ "runtime": "local-openclaw", "agent_id": "", "prompt": "Check browser and memory readiness, then reply in one sentence." }' | jq -r .id) # Stream the response curl -N "http://localhost:4000/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer sk-local" ``` ## Stop the stack ```bash theme={null} docker compose --profile openclaw down ``` ## Hosted or custom bridge Use the same OpenClaw bridge when OpenClaw runs outside the local Compose stack, for example on Render or another container host. Deploy `templates/openclaw` as a web service and point it at your LiteLLM Agent Platform gateway: ```bash theme={null} LITELLM_BASE_URL=https://lap.example.com/v1 LITELLM_API_KEY=sk-lap-api-key RUNTIME_API_KEY=sk-openclaw-runtime-key OPENCLAW_GATEWAY_TOKEN=sk-openclaw-gateway-token OPENCLAW_AGENT_MODEL=litellm/claude-sonnet-4-6 ``` Attach persistent storage for `/data` if you want bridge agent and session state to survive restarts. Then register the bridge as a custom runtime harness. Use the bridge root URL as `api_base`, not its `/v1` OpenClaw Gateway URL: ```bash theme={null} curl -X POST "$LAP_URL/api/runtime-harnesses" \ -H "Authorization: Bearer $LAP_MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "alias": "render-openclaw", "api_spec": "claude_managed_agents", "api_base": "https://.onrender.com", "api_key": "" }' ``` The `alias` becomes the runtime ID for agents and sessions. Create agents with `"runtime": "render-openclaw"` and LAP will drive OpenClaw through the existing Claude Managed Agents-compatible event stream. ## Configuration OpenClaw runtime configuration lives in the `compose.yaml` `openclaw` service block and `templates/openclaw`. By default, the template: * Installs Chromium and starts OpenClaw browser tooling during container boot * Uses FTS-only memory search so no embedding API key is required * Seeds a small runtime memory file and indexes it on boot * Sends OpenClaw model calls to the LAP gateway through the generated `litellm` provider config Override the default backend model with `OPENCLAW_AGENT_MODEL`, and override the registered model list with `OPENCLAW_MODELS`. # OpenCode Source: https://docs.litellm-agent-platform.ai/runtimes/opencode Run agents using the OpenCode runtime via Docker Compose. OpenCode is an open-source coding agent runtime. The Docker Compose profile starts OpenCode alongside the LiteLLM Agent Platform and registers `local-opencode` automatically. ## Prerequisites * Docker Desktop installed and running * LiteLLM Agent Platform repo cloned ## 1. Start the stack ```bash theme={null} docker compose --profile opencode up ``` This starts: * The LiteLLM Agent Platform web/API service * A Postgres database * The OpenCode runtime harness * Registers `local-opencode` in the UI automatically Open [http://localhost:4000](http://localhost:4000) and sign in with your master key (`sk-local` by default). ## 2. Add model provider credentials Before running an agent, add at least one model provider key in **Settings → Credentials** (e.g. an Anthropic or OpenAI API key). OpenCode routes model calls through your LiteLLM gateway. ## 3. Create an agent In the UI, click **New Agent**, choose `local-opencode` as the runtime, select a model, and optionally set a system prompt. Or via the API: ```bash theme={null} curl -X POST http://localhost:4000/api/agents \ -H "Authorization: Bearer sk-local" \ -H "Content-Type: application/json" \ -d '{ "name": "my-opencode-agent", "owner_id": "local-user", "runtime": "local-opencode", "model": "claude-opus-4-5", "system": "You are a senior software engineer." }' ``` ## 4. Run your agent Select your agent in the UI and click **Run**. Type a message in the chat panel and OpenCode will start executing. Or via the API: ```bash theme={null} # Start and run a session SESSION=$(curl -s -X POST http://localhost:4000/session \ -H "Authorization: Bearer sk-local" \ -H "Content-Type: application/json" \ -d '{ "runtime": "local-opencode", "agent_id": "", "prompt": "Add type annotations to all functions in src/" }' | jq -r .id) # Stream the response curl -N "http://localhost:4000/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer sk-local" ``` ## Stop the stack ```bash theme={null} docker compose --profile opencode down ``` ## Run alongside other runtimes You can start multiple runtimes at once: ```bash theme={null} docker compose --profile opencode --profile deepagents up ``` This registers both `local-opencode` and `local-deepagents` in the UI. ## Configuration OpenCode runtime configuration lives in the `compose.yaml` `opencode` service block. You can override the default model and tool settings via environment variables in that block. ## Sandboxed execution with OpenSandbox By default the OpenCode harness runs agent commands directly on the host container. **OpenSandbox** routes every command and file operation into an isolated container sandbox instead, so the agent cannot touch host state. When OpenSandbox is configured: * Native bash and file-edit operations are denied at the harness level. * A `sandbox-exec` MCP server is injected into OpenCode's tool config automatically. * Each session creates a fresh sandbox, executes all commands there, then terminates it. ### How it works ``` opencode → sandbox-exec MCP server │ ├─ POST /v1/sandboxes (create) ├─ GET /v1/sandboxes/{id}/endpoints/44772 (resolve execd URL) ├─ POST {execd}/command (run command, SSE stream) ├─ GET {execd}/files/download (read file) ├─ POST {execd}/files/upload (write file) └─ DELETE /v1/sandboxes/{id} (terminate) ``` The harness talks to the **OpenSandbox controller** (sandbox lifecycle) and **execd** (command execution inside the sandbox) over plain HTTP. No SDK dependency — Node 20 built-ins only. ### Environment variables | Variable | Required | Description | | --------------------- | ----------------- | -------------------------------------------------------------------------------------------------------- | | `OPENSANDBOX_API_URL` | Yes | OpenSandbox controller base URL (e.g. `http://opensandbox-server.opensandbox-system.svc.cluster.local`) | | `OPENSANDBOX_API_KEY` | When auth enabled | API key sent as `OPEN-SANDBOX-API-KEY` header | | `OPENSANDBOX_IMAGE` | Yes | execd container image (e.g. `sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/execd:v1.0.18`) | | `SANDBOX_PROVIDER` | No | Defaults to `opensandbox`. Only `opensandbox` is supported. | Set any of these on the `opencode` service and the harness enables sandbox mode automatically. If `OPENSANDBOX_API_URL` is unset, sandbox mode is skipped and commands run on the host as normal. ### Local Docker Compose For local testing, point the opencode service at an external OpenSandbox instance by adding env vars to `compose.yaml`: ```yaml theme={null} opencode: environment: OPENSANDBOX_API_URL: http://host.docker.internal:8090 OPENSANDBOX_API_KEY: my-local-key OPENSANDBOX_IMAGE: sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/execd:v1.0.18 ``` Then run as normal: ```bash theme={null} docker compose --profile opencode up ``` ### Production deployment on EKS For production, deploy the full OpenSandbox Kubernetes operator alongside the opencode-anthropic-server. The stack runs: * **OpenSandbox controller** — manages sandbox lifecycle via `BatchSandbox` CRDs * **OpenSandbox server** — HTTP API gateway the harness calls * **opencode-anthropic-server** — the OpenCode harness wired to OpenSandbox Key resources: * Kubernetes namespace: `opensandbox-system` * Helm charts: `opensandbox-controller`, `opensandbox-server` * Sandbox images: `opensandbox/execd:v1.0.18`, `opensandbox/egress:v1.0.12` * Default sandbox resource limits: `1 CPU`, `2 GiB RAM` See the [EKS deployment guide](https://github.com/LiteLLM-Labs/litellm-agent-platform-2/blob/main/templates/opencode/docs/eks-deployment.md) for the full step-by-step: cluster setup, EBS CSI driver, OpenSandbox Helm install, ECR push, and Kubernetes manifests. Quick-start: paste the agent prompt from that guide into any coding agent and it will walk through every step, asking for credentials as needed. # Pydantic Deep Agents Source: https://docs.litellm-agent-platform.ai/runtimes/pydantic-deepagents Run Pydantic Deep Agents as a custom Anthropic Managed Agents-compatible runtime. [Pydantic Deep Agents](https://github.com/vstorm-co/pydantic-deepagents) is an open-source deep-agent framework built on top of Pydantic AI by [Vstorm](https://vstorm.co), with planning, subagents, persistent memory, sandboxed execution, live-run forking, and cost control. This runtime exposes it to LiteLLM Agent Platform through a small bridge server that speaks the Anthropic Managed Agents API, so agents and sessions use the same event stream shape as Claude Managed Agents. This page covers the Pydantic Deep Agents bridge template. For the LangChain Deep Agents Compose runtime, see [Deep Agents](/runtimes/deepagents). Pydantic Deep Agents is an independent open-source project built and maintained by [Vstorm](https://vstorm.co), contributors to and partners across the broader [Pydantic AI](https://ai.pydantic.dev) ecosystem. It is built on top of Pydantic AI but is not an official Pydantic library. The bridge stores agents, environments, sessions, and event history in SQLite. Each session gets its own Pydantic Deep `LocalBackend` workspace under `PYDANTIC_DEEP_WORKDIR_ROOT`. ## Prerequisites * Python 3.12 or newer * LiteLLM Agent Platform running locally or deployed * A model provider key, or a LiteLLM gateway the bridge can call ## 1. Start the bridge ```bash theme={null} cd templates/pydantic-deepagents python3 -m venv .venv . .venv/bin/activate pip install -r requirements.txt ANTHROPIC_API_KEY=sk-ant-... \ RUNTIME_API_KEY=local-runtime-key \ DB_PATH=/tmp/pydantic-deepagents.db \ PYDANTIC_DEEP_WORKDIR_ROOT=/tmp/pydantic-deepagents-workspaces \ PORT=8080 \ uvicorn src.server:app --host 0.0.0.0 --port 8080 ``` Run the smoke test against the bridge: ```bash theme={null} BASE=http://localhost:8080 \ RUNTIME_API_KEY=local-runtime-key \ MODEL=anthropic:claude-sonnet-4-6 \ ./scripts/smoke.sh ``` ## 2. Route through LiteLLM For an OpenAI-compatible LiteLLM gateway, set: ```bash theme={null} LITELLM_BASE_URL=http://localhost:4000 LITELLM_API_KEY=sk-... LITELLM_MODELS=claude-sonnet-4-6,gpt-4.1 DEFAULT_MODEL=claude-sonnet-4-6 ``` When `LITELLM_BASE_URL` is set in OpenAI mode, bare model names are normalized to `openai:` for Pydantic AI, and `/v1/models` proxies LiteLLM model discovery with a local fallback. For an Anthropic Messages-compatible gateway, set: ```bash theme={null} LITELLM_BASE_URL=https://litellm-rust.onrender.com LITELLM_API_KEY=sk-... LITELLM_API_FORMAT=anthropic DEFAULT_MODEL=claude-sonnet-4-6 ``` In Anthropic gateway mode, bare model names are normalized to `anthropic:`. The bridge passes the gateway root URL to Pydantic AI's `AnthropicModel`, and the Anthropic SDK appends `/v1/messages` when sending requests. ## 3. Register the custom runtime Register the running bridge in LiteLLM Agent Platform: ```bash theme={null} curl -X POST "$LAP_URL/api/runtime-harnesses" \ -H "Authorization: Bearer $LAP_MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "alias": "pydantic-deepagents", "api_spec": "claude_managed_agents", "api_base": "http://localhost:8080", "api_key": "local-runtime-key" }' ``` The `alias` becomes the runtime ID for agents and sessions. The `api_spec` tells LiteLLM Agent Platform to drive the bridge through the existing Claude Managed Agents protocol. ## 4. Create an agent In the UI, click **New Agent**, choose `pydantic-deepagents` as the runtime, select a model, and set a system prompt. Or via the API: ```bash theme={null} curl -X POST "$LAP_URL/api/agents" \ -H "Authorization: Bearer $LAP_MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-pydantic-deepagent", "owner_id": "local-user", "runtime": "pydantic-deepagents", "model": "claude-sonnet-4-6", "system": "You are an autonomous research agent. Plan carefully before acting." }' ``` ## 5. Start a session and stream events ```bash theme={null} SESSION=$(curl -s -X POST "$LAP_URL/session" \ -H "Authorization: Bearer $LAP_MASTER_KEY" \ -H "Content-Type: application/json" \ -d '{ "runtime": "pydantic-deepagents", "agent_id": "", "prompt": "Inspect this repository and summarize the runtime template layout." }' | jq -r .id) curl -N "$LAP_URL/v1/sessions/$SESSION/events/stream" \ -H "Authorization: Bearer $LAP_MASTER_KEY" ``` The stream emits Anthropic Managed Agents-compatible event frames: ```text theme={null} data: {"type":"session.status_running"} data: {"type":"agent.message","content":"..."} data: {"type":"agent.tool_use","tool_name":"..."} data: {"type":"agent.tool_result","tool_name":"..."} data: {"type":"session.status_idle"} ``` ## MCP verification Use the deterministic Pydantic AI `test` model to verify the bridge and MCP plumbing without a provider API key: ```bash theme={null} cd templates/pydantic-deepagents . .venv/bin/activate RUNTIME_API_KEY=local-runtime-key \ DB_PATH=/tmp/pydantic-deepagents-test.db \ PYDANTIC_DEEP_WORKDIR_ROOT=/tmp/pydantic-deepagents-test-workspaces \ PYDANTIC_DEEP_TODO=false \ PYDANTIC_DEEP_FILESYSTEM=false \ PYDANTIC_DEEP_SUBAGENTS=false \ PYDANTIC_DEEP_SKILLS=false \ PYDANTIC_DEEP_MEMORY=false \ PYDANTIC_DEEP_WEB_SEARCH=false \ PYDANTIC_DEEP_WEB_FETCH=false \ PYDANTIC_DEEP_CONTEXT_MANAGER=false \ PYDANTIC_DEEP_COST_TRACKING=false \ uvicorn src.server:app --host 0.0.0.0 --port 8080 ``` Then run the DeepWiki MCP smoke: ```bash theme={null} BASE=http://localhost:8080 \ RUNTIME_API_KEY=local-runtime-key \ MODEL=test \ ./scripts/mcp-smoke.sh ``` The smoke creates an agent with `https://mcp.deepwiki.com/mcp`, sends a managed-agent event, and asserts the SSE stream contains `agent.tool_use`, `agent.tool_result`, `agent.message`, and `session.status_idle`. ## Configuration | Variable | Default | Purpose | | --------------------------------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | `PORT` | `8080` | HTTP port | | `DB_PATH` | `/data/agents.db` | SQLite persistence path | | `PYDANTIC_DEEP_WORKDIR_ROOT` | `/data/workspaces` | Root directory for per-session workspaces | | `DEFAULT_MODEL` | `anthropic:claude-sonnet-4-6` | Model used when agent creation omits one | | `ANTHROPIC_API_KEY` | none | Direct Anthropic provider key | | `OPENAI_API_KEY` | none | Direct OpenAI provider key | | `OPENAI_BASE_URL` | none | OpenAI-compatible base URL | | `LITELLM_BASE_URL` | none | LiteLLM gateway base URL for model discovery and model calls | | `LITELLM_API_KEY` | none | LiteLLM gateway API key | | `LITELLM_API_FORMAT` | `openai` | Gateway API shape: `openai` for `/v1/chat/completions`, `anthropic` for `/v1/messages` | | `LITELLM_MODELS` | none | Comma-separated fallback model IDs for `/v1/models` | | `RUNTIME_API_KEY` | none | Runtime key expected from LAP via `x-api-key`; when unset, any non-empty key is accepted for local development | | `PYDANTIC_DEEP_EXECUTE` | `true` | Enables the Pydantic Deep execute tool | | `PYDANTIC_DEEP_TODO` | `true` | Enables todo tools | | `PYDANTIC_DEEP_FILESYSTEM` | `true` | Enables filesystem tools | | `PYDANTIC_DEEP_SUBAGENTS` | `true` | Enables subagent tools | | `PYDANTIC_DEEP_SKILLS` | `true` | Enables skill tools | | `PYDANTIC_DEEP_BUILTIN_SUBAGENTS` | `true` | Enables built-in subagent tools | | `PYDANTIC_DEEP_PLAN` | `true` | Enables planning tools | | `PYDANTIC_DEEP_MEMORY` | `true` | Enables persistent memory tools | | `PYDANTIC_DEEP_WEB_SEARCH` | `true` | Enables web search tools | | `PYDANTIC_DEEP_WEB_FETCH` | `true` | Enables web fetch tools | | `PYDANTIC_DEEP_THINKING` | `false` in Anthropic gateway mode, otherwise `high` | Enables Pydantic Deep thinking effort; use `false`, `true`, or an effort like `low`, `medium`, `high` | | `PYDANTIC_DEEP_CONTEXT_MANAGER` | `true` | Enables context management | | `PYDANTIC_DEEP_COST_TRACKING` | `true` | Enables cost tracking | | `PYDANTIC_DEEP_FORKING` | `false` | Enables live run forking tools | | `PYDANTIC_DEEP_CHECKPOINTS` | `false` | Enables checkpoint tools | | `PYDANTIC_DEEP_TEAMS` | `false` | Enables team tools | | `PYDANTIC_DEEP_LITEPARSE` | `false` | Enables LiteParse document tools | Do not put model provider keys in browser-visible LAP config. Provider keys belong in the bridge environment or in the LiteLLM gateway the bridge calls.