AI Gateway Integration

The Beak AI Gateway exposes an OpenAI-compatible /chat/completions endpoint, and under /v1 the OpenAI Responses API and the Anthropic Messages API as well (see Other dialects). External applications point any OpenAI-compatible client at the Gateway, pass a virtual key as the bearer token, and receive a response from whichever underlying provider the team's routing rules selected — without the application needing to know which provider was used.

This page covers the request and response contract, authentication, error handling, streaming, and example clients in Python and TypeScript.

Endpoint

POST {base_url}/chat/completions

Where {base_url} is the environment-specific URL listed in External Agents — Overview.

The path is OpenAI-compatible. The Gateway accepts the same request body as OpenAI's /v1/chat/completions and returns the same response shape.

Authentication

Pass the virtual key as a bearer token:

Authorization: Bearer sk-<vkey>

The Gateway also accepts the key in the x-api-key header, but Authorization: Bearer is preferred because every OpenAI-compatible client sets it automatically.

The Gateway resolves the key, applies the team's budget, rate limit, model allowlist, Security policy, and Observability toggle, then routes the request. None of these are passed by the client — they are stored on the team behind the key.

Request

{
  "model": "gpt-4o-mini",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Summarize the changes in last week's commits."}
  ],
  "temperature": 0.2,
  "max_tokens": 500,
  "stream": false
}

Fields

model — The model display name as registered in the Gateway under Models. Not the underlying provider model id. Must be in the team's allowlist.
messages — Standard OpenAI role / content array. Roles: system, user, assistant, tool.
temperature, top_p, max_tokens, n, stop, presence_penalty, frequency_penalty — Forwarded to the provider as-is.
stream — true for SSE streaming (see Streaming).
tools, tool_choice — OpenAI tool-calling format. Forwarded to providers that support it.
response_format — Forwarded for providers that support structured outputs.

Optional headers for trace stitching

When the team has Observability enabled, the Gateway emits a trace and a span automatically. If your application owns a parent trace, supply these headers so the LLM call appears nested under it:

Header	Description
`x-beak-trace-id`	Your application's trace id. The Gateway uses it as the trace id for this call and does not emit a parent trace row — it emits only spans.
`x-beak-span-id`	The span id for this LLM call. Optional — auto-generated when omitted.
`x-beak-parent-span-id`	The id of the parent span in your trace. Used to position this span in the tree.
`x-beak-agent-id`	Your agent's slug. Surfaces in the Observability Agents view.
`x-beak-agent-name`	Human-readable agent label.
`x-beak-user-id`	The end-user id this call is on behalf of.
`x-beak-session-id`	Conversation or session id for grouping.

Without these headers the Gateway mints its own trace, and the LLM call appears as a one-span trace in Observability.

Response

A non-streaming, successful response is the standard OpenAI completion shape:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1730800000,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "..."},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 42,
    "completion_tokens": 128,
    "total_tokens": 170
  }
}

usage drives spend tracking — every successful response is charged against the key, team, and project budgets in the background.

Error Contract

HTTP	Cause	Response
`401`	Missing or invalid virtual key	`{"detail": "Missing API key"}` or `{"detail": "Invalid API key"}`
`401`	Budget exceeded (key, team, or project)	`{"detail": "<level> budget exceeded: spend=$X.XX, limit=$Y.YY"}`
`403`	Model not in the team's allowlist	`{"detail": "Model '<name>' not allowed for team"}`
`403`	Moderation blocked the request	Beak envelope: `{"module_code": "SECURITY_ERR002", "message": "Request blocked by Beak AI Security. Action: <action>, Risk Score: <score>/100. Triggered by: <validators>", "data": {"action", "legacy_action", "risk_score", "violations", "event_id"}}`
`400`	Model exists but is deactivated	`{"detail": "Model '<name>' is not currently active"}`
`429`	RPM rate limit hit	`{"detail": "RPM limit exceeded: <n>/<limit>"}`
`429`	Parallel request limit hit	`{"detail": "Parallel request limit exceeded"}`
`500`	Provider error or internal failure	`{"detail": "Internal error: <reason>"}`

Note: Moderation rejections return a Beak response envelope, not the OpenAI error format. The data.event_id field links to the entry in the Audit Log for forensic review. Treat 403 with module_code starting SECURITY_ as a policy decision, not a transient error — retrying with the same content will be blocked again.

Streaming

Set stream: true to receive an SSE response:

data: {"id":"...","choices":[{"delta":{"content":"Hello"}}]}

data: {"id":"...","choices":[{"delta":{"content":" world"}}]}

data: [DONE]

The final chunk is the literal string data: [DONE]. Streaming follows the OpenAI SSE format byte-for-byte; any OpenAI-compatible streaming client works without changes.

Spend and token counts are buffered at the end of the stream from the provider's usage field on the final chunk. If the provider does not include usage in stream chunks, spend tracking falls back to a best-effort cost estimate from the prompt and completion token counts.

Other dialects — `/v1`

The Gateway also serves the two other request shapes agent frameworks default to, so an agent reaches Beak without being reconfigured away from its own SDK's defaults. Every dialect runs the same pipeline — key, budget, model access, rate limits, moderation, routing, spend and the completion audit are identical — and only the request and response framing differ.

Path	Dialect	Who uses it
`POST {base_url}/v1/responses`	OpenAI Responses API, non-streaming and streaming	OpenAI Agents SDK (its default API), any Responses client
`POST {base_url}/v1/messages`	Anthropic Messages API, non-streaming and streaming	Claude Code, the Claude Agent SDK, Anthropic SDKs
`GET {base_url}/v1/models`	Model discovery, in a shape both OpenAI and Anthropic clients read	Claude Code gateway discovery, OpenAI SDK `models.list()`
`POST {base_url}/v1/chat/completions`	Alias of `/chat/completions`	Clients whose `base_url` ends in `/v1`

Authentication is the same virtual key as /chat/completions, as Authorization: Bearer (Anthropic clients send it as ANTHROPIC_AUTH_TOKEN). Errors come back in the dialect's own envelope — an OpenAI-style {"error": {...}} on /v1/responses, an Anthropic-style {"type": "error", "error": {...}} on /v1/messages — with the same status codes as the native route; a moderation block is a 403 in either.

OpenAI Agents SDK — point the SDK at the Gateway and leave its default API alone:

from agents import set_default_openai_client
from openai import AsyncOpenAI

set_default_openai_client(AsyncOpenAI(
    base_url="https://<tenant>.beakops.com/api/v1",
    api_key="<virtual-key>",
))

Claude Code / Claude Agent SDK — set the gateway environment (or the managed settings equivalent) on the machine running the agent:
```
ANTHROPIC_BASE_URL=https://<tenant>.beakops.com/api
ANTHROPIC_AUTH_TOKEN=<virtual-key>
```
Claude Code discovers the models the key may use from /v1/models.

Note: Request fields the translation cannot express in Chat Completions are dropped rather than forwarded blind — the Gateway never sends a provider a parameter it did not understand. Streaming on /v1/responses emits Responses events (response.output_text.delta, response.completed); streaming on /v1/messages emits Anthropic events (message_start … message_stop). Spend is settled from the final usage of the stream, as on the native route.

Examples

Python — OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="https://<tenant>.beakops.com/api",
    api_key="sk-<your-virtual-key>",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={
        "x-beak-trace-id": "trace-abc123",
        "x-beak-agent-id": "support-agent-v2",
    },
)
print(response.choices[0].message.content)

Python — LangChain

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    base_url="https://<tenant>.beakops.com/api",
    api_key="sk-<your-virtual-key>",
    default_headers={"x-beak-agent-id": "support-agent-v2"},
)

result = llm.invoke("Hello")

TypeScript — OpenAI SDK

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<tenant>.beakops.com/api",
  apiKey: process.env.BEAK_VIRTUAL_KEY,
  defaultHeaders: {
    "x-beak-agent-id": "support-agent-v2",
  },
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello" }],
});

cURL

curl -X POST "https://<tenant>.beakops.com/api/chat/completions" \
  -H "Authorization: Bearer sk-<your-virtual-key>" \
  -H "Content-Type: application/json" \
  -H "x-beak-agent-id: support-agent-v2" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Webhooks for Budget & Rate-Limit Events

Configure webhooks under AI Gateway > Alerting to receive notifications when a budget or rate-limit event fires. Useful for routing into Slack, PagerDuty, or a custom handler.

Event	Triggered when
`Budget Exceeded`	A team or virtual key crosses its soft or hard budget
`Rate Limited`	A team or virtual key hits its TPM or RPM limit
`Error`	An upstream provider returns an error

Each webhook posts a JSON payload describing the entity, the limit, and the observed value. See Alerting.

Best Practices

Set the OpenAI client's base_url once at process startup. Do not reconstruct it per request — the OpenAI SDK pools connections and the per-request overhead is wasted.
Always set x-beak-agent-id if your team runs more than one application against the same key. The header is what makes the per-agent rollups in Observability meaningful.
Treat 403 SECURITY_ERR002 as a final verdict. Do not retry — the policy will block the same content again, and each retry creates an audit log entry.
Treat 429 as a backoff signal. The rate limiter resets every 60 seconds; an exponential backoff of 1–2s is usually enough.
Surface the event_id from a SECURITY_ERR002 response in your application's error UI — your security team uses it to look up the offending interaction in the Forensic Explorer.
For agents with their own multi-step reasoning, send x-beak-trace-id and emit your own non-LLM spans directly to /aiobservability/ingest/spans so Observability shows the full trace, not just the LLM calls. See AI Security & Observability — Direct API.

Endpoint​

Authentication​

Request​

Fields​

Optional headers for trace stitching​

Response​

Error Contract​

Streaming​

Other dialects — /v1​

Examples​

Python — OpenAI SDK​

Python — LangChain​

TypeScript — OpenAI SDK​

cURL​

Webhooks for Budget & Rate-Limit Events​

Best Practices​

Related Pages​