Skip to main content

External Agents & Applications

This guide is for teams running an AI application or agent outside the Beak AI Runtime — a custom Python or TypeScript app, a LangChain / LangGraph agent, a third-party SaaS, an internal tool — that wants to use Beak as its governance, security, and observability layer for LLM traffic.

The integration surface is intentionally small. In the default path, an external app only needs to point its LLM client at the Beak Gateway and pass a virtual key. Moderation against the team's policy and tracing into Observability happen automatically as part of the Gateway pipeline — no extra calls required.

Architecture​

┌──────────────────────┐      Authorization: Bearer sk-...       ┌────────────────────────┐
│  External Agent or   │  ─────────────────────────────────────▶  │  Beak AI Gateway       │
│  Application         │                                          │  (governance,          │
│                      │  ◀────────────────  response  ─────────  │  routing, security)    │
└──────────────────────┘                                          └──┬─────────────────────┘
                                                                     │
                                  ┌──────────────────────────────────┤
                                  │  inline (when team.enable_security)
                                  ▼
                           ┌─────────────────────┐
                           │  Beak AI Security   │   POST /aisecurity/inline/scan
                           │  (moderation)       │
                           └─────────────────────┘

                                  ┌──────────────────────────────────┐
                                  │  fire-and-forget (when team.enable_observability)
                                  ▼
                           ┌─────────────────────┐
                           │  Beak AI            │   POST /aiobservability/ingest/traces
                           │  Observability      │   POST /aiobservability/ingest/spans
                           └─────────────────────┘

Every external request enters through the Gateway. When an LLM call is made, the Gateway:

  • Validates the virtual key, budget, model allowlist, and rate limits.
  • If the team has Security enabled, calls the assigned policy through /aisecurity/inline/scan and blocks the request on a deny verdict.
  • Routes the request to the chosen provider.
  • Emits a trace and a span to AI Observability when the team has Observability enabled.

External apps never need to call the Security or Observability services directly for the standard LLM-call flow. Direct calls are only needed for the advanced patterns described in AI Security & Observability — Direct API.

Personas​

Pick the integration shape that matches your application.

PersonaDescriptionWhat you need
LLM client onlyYour code calls an LLM. You want governance (budget, rate limit, model allowlist), automatic moderation, and automatic tracing.Virtual key. Point any OpenAI-compatible client (OpenAI SDK, LangChain, custom code) at the Beak AI Gateway. Done.
Agent with own loopYou orchestrate multi-step reasoning, tool calls, and sub-agents in your own code. You want every step in a single trace owned by your application.Virtual key + own trace IDs. Send x-beak-trace-id headers on LLM calls so the Gateway stitches its spans into your trace. Emit your own non-LLM spans directly to /aiobservability/ingest/spans.
Non-LLM moderationYou need to scan content (tool inputs, RAG documents, file uploads) for policy compliance even when no LLM is being called.Virtual key or service JWT + direct calls to /aisecurity/inline/scan.
Self-service adminYour platform manages teams, virtual keys, models, or budgets on behalf of users.Service JWT + admin endpoints under /aigateway/*.

Base URL & Authentication​

Beak exposes one ingress per environment. Sub-paths route to the right backend service.

CapabilityPath prefixAuth
LLM calls (OpenAI-compatible)/chat/completionsVirtual key — Authorization: Bearer sk-<vkey>
Gateway admin (teams, keys, models, spend, alerts)/aigateway/*JWT or API-Key
AI Security (moderation, agents, policies, audit)/aisecurity/*JWT or API-Key
AI Observability (trace ingest, traces, evaluations)/aiobservability/*JWT or API-Key

Note: Virtual keys (sk-...) authenticate only the LLM endpoint. Admin, Security, and Observability APIs require a JWT or API-Key issued through Beak IAM. The two credential types are not interchangeable.

Virtual keys​

Issued in the Gateway UI under AI Gateway > Virtual Keys (see Virtual Keys). Each key inherits its team's budget, rate limits, model allowlist, and Security/Observability toggles. The key value is shown once on creation — store it in a secret manager.

JWT / API-Key​

Issued through Beak IAM. The Gateway's auth middleware accepts the token in either header:

API-Key: <token>
Authorization: Bearer <token>

After validation the Gateway injects tenantid, userid, and roleid headers downstream — your code never sets these.

Environment endpoints​

EnvironmentBase URL
Local devhttp://localhost:5001 (API Gateway), http://localhost:5060 (Gateway Manager direct)
Devhttps://<tenant>-dev.beakops.com/api
QAhttps://<tenant>-qa.beakops.com/api
Productionhttps://<tenant>.beakops.com/api

Replace <tenant> with your tenant slug. The exact production hostname is provisioned per tenant — confirm with your Beak account team.

When to use direct Security / Observability calls​

The default /chat/completions path covers most use cases. Reach for direct calls only when:

  • You own the trace. Your app is the parent of a multi-step agent run and you want LLM spans nested under your trace, not the Gateway's. Use the x-beak-trace-id header pattern.
  • You moderate non-LLM content. Tool arguments, RAG snippets, file contents, or user uploads need policy scanning even when no LLM call is involved.
  • You ingest spans from non-LLM work. Tool calls, retrievals, custom evaluations — anything you want visible in Observability but isn't an LLM call.

Walk-throughs for each are in AI Security & Observability — Direct API.

Best Practices​

  • Issue one virtual key per environment per consumer. A single key shared across staging and production collapses audit trails and makes rotation risky.
  • Set the team's Security and Observability toggles in the Gateway UI before pointing production traffic at the key. The toggles are per-team, not per-key, so changes apply to every key on the team immediately.
  • Treat the virtual key value as a secret. It only appears once at creation; rotate by issuing a new key, deploying it, then deactivating the old one.
  • Resist the urge to call /aisecurity/inline/scan directly when an LLM call is already going through the Gateway — moderation is already happening inline and a second call double-charges the audit log.
  • For multi-tenant SaaS apps integrating Beak on behalf of their own customers, scope each downstream tenant to a separate Beak team so spend, rate limits, and policy isolation are enforced per customer.