Skip to main content

Tenant Data Isolation

AI Observability is moving from one shared database to a database per tenant. This page explains both models, what changes for you when a deployment is cut over, and the OBS_TENANT_DB_MODE setting that controls the move.

If you are only using SaaS and have no self-hosted deployment, the short version is: nothing changes for you until Beak completes the cutover in your environment, and after it completes the user picker in Access Control lists only your own users.

The two models​

Shared database (the original model). Every tenant's traces, spans, roles, and users live in one beakai database. Rows are separated by a tenant_id column, and every query filters on it.

Per-tenant database (the target model). Each tenant's data lives in that tenant's own database, resolved from the platform tenant registry. The tenant_id columns remain, but they are now a consistency check rather than the thing keeping tenants apart.

The second model is the Beak AI platform standard — AI Studio, Agent Runtime, AI Security, Scheduler, and Gateway Manager already work this way. Per-tenant databases are a stronger boundary because separation no longer depends on every query remembering to filter: another tenant's rows are simply not present in the database being queried.

Configuration​

OBS_TENANT_DB_MODE controls which model a deployment uses. It is set per environment.

ValueReads fromWrites toPurpose
offshared beakaishared beakaiThe original model. Default.
readtenant databaseshared beakaiVerify reads against the migrated data while writes stay put, so rollback is a single setting change.
dualtenant databasebothSoak period. No telemetry is lost and rollback remains trivial.
fulltenant databasetenant databaseThe target model.

Related settings, which rarely need changing:

  • DB_TENANT_REGISTRY — the registry database used to resolve a tenant to its own database. Default tenant.
  • DB_TENANT_MIN_POOL_SIZE / DB_TENANT_MAX_POOL_SIZE — connection-pool bounds per tenant database. Defaults 1 and 10. The floor is deliberately low because each replica opens its own pool per active tenant; raise the ceiling only after checking the database's max_connections.

An unrecognised value for OBS_TENANT_DB_MODE stops the service at startup rather than silently falling back to off — a typo cannot be mistaken for a completed cutover.

Access Control and roles are not partially migrated​

The read and dual modes split reads from writes for telemetry only — traces, spans, evaluator scores, and alert events. User accounts and role assignments always read and write in the same database, and move to the tenant database as soon as the mode leaves off.

This matters for rollback: role grants and first-login provisioning that happen during a soak exist only in the tenant database. Returning to off does not lose user accounts — the next sign-in re-provisions them — but explicit role grants made during the soak need re-granting. Plan soaks accordingly, or keep them short.

Migrating an existing deployment​

Existing data must be copied into the tenant databases before the mode is changed. Reading from an empty tenant database produces empty dashboards, not an error.

The service ships a backfill utility for this:

# Report what would be copied, without writing
python scripts/backfill_tenant_dbs.py --dry-run

# Copy every tenant
python scripts/backfill_tenant_dbs.py

# Copy a single tenant
python scripts/backfill_tenant_dbs.py --tenant 20

# Compare source and target row counts
python scripts/backfill_tenant_dbs.py --verify

Properties worth knowing:

  • Non-destructive. It only inserts into tenant databases. The shared database is never modified, so it remains a complete fallback.
  • Re-runnable. Every insert skips rows that already exist, so running it twice is safe — and running it again after a soak picks up whatever was written to the shared database in the meantime.
  • Data only. It never creates tables. The tenant schemas, including the four time-series tables, are provisioned during tenant onboarding.
  • Reports what it will not move. User accounts that hold no role in any tenant cannot be attributed to one, so they stay in the shared database and are listed by name. These are typically stale accounts worth reviewing.

The recommended order per environment is: run the backfill, set read and confirm dashboards match, set dual for a business day, then set full. Environments are cut over one at a time.

What you will notice after the cutover​

  • The user picker in Access Control lists only your tenant's users. Under the shared model the account directory had no tenant column, so the picker listed every user on the platform. Assigning a role was always correctly scoped to your tenant — this affected only who was listed, never who had access — but the list itself is now scoped too.
  • Dashboards, trace search, and analytics are unchanged. Same data, same numbers, read from a different database.
  • Anomaly alert configurations, RCA reports, and evaluation configurations now save. A defect fixed alongside this work pointed those features at the wrong database schema, so creating or listing them failed. If those pages looked permanently empty or errored, that is why; they work now.