Skip to main content

Microsoft Copilot Studio

Copilot Studio lets an organization nominate an external threat detection provider. Once configured, every time a generative agent's orchestrator decides to invoke a tool, Copilot Studio sends the proposed call to Beak AI Security and holds the tool until Beak answers. If Beak blocks it, the agent stops and tells the user their message was blocked.

This is the only Beak control that sees a tool call with its resolved arguments, before the tool runs. Prompt-level scanning sees what a user typed; this sees what the agent is about to do — the address an email is about to be BCC'd to, the path a file tool was told to read, the URL a request tool was told to POST to.

Note: External threat detection is a Microsoft preview feature. Microsoft states preview features are not meant for production use and may change. Treat this integration as a pilot until Microsoft ships it generally.

For the step-by-step setup — the Beak side, the Entra application and Power Platform, in the order they have to be done — see Configuring Copilot Studio protection.

What Beak sees, and what it does not​

SeesThe user's message and recent chat history, the planner's stated reason for choosing this tool, outputs of previously executed tools, the tool's definition, and the resolved argument values
Does not seeConversations that invoke no tool. This hook fires on tool invocation only — prompt-level coverage on Microsoft surfaces stays with Beak's inline scan
Not called at all forClassic agents. Copilot Studio skips external threat detection unless the agent uses generative orchestration
Cannot doRedaction. The protocol's answer is a boolean — a tool call can be blocked, never sanitised

Beak evaluates all of the above against the policies you already use elsewhere in AI Security. No new validators or policy types are involved.

Before you start​

You need:

  • A Power Platform Administrator to configure the connection.

  • Permission to register an application in your Microsoft Entra tenant.

  • The endpoint URL for your Beak environment, from your Beak contact. Use it exactly as given — the path is part of how the request is routed, so an abbreviated version reaches the wrong service and returns a web page instead of a decision. It looks like:

    https://aisecurity.beakops.com/api/aisecurity/agentsecurity/t42-AbCdEfGhIjKlMnOpQrStUv

Note: Register in Beak first. Do the Beak side before you press Save in Power Platform admin center — the Save calls Beak's /validate endpoint and fails if your Entra application is not yet registered, and the error Microsoft shows ("There was a problem connecting to the protection provider") does not say that is the reason.

Step 1 — Register with Beak​

Give your Beak administrator two values from your Entra tenant:

ValueWhere to find it
Directory (tenant) IDEntra ID → Overview
Application (client) IDThe app registration you create in Step 2

Beak registers this pair as the only caller allowed to use your endpoint. A token from any other tenant or application is rejected, even against the correct URL.

Your Beak administrator configures the rest in the same screen — which policy evaluates these tool calls, whether Beak enforces or only observes, what a redaction policy should do here, what happens if Beak cannot answer in time, and optionally which Power Platform environments are in scope. Each is described in Configuring Copilot Studio protection, and all of them remain changeable afterwards without affecting your endpoint URL.

Step 2 — Configure a Microsoft Entra application​

Copilot Studio authenticates using Federated Identity Credentials, which are secret-less: you register an application, Copilot Studio federates into it, and no shared secret is ever created or stored. There is nothing to rotate.

Microsoft provides a script that does the whole step, and it is the recommended path because it computes the encoded values for you:

.\Create-CopilotWebhookApp.ps1 `
  -TenantId "11111111-2222-3333-4444-555555555555" `
  -Endpoint "https://aisecurity.beakops.com/api/aisecurity/agentsecurity/t42-AbCdEfGhIjKlMnOpQrStUv" `
  -DisplayName "Beak AI Security - Production" `
  -FICName "BeakProductionFIC"

Download it from the PowerShell Gallery. Note the App ID it prints — Step 3 needs it, and so does Beak (Step 1).

Doing it by hand​

If you register the application manually, the federated credential's Issuer and Explicit subject identifier must match exactly. Ask your Beak administrator for both strings — Beak computes them from your tenant ID and your endpoint URL, which avoids the encoding mistakes that account for most failed setups:

  • Issuer — https://login.microsoftonline.com/{your tenant ID}/v2.0
  • Type — Explicit subject identifier
  • Value — /eid1/c/pub/t/{encoded tenant ID}/a/{app segment}/{encoded endpoint URL}

The two encoded segments are base64url without padding, and the tenant ID is encoded from its .NET byte order, not the order the GUID is written in. That detail is why hand-encoding usually produces a subject that looks right and never matches.

Step 3 — Configure threat detection in Power Platform​

  1. Sign in to the Power Platform admin center.
  2. Go to Security → Threat detection → Additional threat detection.
  3. Select the environment you want protected, then Set up.
  4. Tick Allow Copilot Studio to share data with a threat detection provider.
  5. Under Azure Entra App ID, enter the App ID from Step 2.
  6. Under Endpoint link, enter the Beak endpoint URL — byte-for-byte the same string you used in Step 2.
  7. Under Set error behavior, choose what happens when Beak does not answer in time. See below.
  8. Save.

Set error behavior — and tell Beak what you chose​

Microsoft's default is Allow the agent to respond: if Beak does not answer within one second, the tool runs. The alternative is Block the query.

Beak cannot read this setting, so tell your Beak administrator which one you picked. Beak has a matching timeout_action that should agree with it: if it does not, a moment where Beak is slow produces one answer while your Power Platform setting expects the other.

Note: This setting decides what happens when the control is unavailable. Leaving it on Allow the agent to respond means a Beak outage silently returns your agents to unprotected operation. Choosing Block the query means a Beak outage stops your agents from using tools. Neither is wrong — but it is a decision about which failure you prefer, and it should be a deliberate one.

Configuration is per environment​

There is no tenant-wide switch. Every Power Platform environment must be configured individually, and a newly created environment is not protected until someone turns this on for it. Microsoft provides no API, PowerShell cmdlet or admin-center setting to enforce it globally.

Put a step in whatever process creates environments in your organization. Otherwise coverage quietly decays as the estate grows.

Start in observe mode​

Copilot Studio has no shadow mode of its own, so Beak provides one, and it is on by default: Beak evaluates every tool call in full and records what it would have blocked, while answering "allow".

Use it. The blast radius of a false positive on this hook is not a warning banner — it is an agent that refuses to complete a task the user asked for. Observe mode lets you see that against real traffic first.

The recommended sequence:

  1. Observe — Beak records would-be blocks. Nothing is blocked; users notice nothing. Review the Audit Log and tune the policy.
  2. Enforce, error behavior = Allow — Beak now blocks. A Beak outage degrades to unprotected rather than broken.
  3. Error behavior = Block — only once you are satisfied with both the block rate and Beak's response times.

Ask your Beak administrator to move between stages; each is a per-tenant setting, changed in the Beak panel described in Configuring Copilot Studio protection.

Decisions your administrator makes​

SettingDefaultWhat it means
EnforceOff (observe)Whether a computed block is actually applied
Redaction behaviorBlockWhat happens when a policy says "allow this with sensitive values removed". This protocol cannot rewrite a tool argument, so there is no way to honour that here. Blocking is the default because allowing would send the tool call with the sensitive value intact while the policy still reported as working. Can be set to allow, as a deliberate choice
Timeout behaviorAllowWhat Beak answers if evaluation approaches its internal deadline. Should match your Set error behavior choice
PolicyAll active in-scope policiesWhich policy evaluates these tool calls
EnvironmentsAllOptionally narrow to specific Power Platform environment IDs. Tool calls from other environments are allowed without evaluation, and recorded as out of scope

What a blocked tool call looks like​

The agent stops processing and tells the user their message was blocked. Beak returns a short explanation naming the kind of content that triggered it — for example "credit card", "SSN" — and never the value itself.

That is deliberate. The detail an investigator needs (what matched, where, which rule fired, which policy) is written to Beak's Audit Log inside your tenant, and never sent to Microsoft. A control that shipped the secret it just detected into a third party's telemetry would be worse than no control.

Every evaluation is recorded, whether it blocked, allowed, would have blocked in observe mode, was out of scope, or could not be answered in time — so "nothing was found" is always distinguishable from "nothing was looked at".

Data handling​

Once configured, Copilot Studio shares the data listed in What Beak sees with Beak AI Security at runtime.

Confirm with your Beak contact which region processes this traffic, and satisfy yourself that it meets your organization's requirements before enabling it in a production environment. You can disconnect at any time by clearing the checkbox in Step 3, which stops the data sharing immediately.

Troubleshooting​

Microsoft's errors when you press Save:

ErrorWhat it usually means
There was a problem connecting to the protection providerBeak did not accept the call. Most often the Entra application is not yet registered in Beak (Step 1), or the integration is not enabled for your Beak environment
The application ID in your authentication token doesn't match the registered applicationThe App ID in Power Platform is not the one registered with Beak. Check for a copy-paste of the object ID instead of the application ID
AADSTS700016: Application with identifier … wasn't foundThe App ID does not exist in your tenant
AADSTS70025: … has no configured federated identity credentialsStep 2 was not completed, or the credential was added to a different application
AADSTS7002111: No matching federated identity record found for presented assertion issuerThe FIC Issuer is wrong. It must be https://login.microsoftonline.com/{tenant ID}/v2.0
AADSTS7002137: No matching federated identity record found for presented assertion subjectThe FIC subject is wrong — usually an encoding mistake, or an endpoint URL that differs from the one entered in Power Platform by a trailing slash. Ask Beak for the exact string
To change a configuration, make sure you have Power Platform admin permissionsYou need the Power Platform Administrator role

Select Copy error info in Power Platform for the full detail.

The endpoint URL cannot change afterwards​

Your endpoint URL is encoded into the federated credential's subject, and Entra matches it exactly. Changing it — even adding a trailing slash — breaks the credential, Copilot Studio can no longer obtain a token, and because the protocol fails open your agents go back to invoking tools unchecked with no error shown anywhere in Power Platform.

If you ever need a different URL, treat it as a coordinated change: update the federated credential and the Endpoint link together, then confirm a tool call still appears in Beak's Audit Log.