Reliability
Agent runs depend on systems that occasionally fail: the LLM gateway rate-limits, a knowledge source is briefly unreachable, an ITSM endpoint returns a 503. The Runtime absorbs these automatically so a momentary blip does not end a user's conversation. This page explains what is retried, what deliberately is not, and how to tune the behaviour.
No configuration is required — every setting described here ships with a working default.
What Gets Retried
The Runtime retries a failed operation only when a retry can plausibly help and repeating the operation is safe. Those are two separate questions, and the answer to the second is what keeps retries from causing damage.
Failures are sorted into three groups:
| Group | Examples | Meaning |
|---|---|---|
| Never arrived | Connection refused, DNS failure, connection-pool timeout, 429 Too Many Requests, 503 Service Unavailable | The request provably never reached the target system, or was rejected before it ran |
| May have run | Read timeout, connection reset mid-request, 500, 502, 504 | The request reached the target system; whether it completed is unknown |
| Will not improve | 400, 401, 403, 404, content-policy refusals, invalid tool arguments | Retrying produces the identical result |
LLM calls retry the first two groups. Generating a completion has no side effect, so repeating one costs tokens and nothing else.
Tool calls retry only the first group by default. This is the important distinction. If a "create incident" tool times out while reading the response, the incident may already exist — replaying it would file a duplicate. The Runtime declines that retry and reports the failure instead.
Knowledge and datasource tools retry both groups. They only ever read data, so repeating a query cannot duplicate anything.
Skill authors can opt a tool into the wider behaviour by marking it as idempotent, which is appropriate for any tool that only reads or whose repeat execution is harmless.
What Is Never Retried
Three cases are excluded by design:
- A reply that has already started streaming. Once the agent has begun writing its answer on screen, restarting the call would print the same text twice. If the connection drops mid-reply, the error is surfaced rather than hidden behind duplicated output.
- Content-policy and security refusals. These are deterministic — the same prompt is refused every time — so retrying only consumes quota.
- A cancelled request. If the user closes the conversation or navigates away, the run stops immediately.
Retry Timing
Retries wait a short, randomised interval that grows with each attempt, starting at around half a second and capping at eight. The randomisation matters when a shared dependency recovers: without it, every agent run that failed at the same moment would retry at the same moment and immediately overwhelm the recovering service.
When a provider returns a Retry-After header, the Runtime honours it in preference to its own calculation.
Retries are transparent to the user. A conversation that succeeds on its second attempt looks like a slightly slower reply, with no error shown.
When Retries Are Exhausted
If every attempt fails, the agent is told what went wrong rather than the run being abandoned. The model receives the error as a tool result and can respond to it — trying a different tool, asking a clarifying question, or explaining what it could not reach. This is often the difference between a partial answer and no answer at all.
Execution Time Budget
Each agent has a maximum execution time (max_execution_time_seconds, default 300 seconds) that bounds a single run end to end, including any time spent on retries.
When a run exceeds its budget it is stopped and the user sees:
This request took too long and was stopped. The conversation is saved — try again, or break the request into smaller steps.
The conversation history is checkpointed continuously, so nothing before the timeout is lost. The user can continue in the same thread.
Note: This budget is enforced from the release that introduced this page onward. Agents that legitimately need longer — deep research agents, or multi-agent orchestrations that call many tools in sequence — should have their
max_execution_time_secondsraised to match their real workload.
Two narrower limits sit inside the run budget:
- LLM request timeout — 120 seconds for a single model call. Replaces a previous provider default of 600 seconds, under which a stalled gateway could hold a conversation open for ten minutes before reporting anything.
- Tool call timeout — 180 seconds for a single tool. Individual skills may specify their own shorter limit.
Configuration Reference
These are set per environment by your platform administrator. The defaults suit normal operation; change them only in response to an observed problem.
| Setting | Default | Purpose |
|---|---|---|
LLM_RETRY_ATTEMPTS | 3 | Total attempts for a model call, including the first |
LLM_RETRY_INITIAL_BACKOFF | 0.5 | Starting wait, in seconds, before the first retry |
LLM_RETRY_MAX_BACKOFF | 8 | Longest wait between attempts, in seconds |
LLM_REQUEST_TIMEOUT_SECONDS | 120 | Time limit for a single model call |
LLM_CONNECT_TIMEOUT_SECONDS | 10 | Time limit to establish the connection |
TOOL_RETRY_ATTEMPTS | 3 | Total attempts for a tool call, including the first |
TOOL_RETRY_INITIAL_BACKOFF | 0.5 | Starting wait, in seconds, before the first retry |
TOOL_RETRY_MAX_BACKOFF | 8 | Longest wait between attempts, in seconds |
TOOL_CALL_TIMEOUT_SECONDS | 180 | Time limit for a single tool call |
ENFORCE_AGENT_EXECUTION_TIMEOUT | true | Whether max_execution_time_seconds is enforced |
Setting either *_RETRY_ATTEMPTS to 1 disables retrying for that layer while leaving its timeout in place.
Best Practices
- Raise
max_execution_time_secondsfor genuinely long-running agents — Rather than disabling the budget globally. A per-agent ceiling still protects every other agent in the tenant. - Mark read-only skills as idempotent — A skill that only queries or reports gains the wider retry behaviour and becomes noticeably more resilient to flaky endpoints.
- Set a shorter timeout on fast skills — A skill that normally answers in two seconds should not be allowed to hang for 180. A tighter per-skill limit surfaces problems sooner.
- Treat repeated timeouts as a signal, not a setting to raise — Consistent timeouts usually mean an unhealthy dependency or an agent taking on too much in one turn, and raising the limit hides that.
- Check whether a failing tool is genuinely unreachable — Because writes are not replayed on ambiguous failures, a tool that reports an error may still have completed its action. Confirm before asking the agent to try again.