An agent calls meter_search with the same query hundreds of times, because nothing in the result
tells it the task is done. Your server forwards every call to the upstream API on one shared key.
The upstream quota trips, so every other agent on the server fails as well, and the looping agent
reads the refusal as an invitation to try again.
What you get
You will end up with a per-identity call budget, a concurrency cap, and a kill switch inside the server, each refusing with an error the agent stops on. This is for you if agents you do not control call an MCP server you run.
Short answer
Count tool calls per authenticated identity per window inside the server, cap how many calls each
identity has in flight, and read a kill switch from a store on every call. Refuse with a tool
result that sets isError, tells the agent to stop, and carries retryable: false in
structuredContent. Never word a permanent refusal as a request to try again, because an agent
that obeys keeps calling until something else stops it.
You will need
Node 22 or later, an MCP server whose tool handlers you control, and bearer tokens that resolve to
an agent identity. Verified 2026-10-15 against Node 22.22.0, @modelcontextprotocol/server and
@modelcontextprotocol/client 2.3.1, and @openai/agents 0.20.0. Rate limits shaped per agent
identity are a separate task; this page adds the ceilings that end a loop. The agent here is a
scripted MCP client that repeats one call and does what each error tells it. No model is called.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| An Envoy circuit breaker | Many services behind one gateway, guarding an upstream from any surge | Counts per cluster and priority, so one looping agent trips it for every caller | One agent’s loop should end that agent’s work and nobody else’s |
| An upstream provider quota | The backstop you have already, set by the provider on your API key | Refuses every agent that shares the key at once, with a 403 or 429 and a reset time that invites a retry | Always put one of the other two in front of it |
| Per-identity budgets in the server | You own the server, and each token resolves to one agent | Protects this server alone, and needs a shared counter once it runs on several instances | Several services share the upstream and need one ceiling across all of them |
The breaker protects everything downstream of the gateway and cannot tell one agent from another, so the agent that caused the trip and the agents that did not are refused alike. The upstream quota costs nothing to set up, but it counts the shared key, so it refuses everyone at once. The in-server budget is precise and narrow: it stops one identity on one server, and a second service needs its own.1
Key every ceiling by identity, not by session
A budget needs a key the agent cannot replace. The
2026-07-28 revision removed
protocol-level sessions and the Mcp-Session-Id header, so a current client sends no session to
key by. Under the
2025-11-25 transport
before it, a session id is optional and minted by the server in answer to an InitializeRequest
sent without one. A client that receives a 404 for its session must send that request again.
Initialization is also the one request a server accepts without a session id, so a client can start
over whenever it likes. Each new session brings a fresh id, so a budget keyed by it resets every
time.
Key it by the identity the bearer token resolves to instead. The server SDK hands that identity
to every tool handler as ctx.http.authInfo.
// 2. The call budget: a fixed number of calls per identity per window.
const now = clock()
const window = Math.floor(now / windowMs)
const entry = used.get(identity)
const calls = entry?.window === window ? entry.calls : 0
if (calls >= budget) {
const resetsAt = new Date((window + 1) * windowMs).toISOString()
return {
ok: false,
refusal: terminal(
'budget_exhausted',
`${identity} has used all ${budget} tool calls allowed until ${resetsAt}. ` +
'Further calls will be refused until then. Stop and tell the user the task is incomplete.',
{ resetsAt },
),
}
}
The clock is a parameter, so the tests move it an hour forward instead of waiting. The window is fixed rather than sliding, which makes the reset time a single value the refusal can state.
Refuse with an error the agent stops on
A refusal has two readers: the model reads the text, and client code reads the data. Give both the same answer.
// A refusal as an MCP tool result, built rather than thrown: the server SDK turns a thrown error
// into bare text and drops its data, the retryable flag with it. isError keeps it in front of
// the model, which reads the message, while client code reads structuredContent.
export function refusalResult(refusal) {
return {
isError: true,
content: [{ type: 'text', text: refusal.message }],
structuredContent: { error: refusal },
}
}
The
tools specification
says clients should hand tool execution errors to the model, so isError is the channel that
reaches it. Throwing does not work as well. The 2.3.1 server SDK catches a thrown ProtocolError
and returns only its message, and one of the tests pins that loss.
Run the same scripted agent against a budget of 20, refused two ways:
node demo.mjs
1. budget of 20 calls an hour, refused with a terminal error
ops-bot 21 calls, 20 served, 1 refused, stopped by the server
last answer: budget_exhausted, retryable false
upstream served 20, refused 0
2. the same budget, refused with "Please try again"
ops-bot 200 calls, 20 served, 180 refused, stopped by its own 200-step cap
last answer: "Rate limited. Please try again."
upstream served 20, refused 0
The budget held both times: the upstream served 20 calls in each run. What changed is where the loop ended. The terminal refusal stopped the agent at call 21. The retryable wording kept it going until its own cap, 180 refusals later, and an agent without a cap would still be calling when the window resets and the budget refills.2
Cap the calls in flight
A budget counts calls over an hour. A cap on concurrency stops a burst inside that hour, and its refusal is the one worth retrying, because it clears as soon as a call finishes.
node demo.mjs
4. at most 2 calls in flight, five sent at once
ops-bot served 2, refused 3: too_many_in_flight, retryable true, retry after 1s
upstream served 2, refused 0
The slot is released in a finally block, so a handler that throws gives it back. A handler that
never returns keeps it, which is why the tool’s own upstream call needs a timeout. The demo holds
the two admitted calls inside the upstream until all five are admitted or refused. A timer in its
place would let a late call find a freed slot, so the hold keeps the count at two however slowly
the machine runs.
Flip a kill switch without a deploy
The kill switch is a list of suspended identities in a store the server reads on every call. Here it is a file. In production, use a key in Workers KV or a row in a database.
export function fileKillSwitch(path) {
let lastGood = []
return {
isSuspended(identity) {
let suspended
try {
suspended = JSON.parse(readFileSync(path, 'utf8')).suspended
} catch (err) {
if (err.code === 'ENOENT') suspended = []
// A half-written or mistyped file keeps the last list that parsed. Failing open would
// un-suspend everyone; failing closed would suspend everyone.
else suspended = lastGood
}
if (!Array.isArray(suspended)) suspended = lastGood
lastGood = suspended
return suspended.includes(identity)
},
}
}
An operator writes ops-bot into the list before the sixth call, then clears it:
node demo.mjs
5. an operator suspends ops-bot before its sixth call
ops-bot 6 calls, 5 served, 1 refused, stopped by the server
last answer: agent_suspended, retryable false
switch cleared, same connection, next call: served
KV is eventually consistent. Cloudflare’s documentation says a change can take 60 seconds or more to reach other locations, so a switch kept there can take a minute or longer to apply everywhere. Keep it in a strongly consistent store if the next call has to see it.
Check it worked
The test that matters runs the agent until the server stops it, then asserts the final error is terminal in its data and in its words.
test('the call after the budget is a terminal error, and the agent stops on it', async () => {
const { upstream, server } = setup({ budget: 5 })
const ops = await connectAs(server, 'tok-ops')
const log = await runAgent(ops)
assert.equal(log.stoppedBy, 'the server')
assert.equal(log.calls, 6, 'five served, then one refusal, then nothing')
assert.equal(upstream.state.served, 5)
assert.equal(log.last.isError, true)
assert.equal(log.last.structuredContent.error.code, 'budget_exhausted')
assert.equal(log.last.structuredContent.error.retryable, false)
assert.doesNotMatch(log.last.content[0].text, /try again|retry/i, 'the words must not contradict the data')
await ops.close()
await server.close()
})
node --test ceilings.test.mjs
1..10
# tests 10
# suites 0
# pass 10
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 741.086207
The other nine pin the rest. The refusal repeats until the window resets and then lifts. A fresh
connection with the same token gets the same refusal, and one identity’s budget leaves another
untouched. A failing handler releases its slot, and a burst of five still gets exactly two slots
when its last three calls reach the server 300 ms late. A half-written kill switch file keeps its
last good list, and a thrown ProtocolError arrives at the client with no structuredContent. The last
answers the client’s POST with a 403, a 429, and a 503, and the client throws after one attempt each
time.
Compare the gateway and the upstream quota
With no budget in the server, the upstream’s quota is the ceiling, and it belongs to everyone who shares the key:
node demo.mjs
3. no server budget, upstream quota of 50 calls for the shared key
ops-bot 200 calls, 50 served, 150 refused, stopped by its own 200-step cap
last answer: "Upstream refused: 429 quota exceeded for this API key. Try again later."
billing-bot first call refused: "Upstream refused: 429 quota exceeded for this API key. Try again later."
The upstream’s words say try later, so the looping agent kept going, and billing-bot failed on a
call it made once. An Envoy breaker in front of the server fails the same way for a different
reason: its
thresholds
count requests per cluster and priority, with no field for the caller.
circuit_breakers:
thresholds:
- priority: DEFAULT
max_connections: 100
max_pending_requests: 20
max_requests: 50
max_retries: 3
That configuration was not run here. What was run is a refusal at the HTTP layer, where a gateway would refuse, seen through the OpenAI Agents SDK:
node edge.mjs
refused inside the tool
the model read: "ops-bot is suspended by an operator. Stop calling tools and tell the user."
run finished: "Stopped."
refused at the HTTP gate
the model read: (nothing, there was no second turn)
run threw SdkHttpError: Error POSTing to endpoint: {"error":"suspended"}
Both stopped the loop. The in-tool refusal let the model finish its turn and report. The 403 at the
gate surfaced as an SdkHttpError from the 2.3.1 client, and the Agents SDK ended the whole run
with it. The same client also throws on a 429 or a 503 without retrying.
When it goes wrong
The budget resets whenever the agent reconnects. It is keyed by a session or a connection. Key it by the identity the token resolves to.
Each instance enforces its own budget. Three instances behind a load balancer allow three times the budget. Keep the counter in one place, such as a Durable Object per identity or an atomic increment in Redis.
An identity is refused as too busy long after its calls ended. A handler that never returns holds its slot. Put a timeout on the upstream call inside the handler.
The client sees retryable: true on a permanent refusal. A helper chose the transient shape, or a
Retry-After value was copied from an HTTP library default. Reserve the transient shape for
conditions that clear by themselves, as
RFC 9110 frames Retry-After: how
long to wait before a follow-up request.
When not to do this
Do not make the in-server budget your only ceiling when several services share one upstream key. Each server counts only the calls it serves, which leaves the total unseen, so keep a gateway limit or the provider quota behind it.
Do not word a permanent refusal as a temporary one. Rate limited. Please try again. on an
exhausted budget turned one refused call into 180 in the second run, and the server answered each
of them.
Do not rely on the agent stopping. The budget held in every run whether the agent obeyed or not, which is the property that protects the upstream. The wording only decides how many refusals you serve before the agent gives up.
Do not suspend an identity by revoking its token alone when a kill switch exists. Revocation ends access, while a suspension can be lifted without reissuing anything, and the refusal says why.
Related how-tos
Last verified
Verified 2026-10-15 against Node 22.22.0, @modelcontextprotocol/server and
@modelcontextprotocol/client 2.3.1, @openai/agents 0.20.0, openai 7.34.0, and Zod 4.6.7.
Every output block is what the command preceding it printed. The agent is scripted and the model
in edge.mjs is a fake Responses endpoint, so no request left the machine. envoy.yaml was not
run.
Footnotes
-
The tools page of the MCP specification lists rate limiting among four things a server must do, beside validating inputs, controlling access, and sanitizing outputs. It names no unit, no window, and no key for the limit. What to count is left to each server, which is the choice this page makes. ↩︎ Back to text
-
The status code for this situation has a worked example of its own. RFC 6585 shows a 429 response with the header
Retry-After: 3600, which is an hour. The body beneath it readsI only allow 50 requests per hour to this Web site per logged in user. Try again soon.in the example’s own words. The same document notes that it does not define how the server identifies the user, nor how it counts requests, which leaves the per-what to whoever writes the limit. ↩︎ Back to text