How-to › Expose your API to agents

How to stop a runaway agent loop from exhausting your MCP server#

Put a call budget, a concurrency cap, and a kill switch inside the server, keyed by identity, and refuse with an error the looping agent stops on.

Audience
Platform team
Level
advanced
Topic
Host, transport and secure an MCP server
Languages
TypeScript
Verified

An agent calls meter_search with the same query hundreds of times, because nothing in the result tells it the task is done. Your server forwards every call to the upstream API on one shared key. The upstream quota trips, so every other agent on the server fails as well, and the looping agent reads the refusal as an invitation to try again.

What you get

You will end up with a per-identity call budget, a concurrency cap, and a kill switch inside the server, each refusing with an error the agent stops on. This is for you if agents you do not control call an MCP server you run.

Short answer

Count tool calls per authenticated identity per window inside the server, cap how many calls each identity has in flight, and read a kill switch from a store on every call. Refuse with a tool result that sets isError, tells the agent to stop, and carries retryable: false in structuredContent. Never word a permanent refusal as a request to try again, because an agent that obeys keeps calling until something else stops it.

You will need

Node 22 or later, an MCP server whose tool handlers you control, and bearer tokens that resolve to an agent identity. Verified 2026-10-15 against Node 22.22.0, @modelcontextprotocol/server and @modelcontextprotocol/client 2.3.1, and @openai/agents 0.20.0. Rate limits shaped per agent identity are a separate task; this page adds the ceilings that end a loop. The agent here is a scripted MCP client that repeats one call and does what each error tells it. No model is called.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
An Envoy circuit breakerMany services behind one gateway, guarding an upstream from any surgeCounts per cluster and priority, so one looping agent trips it for every callerOne agent’s loop should end that agent’s work and nobody else’s
An upstream provider quotaThe backstop you have already, set by the provider on your API keyRefuses every agent that shares the key at once, with a 403 or 429 and a reset time that invites a retryAlways put one of the other two in front of it
Per-identity budgets in the serverYou own the server, and each token resolves to one agentProtects this server alone, and needs a shared counter once it runs on several instancesSeveral services share the upstream and need one ceiling across all of them

The breaker protects everything downstream of the gateway and cannot tell one agent from another, so the agent that caused the trip and the agents that did not are refused alike. The upstream quota costs nothing to set up, but it counts the shared key, so it refuses everyone at once. The in-server budget is precise and narrow: it stops one identity on one server, and a second service needs its own.1

Key every ceiling by identity, not by session

A budget needs a key the agent cannot replace. The 2026-07-28 revision removed protocol-level sessions and the Mcp-Session-Id header, so a current client sends no session to key by. Under the 2025-11-25 transport before it, a session id is optional and minted by the server in answer to an InitializeRequest sent without one. A client that receives a 404 for its session must send that request again. Initialization is also the one request a server accepts without a session id, so a client can start over whenever it likes. Each new session brings a fresh id, so a budget keyed by it resets every time.

Key it by the identity the bearer token resolves to instead. The server SDK hands that identity to every tool handler as ctx.http.authInfo.

    // 2. The call budget: a fixed number of calls per identity per window.
    const now = clock()
    const window = Math.floor(now / windowMs)
    const entry = used.get(identity)
    const calls = entry?.window === window ? entry.calls : 0
    if (calls >= budget) {
      const resetsAt = new Date((window + 1) * windowMs).toISOString()
      return {
        ok: false,
        refusal: terminal(
          'budget_exhausted',
          `${identity} has used all ${budget} tool calls allowed until ${resetsAt}. ` +
            'Further calls will be refused until then. Stop and tell the user the task is incomplete.',
          { resetsAt },
        ),
      }
    }

The clock is a parameter, so the tests move it an hour forward instead of waiting. The window is fixed rather than sliding, which makes the reset time a single value the refusal can state.

Refuse with an error the agent stops on

A refusal has two readers: the model reads the text, and client code reads the data. Give both the same answer.

// A refusal as an MCP tool result, built rather than thrown: the server SDK turns a thrown error
// into bare text and drops its data, the retryable flag with it. isError keeps it in front of
// the model, which reads the message, while client code reads structuredContent.
export function refusalResult(refusal) {
  return {
    isError: true,
    content: [{ type: 'text', text: refusal.message }],
    structuredContent: { error: refusal },
  }
}

The tools specification says clients should hand tool execution errors to the model, so isError is the channel that reaches it. Throwing does not work as well. The 2.3.1 server SDK catches a thrown ProtocolError and returns only its message, and one of the tests pins that loss.

Run the same scripted agent against a budget of 20, refused two ways:

node demo.mjs
1. budget of 20 calls an hour, refused with a terminal error
  ops-bot     21 calls, 20 served, 1 refused, stopped by the server
              last answer: budget_exhausted, retryable false
  upstream served 20, refused 0

2. the same budget, refused with "Please try again"
  ops-bot     200 calls, 20 served, 180 refused, stopped by its own 200-step cap
              last answer: "Rate limited. Please try again."
  upstream served 20, refused 0

The budget held both times: the upstream served 20 calls in each run. What changed is where the loop ended. The terminal refusal stopped the agent at call 21. The retryable wording kept it going until its own cap, 180 refusals later, and an agent without a cap would still be calling when the window resets and the budget refills.2

Cap the calls in flight

A budget counts calls over an hour. A cap on concurrency stops a burst inside that hour, and its refusal is the one worth retrying, because it clears as soon as a call finishes.

node demo.mjs
4. at most 2 calls in flight, five sent at once
  ops-bot     served 2, refused 3: too_many_in_flight, retryable true, retry after 1s
  upstream served 2, refused 0

The slot is released in a finally block, so a handler that throws gives it back. A handler that never returns keeps it, which is why the tool’s own upstream call needs a timeout. The demo holds the two admitted calls inside the upstream until all five are admitted or refused. A timer in its place would let a late call find a freed slot, so the hold keeps the count at two however slowly the machine runs.

Flip a kill switch without a deploy

The kill switch is a list of suspended identities in a store the server reads on every call. Here it is a file. In production, use a key in Workers KV or a row in a database.

export function fileKillSwitch(path) {
  let lastGood = []
  return {
    isSuspended(identity) {
      let suspended
      try {
        suspended = JSON.parse(readFileSync(path, 'utf8')).suspended
      } catch (err) {
        if (err.code === 'ENOENT') suspended = []
        // A half-written or mistyped file keeps the last list that parsed. Failing open would
        // un-suspend everyone; failing closed would suspend everyone.
        else suspended = lastGood
      }
      if (!Array.isArray(suspended)) suspended = lastGood
      lastGood = suspended
      return suspended.includes(identity)
    },
  }
}

An operator writes ops-bot into the list before the sixth call, then clears it:

node demo.mjs
5. an operator suspends ops-bot before its sixth call
  ops-bot     6 calls, 5 served, 1 refused, stopped by the server
              last answer: agent_suspended, retryable false
              switch cleared, same connection, next call: served

KV is eventually consistent. Cloudflare’s documentation says a change can take 60 seconds or more to reach other locations, so a switch kept there can take a minute or longer to apply everywhere. Keep it in a strongly consistent store if the next call has to see it.

Check it worked

The test that matters runs the agent until the server stops it, then asserts the final error is terminal in its data and in its words.

test('the call after the budget is a terminal error, and the agent stops on it', async () => {
  const { upstream, server } = setup({ budget: 5 })
  const ops = await connectAs(server, 'tok-ops')
  const log = await runAgent(ops)
  assert.equal(log.stoppedBy, 'the server')
  assert.equal(log.calls, 6, 'five served, then one refusal, then nothing')
  assert.equal(upstream.state.served, 5)
  assert.equal(log.last.isError, true)
  assert.equal(log.last.structuredContent.error.code, 'budget_exhausted')
  assert.equal(log.last.structuredContent.error.retryable, false)
  assert.doesNotMatch(log.last.content[0].text, /try again|retry/i, 'the words must not contradict the data')
  await ops.close()
  await server.close()
})
node --test ceilings.test.mjs
1..10
# tests 10
# suites 0
# pass 10
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 741.086207

The other nine pin the rest. The refusal repeats until the window resets and then lifts. A fresh connection with the same token gets the same refusal, and one identity’s budget leaves another untouched. A failing handler releases its slot, and a burst of five still gets exactly two slots when its last three calls reach the server 300 ms late. A half-written kill switch file keeps its last good list, and a thrown ProtocolError arrives at the client with no structuredContent. The last answers the client’s POST with a 403, a 429, and a 503, and the client throws after one attempt each time.

Compare the gateway and the upstream quota

With no budget in the server, the upstream’s quota is the ceiling, and it belongs to everyone who shares the key:

node demo.mjs
3. no server budget, upstream quota of 50 calls for the shared key
  ops-bot     200 calls, 50 served, 150 refused, stopped by its own 200-step cap
              last answer: "Upstream refused: 429 quota exceeded for this API key. Try again later."
  billing-bot first call refused: "Upstream refused: 429 quota exceeded for this API key. Try again later."

The upstream’s words say try later, so the looping agent kept going, and billing-bot failed on a call it made once. An Envoy breaker in front of the server fails the same way for a different reason: its thresholds count requests per cluster and priority, with no field for the caller.

    circuit_breakers:
      thresholds:
      - priority: DEFAULT
        max_connections: 100
        max_pending_requests: 20
        max_requests: 50
        max_retries: 3

That configuration was not run here. What was run is a refusal at the HTTP layer, where a gateway would refuse, seen through the OpenAI Agents SDK:

node edge.mjs
refused inside the tool
  the model read: "ops-bot is suspended by an operator. Stop calling tools and tell the user."
  run finished: "Stopped."
refused at the HTTP gate
  the model read: (nothing, there was no second turn)
  run threw SdkHttpError: Error POSTing to endpoint: {"error":"suspended"}

Both stopped the loop. The in-tool refusal let the model finish its turn and report. The 403 at the gate surfaced as an SdkHttpError from the 2.3.1 client, and the Agents SDK ended the whole run with it. The same client also throws on a 429 or a 503 without retrying.

When it goes wrong

The budget resets whenever the agent reconnects. It is keyed by a session or a connection. Key it by the identity the token resolves to.

Each instance enforces its own budget. Three instances behind a load balancer allow three times the budget. Keep the counter in one place, such as a Durable Object per identity or an atomic increment in Redis.

An identity is refused as too busy long after its calls ended. A handler that never returns holds its slot. Put a timeout on the upstream call inside the handler.

The client sees retryable: true on a permanent refusal. A helper chose the transient shape, or a Retry-After value was copied from an HTTP library default. Reserve the transient shape for conditions that clear by themselves, as RFC 9110 frames Retry-After: how long to wait before a follow-up request.

When not to do this

Do not make the in-server budget your only ceiling when several services share one upstream key. Each server counts only the calls it serves, which leaves the total unseen, so keep a gateway limit or the provider quota behind it.

Do not word a permanent refusal as a temporary one. Rate limited. Please try again. on an exhausted budget turned one refused call into 180 in the second run, and the server answered each of them.

Do not rely on the agent stopping. The budget held in every run whether the agent obeyed or not, which is the property that protects the upstream. The wording only decides how many refusals you serve before the agent gives up.

Do not suspend an identity by revoking its token alone when a kill switch exists. Revocation ends access, while a suspension can be lifted without reissuing anything, and the refusal says why.

Last verified

Verified 2026-10-15 against Node 22.22.0, @modelcontextprotocol/server and @modelcontextprotocol/client 2.3.1, @openai/agents 0.20.0, openai 7.34.0, and Zod 4.6.7. Every output block is what the command preceding it printed. The agent is scripted and the model in edge.mjs is a fake Responses endpoint, so no request left the machine. envoy.yaml was not run.

Footnotes

  1. The tools page of the MCP specification lists rate limiting among four things a server must do, beside validating inputs, controlling access, and sanitizing outputs. It names no unit, no window, and no key for the limit. What to count is left to each server, which is the choice this page makes. ↩︎ Back to text

  2. The status code for this situation has a worked example of its own. RFC 6585 shows a 429 response with the header Retry-After: 3600, which is an hour. The body beneath it reads I only allow 50 requests per hour to this Web site per logged in user. Try again soon. in the example’s own words. The same document notes that it does not define how the server identifies the user, nor how it counts requests, which leaves the per-what to whoever writes the limit. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.