How-to › Use AI to do the integration

How to cap what an agent spends on the paid APIs its tools call#

Give each agent run its own client and spend budget, split it across parallel tool calls, and hand an exhausted budget back to the model as a tool error it acts on.

Audience
Platform team
Level
advanced
Topic
Evaluate and observe agents
Languages
TypeScript, Python and Go
Verified

Read first: Set a per-task token budget that stops a runaway agent

Your research agent’s search tool calls a data provider that bills 5 credits a query. One run loops on the same search, and the client’s budget refuses it with an exception. The tool hands the model the exception text, the model reads a failure it can retry, and it retries until the run hits its turn limit.

What you get

You will end up with a budget per agent run, shared across the parallel tool calls of a turn, that the model receives as a tool error and stops on. This is for you if an agent’s tools spend money per call to someone else’s API.

Short answer

Build a fresh sdkgen client for each agent run, with the cost feature’s budget set and onBudget set to deny, so the budget belongs to that run. Reserve each tool call’s price before the turn’s calls start, so parallel calls share what is left. When a reservation or the cost feature refuses, return a tool error with a code, a retryable flag set to false, and an instruction, and the model stops calling.

You will need

Node 22 or later, an agent whose tools call a metered API, and the provider’s price list. Verified 2026-11-22 against Node 22.23.5, @voxgig/sdkgen 4.44.3, NSwag 14.8.0 and Go 1.24.7. The model’s side of the token budget is in Set a per-task token budget that stops a runaway agent. The document, api.json, describes a data provider whose company search costs 5 credits and whose company record costs 1. provider.mjs stands in for it, bills only what it answers with 200, and keeps the bill. No model is called: the model’s turns are data in script.json.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
Hand-written counting wrapperAny client, and the agent harness already holds the run’s stateA price list, a reservation rule, and refund rules you write and test yourselfA generated client already counts the spend for you
Kong or Envoy gateway quotaEvery process egresses through one proxy that code cannot bypassCounts requests per consumer, credential or header, not credits, and a run-id header is whatever the code sendsThe budget has to follow a run, not a credential
NSwagYour clients are already NSwag-generated TypeScript or C#No budget, and a request hook only through a base class you configureYou want spend counted and refused by the client itself
Provider-side spend caps and billing alertsA ceiling for a whole project or account that no client can exceedQuota enforcement lags the threshold, and an alert arrives after the money is spentA run has to stop within seconds of its own limit
sdkgen cost featureAn sdkgen client, one per run, priced from a rate tableA cutoff checked before each call, so a burst or the last call overshootsYour tools do not call an sdkgen client
Speakeasy SDK hooksYour SDK is Speakeasy-generated, on a Business or Enterprise planA before-request hook you write, behind a plan and an accountYou need it on the free plan or offline

Voxgig maintains sdkgen. It is the subject of this page, and its row says what the cutoff costs.

The in-process options know which run spent the money and can stop that run alone, but any code that builds its own client escapes them. The gateway and the provider cannot be bypassed by code. They see a credential, an account, or a header the code itself chose, so neither can tell one agent run from another.

Give each run its own client and budget

The cost feature keeps its budget per client instance, so a client built for a run gives that run a budget of its own. generate.sh scaffolded the client with --feature cost --feature retry.

export function startRun({base, apikey, budget, reservations = true, retry = false}) {
  if (!Number.isFinite(budget) || budget <= 0) throw new RangeError(`not a budget: ${budget}`)
  const client = new FactfindSDK({
    apikey,
    base,
    feature: {
      // The cost feature's own cutoff, a backstop behind the reservations below.
      cost: {active: true, budget, currency: 'credits', onBudget: 'deny', rates: RATES},
      // Off for a run: retry repeats a call after a lost connection inside one reservation,
      // and the provider may have billed both. `retry: true` is the comparison run.
      retry: {active: retry, maxDelay: 2, minDelay: 1, retries: 2},
    },
  })

retry stays off on the run’s client. sdkgen’s retry wrapper repeats a call after a lost connection, which the provider may have billed, so one reservation can pay for two billed attempts, and a test shows exactly that. A retryable tool error lets the model or the harness retry as a new, reserved call. A long-lived client shared across runs would pool their spend, and one run could use up the credits of the next.

Reserve each call’s price before the turn runs

The cost feature checks spend so far, before each call. Two parallel searches with 7 credits left both see 7 credits, and both go out. The run therefore takes each call’s price out of its own total first, from the same rate table, and refuses the call that does not fit.

    reserve(op) {
      const price = RATES[op]
      if (price === undefined) throw new Error(`no price for ${op}`)
      if (!reservations) return null // the comparison run: the cost feature's cutoff alone
      if (reserved + price > budget) return exhausted(op, price)
      reserved += price
      return null
    },

A reservation comes back only on proof that nothing was billed: an error status from this provider, which bills 200 alone, or the cost feature’s own refusal. A lost connection keeps it, because the provider may have billed the request. Checking every attempt inside the retry loop is the client’s side of the budget, which Set a client-side spend budget that stops SDK calls covers.

Return the exhausted budget as a tool error

callTool never throws. An exhausted budget becomes budget_exhausted with retryable: false and a sentence telling the model what to do instead. An MCP server would carry the same object in a result marked isError: true, where the MCP tools specification reports API failures.1

node demo.mjs
1. budget 12, exhaustion returned as a tool error
  turn 1: search_companies -> 3 companies
  turn 2: get_company, get_company, get_company -> c1, c2, c3
  turn 3: search_companies -> budget_exhausted
  stopped: answered after 3 turns
  provider billed 8 credits for 4 calls
2. budget 12, exhaustion returned as an opaque error
  stopped: turn limit of 12, 12 turns, 10 refused tool calls
  provider billed 8 credits for 4 calls
3. budget 7, one turn of two parallel searches, cost feature cutoff alone
  results: 3 companies, 1 company
  provider billed 10 credits for 2 calls
4. budget 7, one turn of two parallel searches, reserved per call
  results: 3 companies, budget_exhausted
  provider billed 5 credits for 1 call

The money holds in run 2 as well. The turns do not, because the scripted model, like a real one given an error it cannot classify, sends the same call again. The reservations refuse every repeat, so the cost is twelve model calls where three were enough.

Put the same budget around NSwag and Speakeasy clients

With its defaults, NSwag’s TypeScript client has no budget and no request hook, and it read no apiKey scheme from this document. Its wiki documents a transformOptions method on a base class you configure. The constructor also takes an http object with a fetch, so here the reservation and the key go there, as nswag-run.mjs does with the client generate.sh compiled.

node nswag-run.mjs
search_companies  ok
get_company       ok
search_companies  budget_exhausted: company.list needs 5 credits, 1 left of 7
reserved 6 of 7, provider billed 6

Speakeasy was not run, since its CLI asks for an account login before it generates. Its SDK hooks page says a before-request hook can cancel an outgoing request, and that hooks are available for Business and Enterprise users only. A budget check there refuses a call before it is sent, and the tool turns that refusal into the same budget_exhausted result.

Check it worked

node --test budget.test.mjs
# tests 13
# suites 0
# pass 13
# fail 0

The central test runs the agent and asserts that the third turn’s result is budget_exhausted with retryable: false, that the loop stopped after three turns, and that the provider billed 8 credits.

When it goes wrong

The run spends past its budget. The cost feature’s budget is a cutoff, and facts.mjs shows each way through it.

node facts.mjs
budget 6, three searches in turn: ok, ok, cost_budget; spent 10, billed 10
budget 6, four searches at once: ok, ok, ok, ok; billed 20
budget 6 on each of two clients in one process: billed 20
a record retried after an unbilled 503, the feature map: 2 requests, billed 1; attempts counted 2, credits counted 2
a record retried after an unbilled 503, [retry, cost]: 2 requests, billed 1; attempts counted 1, credits counted 1

The second search goes out because 5 spent is under 6. Four at once all go, because none has finished when the next one checks. Size the budget one call below the real limit, and reserve as shown earlier. A retry is priced per attempt in the default order, so against a provider that does not bill a 503, the count runs ahead of the bill and the run stops early.

The Go client admits all ten calls of a burst, as the TypeScript one admits four, and its counters race as well. generate.sh built a check into the generated Go tree with -race:

go1.24.7 build -race ./gocheck, on the generated go/ tree, then run once
budget 5 searches, 10 concurrent: 10 reached the provider, 0 refused
race detector exit status: 66
Found 35 data race(s)
files named in the race reports: company_entity.go cost_feature.go feature_hook.go main.go make_request.go runtime_faststr_swiss.go value.go

The race reports name cost_feature.go among others. In Go, reserve under a lock before the call, as the mutex section of the spend-budget page linked earlier does, rather than share one client’s cost budget across goroutines. 4.45.0 ships the same cost feature source in TypeScript and Go.

When not to do this

Do not rely on the sdkgen cost feature alone for an agent that makes parallel calls. Its cutoff admits every call of a burst, and in Go its counters race. Reserve per call in the harness.

Do not put an in-process budget where code you do not control builds its own clients. A Kong gateway counts per consumer, credential or header, and Envoy’s local rate limit holds a token bucket per process or connection. Neither can be bypassed by an agent’s code, which makes either the better floor beneath this page’s run budget.

Speakeasy’s hooks suit a team that already generates with it and has the plan. NSwag suits a team whose clients are NSwag already, at the price of writing the whole budget into the fetch it is given.

Last verified

Verified 2026-11-22 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3 and 4.45.0, @voxgig/apidef 8.32.4, NSwag 14.8.0 on the .NET 10.0.15 runtime, TypeScript 5.9.3 and Go 1.24.7. The sample is JavaScript over the compiled TypeScript target. Python, which the leaf lists, was not run. Every output block is what the command preceding it printed. The generators and the Go race run ran once, in generate.sh, and their output is committed. Speakeasy, Kong, Envoy, and the provider caps were read from their documentation, without a run of any of them.

Footnotes

  1. The specification describes tool execution errors as actionable feedback that language models can use to self-correct and retry with adjusted parameters. For an exhausted budget the right adjustment is to stop. So the error here says it in words, beside a flag that says the same thing to code. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.