How-to › Test and mock integrations

How to run a live SDK suite without tripping sandbox rate limits#

Count a live suite's requests and give each CI job its share of the sandbox limit, so that parallel language jobs on one key finish green, not on a 429.

Audience
API consumer
Level
advanced
Topic
Run SDK test suites
Languages
TypeScript, Go and Python
Verified

Read first: Seed a sandbox account for SDK integration tests

The live suite for your TypeScript SDK passes when its job runs alone. The Go and Python jobs start at the same minute with the same sandbox key, the sandbox counts all three against one quota, and it answers 429 before the suites finish. Two jobs go red on calls that were correct.

What you get

You will end up with a live suite whose request count you know, and a pacing setting per CI job that keeps every job inside the sandbox limit. This is for you if several SDK jobs share one sandbox key.

Short answer

Count the requests one run of the live suite makes, then divide the sandbox’s documented limit among the jobs that share its key. In an SDK generated by sdkgen, set the ratelimit feature in each job’s test.client.options, since each client has its own bucket. Kiota’s default RetryHandler waits out Retry-After but paces nothing, and an ogen client takes a rate.Limiter through WithClient.

You will need

The vendor’s documented limit for the sandbox, the sandbox key as a CI secret, Node 22 or later, and Go 1.25 or later for the ogen sample. There is no vendor here: sandbox.mjs stands in for one on 127.0.0.1, and api.json documents its limit as 15 requests per key in each 5-second window. Every answer carries RateLimit-Policy and RateLimit from the IETF RateLimit header fields draft.1 A request over the quota gets 429 with Retry-After in seconds.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
GitHub Actions max-parallelA matrix of language jobs whose suites fit the limit one at a timeThe jobs run in turn, so the nightly run takes as long as all suites added upThe suites need to finish together
Kiota RetryHandlerA Kiota client, where a 429 should become a wait instead of a failureIt waits only after a 429, so every refused request is sent and refused firstThe vendor locks keys that keep hitting the limit
ogen client with golang.org/x/time/rateA Go suite on an ogen client, paced by a limiter passed in through WithClientA limiter you write and share, because ogen generates no retry and no pacingThe suite is not in Go
sdkgen run-a-live-suite with the ratelimit featureAn sdkgen SDK whose generated live suite runs per language jobOne bucket per client, so you divide the quota by hand across jobsThe jobs cannot agree on a share in advance
Stripe sandbox limitsYour vendor publishes separate sandbox limits you can plan againstThe sandbox limit is lower than live, so a suite sized for live overshoots itThe vendor documents no sandbox limit

Voxgig maintains sdkgen. It is one of five options here, not the recommendation.

Pacing keeps the jobs under the quota, and a retry recovers after they go over it. The sdkgen feature and the Go limiter pace one process each, so three jobs need three shares. Kiota’s handler is on by default and needs no share, at the price of the refused requests it waits out.

Count the suite and split the quota

generate.sh scaffolds an sdkgen project from api.json with the ratelimit and retry features and keeps the compiled SDK with its entity live suite. That suite runs live when WAREHOUSE_TEST_LIVE=TRUE, with the key in WAREHOUSE_APIKEY and the host in WAREHOUSE_SERVER_SANDBOX, the three inputs the run-a-live-suite how-to names. jobs.mjs copies it into a temporary directory per job, writes that job’s control file, and runs the jobs at once on one key. All three jobs are copies of the TypeScript suite, standing in for the TypeScript, Go, and Python jobs of the opening paragraph: no Python job was run here.

node jobs.mjs
sandbox limit: 15 requests per key in 5 s
one job, no pacing
  answered 429: no, every job green: yes, requests served: 6
three jobs, one key, no pacing
  answered 429: yes, every job green: no
three jobs, one key, control/retry.json in each
  answered 429: yes, every job green: yes, requests served: 18
three jobs, one key, control/paced.json in each
  answered 429: no, every job green: yes, requests served: 18

One run makes six requests: create, list, update, load, remove, and a list that confirms the remove. Three jobs started together send all of theirs inside one window, so the sandbox refuses every request after the fifteenth. Which job gets the 429 varies from run to run. The output prints only what the sandbox’s count decides: whether any request was refused, whether every job passed, and how many requests a green run was served.

The share is the limit divided by the jobs: 15 requests in 5 seconds is 3 a second, and one job gets 1. control/paced.json sets half of that, with a bucket of one. A job then sends at most three requests in any 5-second window, nine for the three jobs, which leaves six of the 15 spare.

{
  "version": 1,
  "test": {
    "live": { "delayMs": 0 },
    "client": {
      "options": {
        "feature": { "ratelimit": { "active": true, "rate": 0.5, "burst": 1 } }
      }
    }
  }
}

The suite merges test.client.options under its own key and host, so the feature reaches the live client. Each client gets its own bucket, which is why the share is per job. test.live.delayMs spaces the steps by 500 ms by default, and 0 here leaves the pacing to the feature.

Wait out Retry-After when pacing is not enough

The retry feature reads Retry-After, but caps the wait at maxDelay, 2000 ms by default. The sandbox here asks for up to 5 seconds, so a default retry comes back inside the same window and is refused again. control/retry.json raises maxDelay to 6000, and the third row of the output shows a 429 answered and every job green. No generated file reads RateLimit or RateLimit-Policy by name, which the test file checks.

Pace the same calls in Kiota and in Go

Kiota’s generated client has no retry code of its own. Its default chain starts with the runtime’s RetryHandler, which retries a 429 after the seconds Retry-After gives.2 kiota.mjs sends three sequences of six items.get() calls at once through one client.

node kiota.mjs
Kiota client, default middleware, 3 sequences x 6 items.get() calls, one key
  answered 429: yes, lists returned: 18

ogen returns a declared 429 as a value, not an error, because it generates a sum type of the responses. With Retry-After declared on that response, the value carries it as RetryAfter. Nothing in the generated client retries or paces. Of the 18 documentation pages ogen’s sitemap lists, only the feature list mentions retries, for reconnecting a server-sent event stream. The seam is WithClient, which takes anything with a Do method, as ogen’s http package defines Client.

func (l limited) Do(req *http.Request) (*http.Response, error) {
	if err := l.limiter.Wait(req.Context()); err != nil {
		return nil, err
	}
	return l.inner.Do(req)
}
node ogen.mjs
ogen client, 3 goroutines x 6 ListItems calls, one key
  no limiter, wait out each Retry-After    answered 429: yes, lists returned: 18
  rate.NewLimiter(2, 1) in WithClient      answered 429: no, lists returned: 18

One limiter shared by three goroutines is one budget for the process. A Go job and a TypeScript job are two processes, so each still needs its share.

Check it worked

The suite runs every scenario on this page against a fresh sandbox and asserts the counts the sandbox recorded.

test('sdkgen: the ratelimit feature in each run keeps three runs under the limit', async () => {
  const seen = await runJobs(variant('three jobs, one key, control/paced.json in each'))
  assert.equal(seen.refused, 0)
  assert.deepEqual(seen.results.map((r) => r.green), [true, true, true])
  assert.deepEqual(seen.perJob, [6, 6, 6], 'every job sent its own six')
})
node --test live.test.mjs | tail -8
# tests 11
# suites 0
# pass 11
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 38104.919532

Another test holds the stand-in to its policy on a controlled clock: 15 answers, then 429 with Retry-After: 4 and RateLimit: "sandbox";a=0;w=4 1.2 seconds into the window.

When it goes wrong

Each job passes alone, but two go red when they run together. The jobs share a key, and the sandbox counts by key.3 Give each job its share of the quota, run the matrix with max-parallel: 1, or ask the vendor for a key per language.

The suite still goes red with the retry feature on. Its default maxDelay of 2000 ms is shorter than the sandbox’s window, so the retry lands inside it. Raise maxDelay past the window, or pace instead.

The pacing holds on one machine but fails across two. A bucket lives in one process, and a second workflow on the same key, such as a pull request run beside the nightly one, adds its own. Put both workflows in one concurrency group and run the live suite on a schedule or a label, not on every push.

When not to do this

Do not pace a suite that fits the limit three times over. If every job together stays under the quota, the share is wasted minutes.

Do not rely on sdkgen’s ratelimit feature as a shared budget. Each client instance holds its own bucket and reads no RateLimit field, so a vendor that lowers the quota mid-run is not heard. Where the vendor sends Retry-After and you cannot fix a share in advance, Kiota’s default handler waits it out without any arithmetic, and that is the better fit.

Do not count on ogen to pace anything. Its limiter sits in your Go code rather than in the generated client, which suits a Go suite that already owns its http.Client, and leaves the other languages without it.

Do not size a sandbox suite from the live limit. Stripe documents 25 requests a second in a sandbox and 100 in live mode.

Last verified

Verified 2026-11-21 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, Kiota 1.39.0 with its runtime packages at 1.0.0-preview.113, ogen v1.24.0, golang.org/x/time v0.15.0, and Go 1.25.0. Every output block is what the command preceding it printed. ogen v1.24.0 was chosen over v1.25.0 because v1.25.0 needs Go 1.26. The Python job the problem names was not run; the TypeScript suite stands in for every job.

Footnotes

  1. The fields here follow revision 12 of the draft, dated November 20, 2026, which names the available quota a and the effective window w. Revision 11, dated May 23, 2026, named them r and t. One example in revision 12 still says the server advertises only r=20, over a field that reads a=20, and its list of changes does not mention the rename. ↩︎ Back to text

  2. The handler’s TypeScript source reads Retry-After as seconds or as a date, and caps the wait at 180 seconds. The middleware page describes the same handler as retrying a 429 or a 503 and does not mention the header. ↩︎ Back to text

  3. RFC 6585 defined 429 in 2012 and declined to say how a server identifies the user or counts requests. Its example limit is 50 requests an hour for each logged-in user, and a sandbox key is the closest thing a CI job has to one. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.