# How to run a live SDK suite without tripping sandbox rate limits

> Count a live suite's requests and give each CI job its share of the sandbox limit, so that parallel language jobs on one key finish green, not on a 429.

Source: https://voxgig.com/howto/live-sdk-suite-under-sandbox-rate-limits

- Audience: api-consumer
- Level: advanced
- Languages: typescript, go, python
- Verified: 2026-11-21
- Published: 2026-11-21

## Short answer

Count the requests one run of the live suite makes, then divide the sandbox's documented limit among the jobs that share its key. In an SDK generated by sdkgen, set the `ratelimit` feature in each job's `test.client.options`, since each client has its own bucket. Kiota's default RetryHandler waits out `Retry-After` but paces nothing, and an `ogen` client takes a `rate.Limiter` through `WithClient`.

---
## You will need

The vendor's documented limit for the sandbox, the sandbox key as a CI secret, Node 22 or later,
and Go 1.25 or later for the `ogen` sample. There is no vendor here: `sandbox.mjs` stands in for
one on 127.0.0.1, and `api.json` documents its limit as 15 requests per key in each 5-second
window. Every answer carries `RateLimit-Policy` and `RateLimit` from the IETF
[RateLimit header fields draft](https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/).[^1]
A request over the quota gets [429](https://www.rfc-editor.org/rfc/rfc6585.html#section-4) with
[`Retry-After`](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.3) in seconds.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [GitHub Actions `max-parallel`](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations) | A matrix of language jobs whose suites fit the limit one at a time | The jobs run in turn, so the nightly run takes as long as all suites added up | The suites need to finish together |
| [Kiota RetryHandler](https://learn.microsoft.com/en-us/openapi/kiota/middleware) | A Kiota client, where a 429 should become a wait instead of a failure | It waits only after a 429, so every refused request is sent and refused first | The vendor locks keys that keep hitting the limit |
| [`ogen` client with `golang.org/x/time/rate`](https://pkg.go.dev/golang.org/x/time/rate) | A Go suite on an `ogen` client, paced by a limiter passed in through `WithClient` | A limiter you write and share, because `ogen` generates no retry and no pacing | The suite is not in Go |
| [sdkgen run-a-live-suite with the `ratelimit` feature](https://github.com/voxgig/sdkgen/blob/main/docs/how-to/run-a-live-suite.md) | An sdkgen SDK whose generated live suite runs per language job | One bucket per client, so you divide the quota by hand across jobs | The jobs cannot agree on a share in advance |
| [Stripe sandbox limits](https://docs.stripe.com/rate-limits) | Your vendor publishes separate sandbox limits you can plan against | The sandbox limit is lower than live, so a suite sized for live overshoots it | The vendor documents no sandbox limit |

Voxgig maintains sdkgen. It is one of five options here, not the recommendation.

Pacing keeps the jobs under the quota, and a retry recovers after they go over it. The sdkgen
feature and the Go limiter pace one process each, so three jobs need three shares. Kiota's
handler is on by default and needs no share, at the price of the refused requests it waits out.

## Count the suite and split the quota

`generate.sh` scaffolds an sdkgen project from `api.json` with the `ratelimit` and `retry`
features and keeps the compiled SDK with its entity live suite. That suite runs live when
`WAREHOUSE_TEST_LIVE=TRUE`, with the key in `WAREHOUSE_APIKEY` and the host in
`WAREHOUSE_SERVER_SANDBOX`, the three inputs
[the run-a-live-suite how-to](https://github.com/voxgig/sdkgen/blob/main/docs/how-to/run-a-live-suite.md)
names. `jobs.mjs` copies it into a temporary directory per job, writes that job's control file,
and runs the jobs at once on one key. All three jobs are copies of the TypeScript suite, standing
in for the TypeScript, Go, and Python jobs of the opening paragraph: no Python job was run here.

```bash
node jobs.mjs
```

```text output
sandbox limit: 15 requests per key in 5 s
one job, no pacing
  answered 429: no, every job green: yes, requests served: 6
three jobs, one key, no pacing
  answered 429: yes, every job green: no
three jobs, one key, control/retry.json in each
  answered 429: yes, every job green: yes, requests served: 18
three jobs, one key, control/paced.json in each
  answered 429: no, every job green: yes, requests served: 18
```

One run makes six requests: create, list, update, load, remove, and a list that confirms the
remove. Three jobs started together send all of theirs inside one window, so the sandbox refuses
every request after the fifteenth. Which job gets the 429 varies from run to run. The output prints
only what the sandbox's count decides: whether any request was refused, whether every job passed,
and how many requests a green run was served.

The share is the limit divided by the jobs: 15 requests in 5 seconds is 3 a second, and one job
gets 1. `control/paced.json` sets half of that, with a bucket of one. A job then sends at most
three requests in any 5-second window, nine for the three jobs, which leaves six of the 15 spare.

```json title="control/paced.json"
{
  "version": 1,
  "test": {
    "live": { "delayMs": 0 },
    "client": {
      "options": {
        "feature": { "ratelimit": { "active": true, "rate": 0.5, "burst": 1 } }
      }
    }
  }
}
```

The suite merges `test.client.options` under its own key and host, so the feature reaches the live
client. Each client gets its own bucket, which is why the share is per job.
`test.live.delayMs` spaces the steps by 500 ms by default, and 0 here leaves the pacing to the
feature.

## Wait out Retry-After when pacing is not enough

The `retry` feature reads `Retry-After`, but caps the wait at `maxDelay`, 2000 ms by default. The
sandbox here asks for up to 5 seconds, so a default retry comes back inside the same window and
is refused again. `control/retry.json` raises `maxDelay` to 6000, and the third row of the output
shows a 429 answered and every job green. No generated file reads `RateLimit` or
`RateLimit-Policy` by name, which the test file checks.

## Pace the same calls in Kiota and in Go

Kiota's generated client has no retry code of its own. Its default chain starts with the
runtime's `RetryHandler`, which retries a 429 after the seconds `Retry-After` gives.[^2]
`kiota.mjs` sends three sequences of six `items.get()` calls at once through one client.

```bash
node kiota.mjs
```

```text output
Kiota client, default middleware, 3 sequences x 6 items.get() calls, one key
  answered 429: yes, lists returned: 18
```

`ogen` returns a declared 429 as a value, not an error, because it generates
[a sum type of the responses](https://ogen.dev/docs/concepts/interface_responses/). With
`Retry-After` declared on that response, the value carries it as `RetryAfter`. Nothing in the
generated client retries or paces. Of the 18 documentation pages `ogen`'s sitemap lists, only the
feature list mentions retries, for reconnecting a server-sent event stream. The seam is `WithClient`, which takes anything
with a `Do` method, as [`ogen`'s `http` package](https://pkg.go.dev/github.com/ogen-go/ogen/http)
defines `Client`.

```go title="ogen/main.go"
func (l limited) Do(req *http.Request) (*http.Response, error) {
	if err := l.limiter.Wait(req.Context()); err != nil {
		return nil, err
	}
	return l.inner.Do(req)
}
```

```bash
node ogen.mjs
```

```text output
ogen client, 3 goroutines x 6 ListItems calls, one key
  no limiter, wait out each Retry-After    answered 429: yes, lists returned: 18
  rate.NewLimiter(2, 1) in WithClient      answered 429: no, lists returned: 18
```

One limiter shared by three goroutines is one budget for the process. A Go job and a TypeScript
job are two processes, so each still needs its share.

## Check it worked

The suite runs every scenario on this page against a fresh sandbox and asserts the counts the
sandbox recorded.

```js title="live.test.mjs"
test('sdkgen: the ratelimit feature in each run keeps three runs under the limit', async () => {
  const seen = await runJobs(variant('three jobs, one key, control/paced.json in each'))
  assert.equal(seen.refused, 0)
  assert.deepEqual(seen.results.map((r) => r.green), [true, true, true])
  assert.deepEqual(seen.perJob, [6, 6, 6], 'every job sent its own six')
})
```

```bash
node --test live.test.mjs | tail -8
```

```text output
# tests 11
# suites 0
# pass 11
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 38104.919532
```

Another test holds the stand-in to its policy on a controlled clock: 15 answers, then 429 with
`Retry-After: 4` and `RateLimit: "sandbox";a=0;w=4` 1.2 seconds into the window.

## When it goes wrong

Each job passes alone, but two go red when they run together. The jobs share a key, and the sandbox counts by
key.[^3] Give each job its share of the quota, run the matrix with `max-parallel: 1`, or ask the
vendor for a key per language.

The suite still goes red with the `retry` feature on. Its default `maxDelay` of 2000 ms is
shorter than the sandbox's window, so the retry lands inside it. Raise `maxDelay` past the
window, or pace instead.

The pacing holds on one machine but fails across two. A bucket lives in one process, and a second
workflow on the same key, such as a pull request run beside the nightly one, adds its own. Put
both workflows in one
[concurrency group](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency)
and run the live suite on a schedule or a label, not on every push.

## When not to do this

Do not pace a suite that fits the limit three times over. If every job together stays under the
quota, the share is wasted minutes.

Do not rely on sdkgen's `ratelimit` feature as a shared budget. Each client instance holds its
own bucket and reads no `RateLimit` field, so a vendor that lowers the quota mid-run is not
heard. Where the vendor sends `Retry-After` and you cannot fix a share in advance, Kiota's default
handler waits it out without any arithmetic, and that is the better fit.

Do not count on `ogen` to pace anything. Its limiter sits in your Go code rather than in the generated
client, which suits a Go suite that already owns its `http.Client`, and leaves the other
languages without it.

Do not size a sandbox suite from the live limit. Stripe documents 25 requests a second in a
sandbox and 100 in live mode.

## Related how-tos

- [Seed a sandbox account for SDK integration tests](/howto/seed-a-sandbox-account-for-sdk-integration-tests)

- [Run a scheduled live suite that gates an SDK release](/howto/scheduled-live-suite-before-an-sdk-release)

- [Read RateLimit headers and slow down before you get a 429](/howto/read-ratelimit-headers-and-pace-before-429)

- [Layer unit, integration, and live tests for an SDK](/howto/layer-unit-integration-and-live-sdk-tests)

- [Share one rate limiter across many workers with Redis](/howto/share-a-rate-limiter-across-workers-with-redis)

- [Build a CI matrix for an SDK across six languages](/howto/sdk-ci-matrix-across-six-languages)

## Last verified

Verified 2026-11-21 against Node 22.23.5, `@voxgig/create-sdkgen` 0.31.9, `@voxgig/sdkgen` 4.44.3,
Kiota 1.39.0 with its runtime packages at 1.0.0-preview.113, `ogen` v1.24.0, `golang.org/x/time`
v0.15.0, and Go 1.25.0. Every output block is what the command preceding it printed. `ogen` v1.24.0
was chosen over v1.25.0 because v1.25.0 needs Go 1.26. The Python job the problem names was not
run; the TypeScript suite stands in for every job.

[^1]: The fields here follow
[revision 12](https://www.ietf.org/archive/id/draft-ietf-httpapi-ratelimit-headers-12.txt) of the
draft, dated November 20, 2026, which names the available quota `a` and the effective window `w`.
[Revision 11](https://www.ietf.org/archive/id/draft-ietf-httpapi-ratelimit-headers-11.txt), dated
May 23, 2026, named them `r` and `t`. One example in revision 12 still says the server advertises
only r=20, over a field that reads `a=20`, and its list of changes does not mention the rename.

[^2]: The handler's [TypeScript source](https://github.com/microsoft/kiota-typescript/blob/main/packages/http/fetch/src/middlewares/retryHandler.ts)
reads `Retry-After` as seconds or as a date, and caps the wait at 180 seconds. The middleware
page describes the same handler as retrying a 429 or a 503 and does not mention the header.

[^3]: [RFC 6585](https://www.rfc-editor.org/rfc/rfc6585.html#section-4) defined 429 in 2012 and
declined to say how a server identifies the user or counts requests. Its example limit is 50
requests an hour for each logged-in user, and a sandbox key is the closest thing a CI job has
to one.