# How to ship a client-side rate limiter inside an SDK you generate

> Turn on a token bucket in the SDK you publish, say whose budget it is, and prove its rate in each generated language on a clock the test controls.

Source: https://voxgig.com/howto/ship-a-rate-limiter-in-a-generated-sdk

- Audience: library-maintainer
- Level: advanced
- Languages: typescript, go, python
- Verified: 2026-11-21
- Published: 2026-11-21

## Short answer

Switch on sdkgen's ratelimit feature in the constructor with a rate and burst: 1, and document that the bucket belongs to one client instance and paces calls made one after another. Calls started together on one client leave together, so queue them. In the default order retry wraps ratelimit. Prove the rate in every target through the feature's now and sleep options, because the shared corpus covers only cost.

---
## You will need

Node 22 or later, and Go 1.24 or later and Python 3.12 or later for the other two targets. The
sample's `api.json` is a quotes API with one operation, `GET /quotes`. `generate.sh` scaffolds it
with `-t ts -t go -t py --feature ratelimit,retry` and commits the three SDKs, so every command
below runs offline against a transport that records when each request left.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [bottleneck or golang.org/x/time/rate](https://pkg.go.dev/golang.org/x/time/rate) | Your users already run a limiter, or need one shared across clients | Every user wires it in, and the SDK's docs cannot promise a rate | You want the limit on by default for every user |
| [OpenAPI Generator](https://openapi-generator.tech/docs/generators/typescript-fetch/) | You generate many languages and can ship a hand-written limiter beside each client | No limiter outside its C# `generichost` library, so each language's bucket is yours to write and test | You want one option block across languages |
| [sdkgen ratelimit feature](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md) | One option block in each target's constructor, tested on an injected clock | Calls started together on one client leave together, the burst defaults to the rate, and trimmed Go ships no test | Users share one key across processes or clients |
| [Speakeasy](https://www.speakeasy.com/docs/sdks/customize/runtime/retries) | Your SDKs come from Speakeasy, and honoring `Retry-After` on retry is enough | Its runtime customization pages list retries, timeouts, and a custom HTTP client, but no limiter | You need sends paced before the first 429 |
| [Swagger Codegen](https://github.com/swagger-api/swagger-codegen) | A Swagger Codegen 3 client, whose constructor takes the `fetch` it sends with | A seam and nothing else, so the bucket and its scope are yours | You need retries in the same chain as the limiter |

Voxgig maintains sdkgen, the subject here; its row says when another fits.

A built-in feature costs you a constructor line and gives every target the same options, and
its budget belongs to one client instance. A hand-written limiter beside an OpenAPI Generator or
Swagger Codegen client lives wherever you construct it, so a module-level bucket covers a whole
process. In a Speakeasy SDK the seam for one is the
[custom HTTP client](https://www.speakeasy.com/docs/sdks/customize/runtime/custom-http-client)
its users can pass in. None of these holds a fleet to one quota; that needs the shared limiter in Redis.

## Turn the feature on and pin it to a controlled clock

The [features reference](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md)
gives `ratelimit` a `rate` in tokens per second, a `burst` that defaults to `rate`, and injectable
`now()` and `sleep(ms)`, and calls the budget per client instance. The sample turns it on with
`burst: 1` and hands it the test's clock.

```js title="pace.mjs"
const RATE = 2
const limit = (clock, extra = {}) => ({ active: true, rate: RATE, burst: 1, now: clock.now, sleep: clock.sleep, ...extra })
const retry = (clock) => ({ active: true, retries: 2, minDelay: 50, jitter: false, sleep: clock.sleep })

export const SCENARIOS = [
  { label: 'rate 2, burst 1, 5 calls', calls: 5, feature: (c) => ({ ratelimit: limit(c) }) },
  { label: 'rate 2, default burst, 5 calls', calls: 5, feature: (c) => ({ ratelimit: limit(c, { burst: undefined }) }) },
  { label: 'rate 2, burst 1, retry on, 2 calls, one 503', calls: 2, statuses: [503], feature: (c) => ({ ratelimit: limit(c), retry: retry(c) }) },
  { label: 'rate 2, burst 1, a client per call, 5 calls', calls: 5, perCall: true, feature: (c) => ({ ratelimit: limit(c) }) },
  {
    label: 'the same retry case, retry listed first',
    calls: 2,
    statuses: [503],
    feature: (c) => [{ name: 'retry', ...retry(c) }, { name: 'ratelimit', ...limit(c) }],
  },
  { label: 'rate 2, burst 1, 5 calls at once', calls: 5, together: true, feature: (c) => ({ ratelimit: limit(c) }) },
  { label: 'the same 5 through oneAtATime()', calls: 5, together: true, queued: true, feature: (c) => ({ ratelimit: limit(c) }) },
]
```

Each `sleep` moves the clock instead of waiting, so the send times are exact and the run takes
milliseconds.

```bash
node pace.mjs
```

```text output
sdkgen ratelimit in TypeScript, controlled clock, sends in ms, rate 2 per second
  rate 2, burst 1, 5 calls                     0 500 1000 1500 2000     2, inside the rate
  rate 2, default burst, 5 calls               0 0 500 1000 1500        3, over the rate
  rate 2, burst 1, retry on, 2 calls, one 503  0 500 1000               2, inside the rate
  rate 2, burst 1, a client per call, 5 calls  0 0 0 0 0                5, over the rate
  the same retry case, retry listed first      0 50 500                 3, over the rate
  rate 2, burst 1, 5 calls at once             0 500 500 500 500        5, over the rate
  the same 5 through oneAtATime()              0 500 1000 1500 2000     2, inside the rate
```

The default burst starts with two tokens, so the first second holds three sends. The client per
call row is the bug from the top of this page: five buckets, five sends at 0 ms. The last two rows
start five calls together on one client.

## Repeat the four cases in the other two targets

The Go and Python targets take the same option names, `now` and `sleep` as functions, and the
transport in `system.fetch`. `go/main.go` and `pace.py` run the first four cases.

```bash
cd go && go run .
```

```text output
sdkgen ratelimit in Go, controlled clock, sends in ms, rate 2 per second
  rate 2, burst 1, 5 calls                     0 500 1000 1500 2000     2, inside the rate
  rate 2, default burst, 5 calls               0 0 500 1000 1500        3, over the rate
  rate 2, burst 1, retry on, 2 calls, one 503  0 500 1000               2, inside the rate
  rate 2, burst 1, a client per call, 5 calls  0 0 0 0 0                5, over the rate
```

```bash
python3 pace.py
```

```text output
sdkgen ratelimit in Python, controlled clock, sends in ms, rate 2 per second
  rate 2, burst 1, 5 calls                     0 500 1000 1500 2000     2, inside the rate
  rate 2, default burst, 5 calls               0 0 500 1000 1500        3, over the rate
  rate 2, burst 1, retry on, 2 calls, one 503  0 500 1000               2, inside the rate
  rate 2, burst 1, a client per call, 5 calls  0 0 0 0 0                5, over the rate
```

The three targets agree send for send. In the retry row the 503 leaves at 0 ms, retry sleeps
50 ms, and the limiter holds the second attempt to 500. That is the wrap order the reference
draws, `call`, `timeout`, `retry`, `ratelimit`, then `proxy`, `netsim`, and `cache` before the
transport, which comes from installing a map of features in sorted order.[^1] Pass the features as an array with `retry` first and the retry
skips the limiter: the last TypeScript row sent three in one second.

## Queue the calls that share one client

The bucket paces calls made one after another. The last two rows of `node pace.mjs` start five
calls together, on a clock that wakes each sleeper at its own time. Each call finds the bucket
empty, waits out the same 500 ms, and sends, so four leave at 500. Held in line by `queue.mjs`,
the same five leave one token apart.

```js title="queue.mjs"
// One call at a time on a shared client. Each call starts when the one before it has settled,
// so the limiter inside the SDK sees serial calls, which it paces. A failed call does not stop
// the queue.
export function oneAtATime() {
  let tail = Promise.resolve()
  return (call) => {
    const result = tail.then(() => call())
    tail = result.catch(() => {})
    return result
  }
}
```

The real clock gives the same result in all three targets, as `concurrency-run.txt` records. At
`rate: 10, burst: 1`, 15 concurrent calls on one client left inside 140 ms in each, and
`go run -race` reported data races in `ratelimit_feature.go`. The queue costs overlap, because
each call waits for the one before it to finish. Go and Python callers need the same thing, a mutex
or a lock around each call, which the sample does not run.

## Check what the generated suites already cover

The reference says parity between languages is proved by a shared corpus of cases in
`.sdk/test/feature`. `coverage.mjs` reads the copies `generate.sh` took of that corpus, of each
target's ratelimit tests, and of each README's ratelimit section.

```bash
node coverage.mjs
```

```text output
shared feature corpus, .sdk/test/feature: cost
ratelimit tests in each target of this project
  ts  throttles once the burst is spent                1 client, no retry
  ts  burst defaults to rate and refills over time     1 client, no retry
  ts  inactive ratelimit does not wrap                 1 client, no retry
  ts  real clock throttle path                         1 client, no retry, asserts throttled >= 0
  py  throttles once the burst is spent                1 client, no retry
  py  burst defaults to rate and refills over time     1 client, no retry
  py  inactive ratelimit does not wrap                 1 client, no retry
  go  none: test/feature_test.go was dropped with the trimmed feature set
ratelimit section of each generated README
  ts  says whose budget it is: no, burst default given as 5
  py  says whose budget it is: no, burst default given as 5
  go  says whose budget it is: no, burst default given as 5
```

The corpus that would hold the three targets to one behavior has one section, `cost`. The Go
target's model drops its cross-feature suite whenever the project selects fewer than every
feature, so a Go SDK with `ratelimit` ships no test of it. No ratelimit test in either suite pairs it
with `retry`, builds a second client, or starts two calls at once.[^2] None of the three README sections says the budget is per client.
Their tables give `burst` a default of 5, but the code takes the rate when `burst` is unset. `@voxgig/sdkgen` 4.45.0 generated
the same three SDKs, tests, READMEs, and corpus, as `sdkgen-4.45-diff.txt` records.

## Ship a limiter beside OpenAPI Generator and Swagger Codegen clients

Neither generator writes a limiter for TypeScript, Go, or Python. The option pages of OpenAPI
Generator's [`typescript-fetch`](https://openapi-generator.tech/docs/generators/typescript-fetch/),
`go`, and `python` generators name no rate or throttle setting. A search of every template in
both jars finds a server-side interceptor in the `java-pkmst` server stub, and one client
limiter, in OpenAPI Generator's C# `generichost` library. Its
[README template](https://github.com/OpenAPITools/openapi-generator/blob/v7.27.0/modules/openapi-generator/src/main/resources/csharp/libraries/generichost/README.client.mustache)
makes `RateLimitProvider` the default token provider, rationing each credential by a timeout.
Both `typescript-fetch` clients have a seam, so `limiter.mjs` ships a token bucket beside the
generated tree.

```js title="limiter.mjs"
// OpenAPI Generator's typescript-fetch runs every middleware's pre before each request.
export const rateLimitMiddleware = (bucket) => ({
  async pre() {
    await bucket.acquire()
  },
})

// Swagger Codegen's typescript-fetch takes the fetch it sends with as a constructor argument.
export const rateLimitedFetch = (bucket, fetch) => async (url, init) => {
  await bucket.acquire()
  return fetch(url, init)
}
```

OpenAPI Generator takes it as `new Configuration({ middleware: [rateLimitMiddleware(bucket)] })`.
Swagger Codegen takes `new DefaultApi(undefined, basePath, rateLimitedFetch(bucket, fetch))`.
`limiter.mjs` sits outside the generated tree, so regeneration never touches it, and the
[customization page](https://openapi-generator.tech/docs/customization/) documents templates and
`.openapi-generator-ignore` for a maintainer who wants it inside the generated sources instead.

```bash
node others.mjs
```

```text output
limiter.mjs beside two generated clients, controlled clock, sends in ms, rate 2 per second
OpenAPI Generator
  a client per call, one shared bucket         0 500 1000 1500 2000     2, inside the rate
  a client and a bucket per call               0 0 0 0 0                5, over the rate
Swagger Codegen
  a client per call, one shared bucket         0 500 1000 1500 2000     2, inside the rate
  a client and a bucket per call               0 0 0 0 0                5, over the rate
```

The scope moves with the bucket. Built once per process, it holds a client per call to the rate,
which the sdkgen feature cannot, though a test shows the same bucket in sdkgen's `system.fetch`
does. Its line of waiting callers also takes tokens one at a time when calls start together.

## Check it worked

```bash
node --test pace.test.mjs
```

```text output
1..18
# tests 18
# suites 0
# pass 18
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 813.698151
```

One test runs `go run .` and `python3 pace.py` and asserts their four rows equal the TypeScript
ones, so a target that drifts fails here, Go included, whose own suite has no ratelimit test.

## When it goes wrong

Users get 429 responses from a client you rate-limited. They build a client per request. Say in the
README that the budget belongs to one client instance, and show a client made once and shared.

A release starts sending two at once. Someone passed `feature` as an array with `retry` first,
or added a limiter after retry. Pin the order with a test like the retry rows here, which fails on
`0 50 500`.

A shared client still sends in bursts under load. Calls started together each wait out the same
gap, then leave together. Put them through `oneAtATime()` from `queue.mjs`, or a lock in Go and
Python.

The Go SDK sends in bursts and CI stays green. Its suite has no ratelimit test to fail. Copy the
four cases into the Go module you publish, as `go/main.go` does, and run them in CI.

## When not to do this

Do not promise a rate that the SDK cannot keep. sdkgen's feature holds one client's serial calls to
its rate; it cannot hold concurrent calls, a process of many clients, or a fleet sharing a key. Point those users at a shared
limiter, and keep the promise in the README to serial calls on one client.

Do not ship sdkgen's feature for a language you have not tested it in. The corpus will not catch a
difference between targets, so a cross-language test like the one here has to.

Choose OpenAPI Generator when your users are on C#, where its `generichost` library already ships
a limiter, or when you already generate with it and a bucket in a middleware is one file. Choose Swagger Codegen when a client you already
publish comes from it and its `fetch` argument is the one seam you need.

## Related how-tos

- [Throttle outbound API calls with a token bucket](/howto/throttle-outbound-calls-with-a-token-bucket)

- [Test a backoff schedule with a fake clock](/howto/test-a-backoff-schedule-with-a-fake-clock)

- [Share one rate limiter across many workers with Redis](/howto/share-a-rate-limiter-across-workers-with-redis)

- [Read RateLimit headers and slow down before you get a 429](/howto/read-ratelimit-headers-and-pace-before-429)

- [Choose between 429 and 503 when you throttle a request](/howto/choose-429-or-503-when-throttling)

## Last verified

Verified 2026-11-21 against Node 22.23.5, Go 1.24.7, Python 3.13.18, `@voxgig/create-sdkgen`
0.31.9, `@voxgig/sdkgen` 4.44.3, `@voxgig/apidef` 8.32.4, OpenAPI Generator 7.27.0, Swagger Codegen
3.0.85, and TypeScript 5.9.3. Every output block is what the command preceding it printed. The
three generators ran once, in `generate.sh`, and their output is committed under `generated/`.
sdkgen 4.45.0 ran once, in `compare-sdkgen.sh`, and the real-clock concurrency programs once, in
`record-concurrency.sh`. Speakeasy was not run. Its row comes from its retries page and the list
of its runtime customization pages, none of which covers rate limiting.

[^1]: The reference orders a map test first and then by name, which puts `cache` innermost and
`timeout` outermost by the accident of the alphabet. In Go that order has to be made. The
[language specification](https://go.dev/ref/spec#For_range) says the iteration order over
maps is not specified and is not guaranteed to be the same from one iteration to the next. The
generated `make_options.go` sorts the names before it installs anything.

[^2]: The real clock test in the TypeScript suite of
[`@voxgig/sdkgen` 4.44.3](https://unpkg.com/@voxgig/sdkgen@4.44.3/project/.sdk/tm/ts/test/feature.test.ts)
makes two calls at a rate of 1000 a second with a burst of one. It then asserts that the count of
throttled calls is at least zero. A count starts at zero and only goes up.