Your SDK’s README says the client rate-limits itself, and a user’s serverless handler builds a fresh client on every request. Each client arrives with a full token bucket, so every request leaves at once. The API answers 429, and the retries go out into the same second.
What you get
You will end up with a rate limiter in the SDK you publish, a README line on whose budget it is, and its send times tested in each language. This is for you if you publish a generated SDK.
Short answer
Switch on sdkgen’s ratelimit feature in the constructor with a rate and burst: 1, and document
that the bucket belongs to one client instance and paces calls made one after another. Calls
started together on one client leave together, so queue them. In the default order retry wraps
ratelimit. Prove the rate in every target through the feature’s now and sleep options, because
the shared corpus covers only cost.
You will need
Node 22 or later, and Go 1.24 or later and Python 3.12 or later for the other two targets. The
sample’s api.json is a quotes API with one operation, GET /quotes. generate.sh scaffolds it
with -t ts -t go -t py --feature ratelimit,retry and commits the three SDKs, so every command
below runs offline against a transport that records when each request left.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| bottleneck or golang.org/x/time/rate | Your users already run a limiter, or need one shared across clients | Every user wires it in, and the SDK’s docs cannot promise a rate | You want the limit on by default for every user |
| OpenAPI Generator | You generate many languages and can ship a hand-written limiter beside each client | No limiter outside its C# generichost library, so each language’s bucket is yours to write and test | You want one option block across languages |
| sdkgen ratelimit feature | One option block in each target’s constructor, tested on an injected clock | Calls started together on one client leave together, the burst defaults to the rate, and trimmed Go ships no test | Users share one key across processes or clients |
| Speakeasy | Your SDKs come from Speakeasy, and honoring Retry-After on retry is enough | Its runtime customization pages list retries, timeouts, and a custom HTTP client, but no limiter | You need sends paced before the first 429 |
| Swagger Codegen | A Swagger Codegen 3 client, whose constructor takes the fetch it sends with | A seam and nothing else, so the bucket and its scope are yours | You need retries in the same chain as the limiter |
Voxgig maintains sdkgen, the subject here; its row says when another fits.
A built-in feature costs you a constructor line and gives every target the same options, and its budget belongs to one client instance. A hand-written limiter beside an OpenAPI Generator or Swagger Codegen client lives wherever you construct it, so a module-level bucket covers a whole process. In a Speakeasy SDK the seam for one is the custom HTTP client its users can pass in. None of these holds a fleet to one quota; that needs the shared limiter in Redis.
Turn the feature on and pin it to a controlled clock
The features reference
gives ratelimit a rate in tokens per second, a burst that defaults to rate, and injectable
now() and sleep(ms), and calls the budget per client instance. The sample turns it on with
burst: 1 and hands it the test’s clock.
const RATE = 2
const limit = (clock, extra = {}) => ({ active: true, rate: RATE, burst: 1, now: clock.now, sleep: clock.sleep, ...extra })
const retry = (clock) => ({ active: true, retries: 2, minDelay: 50, jitter: false, sleep: clock.sleep })
export const SCENARIOS = [
{ label: 'rate 2, burst 1, 5 calls', calls: 5, feature: (c) => ({ ratelimit: limit(c) }) },
{ label: 'rate 2, default burst, 5 calls', calls: 5, feature: (c) => ({ ratelimit: limit(c, { burst: undefined }) }) },
{ label: 'rate 2, burst 1, retry on, 2 calls, one 503', calls: 2, statuses: [503], feature: (c) => ({ ratelimit: limit(c), retry: retry(c) }) },
{ label: 'rate 2, burst 1, a client per call, 5 calls', calls: 5, perCall: true, feature: (c) => ({ ratelimit: limit(c) }) },
{
label: 'the same retry case, retry listed first',
calls: 2,
statuses: [503],
feature: (c) => [{ name: 'retry', ...retry(c) }, { name: 'ratelimit', ...limit(c) }],
},
{ label: 'rate 2, burst 1, 5 calls at once', calls: 5, together: true, feature: (c) => ({ ratelimit: limit(c) }) },
{ label: 'the same 5 through oneAtATime()', calls: 5, together: true, queued: true, feature: (c) => ({ ratelimit: limit(c) }) },
]
Each sleep moves the clock instead of waiting, so the send times are exact and the run takes
milliseconds.
node pace.mjs
sdkgen ratelimit in TypeScript, controlled clock, sends in ms, rate 2 per second
rate 2, burst 1, 5 calls 0 500 1000 1500 2000 2, inside the rate
rate 2, default burst, 5 calls 0 0 500 1000 1500 3, over the rate
rate 2, burst 1, retry on, 2 calls, one 503 0 500 1000 2, inside the rate
rate 2, burst 1, a client per call, 5 calls 0 0 0 0 0 5, over the rate
the same retry case, retry listed first 0 50 500 3, over the rate
rate 2, burst 1, 5 calls at once 0 500 500 500 500 5, over the rate
the same 5 through oneAtATime() 0 500 1000 1500 2000 2, inside the rate
The default burst starts with two tokens, so the first second holds three sends. The client per call row is the bug from the top of this page: five buckets, five sends at 0 ms. The last two rows start five calls together on one client.
Repeat the four cases in the other two targets
The Go and Python targets take the same option names, now and sleep as functions, and the
transport in system.fetch. go/main.go and pace.py run the first four cases.
cd go && go run .
sdkgen ratelimit in Go, controlled clock, sends in ms, rate 2 per second
rate 2, burst 1, 5 calls 0 500 1000 1500 2000 2, inside the rate
rate 2, default burst, 5 calls 0 0 500 1000 1500 3, over the rate
rate 2, burst 1, retry on, 2 calls, one 503 0 500 1000 2, inside the rate
rate 2, burst 1, a client per call, 5 calls 0 0 0 0 0 5, over the rate
python3 pace.py
sdkgen ratelimit in Python, controlled clock, sends in ms, rate 2 per second
rate 2, burst 1, 5 calls 0 500 1000 1500 2000 2, inside the rate
rate 2, default burst, 5 calls 0 0 500 1000 1500 3, over the rate
rate 2, burst 1, retry on, 2 calls, one 503 0 500 1000 2, inside the rate
rate 2, burst 1, a client per call, 5 calls 0 0 0 0 0 5, over the rate
The three targets agree send for send. In the retry row the 503 leaves at 0 ms, retry sleeps
50 ms, and the limiter holds the second attempt to 500. That is the wrap order the reference
draws, call, timeout, retry, ratelimit, then proxy, netsim, and cache before the
transport, which comes from installing a map of features in sorted order.1 Pass the features as an array with retry first and the retry
skips the limiter: the last TypeScript row sent three in one second.
Queue the calls that share one client
The bucket paces calls made one after another. The last two rows of node pace.mjs start five
calls together, on a clock that wakes each sleeper at its own time. Each call finds the bucket
empty, waits out the same 500 ms, and sends, so four leave at 500. Held in line by queue.mjs,
the same five leave one token apart.
// One call at a time on a shared client. Each call starts when the one before it has settled,
// so the limiter inside the SDK sees serial calls, which it paces. A failed call does not stop
// the queue.
export function oneAtATime() {
let tail = Promise.resolve()
return (call) => {
const result = tail.then(() => call())
tail = result.catch(() => {})
return result
}
}
The real clock gives the same result in all three targets, as concurrency-run.txt records. At
rate: 10, burst: 1, 15 concurrent calls on one client left inside 140 ms in each, and
go run -race reported data races in ratelimit_feature.go. The queue costs overlap, because
each call waits for the one before it to finish. Go and Python callers need the same thing, a mutex
or a lock around each call, which the sample does not run.
Check what the generated suites already cover
The reference says parity between languages is proved by a shared corpus of cases in
.sdk/test/feature. coverage.mjs reads the copies generate.sh took of that corpus, of each
target’s ratelimit tests, and of each README’s ratelimit section.
node coverage.mjs
shared feature corpus, .sdk/test/feature: cost
ratelimit tests in each target of this project
ts throttles once the burst is spent 1 client, no retry
ts burst defaults to rate and refills over time 1 client, no retry
ts inactive ratelimit does not wrap 1 client, no retry
ts real clock throttle path 1 client, no retry, asserts throttled >= 0
py throttles once the burst is spent 1 client, no retry
py burst defaults to rate and refills over time 1 client, no retry
py inactive ratelimit does not wrap 1 client, no retry
go none: test/feature_test.go was dropped with the trimmed feature set
ratelimit section of each generated README
ts says whose budget it is: no, burst default given as 5
py says whose budget it is: no, burst default given as 5
go says whose budget it is: no, burst default given as 5
The corpus that would hold the three targets to one behavior has one section, cost. The Go
target’s model drops its cross-feature suite whenever the project selects fewer than every
feature, so a Go SDK with ratelimit ships no test of it. No ratelimit test in either suite pairs it
with retry, builds a second client, or starts two calls at once.2 None of the three README sections says the budget is per client.
Their tables give burst a default of 5, but the code takes the rate when burst is unset. @voxgig/sdkgen 4.45.0 generated
the same three SDKs, tests, READMEs, and corpus, as sdkgen-4.45-diff.txt records.
Ship a limiter beside OpenAPI Generator and Swagger Codegen clients
Neither generator writes a limiter for TypeScript, Go, or Python. The option pages of OpenAPI
Generator’s typescript-fetch,
go, and python generators name no rate or throttle setting. A search of every template in
both jars finds a server-side interceptor in the java-pkmst server stub, and one client
limiter, in OpenAPI Generator’s C# generichost library. Its
README template
makes RateLimitProvider the default token provider, rationing each credential by a timeout.
Both typescript-fetch clients have a seam, so limiter.mjs ships a token bucket beside the
generated tree.
// OpenAPI Generator's typescript-fetch runs every middleware's pre before each request.
export const rateLimitMiddleware = (bucket) => ({
async pre() {
await bucket.acquire()
},
})
// Swagger Codegen's typescript-fetch takes the fetch it sends with as a constructor argument.
export const rateLimitedFetch = (bucket, fetch) => async (url, init) => {
await bucket.acquire()
return fetch(url, init)
}
OpenAPI Generator takes it as new Configuration({ middleware: [rateLimitMiddleware(bucket)] }).
Swagger Codegen takes new DefaultApi(undefined, basePath, rateLimitedFetch(bucket, fetch)).
limiter.mjs sits outside the generated tree, so regeneration never touches it, and the
customization page documents templates and
.openapi-generator-ignore for a maintainer who wants it inside the generated sources instead.
node others.mjs
limiter.mjs beside two generated clients, controlled clock, sends in ms, rate 2 per second
OpenAPI Generator
a client per call, one shared bucket 0 500 1000 1500 2000 2, inside the rate
a client and a bucket per call 0 0 0 0 0 5, over the rate
Swagger Codegen
a client per call, one shared bucket 0 500 1000 1500 2000 2, inside the rate
a client and a bucket per call 0 0 0 0 0 5, over the rate
The scope moves with the bucket. Built once per process, it holds a client per call to the rate,
which the sdkgen feature cannot, though a test shows the same bucket in sdkgen’s system.fetch
does. Its line of waiting callers also takes tokens one at a time when calls start together.
Check it worked
node --test pace.test.mjs
1..18
# tests 18
# suites 0
# pass 18
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 813.698151
One test runs go run . and python3 pace.py and asserts their four rows equal the TypeScript
ones, so a target that drifts fails here, Go included, whose own suite has no ratelimit test.
When it goes wrong
Users get 429 responses from a client you rate-limited. They build a client per request. Say in the README that the budget belongs to one client instance, and show a client made once and shared.
A release starts sending two at once. Someone passed feature as an array with retry first,
or added a limiter after retry. Pin the order with a test like the retry rows here, which fails on
0 50 500.
A shared client still sends in bursts under load. Calls started together each wait out the same
gap, then leave together. Put them through oneAtATime() from queue.mjs, or a lock in Go and
Python.
The Go SDK sends in bursts and CI stays green. Its suite has no ratelimit test to fail. Copy the
four cases into the Go module you publish, as go/main.go does, and run them in CI.
When not to do this
Do not promise a rate that the SDK cannot keep. sdkgen’s feature holds one client’s serial calls to its rate; it cannot hold concurrent calls, a process of many clients, or a fleet sharing a key. Point those users at a shared limiter, and keep the promise in the README to serial calls on one client.
Do not ship sdkgen’s feature for a language you have not tested it in. The corpus will not catch a difference between targets, so a cross-language test like the one here has to.
Choose OpenAPI Generator when your users are on C#, where its generichost library already ships
a limiter, or when you already generate with it and a bucket in a middleware is one file. Choose Swagger Codegen when a client you already
publish comes from it and its fetch argument is the one seam you need.
Related how-tos
Last verified
Verified 2026-11-21 against Node 22.23.5, Go 1.24.7, Python 3.13.18, @voxgig/create-sdkgen
0.31.9, @voxgig/sdkgen 4.44.3, @voxgig/apidef 8.32.4, OpenAPI Generator 7.27.0, Swagger Codegen
3.0.85, and TypeScript 5.9.3. Every output block is what the command preceding it printed. The
three generators ran once, in generate.sh, and their output is committed under generated/.
sdkgen 4.45.0 ran once, in compare-sdkgen.sh, and the real-clock concurrency programs once, in
record-concurrency.sh. Speakeasy was not run. Its row comes from its retries page and the list
of its runtime customization pages, none of which covers rate limiting.
Footnotes
-
The reference orders a map test first and then by name, which puts
cacheinnermost andtimeoutoutermost by the accident of the alphabet. In Go that order has to be made. The language specification says the iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next. The generatedmake_options.gosorts the names before it installs anything. ↩︎ Back to text -
The real clock test in the TypeScript suite of
@voxgig/sdkgen4.44.3 makes two calls at a rate of 1000 a second with a burst of one. It then asserts that the count of throttled calls is at least zero. A count starts at zero and only goes up. ↩︎ Back to text