How-to › Make calls that survive failure

How to test a backoff schedule with a fake clock#

Assert the exact waits a retry policy asks for, with the jitter seeded, by injecting its sleep or by driving a fake clock, so the test never sleeps.

Audience
Library maintainer
Level
intermediate
Topic
Set retries and timeouts
Languages
TypeScript, Python and Go
Verified

Read first: Retry fetch calls with exponential backoff in Node.js

Your retry test waits for real. Three attempts at 100, 200, and 400 milliseconds cost the suite most of a second per case, so somebody shrinks the delays to one millisecond. The assertion then checks that the call eventually succeeded, and a policy that waits the wrong intervals passes it. On a loaded CI runner the short intervals still turn flaky.

What you get

You will end up with a test that asserts the exact sequence of waits for three retries, jitter included, in milliseconds of fake time. This is for you if you maintain a client that retries and want its schedule pinned rather than observed.

Short answer

Make the retry policy take its sleep function and its random source as inputs, then record the waits in the test. sdkgen’s retry feature takes sleep and jitter: false in the constructor, but draws its jitter from Math.random, so seed that around the call. Without an injectable sleep, install @sinonjs/fake-timers and advance it with tickAsync, which yields to the event loop between timers.

You will need

Node 22 or later, and a retry policy or a generated SDK that retries. sdkgen’s packages declare Node 24, so npm prints EBADENGINE warnings on Node 22, where the scaffold and the generation for this page both completed. The document is api.json, a Tides API with two GET operations. The retry loop itself is the subject of retry fetch calls with exponential backoff in Node.js. Producing the failures in an SDK’s own tests is a mocking task, and the stand-ins here do the least that task needs.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
@sinonjs/fake-timers and vitest fake timersThe code under test calls setTimeout and offers no sleep optionA global clock swap, and a test that hangs if it advances time synchronouslyThe policy already takes a sleep function you can record
Fern retriesYou want retries on by default in a hosted SDK pipelineNo documented clock or sleep option, so tests rely on fake timers or retries turned offYour tests must assert the schedule the SDK itself computes
jonboulle/clockworkGo code where the policy takes a clockwork.ClockA clock parameter threaded through every constructor, and a goroutine to synchronize withThe Go client is generated and takes no clock
orval mutatorA TypeScript client from orval, with retry written in the mutatorYou write and test the loop, and its inputs sit in module scopeYou need the same policy in more languages than TypeScript
sdkgen retry sleep optionA generated SDK whose schedule you assert in each target languageJitter comes from each language’s global random source, and Go 1.24 made rand.Seed a no-opYou need a random source passed as an option
tenacity sleep argument and freezegunPython code that retries with tenacityTwo libraries, since time.sleep is not among the clock functions the freezegun README says it freezesThe retry loop is not tenacity’s

Voxgig maintains sdkgen, one of six options in this table, and the table does not rank them.

orval writes no retry code, so its row is a loop you own, with every input you choose to expose. sdkgen gives the loop and a sleep option in each target, but its jitter is not an option. Fern retries by default and documents no clock or sleep option, which suits a team that tests through fake timers.

Record the waits through the sleep option

generate.sh scaffolds the project with the retry feature and generates the TypeScript SDK. The committed generated/ folder is its compiled output.

cp api.json "$work/"
cd "$work"
npx --yes @voxgig/create-sdkgen@0.31.9 tides --def api.json --target ts --feature retry
cd tides-sdk/.sdk
npm run generate

The feature reference says every feature that measures time takes an injectable version of it, and for retry that is sleep(ms). Pass a function that records its argument and resolves at once.

  const sleeps = []
  const client = new TidesSDK({
    base: api.base,
    feature: {
      retry: { active: true, ...retry, sleep: (ms) => { sleeps.push(ms) } },
    },
  })

schedule.mjs makes one list() call per row against a stand-in on 127.0.0.1 that answers 503 a set number of times, then 200.

node schedule.mjs
retry { retries: 3, minDelay: 100, maxDelay: 1000, factor: 2 }, three 503s then 200
  jitter: false                          waits [100,200,400]    requests 4   resolved, 1 station
  jitter on, Math.random seeded with 7   waits [169,252,440]    requests 4   resolved, 1 station
  jitter on, seeded with 7 again         waits [169,252,440]    requests 4   resolved, 1 station
  jitter on, unseeded                    each wait in [base, base + 100): true
the same policy, four 503s, so the attempts run out
  jitter: false                          waits [100,200,400]    requests 4   request_status
the same policy, each 503 sent with Retry-After: 2
  jitter: false                          waits [1000,1000,1000] requests 4   resolved, 1 station

Three retries ask for 100, 200, and 400 ms, which is minDelay * factor ^ attempt. The whole run spends no wall-clock time waiting, because the recorded sleep returns at once. The client also records each wait under client._retry.retries, and the test checks that both lists agree.

Seed the jitter the feature draws

With jitter on, each wait gains up to minDelay milliseconds from Math.random. The feature takes no random source as an option, and JavaScript gives no way to seed the built-in one,1 so seeded.mjs replaces it for the length of one call and restores it afterwards.

export async function withSeededRandom(seed, fn) {
  const real = Math.random
  Math.random = mulberry32(seed)
  try {
    return await fn()
  } finally {
    Math.random = real
  }
}

The same seed gave the same three waits twice. The Python and Go targets, run once in a scratch project, take sleep too and draw jitter from the global random.random() and rand.Intn. Python’s waits repeat under random.seed(7).

Go’s math/rand documentation says Seed is a no-op as of Go 1.24. It took effect here because the generated go.mod says go 1.21, and GODEBUG=randseednop=1 turned it off. Under a go.mod that says 1.24 or later, only jitter: false gives exact waits in Go, unless the test sets GODEBUG=randseednop=0.

Write the same seam into an orval mutator

orval generates one function per operation and no retry. With override.mutator in its output configuration, every generated function calls your function in place of fetch, so the loop and its clock live there.

export const retryFetch = async <T>(url: string, options: RequestInit): Promise<T> => {
  const p = policy
  const repeatable = p.methods.includes((options.method ?? 'GET').toUpperCase())
  for (let attempt = 0; ; attempt++) {
    const res = await p.fetch(p.baseUrl + url, options)
    if (!repeatable || !p.statuses.includes(res.status) || attempt >= p.retries) {
      const body = res.status === 204 ? undefined : await res.json().catch(() => undefined)
      return { data: body, status: res.status, headers: res.headers } as T
    }
    await res.body?.cancel()
    await p.sleep(backoff(attempt, p))
  }
}

The generated module imports retryFetch by name, so the policy is module state that a test sets through configureRetry. The base URL lives there too, because orval’s fetch client writes relative paths such as /stations.

node orval-schedule.mjs
orval fetch client, retryFetch mutator, three 503s then 200
  random: () => 0          waits [100,200,400]    requests 4   status 200
  random: mulberry32(7)    waits [101,206,497]    requests 4   status 200

Fall back to fake timers when Fern owns the loop

Fern’s retries page documents maxRetries for TypeScript, max_retries for Python, and option.WithMaxAttempts for Go. The page documents no clock or sleep option for any of the three. Only the Java SDK tunes the schedule, through initialRetryDelayMillis, maxRetryDelayMillis, retryJitterFactor, and retryDelays, an explicit list used without jitter. Fern’s pricing page lists Python and TypeScript on the Free plan, so Java needs Enterprise. Fern was not run here.

In TypeScript and Python, a Fern client is a case for the first table row: install fake timers around the call. In TypeScript, allowCustomFetcher: true in sdk-config.yml, the configuration format described here, adds a fetcher option that wraps every request. With maxRetries: 0, a fetcher of your own can carry the loop and its sleep, as the orval mutator does.

Check it worked

The test file asserts each schedule on this page, and derives the seeded waits from the generator rather than copying them.

test('sdkgen: the seeded waits sit three draws into the stream, after the SDK used three', async () => {
  const r = await withSeededRandom(7, () => schedule({ failures: 3, retry: POLICY }))
  const jitter = r.sleeps.map((ms, i) => ms - 100 * 2 ** i)
  assert.deepEqual(jitter, draws(7, 3))
  assert.notDeepEqual(jitter, draws(7, 0))
})
node --test schedule.test.mjs
1..10
# tests 10
# suites 0
# pass 10
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 606.327873

# pass 10 means [100,200,400] holds with jitter off, and the seeded run repeats exactly.

When it goes wrong

The test hangs after the first request. A fake clock advanced with a synchronous tick fires timers that exist at that moment, and the retry has not set one yet, because it is still awaiting the first response. fake-clock.mjs installs @sinonjs/fake-timers around the SDK’s default setTimeout sleep, with a fetch stand-in that answers without a socket.

node fake-clock.mjs
three 503s then 200, retry { retries: 3, minDelay: 100, jitter: false }, default sleep
  clock.tick(700)              pending   requests sent at [0]
  await clock.tickAsync(700)   resolved  requests sent at [0,100,300,700]
  await clock.runAllAsync()    resolved  requests sent at [0,100,300,700]

Use tickAsync, which the fake-timers README says lets scheduled promise callbacks run before the timers. In vitest the same rule reads vi.advanceTimersByTimeAsync. In Go, call BlockUntilContext on a clockwork FakeClock before Advance, for the same reason.

The seeded waits are not the generator’s first three numbers. In this build, constructing the client and starting list() create three Context objects, and each draws once from Math.random for its id, so the retry feature’s jitter starts at the fourth draw. orval’s mutator used the first three and gave [101,206,497]. Assert the bands, or switch jitter off, when exact numbers would tie the test to the SDK’s internals.

Every wait reads 1000. The stand-in sent Retry-After: 2, which replaces the computed delay, capped at maxDelay.

When not to do this

Do not seed Math.random in a test that runs other code in parallel. The replacement is global, so anything else drawing from it during the call shifts the stream, and its numbers stop being random.

Do not adopt sdkgen to get a testable schedule for one TypeScript client. orval’s mutator gives the same seam, and you choose every input, the random source included. Pick sdkgen when the same schedule must hold in several generated languages, where one sleep option reads the same in each.

Do not build a clock seam into a Fern client you test once. Its documented retries, on by default with two retries, suit a team that trusts the vendor’s schedule and tests its own code under fake timers.

Last verified

Verified 2026-11-21 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, @voxgig/apidef 8.32.4, @sinonjs/fake-timers 15.5.0, orval 8.48.0 and TypeScript 5.9.3. Every output block is what the command preceding it printed. sdkgen and orval ran on api.json, and their compiled output is committed. sdkgen 4.45.0 generated the same retry source for all three targets. The Python and Go targets of the same project ran once in a scratch directory, with an injected sleep and a seeded generator, and are not committed. Fern was not run, and its retry settings and custom fetcher are read from its documentation.

Footnotes

  1. The MDN page on Math.random() says the implementation selects the initial seed, and that it cannot be chosen or reset by the user. A test that wants a repeatable stream therefore brings its own generator and puts it where the built-in one was, which is the whole of seeded.mjs. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.