How-to › Make calls that survive failure

How to test cache hits, misses, and expiry without sleeping#

Prove a client cache serves the second call, misses on another key, and expires at its TTL, by injecting the clock the cache reads instead of waiting for it.

Audience
Library maintainer
Level
intermediate
Topic
Cache API responses
Languages
TypeScript, Python and Go
Verified

Read first: Choose a TTL cache or an HTTP cache for a read-heavy API

Your cache test calls sleep(1100) to watch a one-second entry expire, and the suite gains a second for every case. On a loaded CI machine the sleep sometimes ends before the entry has expired, and the test fails. Swapping the sleep for a faked Date makes the expiry test fail every time, because the cache never read Date.

What you get

You will end up with tests that prove a hit, a miss on another key, and expiry at the TTL in milliseconds, on a clock you move by hand. This is for you if you ship an SDK with a cache and want its tests to stop waiting.

Short answer

Inject the clock the cache reads, then advance it between calls. With an sdkgen SDK, pass now to the cache feature and run the client in test mode. The second load is then a hit, another id is a miss, and the entry expires at exactly the TTL with no network. Check which clock a cache reads by default first: lru-cache and cachetools read a monotonic clock, which a faked Date or time.time never moves.

You will need

Node 22 or later, and for the other two languages Python 3.12 or later and Go 1.24 or later. Verified 2026-11-22 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9 and @voxgig/sdkgen 4.44.3. The other runs used Python 3.13.18, Go 1.24.7, Jest 30.5.2, lru-cache 11.5.3, cachetools 8.0.2, clockwork v0.5.0 and OpenAPI Generator 7.27.0. The choice between a TTL cache and an HTTP cache is its own page; this one tests the TTL kind.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
cachetools timer with respxA Python client you wrote, with an httpx transport respx can mockA cache and its key you own, and a default timer that a patched time.time never movesThe SDK already ships a cache
clockwork behind a Go cacheA Go client whose cache you wrote and can hand a clockEvery time read routed through the clock, and one time.Since defeats itYou would rather not keep a cache
Fern custom fetcherA Fern TypeScript SDK that needs a cache its generator does not writeA cache you write and test yourself, behind the allowCustomFetcher generator optionYou need the cache in every language
lru-cache with nock and Jest fake timersA Node client on lru-cache, tested in JestFake timers that never reach lru-cache’s clock, and nock stalls unless nextTick stays realYou can pass perf and ttlResolution: 0 instead
OpenAPI Generator fetchApiA typescript-fetch client, and a cache you can keep in a fetchApi wrapperThe whole cache is yours: key, replay, write invalidation, and its testsYou want the generator to supply the cache
sdkgen cache and test featuresA generated SDK in several languages, tested against its in-memory mockA plain TTL cache with no Cache-Control, ETag or invalidation on writeThe API’s caching headers must be honored

Voxgig maintains sdkgen. It is the subject of this page, and the last row’s cost is the reason another row may suit you better.

Each of the six reads a clock, and the test is only as good as the injection into that clock. sdkgen and cachetools take one as an option; lru-cache takes one but caches its reading unless told not to. The two generators other than sdkgen write no cache at all, so the cache and the clock are the reader’s code.

Turn on the cache and test features with an injected clock

Scaffold with both features, then generate. generate.sh runs npx --yes @voxgig/create-sdkgen@0.31.9 bookshelf --def api.json -t ts -t py -t go -f cache -f test and commits the compiled TypeScript SDK. The cache feature’s model file lists the options:

  config: {
    options: {
      active: false
      # time-to-live in milliseconds
      ttl: 5000
      # maximum cached entries (oldest evicted)
      max: 256
      # cacheable request methods
      methods: ['GET']
    }
  }

  # Cache wraps the transport in init(); it dispatches no pipeline hooks.
  hook: {}
}


main: kit: feature: cache: config: optspec: {
  # injected clock, for TTL expiry without waiting
  now: '`$FUNCTION`'
}

now is a function the cache calls instead of the wall clock. sdkgen-demo.mjs builds the client with BookshelfSDK.test, which serves the seeded books from memory, and passes { cache: { active: true, ttl: 1000, now: () => clock.t } }:

node sdkgen-demo.mjs
ts: BookshelfSDK.test, cache ttl 1000 on an injected clock
  load b1                 Dune  hit 0 miss 1
  load b1 again           Dune  hit 1 miss 1
  load b2                 Emma  hit 1 miss 2
  clock +999 ms, load b1  Dune  hit 2 miss 2
  clock +1 ms, load b1    Dune  hit 2 miss 3
  clock and timer calls during the loads: Date.now 0, new Date 0, performance.now 0, process.hrtime 0, setTimeout 0, setInterval 0, setImmediate 0
  with no now option, two loads called Date.now 2 times

The counters are client._cache, which the SDK keeps. The second load is a hit, b2 is a miss, and at exactly 1000 ms the entry is gone, because the feature keeps an entry only while hit.expiry > now. The demo replaced every clock and timer with a counter while the loads ran, and all seven stayed at zero, so no sleep and no wall-clock read decided a line. The last line is the counter’s control, and a test calls each of the seven once and sees each counted.

The Python and Go SDKs take the same option, now returning milliseconds. Their generated trees would more than double the sample, so they are not committed, and record.sh ran the same five loads against both once. The hit and miss columns match:

cat recorded/go.txt
go: bookshelf-sdk/go in test mode, cache ttl 1000 on an injected clock
  load b1                 Dune  hit 0 miss 1
  load b1 again           Dune  hit 1 miss 1
  load b2                 Emma  hit 1 miss 2
  clock +999 ms, load b1  Dune  hit 2 miss 2
  clock +1 ms, load b1    Dune  hit 2 miss 3

In Go the option is a func() int64 in the feature map, and the counts are the Hit and Miss fields of the CacheFeature on client.Features.

Check which clock each cache reads

A faked clock helps only if the cache reads that clock. clocks.mjs quotes the default and the expiry test from each source that ran:

node clocks.mjs
cache        default clock                 expiry test
sdkgen ts    Date.now()                    hit.expiry > now
sdkgen py    time.time()                   hit["expiry"] > now
sdkgen go    time.Now().UnixMilli()        hit.expiry > now
lru-cache    performance, as loaded        !!t && !!s && (cachedNow || getNow()) - s > t
cachetools   time.monotonic                self.timer() < link.expires
lru-cache    holds each reading until setTimeout(() => (cachedNow = 0), this.ttlResolution)

sdkgen’s default is the wall clock in all three languages, so Node’s mock.timers with Date faked also expires it, which a test covers. lru-cache keeps the performance object it found when it was loaded, and cachetools defaults to time.monotonic. lru-cache also holds an entry stored at time 0 forever, through !!s,1 and keeps an entry until one millisecond past its TTL.

Do the same in Fern and OpenAPI Generator

Neither generator gave this page a response cache. Fern’s feature table lists retries, timeouts and a custom HTTP client, with no row for a cache, and its index of the current SDK pages names no cache either. Its TypeScript generator takes allowCustomFetcher: true in sdk-config.yml, the current configuration format, which adds a fetcher option that wraps every request. A cache with a now argument belongs there, tested the way oag-cache.mjs is. Fern was not run here, so this paragraph comes from those two pages.

OpenAPI Generator’s typescript-fetch templates mention no cache, and the client takes a fetchApi in its Configuration. oag-cache.mjs passes one that caches GETs on an injected now and replays the stored status, headers, and body. Its key is the URL and an HMAC of the request headers, so a response cached for one Authorization is never served to another caller, as the page on cache keys explains. Any other method clears it as it starts and again as it ends, and a GET that a write overtook is returned without being stored, which a test checks:

node oag-cache.mjs
OpenAPI Generator 7.27.0 typescript-fetch, fetchApi cache ttl 1000 on an injected clock
  getBook b1                  Dune    requests 1
  getBook b1 again            Dune    requests 1
  getBook b2                  Emma    requests 2
  clock +999 ms, getBook b1   Dune    requests 2
  clock +1 ms, getBook b1     Dune    requests 3
  updateBook b1               Dune 2  requests 4
  getBook b1 after the write  Dune 2  requests 5
  hits 2, misses 4

The requests column is what reached a stand-in on 127.0.0.1. The read after the write went to the server, which the sdkgen cache would not have done inside its TTL.

Check it worked

Eighteen tests run the TypeScript SDK and the OpenAPI Generator cache in process, ttl_check.py for cachetools, go test for the clockwork cache, and the Jest suite for lru-cache:

node --test cache.test.mjs
# tests 18
# suites 0
# pass 18
# fail 0
# cancelled 0
# skipped 0
# todo 0

When it goes wrong

The expiry test passes on the real clock and never expires under Jest’s fake timers. Jest swaps the global performance object, and lru-cache kept the one it loaded with, as its README warns. Import it after the timers are faked, or pass perf: { now } with ttlResolution: 0. The default of 1 holds each reading until a setTimeout of 1 ms fires:

node run-jest.mjs
jest 30.5.2, lru-cache 11.5.3: 8 of 8 passed
  ok   an injected perf clock with ttlResolution 0: hit, miss, and expiry one ms after the ttl
  ok   pitfall: fake timers that leave performance real never expire the entry
  ok   pitfall: fake timers swap the global performance, and lru-cache kept the one it loaded with
  ok   lru-cache loaded after the fake timers reads the fake performance and expires one ms after the ttl
  ok   pitfall: an injected clock that starts at 0 stores entries that never expire
  ok   pitfall: with every timer faked, a fetch through nock does not settle
  ok   with only nextTick kept real, a fetch through nock settles
  ok   with every timer faked and no nock, a fetch to a local server settles

The test hangs once fake timers are on. nock 14 connects its fake socket in a process.nextTick callback, and Jest’s useFakeTimers fakes nextTick unless doNotFake names it. Keep nextTick real. The same fetch with nock taken out settles with every timer faked.

A Python test that patches time.time sees no expiry, because TTLCache reads time.monotonic. Pass timer=. In Go, one time.Since in the cache measures against the real clock, so every read missed against clockwork’s fake clock.

When not to do this

Do not inject a clock into a cache you do not own. Test the SDK’s cache through its option, as sdkgen’s now, or wrap the transport and own the cache, as the OpenAPI Generator sample does.

Do not take sdkgen’s cache for an HTTP cache because its tests pass. It ignores Cache-Control and ETag, and a client’s own update leaves the cached read in place until the TTL runs out, which a test shows. Its key is the method and the URL, so a direct call that sends another caller’s headers gets the first caller’s cached response, which a test also shows. @voxgig/sdkgen 4.45.0 generated the same cache feature source in all three languages. Fern’s fetcher or OpenAPI Generator’s fetchApi suit a team that needs a cache which clears on its own writes, because that cache is theirs to write.

Last verified

Verified 2026-11-22 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3 and @voxgig/apidef 8.32.4. The other runs used Python 3.13.18 with requests 2.34.2, Go 1.24.7, Jest 30.5.2, nock 14.0.17, lru-cache 11.5.3, cachetools 8.0.2, respx 0.23.1 and httpx 0.28.1, clockwork v0.5.0 and OpenAPI Generator 7.27.0. Every output block is what the command preceding it printed. The SDKs were generated in generate.sh. The cachetools and clockwork tests run with the page’s checks, and the generated Python and Go SDKs ran once, in record.sh, committed under recorded/. Fern is described from its feature table and dynamic authentication guide.

Footnotes

  1. The !!s in lru-cache’s expiry test treats a start time of zero as no start time at all, so an entry stored while the clock reads 0 is never stale. The README offers a bare-bones time-mocking system for tests, a perf object over let myClockTime = 0. Run as printed, with no advance before the first store, it stores entries that do not expire, which lru.jest.test.cjs shows with a clock of its own. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.