How to share an API response cache across many workers with Redis#

Forty workers each keep their own response cache, so a rate table the origin serves once a minute is fetched forty times a minute, and its rate limit counts every one. When a worker restarts its cache is empty, and the first request after a deploy is forty cold misses arriving together.

What you get

You will end up with one shared cache that workers in two languages read, and one origin request per key however many miss at once. A test proves no stored entry carries a cookie. This is for you if each worker keeps a cache of its own.

Short answer

Key each response on a namespace, the tenant or public, the method, and the URL, and store the status, headers, and body bytes under one key with SET and EX. Claim the miss with SET NX EX so one worker fetches while the rest wait for its entry. Store only 2xx responses that carry no Set-Cookie, no private, no Vary other than Accept-Encoding, and no per-caller request header under a shared key.

You will need

Node 22 or later, Python 3 for the second worker, and a Redis you can reach, or the stand-in in the code directory. Verified 2026-09-25 against Node 22.22.2, got 16.0.0, and undici 8.11.2, with the Python worker on Python 3.11. No Redis ran here. redis-standin.mjs is a TCP server that implements the six commands the cache and its demo use. It gives them the arguments and answers Redis documents, and the page says which claims are about the stand-in. The SET command defines the EX and NX options the cache relies on.¹

Approaches compared

The in-process caches are the fastest option and the incoherent one: each worker pays its own miss and forgets everything on restart. Every shared option adds a network hop and a memory budget that all workers draw on. The HTTP-aware caches, got and undici, do the storing decision for you from the origin’s headers, and refuse to store what the headers do not allow. The hand-rolled key does whatever your policy says, which is the whole of its cost. Memcached is the plainer of the two shared stores.²

Key on the tenant, the method, and the URL

The key is the contract between the workers, so it is built by one function they all call. Redis has no namespaces, and its documentation on keys names the colon as the convention for splitting a key into sections, so the key is a colon-separated path.

export function cacheKey({ method = 'GET', url, tenant = null }) {
  const u = new URL(url)
  return `${NAMESPACE}:${tenant ?? 'public'}:${method.toUpperCase()}:${u.host}${u.pathname}${u.search}`
}

// Why a response must not be stored where other callers will read it. Returns
// the reason, or null when the response may be shared.
export function storable(request, response) {
  if (response.status < 200 || response.status > 299) return `status ${response.status}`
  if (response.headers.has('set-cookie')) return 'set-cookie'
  const directives = (response.headers.get('cache-control') ?? '').toLowerCase()
  if (/\bno-store\b/.test(directives)) return 'cache-control: no-store'
  if (/\bprivate\b/.test(directives)) return 'cache-control: private'
  // Vary names request headers that pick the representation, and the key holds
  // none of them. Accept-Encoding is safe: the body is stored decoded.
  const vary = (response.headers.get('vary') ?? '').toLowerCase().split(',').map((name) => name.trim())
  const unkeyed = vary.filter((name) => name && name !== 'accept-encoding')
  if (unkeyed.length) return `vary: ${unkeyed.join(', ')}`
  const sent = new Headers(request.headers ?? {})
  const perCaller = sent.has('authorization') || sent.has('cookie')
  if (perCaller && !request.tenant) return 'per-caller request under a shared key'
  return null
}

resp:v1 is the namespace, and v1 is there for the day the frame changes: bump it and the old entries expire on their own. public stands where a tenant would, so a per-caller entry and a shared one can never collide. The policy returns a reason rather than a boolean, because the reason is what an operator reads in a log.

Three rules in storable are conditions RFC 9111 sets on what a shared cache may store. A response marked no-store or private is refused, and so is the answer to a request that carried Authorization, unless the key is scoped to that caller. Then the store acts as that caller’s private cache. The Set-Cookie rule is stricter than the RFC, which does not forbid caching such a response and says so.³ The 2xx rule is stricter too, and it keeps the one status code the RFCs forbid a cache to store out by construction.⁴

The Vary rule is about reuse, not storage. Section 4.1 forbids answering a request from a stored response, without revalidation, when the headers its Vary names differ. The key holds none of them and the cache does no revalidation, so storable refuses a response with Vary. Accept-Encoding is the exception: it only chooses a content coding, and the body is stored without one.

Pay one miss per key, not one per worker

Forty workers that miss the same key at the same moment send forty requests. A shared store on its own does not stop that, because all forty see the miss before any of them stores. The fix is a claim on the key, made with the NX option, which sets only when the key is absent.

  const lock = `${key}:lock`
  const mine = `${process.pid}-${Math.random().toString(16).slice(2)}`
  const acquired = await store.set(lock, mine, 'EX', lockSeconds, 'NX')
  if (acquired !== 'OK') {
    const until = Date.now() + lockSeconds * 1000
    while (Date.now() < until) {
      await sleep(waitMs)
      const entry = await store.getBuffer(key)
      if (entry) return unpack(entry)
      // The lock is gone. The winner stores before it unlocks, so read once more:
      // no entry now means it finished without storing, and the fetch is ours.
      if ((await store.ttl(lock)) === -2) {
        const stored = await store.getBuffer(key)
        if (stored) return unpack(stored)
        break
      }
    }
  }

EX on the lock is what makes a crashed worker harmless: its claim expires and the next worker takes over. The winner fetches, stores the entry with its own EX, and deletes the lock. The others poll for the entry and read it, so the origin sees one request. The demo runs four Node workers and one Python worker as separate processes against one stand-in, started at once.

node demo.mjs

node 22.22.2, five workers against one stand-in for Redis

shared store, all workers at once
  w1         /rates 2c527902a1b8acdf   /me user=w1 stored=false (set-cookie)
  w2         /rates 2c527902a1b8acdf   /me user=w2 stored=false (set-cookie)
  w3         /rates 2c527902a1b8acdf   /me user=w3 stored=false (set-cookie)
  w4         /rates 2c527902a1b8acdf   /me user=w4 stored=false (set-cookie)
  w5-python  /rates 2c527902a1b8acdf   /me user=w5-python stored=false (set-cookie)
  /rates bodies distinct: 1, served from cache: 4 of 5
  origin requests: /rates 1, /me 5
  keys in the store:
    resp:v1:public:GET:127.0.0.1:44501/rates  ttl 60s

per-process store, all workers at once
  w1         /rates 2c527902a1b8acdf   /me user=w1 stored=false (set-cookie)
  w2         /rates 2c527902a1b8acdf   /me user=w2 stored=false (set-cookie)
  w3         /rates 2c527902a1b8acdf   /me user=w3 stored=false (set-cookie)
  w4         /rates 2c527902a1b8acdf   /me user=w4 stored=false (set-cookie)
  /rates bodies distinct: 1, served from cache: 0 of 4
  origin requests: /rates 4, /me 4

One origin request for five workers, one body hash across two languages, and one key in the store with a 60 second TTL. The same four Node workers with a Map each pay four misses, which is where the page started. The Python worker reads and writes the same frame as the Node ones because the frame is bytes. It is a line of JSON for the status and headers, then the body without its content coding, since fetch decodes a compressed body and the Python worker asks for none. Both drop the headers that described the connection, the coding, and the length. Parse the body before storing it and the next language in the fleet inherits your parser’s opinions.

Keep Set-Cookie out of the shared store

The /me route answers with Set-Cookie and Cache-Control: private, and both fleets refused to store it, five times over. The last two blocks of the demo show what the refusal prevents.

node demo.mjs

the pitfall: store any 2xx under a key with no tenant
  acme  from origin: user=acme set-cookie=session=acme; HttpOnly
  birch from cache: user=acme set-cookie=session=acme; HttpOnly  LEAK

the fix: refuse the response, and key the ones you keep by tenant
  acme  from origin: user=acme stored=false (set-cookie)
  birch from origin: user=birch stored=false (set-cookie)
  keys in the store: 0

Under a policy that stores any 2xx, birch’s first request is answered from the store with acme’s profile and acme’s session cookie. The origin’s response was not wrong: it said private, and the policy did not read it. The fix has two parts. The response is refused because it carries Set-Cookie, and the responses you do keep for a caller go under that caller’s tenant in the key, as the cache keys page works through.

What the HTTP-aware caches do with the same responses

Both library rows make the storing decision from the origin’s headers, and both run here against the same origin, offline.

node libraries.mjs

got 16.0.0 | undici 8.11.2

got, cache: new Map()
  /rates  two calls, origin requests 1, second isFromCache true
  /me     two calls, origin requests 2, second isFromCache false
undici, interceptors.cache with a MemoryCacheStore
  /rates  two calls, origin requests 1
  /me     two calls, origin requests 2

Both stored the public rate table but not the private profile, which is what storable decides from the headers alone. The cache documentation for got accepts anything with the Map interface, and its Redis path is @keyv/redis, whose adapter page documents a namespace and a separator. undici’s cache store selects an entry by the stored Vary map, and a Redis-backed store is three methods away. The Python row, requests-cache, takes backend='redis' and uses the cache name as the namespace. Each stores a frame of its own, which is the case for the hand-rolled key once languages mix.

Check it worked

Fifteen tests run, most of them against the stand-in, and the first is the one the page promises: six callers at once, one origin request, and bytes that equal a direct fetch.

node --test cache.test.mjs

1..15
# tests 15
# suites 0
# pass 15
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 2770.097014

One asserts a failure on purpose. It stores any 2xx under a tenant-free key and asserts that birch receives acme, so the leak cannot leave the demo by accident. Another runs the Python worker and checks that its stored frame has the same header names and body bytes as a Node worker’s. A third checks that the lock key is gone once the entry is stored, because a lock that outlives its miss serializes every reader for lockSeconds. Two more check that both languages refuse a response that varies on Accept-Language or on *.

When it goes wrong

Two workers both fetch the same key at the same moment. The claim is made with SET and NX in one command, and a client that does GET then SET in two has a gap between them. Keep the claim atomic, and keep EX on it.

Every request waits lockSeconds and then fetches anyway. The winner’s response could not be stored, so the waiters polled until the lock expired. The waiting loop breaks as soon as the lock is gone, which is why the winner deletes it in finally rather than leaving it to expire. The winner stores before it unlocks, so a waiter that finds the lock gone reads the key once more before it fetches.

A stored entry outlives the data it describes. The TTL is the only invalidation this page has. Set it from the origin’s max-age when there is one, and delete the key on the write path when you own both sides.

When not to do this

Do not share a cache between workers that need to disagree. A canary worker that reads the rate table from the same key as the fleet reads the fleet’s version, and the namespace in the key is where that distinction belongs.

Do not put a Redis hop in front of a response that costs less than the hop. A per-process lru-cache or a TTLCache from cachetools, as its documentation describes it, answers in microseconds, and a shared store answers in the time it takes to cross the network.

Do not store a response because it was a 200. The status says whether the request worked, not whether the answer belongs to the next caller. Read the headers, and when the headers are silent, treat a per-caller request as private.

Do not treat the stand-in as Redis. It implements six commands and one process. It has no persistence, no eviction, and no replication, and the persistence options Redis documents are exactly what a cache that survives a restart depends on.

Related how-tos

• Build cache keys that include the caller and Vary headers

• Retry fetch calls with exponential backoff in Node.js

Last verified

Verified 2026-09-25 against Node 22.22.2, got 16.0.0, and undici 8.11.2, with the Python worker on Python 3.11. Every output block is what the command preceding it printed. No Redis ran: every command went to redis-standin.mjs, which implements SET with EX and NX, GET, DEL, TTL, KEYS, and FLUSHALL over TCP with the answers Redis documents for them.