How to decide which layer retries a failed vendor call#

Stripe answers 503 for a minute, and your vendor log shows 48 customer creates for one order. stripe-node retries twice inside each call, the p-retry wrapper in your provider retries every call three times, and JetStream redelivers the whole message after each nak. When a worker dies after a create that succeeded, the redelivery creates a second customer under a fresh idempotency key.

What you get

You will end up with one retry owner per failure class, across network errors, 429 responses, 5xx, and handler crashes, and a fake vendor that counts calls to prove it. This is for you if your provider sits between a message bus and a vendor SDK.

Short answer

Give each failure class one retry owner among stripe-node, a p-retry provider, and NATS JetStream, and switch the other two off for it. Let maxNetworkRetries in stripe-node own network errors and 5xx, since it reuses one idempotency key across its attempts. Let p-retry own 429 responses, within a shared retry budget. Let JetStream redeliver only after a crash, and end the message with term after any vendor failure. Derive the idempotency key from the message.

You will need

Node 22.18 or later, which runs TypeScript files without a build step, plus stripe-node and p-retry. A JetStream consumer whose ack_wait and max_deliver you control, and an idempotency key for vendor writes. Verified 2026-10-21 against Node 22.22.0, stripe 23.0.0, and p-retry 8.0.1. The sample calls a local fake of the Stripe API in place of Stripe, and drives an in-process model of a JetStream consumer in place of a NATS server. Turning vendor errors into outcomes a caller can branch on is the subject of mapping vendor error responses onto message errors.

Approaches compared

The three layers differ in what they know. stripe-node knows Stripe’s errors and keeps one key across its attempts, but a single setting covers network errors and 5xx together. The provider sees every call to the vendor, so it can hold a budget for the whole process, at the price of writing the classification yourself. JetStream outlives a dead worker, and pays for that by re-running everything the handler does.

Count what one failure costs

The stacked setup in stacked.ts retries at every layer. stripe-node keeps its default of two retries,¹ a p-retry wrapper retries any error three times, and the handler answers every failure with a bare nak, which JetStream redelivers at once. JetStream’s own default for max_deliver is -1, no limit at all, so the harness sets it to four to make the count finite.

vendor.ts is a fake of Stripe’s customer endpoint that fails on demand and counts every call. bus.ts models a JetStream consumer from its documented rules: an unanswered delivery comes back after ack_wait, nak comes back at once, and term ends it.

node calls.ts stacked

every layer retries: stripe-node 2 retries, p-retry 3, JetStream max_deliver 4

failure                           vendor calls  customers created
connection reset on every call    48            0
429 on every call                 16            0
503 on every call                 48            0
worker crashes once after create  2             2

The counts multiply. A reset or a 503 costs three attempts inside stripe-node, four calls from p-retry, and four deliveries from JetStream, which is 48.² A 429 costs 16, because stripe-node never retries a 429 unless Stripe’s Stripe-Should-Retry header asks it to. The last row is the expensive one. Stripe created the customer, the worker died before its ack, and the redelivered message created a second customer under a new key.

Give each failure class one owner

Give each class to the layer that knows most about it, and switch it off everywhere else. Network errors and 5xx go to stripe-node, which reuses the first attempt’s idempotency key and obeys Stripe-Should-Retry. 429 responses go to the provider, because stripe-node leaves them alone and Stripe’s rate limit documentation tells you to retry them with backoff and jitter. Crashes go to JetStream, the only layer that outlives the process.

// The provider owns 429 and nothing else. stripe-node has already retried network errors
// and 5xx inside the call, so a retry here would multiply its attempts.
export function customersProvider(stripe: Stripe, { budget = new RetryBudget(), minTimeout = 500 }: Options = {}) {
  return {
    async create(params: { email: string }, idempotencyKey: string) {
      const customer = await pRetry(() => stripe.customers.create(params, { idempotencyKey }), {
        retries: 3,
        minTimeout,
        randomize: true,
        onFailedAttempt: ({ error }) => { if (isRateLimit(error)) budget.spend() },
        shouldRetry: ({ error }) => isRateLimit(error) && budget.allows(),
      })
      budget.earn()
      return customer
    },
  }
}

shouldRetry is the switch. It answers yes only for a 429 and only while the budget allows, so a 503 that reaches the provider goes straight back to the handler. The key is passed in rather than left to stripe-node, which mints a new one for every call it makes, and a p-retry attempt is a new call.

The budget caps the process, not the call. RetryBudget is a token bucket: each 429 spends a token, each success earns a tenth of one back, and retries stop while half the bucket is spent.³

node storm.ts

20 orders while Stripe answers 429 to every call
  per-call cap only            80 vendor calls
  per-call cap and budget      23 vendor calls

With the per-call cap alone, every order makes four calls to a vendor that is already refusing them. With the budget, the first order uses its three retries and the other nineteen give up after one call each.

Settle vendor failures before the bus sees them

The handler is where JetStream gets switched off for everything but a crash. It derives the key from the order, records a vendor failure, and answers it with term, which tells the server never to deliver that message again.

export function onOrder(customers: Customers, store: Store) {
  return async (msg: Msg) => {
    const order = msg.json<Order>()
    // One key per order and operation, never per attempt or per delivery. A redelivered
    // message sends the key the first delivery sent, so Stripe replays the first answer.
    const key = `order-${order.id}-create-customer`
    let customer
    try {
      customer = await customers.create({ email: order.email }, key)
    } catch (err) {
      // stripe-node and the provider have spent their retries. A redelivery repeats them all.
      await store.failed(order.id, (err as Error).message)
      return msg.term('vendor call failed')
    }
    // A crash between the create and the ack leaves the message unanswered, so it comes back.
    await store.save(order.id, customer.id)
    msg.ack()
  }
}

A crash sends no answer at all, so it is the one case that still reaches redelivery. The message returns after ack_wait and runs the create again with the same key.

node calls.ts owners

one owner per class: stripe-node for network and 5xx, p-retry for 429, JetStream for crashes

failure                           vendor calls  customers created
connection reset on every call    3             0
429 on every call                 4             0
503 on every call                 3             0
worker crashes once after create  2             1

Every class now costs what its one owner allows: three attempts from stripe-node, four from p-retry, two deliveries for the crash. The second create in the crash row carried the first one’s key, so the fake replayed cus_1 instead of creating cus_2, as Stripe does when it answers with Idempotent-Replayed: true.

Check it worked

Run the suite. It asserts the counts for both setups, the budget’s 23 calls, and the replay after a crash. Two more tests pin behavior of the tools themselves: stripe-node retries a closed connection once even at maxNetworkRetries: 0, and a slow handler gets a second delivery unless it calls working().

node --test retries.test.ts

1..7
# tests 7
# suites 0
# pass 7
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 1903.093148

# pass 7 with # fail 0 is the result to expect.

When it goes wrong

A customer appears twice after a worker restart. The redelivered message ran the create with a key stripe-node minted for that call, so Stripe saw a new request. Derive the key from the order and the operation, as Stripe’s guidance on handling errors at the HTTP level suggests with the id of a shopping cart.

node pitfall.ts

key minted by stripe-node for each call
  call 1  stripe-node-retry-1036b498-2848-4597-b7d8-1761feaeaedd  created cus_1
  call 2  stripe-node-retry-fec5a937-4e53-44eb-96e8-78d819a7370f  created cus_2
  customers created: 2
key derived from the order
  call 1  order-1001-create-customer  created cus_1
  call 2  order-1001-create-customer  replayed cus_1
  customers created: 1

A second worker runs the same message while the first is still retrying. stripe-node’s defaults allow one call three attempts of up to 80 seconds each, over four minutes in all, against a default ack_wait of 30 seconds. Lower stripe-node’s timeout, raise ack_wait past the worst case, or call msg.working() while the handler waits.

Setting maxNetworkRetries: 0 does not switch stripe-node off. A closed connection is still retried once, and its README says so in a note. Count two calls per reset when you size a budget.

When not to do this

Do not add a retry layer to fix a count you have not measured. If stripe-node is the only thing calling Stripe and nothing redelivers, its retries are the whole story, and a p-retry wrapper adds a classification to maintain for no gain.

Do not give 429 responses to the provider when the vendor asks for long waits. A wait longer than ack_wait belongs to the bus, where a nak with a delay frees the worker while it waits. The provider must then stop retrying them altogether.

Do not switch JetStream redelivery off. A worker that dies mid-message has no other way back, and with term on every vendor failure, redelivery only ever answers a crash.

Do not copy this split to a vendor whose SDK does not retry. The provider then owns network errors and 5xx as well, under the same rule: one owner per class, and a key that survives every attempt. If the vendor accepts no idempotency key, retry its reads only, because a write that timed out may already have landed.

Related how-tos

• Make queue consumer writes safe under redelivery

• Audit SDK retry and timeout defaults before adoption

• Enforce one total deadline across all retry attempts

• Migrate a hand-written vendor client to a provider plugin

• Map vendor error responses onto message errors

Last verified

Verified 2026-10-21 against Node 22.22.0, stripe 23.0.0, and p-retry 8.0.1. Every output block is what the command preceding it printed, run in the page’s code directory after npm ci. The vendor is a local fake of Stripe’s customer endpoint, so no request reached Stripe. JetStream is modelled in process from its documented consumer rules, so no NATS server ran. The harness shortens stripe-node’s sleep between retries to keep the run short, which changes when its retries happen but not how many there are.