How to handle webhooks that arrive out of order#

A customer.subscription.updated arrives and your handler writes the plan it carries. The created event that should have inserted the row lands a minute later and overwrites it with the state from before the update. Three days after that, a retry of an earlier event does it again. The database holds whatever the sender said last, which is not the sender’s current state.

What you get

You will end up with a handler that lands on the sender’s current state whatever order deliveries take, and a test that feeds it the wrong order and a late retry. This is for you if your handler trusts whichever payload arrived last.

Short answer

Treat each delivery as a hint and fetch the object from the sender’s API before you write: one call per event, and immune to ordering. Or store the version the sender puts on the object and apply an event only when its version is higher; Stripe’s created is in seconds and cannot do that job. An SQS FIFO queue keyed by object id serializes workers per object and keeps arrival order, so it fixes concurrency, not the sender’s order.

You will need

Node 22.18 or later, which runs the TypeScript files here directly. A sender whose events name an object you can read back, or that stamps a version on the object; the sample’s Sender does both and stands in for Stripe. Verified 2026-10-21 against Node 22.22.0. No queue service and no sender API were called. fifo.ts models the message groups of an SQS FIFO queue in memory, and the Stripe and SQS behavior this page quotes was read from their documentation.

Approaches compared

Fetching on receipt costs a round trip per event, metered by Stripe against your rate limit and read allowance, and in return the handler never reasons about order. A version compare costs a column in your store and spares the wire entirely, but it needs a field the sender promises will only grow. The FIFO queue costs an AWS resource with a per-partition request cap, and buys serialization per object rather than ordering: it replays arrival order faithfully, stale retry included.

Fetch the object the event names

The payload is where the object was when the event was generated. Stripe’s documentation says so of its snapshot events: the snapshot can be stale by the time you process it, so fetch the current version before acting. Its thin events carry only the id and type and are built around that fetch. fetchOnReceipt does the same with a snapshot: it reads the id, asks the API, and writes what comes back.

export function fetchOnReceipt(
  event: Event, api: Pick<Sender, 'retrieve'>, store: Store, calls: { count: number },
): 'fetched' | 'gone' {
  calls.count++
  const current = api.retrieve(event.object.id)
  if (!current) {
    store.delete(event.object.id)
    return 'gone'
  }
  store.set(current.id, rowOf(current, event.created))
  return 'fetched'
}

Order cannot hurt this handler, because it never uses the state in the payload. Two handlers fetching one object at once can still write in the wrong order, so give each object one handler at a time, as the queue below does.

What it costs is on the wire. Stripe’s rate limits cap each endpoint at twenty-five requests a second, and every subscription you fetch counts against the same one; reads are also metered against your transaction count.¹ A burst of events, such as every subscription renewing on the first of the month, becomes the same burst of reads. An object the API no longer has comes back as gone, and the row goes with it.

Compare a version before you write

When the payload carries the whole object and the sender stamps it with something that only grows, the handler can decide locally. applyIfNewer keeps the last applied value per object and skips anything earlier or equal.

export function applyIfNewer(
  event: Event, store: Store, { key = 'version', onTie = 'skip' }: { key?: 'version' | 'created'; onTie?: 'skip' | 'apply' } = {},
): 'applied' | 'skipped' {
  const have = store.get(event.object.id)
  const incoming = key === 'version' ? event.object.version : event.created
  if (have) {
    if (incoming < have[key]) return 'skipped'
    if (incoming === have[key] && onTie === 'skip') return 'skipped'
  }
  store.set(event.object.id, rowOf(event.object, event.created))
  return 'applied'
}

The field is the whole question. The sample’s sender increments version on every change. Stripe records an event’s created in whole seconds, and its event ordering guidance says distinct events can share a timestamp and that you should not use created to determine order.² The key and onTie parameters exist so the pitfall run can show what happens when you try anyway.

In a database, make the compare part of the write, an UPDATE whose WHERE clause requires a lower stored version, so two deliveries handled at once cannot both pass it.

Serialize one object per worker with a message group

An SQS FIFO queue takes a MessageGroupId on every message. AWS documents that messages in one group are processed one at a time, in strict order. While a message of a group is in flight, no other consumer receives from that group until it is deleted or its visibility timeout expires. With the object id as the group, two workers can never race on one subscription.

  // The oldest message whose group has nothing in flight. A consumer that
  // asks while another holds a message of the same group gets a different
  // group, or nothing.
  receive(): Message<T> | undefined {
    const message = this.messages.find((m) => !this.inFlight.has(m.groupId))
    if (message) this.inFlight.add(message.groupId)
    return message
  }

The order a group keeps is the order you sent, which for a webhook receiver is arrival order. The queue has no view of the sender’s history, so a stale retry sent to it today is processed today. demo.ts runs one subscription history, delivered in the wrong order with a retry three days late, through every approach on this page, then through a two-worker drain of the model queue.

node demo.ts

truth at the API    sub_1 active pro v3 | sub_2 canceled basic v2
arrival order       evt_003 sub_1 v3, evt_001 sub_1 v1, evt_004 sub_2 v1, evt_002 sub_1 v2, evt_005 sub_2 v2, evt_002 sub_1 v2 (retry, three days later)

apply in arrival order  sub_1 active basic v2 | sub_2 canceled basic v2   WRONG, api calls 0
fetch on receipt        sub_1 active pro v3 | sub_2 canceled basic v2    matches the API, api calls 6
version compare         sub_1 active pro v3 | sub_2 canceled basic v2    matches the API, api calls 0, skipped evt_001 evt_002 evt_002

FIFO queue, MessageGroupId = subscription id, two workers
  step 1  A: evt_003 sub_1 v3   B: evt_004 sub_2 v1
  step 2  A: evt_001 sub_1 v1   B: evt_005 sub_2 v2
  step 3  A: evt_002 sub_1 v2   B: nothing free
  step 4  A: evt_002 sub_1 v2   B: nothing free
  payload applied in queue order   sub_1 active basic v2 | sub_2 canceled basic v2   WRONG
  applyIfNewer inside the worker   sub_1 active pro v3 | sub_2 canceled basic v2    matches the API

Arrival order ends with sub_1 on the basic plan, because the stale evt_002 retry was the last word. Fetching lands on the API’s state at six calls. The version compare lands there at none and names the three deliveries it refused. The queue trace shows the group at work: in step one worker B skips sub_1, which worker A holds, and takes sub_2; in steps three and four B finds nothing free. Applying payloads in that order repeats the arrival-order mistake; applyIfNewer inside the worker fixes it, which is the division of labor between the two mechanisms.

A queue of this kind also costs capacity. AWS caps a FIFO partition at three hundred requests a second per API action without batching, three thousand messages a second with it, and more in high throughput mode. Its duplicate detection spans five minutes, which the sender’s retries outlast.³

Check it worked

order.test.ts pins each claim in the steps. It covers the stale end state of arrival order, the fetch at one call per event, the version compare with its three skips, the group lock, and the late retry. The queue test matters most, because it is the claim a reader is most likely to assume the other way.

test('the queue preserves arrival order, so a stale retry still wins unless the worker compares versions', () => {
  const { sender, delivered } = scenario()
  const queue = new FifoQueue<Event>()
  for (const e of delivered) queue.send(e.object.id, e)
  const arrival: Store = new Map()
  const versioned: Store = new Map()
  const trace = drainWithTwoWorkers(queue, (e) => { applyInArrivalOrder(e, arrival); applyIfNewer(e, versioned) }, label)
  assert.equal(trace.length, 4)
  assert.equal(arrival.get('sub_1')!.version, 2)
  assert.equal(describe(versioned, ['sub_1', 'sub_2']), truthOf(sender, ['sub_1', 'sub_2']))
})

node --test order.test.ts

1..8
# tests 8
# suites 0
# pass 8
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 186.37272

Eight tests pass. describe(versioned, ...) equal to truthOf(sender, ...) is the assertion the whole page rests on: the store reads the same as the sender’s API after the wrong order.

When it goes wrong

The store holds an earlier state than the sender, and the handler compared timestamps. pitfall.ts feeds two events generated in the same second to applyIfNewer with key: 'created', in both arrival orders and with both tie rules.

node pitfall.ts

1. two events in one second: evt_001 v1 and evt_002 v2, both created=1770000000
   arrival v1 then v2, compare created, tie skip  -> sub_1 incomplete basic v1   WRONG
   arrival v1 then v2, compare created, tie apply -> sub_1 active basic v2   ok
   arrival v1 then v2, compare version          -> sub_1 active basic v2   ok
   arrival v2 then v1, compare created, tie skip  -> sub_1 active basic v2   ok
   arrival v2 then v1, compare created, tie apply -> sub_1 incomplete basic v1   WRONG
   arrival v2 then v1, compare version          -> sub_1 active basic v2   ok

2. a retry of evt_002 (v2) arrives 3 days after evt_003 (v3) was applied
   apply in arrival order -> sub_1 active basic v2   WRONG, the retry was applied
   version compare        -> sub_1 active pro v3   ok, the retry was skipped

Skipping ties loses the second update when the events arrive in order; applying ties lets the earlier one win when they arrive reversed. No rule for equal timestamps is right in both orders, so a field in seconds is not a version. Ask the sender for one, or fetch.

A subscription goes back to a plan the customer left days ago. A retry carries the original payload and the original created, and arrived after the events it predates. Stripe retries for up to three days in live mode, and a manual resend reaches further. The second half of the pitfall run shows arrival order taking the retry and the version compare skipping it. Keep the last applied version for every object you store, with no expiry.

Two workers write the same object at once and the later write loses a field. The queue is a standard one, or the group id is something other than the object id. Set MessageGroupId to the object id on every message, and check that the queue is FIFO, because a standard queue uses the group only for fair queuing.

When not to do this

Do not fetch on receipt for an event that is a fact rather than a state. A payment that succeeded or a refund that was issued happened once; the API’s current object has moved on and the fetch tells you nothing about the moment the event records. Apply those once by event id, and keep the state mechanisms for objects.

Do not treat created as a version because it is usually right. Two changes in one second are common, and the pitfall run shows that no tie rule saves you.

Do not add a FIFO queue to fix the sender’s order. It fixes your workers’ concurrency, and it replays arrival order with every stale retry in place, so a version compare or a fetch still belongs inside the worker.

Do not rely on the queue’s duplicate detection to absorb retries. Its window is five minutes, while the sender’s retry schedule runs for days; a store of the delivery ids you have processed is what recognizes a retry.

Related how-tos

• Acknowledge a webhook fast and process it from a queue

• Make queue consumer writes safe under redelivery

Last verified

Verified 2026-10-21 against Node 22.22.0. Every output block is what the command preceding it printed. The sender, its API, and the FIFO queue are in-memory models in the sample’s own files, written to the documented behavior of Stripe events and SQS message groups. The run called no network service, and the limits and retry schedules quoted were read from the two vendors’ documentation.