When one customer’s key spends its quota, your API answers 503, which the load balancer reads as a sick backend. It pulls the instance, so every other customer loses capacity because one key ran hot. Elsewhere your 429 responses carry no Retry-After, and an agent that hits one retries in a tight loop until its budget is gone.
What you get
You will end up with a throttle that returns 429 for a spent quota and 503 for a full server, each with Retry-After, tested against a load balancer. This is for you if your API refuses requests to protect capacity.
Short answer
Return 429 Too Many Requests when one caller has spent its own quota, and 503 Service Unavailable
when the server is out of capacity whoever is calling. Send Retry-After on both, as seconds rather
than a date, with a problem details body that names the policy. Check the quota first and charge
it only for admitted requests, so a 503 never costs a caller part of their allowance.
You will need
Node 22 or later, and an HTTP API with at least one endpoint you can throttle. Verified 2026-10-15 against Node 22.22.0. The sample keeps its counters in memory so the decision is visible. Where the counters live, and how a key and a tenant share a budget, is the subject of rate limiting by API key and tenant, linked at the end.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| 429 Too Many Requests (RFC 6585) | One caller has spent its own quota, and every other caller is unaffected | Outlier detectors count 5xx answers, so a 429 never sheds load from a full server | The server itself is short of capacity |
| 503 Service Unavailable (RFC 9110) | The server is overloaded or in maintenance, whoever is calling | Load balancers eject on it, and CloudFront caches it for ten seconds by default | Only one caller is over a limit |
| Retry-After as an HTTP-date | A refusal with a known end, such as a maintenance window | The client’s clock sets the wait, so a skewed clock waits the wrong time | The wait is a duration you computed |
| Retry-After as delta-seconds | Any throttling refusal, where the wait is computed per request | Counts from when the response arrives, so time spent in transit is added to it | The refusal ends at a fixed moment on the calendar |
A 429 is addressed to one caller, and the infrastructure between you and that caller lets it through without drawing conclusions. A 503 is addressed to everyone, including the load balancer, which is the point when the server is full and the damage when it is not. Seconds survive clock skew and drift by the transit time. A date survives a slow proxy but fails on any client whose clock is wrong.
Decide by whose limit was reached
The status follows from whose limit ran out. The throttle checks the caller’s quota first, then the server’s capacity, and charges the quota only once both have passed.
if (window.used >= quota.limit) {
const retryAfter = Math.ceil((window.resetAt - now) / 1000)
return refuse(429, retryAfter, {
type: 'https://api.example.com/problems/quota-exceeded',
title: 'Your key has used its quota for this window',
policy: 'key-quota',
detail: `${window.used} of ${quota.limit} requests used in a ${quota.windowSeconds} s window.`,
})
}
if (inFlight >= maxInFlight) {
// Two to five seconds, spread so refused callers do not all return at once.
const retryAfter = 2 + Math.floor(random() * 4)
return refuse(503, retryAfter, {
type: 'https://api.example.com/problems/server-overloaded',
title: 'The service is at capacity',
policy: 'server-capacity',
detail: 'Retry after the delay in Retry-After. Your quota was not charged.',
})
}The order matters. A key that has spent its quota gets 429 even while the server is full, because the caller would be refused on a quiet day too. The 429’s wait is exact, the time until the window resets. The 503’s wait is short and spread, because a fixed value sends every refused caller back in the same second.
Both bodies follow RFC 9457, the problem details
format, with a policy member an agent can branch on. The 503 detail names no queue depth or
limit, since those tell a caller how hard to push. The
RateLimit header fields draft
can carry the quota on every response as well, which spares a client from discovering the limit by
hitting it.
Run both scenarios behind a load balancer
The sample has no Envoy or ALB, so balancer.mjs models the two ways one judges a backend. The
passive half ejects a host after five consecutive 5xx answers, the default of
Envoy’s outlier detection.
The active half probes /health and counts 200 to 399 as success, the rule a
Kubernetes HTTP probe
applies. /health is outside the quota and fails only when the server is full.
node demo.mjsscenario 1: key_a goes past its quota of 5 a minute, key_b stays under it
key_a 200 200 200 200 200 429 429 Retry-After: 60, 60
key_b 200 200
balancer: 0 consecutive 5xx, ejected false; probes 200 200 200, ready true
{
"type": "https://api.example.com/problems/quota-exceeded",
"title": "Your key has used its quota for this window",
"policy": "key-quota",
"detail": "5 of 5 requests used in a 60 s window.",
"status": 429,
"retry_after": 60
}
scenario 2: three slow requests fill the server, then key_a and key_b call
key_a 503 503 503 Retry-After: 4, 2, 5
key_b 503 503 Retry-After: 3, 4
balancer: 5 consecutive 5xx, ejected true; probes 503 503 503, ready false
{
"type": "https://api.example.com/problems/server-overloaded",
"title": "The service is at capacity",
"policy": "server-capacity",
"detail": "Retry after the delay in Retry-After. Your quota was not charged.",
"status": 503,
"retry_after": 4
}
key_a, after the load clears 200 200 200 200 200In the first scenario only key_a is refused, and the balancer keeps the host. In the second,
both keys are refused, the passive detector ejects the host after the fifth 503, and the probe
marks it unready. The last line shows the quota untouched: key_a was refused three times, all for
capacity, and still has five requests left.
Send seconds, not a date
Both forms are legal, and they fail differently. A date is read against the client’s clock; seconds are counted from when the response arrives.
node demo.mjsRetry-After for a 60 s wait, read by a client whose clock runs 30 s fast
seconds 60 client waits 60 s
HTTP-date Thu, 15 Oct 2026 12:01:00 GMT client waits 30 sThe client with a fast clock waits half as long, so it comes back while the window is still closed and is refused again. Use the date form for a refusal that ends at a moment you know in advance, such as a maintenance window, and seconds for everything a throttle computes.1
Check it worked
The test runs both scenarios. It asserts that only the spent key sees 429, that every refusal
carries Retry-After and a policy, and that the balancer ejects the host on 503 responses alone.
It also checks that a Retry-After in neither form, such as 1.5, parses as absent rather than as a
wait of zero. Another sends a flood of distinct keys and checks that the key map is walked once per window, not
on every request.
node --test throttle.test.mjs1..8
# tests 8
# suites 0
# pass 8
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 252.931702Against your own stack, send a burst on one key past its quota and confirm the target health in your load balancer does not change. Then saturate the service and confirm it does.
When it goes wrong
An agent hammers the API after its first refusal. The 429 carries no Retry-After, and a client
that does what the response says has nothing to wait for.
node demo.mjsan agent retrying on its own after its quota is spent, ten attempts at most
without Retry-After 10 attempts in 0 s, last status 429
with Retry-After 2 attempts in 60 s, last status 200The agent sent ten requests in no time at all, had every one refused, and gave up a minute before its quota would have let it through. RFC 6585 makes the header optional, so a client that retries at once when it is missing is following the response, not breaking a rule. Send it every time.
The container restarts under load. The overload check was wired to the liveness probe, and a Kubernetes liveness failure kills the container, which removes capacity at the moment it is short. Report overload on readiness only.
Every instance is full and the load balancer sends traffic to all of them anyway. The AWS Elastic Load Balancing documentation says its load balancer fails open when every target in the group fails its health check.2 A 503 can move load only onto instances that have room, so a fleet that is full everywhere needs more capacity, not a different status code.
When not to do this
Do not answer 503 for a per-caller limit to sound less blunt. It tells every proxy, balancer, and CDN that the service is unhealthy. CloudFront caches a 503 for ten seconds by default and serves it to every caller of that URL. RFC 6585 forbids caches to store a 429 at all.
Do not answer 429 when the server is overloaded. The caller did nothing wrong, and a balancer that cannot see the overload keeps sending traffic to the instance that is refusing it.
Do not send 503 without a body or with an HTML error page. An agent needs policy and
retry_after to choose between waiting and failing, and a gateway’s default page carries
neither.3
Related how-tos
Last verified
Verified 2026-10-15 against Node 22.22.0. Every output block is what the command preceding it printed. The load balancer is a model with Envoy’s consecutive 5xx rule and a Kubernetes-style probe, not Envoy or Kubernetes; their behavior is quoted from their documentation.
Footnotes
-
The grammar in RFC 9110 calls the seconds form
delay-seconds, and its example of the date form isFri, 31 Dec 1999 23:59:59 GMT. The specification was published in June 2022, so a client that obeys its own example waits no time at all. ↩︎ Back to text -
The fail-open takes three sentences of the health check documentation, placed between how a health check connection is closed and the fact that health checks do not support WebSockets. The same page sets the default success code at 200 alone, so a 429 on a throttled health check path reads as unhealthy as surely as a 503. ↩︎ Back to text
-
RFC 9110 notes that the 503 status code does not oblige an overloaded server to use it, and that some servers simply refuse the connection instead. A refused connection has no status, no body, and no Retry-After, which makes it the one refusal with less to say than a bare 429. ↩︎ Back to text