Add a circuit breaker to an outbound HTTP client
Stop calling a failing dependency, try one request after a cool-off, and close the circuit only if it succeeds, so an outage costs one timeout not thousands.
How-to › Section 2
Make calls that survive failure#
Timeouts, retries, backoff, rate limits, idempotency, caching and error handling, on the client and server side of any API. These branches own the behavior; an SDK feature that implements it is one option here.
14 guides in 5 topics. Every guide compares its approaches and shows the output its commands printed.
- Set retries and timeouts 4
- Set and respect rate limits 1
- Make writes safe to retry 1
- Cache API responses 2
- Handle and design API errors 6
Set retries and timeouts 4 guides
Deadline and retry policy for outbound calls: connect versus read timeouts, exponential backoff with jitter, retry budgets, which status codes and transport errors are retryable, retrying streaming and non-idempotent calls, circuit breakers, and adding the behavior to an SDK you ship.
Enforce one total deadline across all retry attempts
Create the deadline once before the first attempt and compose it with each per-attempt timeout, so retries and their waits come out of one budget.
Retry fetch calls with exponential backoff in Node.js
Retry only the statuses the server meant as temporary, back off with full jitter, and honor Retry-After, so a shared outage does not become a stampede.
Time out fetch calls with AbortSignal in Node.js
Put a hard deadline on every fetch call with AbortSignal.timeout, merge it with a caller's own signal, and tell a deadline apart from a cancellation.
Set and respect rate limits 1 guide
Both sides of a quota: publishing RateLimit header fields (IETF draft) and Retry-After, 429 versus 503, per-key and per-tenant limits, burst allowances; and on the client, reading the headers, client-side token buckets, shared limiters in Redis for many workers, concurrency limits, queueing for bursty agent traffic.
Rate limit by API key and tenant, not client IP
Enforce one limit per API key and a larger one per tenant, so a noisy key cannot spend a customer's whole allowance and one customer cannot spend yours.
Make writes safe to retry 1 guide
Making a retried write safe on both sides: the Idempotency-Key header (IETF draft), key generation and storage, fingerprinting request bodies, server-side replay windows, conflicts on mismatched payloads, and at-least-once delivery semantics.
Make queue consumer writes safe under redelivery
Derive an idempotency key from the message rather than from the attempt, so a broker that delivers at least once cannot charge a customer twice.
Cache API responses 2 guides
Client and server caching: ETag and If-None-Match, Cache-Control, stale-while-revalidate, TTL caches, cache keys that include auth, invalidation on write, CDN caching of API responses.
Build cache keys that include the caller and Vary headers
Stop a shared cache serving one tenant's response to another: key on the caller, the negotiated representation, and the query parameters that matter.
Share an API response cache across many workers with Redis
Move a per-process response cache into Redis so forty workers pay one miss per key, with a lock on the miss and a policy that keeps cookies out of the shared store.
Handle and design API errors 6 guides
Errors a client and an agent can act on, from both sides: RFC 9457 problem details, stable error codes, field-level validation errors, retryable versus terminal taxonomies, partial failures in batches, mapping HTTP status to typed errors per language, and fail-safe semantics for autonomous callers (no vague 500s, 429 always with Retry-After).
Design error responses an autonomous agent can act on
Give an LLM caller a stable code, the parameter at fault, a next step, and a structured delay, so it fixes, waits, or stops instead of retrying a 500 forever.
Evolve an error contract without breaking clients
Add codes, retire codes and reword messages while clients in production keep working, with a catalog diff that fails the build on the changes that break them.
Report partial failure in a batch endpoint
Return one outcome per item when a batch of writes half succeeds, name each failed item by the id the client sent, and never answer 200 when nothing was written.
Return validation errors a client can map to a form
Answer a bad request with an RFC 9457 problem document carrying one JSON Pointer entry per failing field, so a client can attach each message to an input.
Translate upstream API errors behind your own API
Decide what your API returns when a provider it depends on fails, without leaking the provider's status codes, messages, or credential problems to callers.
Verify every error response matches one schema
Prove that no endpoint can return an error body outside your problem schema, using a lint over the description and a runtime check against the running service.
Generate the client instead of writing it#
Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.