How to wrap an SDK transport in a middleware chain#

Retry logic sits in three operations, the auth header is set in five, and the caching somebody added last quarter lives in a decorator on two of them. A new operation ships without the retry. Nobody notices until a vendor spends an afternoon refusing requests and one code path gives up on the first attempt.

What you get

You will end up with each concern written once, as a layer around a single transport. Adding an operation gets the whole stack for free, and the order of the stack is one readable line. This is for you if the same cross-cutting logic is copied across your client.

Short answer

Write each concern as a function that takes the next call and returns a call, then compose them outermost first so the declaration order is the order a request travels. A layer may change the request, inspect the response, call the next layer more than once, or answer without calling it at all. Nesting order decides what a retry counts and what a cache hit skips.

You will need

Python 3.11 or later, and an SDK or client whose transport you can wrap. The shape is the same one WSGI uses, and the same one the Guzzle handler stack uses in PHP.¹ The idea transfers to whichever language your client is written in, and so does the ordering argument below.

Approaches compared

The difference is whether the ordering is yours. A library stack gives you tested behavior and decides for you where retry sits relative to everything else.² A chain you compose makes that a line in your code, and it makes the wrong order equally easy to write. Take the library stack when it covers the concerns you have, and compose when it does not.

Compose outermost first

One function builds the chain, and the order reads the way a request travels.

def chain(transport, *layers):
    """Compose layers around a transport. The first layer is the outermost."""
    call = transport
    for layer in reversed(layers):
        call = layer(call)
    return call

Every layer has the same shape: take the next call, return a call. That uniformity is what lets you reorder the stack without editing a layer, and it is why a layer can do things a hook cannot. A retry layer calls the next one several times. A cache layer sometimes calls it not at all. A hook that fires before a request and another that fires after cannot express either of those, which is why a hook API turns into a chain as soon as retry arrives.

Put the cache outside the retry

Nesting is not a style question, it changes what happens.

call = chain(transport, trace(log), cache(store, log), retry(3, log=log), auth("sk_live_1"))

With the cache outside, a hit skips the retry layer entirely and costs nothing. Flip the two, and every cache hit still walks through the retry machinery, and a retried request writes its intermediate failures past a cache that is now inside the loop.

Auth sits innermost, next to the transport. A retried attempt then gets a header applied again, which matters the moment that header is a signature over the request or a token that can expire between attempts.

Both layers ask the method before they act. A 503 can arrive after the server already processed the request, so a retried POST is how one charge becomes two. A cache keyed on method and path is no safer: it would hand the first response to a second POST carrying a different body, and that second write would never leave the client. Retry the idempotent methods, cache the safe ones, and let a write through untouched.³ Add POST to the retry list only for an API that takes an idempotency key.

Check it worked

Run the chain against a transport that fails twice, then succeeds, then gets asked the same question again.

python3 demo.py

order: trace cache retry auth transport
  -> /meters
  retry after 503
  retry after 503
  <- 200
  -> /meters
  hit /meters
  <- 200
  -> /meters/mtr_8f2
  <- 200
first : 200 {'path': '/meters', 'auth': 'Bearer sk_live_1'}
second: 200 served from the cache
third : 200 {'path': '/meters/mtr_8f2', 'auth': 'Bearer sk_live_1'}

The trace layer logs one arrow pair per call from the caller’s point of view, and the two retries happen inside it. That is the trace you want: a caller made one request and it took three attempts, rather than three requests that each look independent.

The third call shows the layers doing nothing interesting, which is the common case. One path through a stack of five layers, one transport call, the auth header applied. A chain earns its place by being unremarkable on the requests that work.

python3 test_middleware.py

ok declaration_order_is_request_order
ok a_layer_can_answer_without_calling_the_transport
ok retry_sits_inside_cache_so_a_hit_costs_no_attempts
ok a_failure_is_not_cached
ok every_layer_sees_the_same_request_object
ok a_write_is_not_retried
ok a_write_is_not_served_from_the_cache
7 cases, 0 failures

When it goes wrong

Retries are invisible in your traces. The trace layer sits inside the retry layer, so each attempt opens its own span with no parent. Move tracing outward, and record the attempt count on the span.

A signed request fails on the second attempt. The signature was computed once, outside the retry, and the timestamp inside it has aged. Sign innermost, per attempt, and make the signing layer read the request as it stands rather than a copy taken earlier.

A cache serves an error. The cache layer stores whatever the next layer returned. Store only the statuses you meant to store, and assert that in a test.

The chain works in tests and not in production. The layers are built per request rather than once, so the cache and any circuit breaker start empty every time. Build the chain when the client is built, and hand the same instance to every operation.

When not to do this

Do not make a layer that knows about one operation. The value of the chain is that every call gets the same treatment, and a layer with a path check in it is a special case pretending to be infrastructure. Put that logic in the operation, where the next reader will look for it.

Do not stack layers you cannot explain the order of. Six layers whose nesting nobody can justify produce behavior nobody predicted, and the failure appears under load. Write the reason for the order in a comment beside the chain, because the chain is one line and the reasoning is not.

Do not use the chain for business rules. Anything that changes what an operation means belongs where a reader of that operation will find it. A layer that rewrites a field for one customer is the start of a client nobody can debug.

Related how-tos

• Retry fetch calls with exponential backoff in Node.js

• Add a circuit breaker to an outbound HTTP client

Last verified

Verified 2026-09-14 against Python 3.11.15. Both output blocks are what the preceding command printed.