# How to wrap an SDK transport in a middleware chain > Compose retry, caching, authentication and tracing as layers around one transport, so each concern is written once and the order is declared rather than implied. Source: https://voxgig.com/howto/wrap-an-sdk-transport-with-middleware - Audience: api-producer - Level: intermediate - Languages: python - Verified: 2026-09-14 - Published: 2026-09-14 - Updated: 2026-09-24 ## Short answer Write each concern as a function that takes the next call and returns a call, then compose them outermost first so the declaration order is the order a request travels. A layer may change the request, inspect the response, call the next layer more than once, or answer without calling it at all. Nesting order decides what a retry counts and what a cache hit skips. --- ## You will need Python 3.11 or later, and an SDK or client whose transport you can wrap. The shape is the same one [WSGI](https://peps.python.org/pep-3333/) uses, and the same one the [Guzzle handler stack](https://docs.guzzlephp.org/en/stable/handlers-and-middleware.html) uses in PHP.[^1] The idea transfers to whichever language your client is written in, and so does the ordering argument below. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [A chain of wrapping functions](https://peps.python.org/pep-3333/) | You own the transport and want the order visible where the client is built | Code you maintain, and a chain that can be assembled wrongly | The client library already exposes a stack of its own | | [Guzzle handler stack middleware](https://docs.guzzlephp.org/en/stable/handlers-and-middleware.html) | A PHP client built on Guzzle, where the stack is already there | Named positions and a push API, which is more machinery than a list | The client is not built on Guzzle | | [requests transport adapters](https://requests.readthedocs.io/en/latest/user/advanced/#transport-adapters) | Per-host behavior such as pooling, TLS settings or a mounted retry | One adapter per host prefix, and a hook model that is not a chain | The behavior belongs to every host the client calls | | [urllib3 retries](https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html) | Retry alone, configured once, with backoff and status lists included | Only retry, so every other concern still needs somewhere to live | You need caching, tracing or auth in the same place | The difference is whether the ordering is yours. A library stack gives you tested behavior and decides for you where retry sits relative to everything else.[^2] A chain you compose makes that a line in your code, and it makes the wrong order equally easy to write. Take the library stack when it covers the concerns you have, and compose when it does not. ## Compose outermost first One function builds the chain, and the order reads the way a request travels. ```python title="middleware.py" def chain(transport, *layers): """Compose layers around a transport. The first layer is the outermost.""" call = transport for layer in reversed(layers): call = layer(call) return call ``` Every layer has the same shape: take the next call, return a call. That uniformity is what lets you reorder the stack without editing a layer, and it is why a layer can do things a hook cannot. A retry layer calls the next one several times. A cache layer sometimes calls it not at all. A hook that fires before a request and another that fires after cannot express either of those, which is why a hook API turns into a chain as soon as retry arrives. ## Put the cache outside the retry Nesting is not a style question, it changes what happens. ```python title="demo.py" call = chain(transport, trace(log), cache(store, log), retry(3, log=log), auth("sk_live_1")) ``` With the cache outside, a hit skips the retry layer entirely and costs nothing. Flip the two, and every cache hit still walks through the retry machinery, and a retried request writes its intermediate failures past a cache that is now inside the loop. Auth sits innermost, next to the transport. A retried attempt then gets a header applied again, which matters the moment that header is a signature over the request or a token that can expire between attempts. Both layers ask the method before they act. A 503 can arrive after the server already processed the request, so a retried `POST` is how one charge becomes two. A cache keyed on method and path is no safer: it would hand the first response to a second `POST` carrying a different body, and that second write would never leave the client. Retry the idempotent methods, cache the safe ones, and let a write through untouched.[^3] Add `POST` to the retry list only for an API that takes an idempotency key. ## Check it worked Run the chain against a transport that fails twice, then succeeds, then gets asked the same question again. ```bash python3 demo.py ``` ```text output order: trace cache retry auth transport -> /meters retry after 503 retry after 503 <- 200 -> /meters hit /meters <- 200 -> /meters/mtr_8f2 <- 200 first : 200 {'path': '/meters', 'auth': 'Bearer sk_live_1'} second: 200 served from the cache third : 200 {'path': '/meters/mtr_8f2', 'auth': 'Bearer sk_live_1'} ``` The trace layer logs one arrow pair per call from the caller's point of view, and the two retries happen inside it. That is the trace you want: a caller made one request and it took three attempts, rather than three requests that each look independent. The third call shows the layers doing nothing interesting, which is the common case. One path through a stack of five layers, one transport call, the auth header applied. A chain earns its place by being unremarkable on the requests that work. ```bash python3 test_middleware.py ``` ```text output ok declaration_order_is_request_order ok a_layer_can_answer_without_calling_the_transport ok retry_sits_inside_cache_so_a_hit_costs_no_attempts ok a_failure_is_not_cached ok every_layer_sees_the_same_request_object ok a_write_is_not_retried ok a_write_is_not_served_from_the_cache 7 cases, 0 failures ``` ## When it goes wrong Retries are invisible in your traces. The trace layer sits inside the retry layer, so each attempt opens its own span with no parent. Move tracing outward, and record the attempt count on the span. A signed request fails on the second attempt. The signature was computed once, outside the retry, and the timestamp inside it has aged. Sign innermost, per attempt, and make the signing layer read the request as it stands rather than a copy taken earlier. A cache serves an error. The cache layer stores whatever the next layer returned. Store only the statuses you meant to store, and assert that in a test. The chain works in tests and not in production. The layers are built per request rather than once, so the cache and any circuit breaker start empty every time. Build the chain when the client is built, and hand the same instance to every operation. ## When not to do this Do not make a layer that knows about one operation. The value of the chain is that every call gets the same treatment, and a layer with a path check in it is a special case pretending to be infrastructure. Put that logic in the operation, where the next reader will look for it. Do not stack layers you cannot explain the order of. Six layers whose nesting nobody can justify produce behavior nobody predicted, and the failure appears under load. Write the reason for the order in a comment beside the chain, because the chain is one line and the reasoning is not. Do not use the chain for business rules. Anything that changes what an operation means belongs where a reader of that operation will find it. A layer that rewrites a field for one customer is the start of a client nobody can debug. ## Related how-tos - [Retry fetch calls with exponential backoff in Node.js](/howto/retry-fetch-with-backoff-in-node) - [Add a circuit breaker to an outbound HTTP client](/howto/add-a-circuit-breaker-to-an-http-client) ## Last verified Verified 2026-09-14 against Python 3.11.15. Both output blocks are what the preceding command printed. [^1]: [PEP 3333](https://peps.python.org/pep-3333/#middleware-components-that-play-both-sides) describes itself as an updated version of PEP 333, modified slightly to improve usability under Python 3. Its section on the pattern is headed Middleware: Components that Play Both Sides. [PEP 333](https://peps.python.org/pep-0333/) is dated December 2003, and Phillip J. Eby is the author of both. A function that takes the next call and returns a call has been a Python convention for twenty-two years, with one revision. The revision was for the language rather than the idea. [^2]: Guzzle's [handler stack](https://docs.guzzlephp.org/en/stable/handlers-and-middleware.html#handlerstack) is the chain with a management layer. Its documentation says middleware functions return a function that accepts the next handler to invoke, which is this page's shape exactly. It then offers `push`, `unshift`, `before`, `after` and `remove` for placing one, by position or by the name of a neighbor, and `create` installs four handlers of its own before you add any. The order is decided for you, and then it is negotiable. [^3]: urllib3 reached the same list. Its [`Retry`](https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry.DEFAULT_ALLOWED_METHODS) retries `DELETE`, `GET`, `HEAD`, `OPTIONS`, `PUT` and `TRACE` by default, six methods with `POST` the notable absentee, and the parameter documentation says the set is the methods considered idempotent. The class signature defaults `total` to 10, and the [user guide](https://urllib3.readthedocs.io/en/stable/user-guide.html) says urllib3 will retry requests 3 times by default, which leaves a reader who wanted one number holding two.