Advertise next and previous pages with Link headers
Send the next, previous, first and last page URLs in a Link header, so clients follow links you build rather than assembling query strings themselves.
How-to › Section 3
Move data in and out#
Reading collections page by page, streaming, bulk jobs, file transfer, loading API data into notebooks, and scheduled pipelines that keep a copy in sync.
14 guides in 3 topics. Every guide compares its approaches and shows the output its commands printed.
Paginate collections 9 guides
Cursor versus offset versus keyset versus Link header (RFC 8288) pagination, on both sides: designing a list endpoint that survives inserts and deletes, page-size negotiation, stable ordering, total counts, backwards paging; and on the client, auto-pagination iterators, parallel page fetching with backpressure, paging over GraphQL connections.
Choose a pagination style for a list endpoint
Compare the four ways to page a list on what decides it: whether a caller can miss a row when the data shifts, and what the query costs at page 900.
Describe pagination so generators can follow it
Put enough in the OpenAPI document that a generated client can walk a collection by itself, instead of handing every consumer a single-page call.
Encode and sign opaque pagination cursors
Encode the page position as base64url JSON and sign it with HMAC, so callers carry a cursor without reading it and a tampered one never reaches your query.
Loop over every page of a REST collection
Walk a paginated list endpoint to the end, read the three stop signals APIs actually send, and refuse to loop forever when a server repeats its cursor.
Migrate an API from offset to cursor pagination
Move a list endpoint from offset to cursor pagination without breaking the clients still sending an offset, and show why the change is worth making.
Negotiate page size between an API and its clients
Clamp an over-large page size to your ceiling rather than rejecting it, say in the response what was served, and refuse only values that are not sizes.
Paginate an API with a Ruby enumerator
Wrap a paged list in an Enumerator that fetches pages as rows are pulled, so first(50) stays cheap, lazy.select stops early, and a 503 surfaces where you iterate.
Paginate an API with PHP generators
Walk a paginated collection with a generator, so a caller writes one foreach, pages are fetched only as they are consumed, and an early break costs nothing.
Stream responses and subscriptions 3 guides
Server-Sent Events, WebSockets, chunked and NDJSON responses, long polling, resumable streams with last-event ids, backpressure, reconnection, and streaming LLM tokens to a client, from both the producer and consumer side.
Add backpressure to a WebSocket consumer
Keep a slow sink from being overrun by a fast WebSocket feed: pause the socket in Node, read a WebSocketStream where it exists, or window acknowledgements.
Consume Server-Sent Events with fetch and custom headers
Read an SSE endpoint that needs a bearer token or a POST body, which EventSource cannot send, and own reconnection, the retry field and Last-Event-ID yourself.
Pick SSE, WebSockets, or long polling for a live endpoint
Serve one job-progress feed as SSE, WebSocket, long poll, and short poll, restart the proxy in front of it, and choose by what survives and what each event costs.
Run bulk jobs and move files 2 guides
Bulk import and export, batch endpoints, async jobs with status polling, multipart and resumable uploads (tus, S3 multipart), presigned URLs, and large downloads.
Poll a job status endpoint with backoff
Poll a job until it reaches a terminal state: honor Retry-After, back off with jitter when the server sends none, and stop at a deadline with the job id in hand.
Upload a large file with resumable chunks
Resume an upload from the last byte the server confirmed after a dropped connection or a closed tab, with tus, S3 multipart, Google resumable uploads, or your own.
Generate the client instead of writing it#
Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.