# How to layer unit, integration, and live tests for an SDK > Split an SDK suite into unit tests on an injected transport, integration tests against a sandbox, and a guarded live suite. Each layer gets a command and a trigger. Source: https://voxgig.com/howto/layer-unit-integration-and-live-sdk-tests - Audience: api-producer - Level: beginner - Languages: typescript, javascript - Verified: 2026-10-21 - Published: 2026-10-19 ## Short answer Give the client an injected `fetch` and test request building, error mapping, paging, and retry against a scripted one, with no socket. Run the same client over HTTP against a sandbox, or a committed stand-in, in a second command. Put the real API behind a third command that skips without `ORDERS_LIVE_URL` and `ORDERS_LIVE_KEY`, and trigger it on a schedule rather than on every pull request, so one red layer names one cause. --- ## You will need Node 22 or later, an SDK with at least one working operation, and a CI pipeline that runs one test command. Verified 2026-10-21 against Node 22.22.0. The sample SDK is `src/client.mjs`, an eighty-line orders client with paging, error mapping, and one retry. The sandbox is `sandbox/server.mjs`, a stand-in on 127.0.0.1 that speaks the vendor's wire format from memory. No vendor API is called: the live layer is shown skipping, which is what it does without credentials. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [Hand-layered test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html) | A hand-written client, or any client whose retry, paging, and error mapping are yours | Every layer is code you write and keep, and the injected transport is a seam you must design in | The client is generated and the generator ships a suite that covers the wire format | | [sdkgen generated suites and live flow](https://voxgig.com/sdk) | A client generated by sdkgen, with its in-memory test transport and entity flows | A generated suite tests what the generator emits, so behavior you added around it still needs tests you write | You did not generate the client, or your contributors want a framework they know | | [Speakeasy generated contract tests](https://www.speakeasy.com/docs/sdks/sdk-contract-testing) | A Speakeasy SDK in one of its six languages, with examples in the document | An Enterprise tier and an add-on, a Docker runtime for the mock locally, and success paths only, since error assertions are not generated | You need error paths covered, or the SDK is not Speakeasy's | | [Stainless generated tests against a mock server](https://www.stainless.com/docs/sdks/test/generated-tests/) | A Stainless SDK whose endpoint tests can run against the mock built from the document | A paid mock server feature, tests that stay off until you remove `disable_mock_tests`, and a mock that may not handle every edge case | Your own retry or paging code, which the generated tests never reach | Voxgig maintains sdkgen. It is one of four options here, not the recommendation. The three generated suites cover the wire format for the code the generator wrote, at a price each. Speakeasy's needs an Enterprise plan and stops at success paths, Stainless's needs its paid mock server and ships switched off,[^1] and sdkgen's live tests need the project's credentials in the environment. None of them tests a retry loop, a paging helper, or an error mapping you wrote yourself, which is what the hand-layered pyramid[^2] below is for. ## Inject the transport and keep the unit layer off the network The seam is the `fetch` parameter. The client takes it as an option, defaults to the global one, and never imports it. A test can therefore hand it a function that answers from memory and records what it was asked. ```js title="src/client.mjs" export function createClient({ baseUrl, apiKey, fetch = globalThis.fetch, sleep = wait }) { if (!baseUrl) throw new Error('baseUrl is required') // The trailing slash keeps a base path: orders/1 against https://host/v1/ is /v1/orders/1. const base = new URL(String(baseUrl).replace(/\/*$/, '/')) async function call(method, path, body, attempt = 1) { const url = new URL(path, base) const res = await fetch(url, { method, headers: { authorization: `Bearer ${apiKey}`, 'content-type': 'application/json', accept: 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body), }) // One retry on 429, after the pause the API asked for. The pause is injected too. if (res.status === 429 && attempt === 1) { await res.body?.cancel() await sleep(retryAfterMs(res.headers.get('retry-after'))) return call(method, path, body, attempt + 1) } ``` The pause is injected the same way, which is what lets the retry test assert a three-second wait without waiting three seconds. ```js title="test/unit/client.test.mjs" test('retries once on 429 after the Retry-After pause, with the pause injected', async () => { const { fetch, calls } = scriptedFetch([ { status: 429, headers: { 'retry-after': '3' }, body: { code: 'rate_limited', message: 'slow down' } }, { status: 200, body: { id: 'ord_1', sku: 'SKU-1', quantity: 1, status: 'placed' } }, ]) const pauses = [] const client = createClient({ baseUrl: BASE, apiKey: 'sk_test_9', fetch, sleep: async (ms) => { pauses.push(ms) } }) const order = await client.getOrder('ord_1') assert.equal(order.id, 'ord_1') assert.equal(calls.length, 2) assert.deepEqual(pauses, [3000]) }) ``` ```bash node --test "test/unit/*.test.mjs" ``` ```text output 1..11 # tests 11 # suites 0 # pass 11 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 142.318178 ``` Eleven tests, no port, and well under a second. This layer proves the code that is yours: the path and header a request carries, the error an unhappy status becomes, how paging follows `next_cursor`, and what a 429 costs. Two tests keep an id of `..` from resolving to the API root with the key attached. The layer cannot prove that the vendor's JSON looks the way the script says it does, which is the next layer's job. ## Point the integration layer at a sandbox, or the stand-in The integration layer uses the same client with the real `fetch`, over a socket, against the vendor's sandbox when `ORDERS_SANDBOX_URL` and `ORDERS_SANDBOX_KEY` are set. Without them it starts the committed stand-in, which answers with the vendor's wire format from memory, so the layer runs on every pull request with no credential. Node's test runner has [`before` and `after` hooks](https://nodejs.org/docs/latest-v22.x/api/test.html#beforefn-options) for the lifecycle. ```js title="test/integration/orders.test.mjs" let sandbox let client before(async () => { sandbox = process.env.ORDERS_SANDBOX_URL ? { url: process.env.ORDERS_SANDBOX_URL, apiKey: process.env.ORDERS_SANDBOX_KEY, stop: async () => {} } : await startSandbox() client = createClient({ baseUrl: sandbox.url, apiKey: sandbox.apiKey }) }) after(() => sandbox.stop()) ``` ```bash node --test "test/integration/*.test.mjs" ``` ```text output 1..3 # tests 3 # suites 0 # pass 3 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 220.3894 ``` Three tests, each of which catches a wire-format change the unit layer cannot see. They check the id a create answers with, the shape of a page, and the body and `x-request-id` header of a 404. Against a real sandbox the same tests need records to read and a way to remove the ones they create, which is a discipline of its own. ## Guard the live layer and give it its own trigger The live layer is the only one that talks to the vendor. It is also the one that costs: a request against quota, a credential in the environment, and a record left in the account. Three decisions keep it cheap. It skips, with a reason, when the two variables are absent. It runs on a schedule and on demand, never on a pull request. Its credentials sit in a GitHub [environment](https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments), where a required reviewer can gate them. ```js title="test/live/orders.test.mjs" const url = process.env.ORDERS_LIVE_URL const key = process.env.ORDERS_LIVE_KEY const skip = url && key ? false : 'ORDERS_LIVE_URL and ORDERS_LIVE_KEY are not set' if (skip && process.env.ORDERS_LIVE_REQUIRED === 'true') { throw new Error(`${skip}, and ORDERS_LIVE_REQUIRED says this run must reach the API`) } const client = skip ? null : createClient({ baseUrl: url, apiKey: key }) test('an authenticated read against the live API', { skip }, async () => { const order = await client.getOrder(process.env.ORDERS_LIVE_ORDER_ID) assert.equal(order.status, 'placed') }) ``` ```bash node --test "test/live/*.test.mjs" ``` ```text output TAP version 13 # Subtest: an authenticated read against the live API ok 1 - an authenticated read against the live API # SKIP ORDERS_LIVE_URL and ORDERS_LIVE_KEY are not set --- duration_ms: 0.757202 type: 'test' ... # Subtest: a write against the live API, with the record it leaves behind named ok 2 - a write against the live API, with the record it leaves behind named # SKIP ORDERS_LIVE_URL and ORDERS_LIVE_KEY are not set --- duration_ms: 0.37624 type: 'test' ... 1..2 # tests 2 # suites 0 # pass 0 # fail 0 # cancelled 0 # skipped 2 # todo 0 # duration_ms 94.870379 ``` `# SKIP` with the reason is what a contributor without credentials sees, and the exit code is 0. In CI the live job runs on `schedule` and `workflow_dispatch`, two [events](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows) a pull request never fires, and takes its key from an environment secret. It also sets `ORDERS_LIVE_REQUIRED`, so a missing secret fails the job instead of skipping it. ```yaml title="ci/sdk-tests.yml" live: # Never on a pull request: a fork gets no secrets, and every push would spend quota. if: github.event_name != 'pull_request' runs-on: ubuntu-latest environment: sandbox steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22 - run: npm ci - run: npm run test:live env: ORDERS_LIVE_REQUIRED: 'true' ORDERS_LIVE_URL: ${{ vars.ORDERS_LIVE_URL }} ORDERS_LIVE_KEY: ${{ secrets.ORDERS_LIVE_KEY }} ORDERS_LIVE_ORDER_ID: ${{ vars.ORDERS_LIVE_ORDER_ID }} ``` sdkgen's generated suite draws the same line with an environment variable. Its live tests run when `_TEST_LIVE` is `TRUE` and read the credential from `_APIKEY`, pacing their calls 500 ms apart by default. `main.kit.test.live.strict` decides whether a failing live request fails the test or skips it with the reason.[^3] ## Check it worked `layers.mjs` runs the three commands the way CI would and prints one line per layer, with the counts read from the TAP summary and the reason for any skip. ```bash node layers.mjs ``` ```text output unit pass 11 fail 0 skip 0 proves request building, error mapping, paging, retry integration pass 3 fail 0 skip 0 proves the wire format, against a sandbox live pass 0 fail 0 skip 2 skipped: ORDERS_LIVE_URL and ORDERS_LIVE_KEY are not set red: none ``` `pass 11`, `pass 3`, and `skip 2` with the reason spelled out is the green state for a machine without credentials. `red: none` is the literal to look for. `layers.test.mjs` checks that the script exits 1 on a red layer and on one that ran no tests, since `node --test` exits 0 on a glob that matches nothing. It also checks `ORDERS_LIVE_REQUIRED`, and that a layer run from inside another test still reports its failure. ```bash node --test layers.test.mjs ``` ```text output 1..5 # tests 5 # suites 0 # pass 5 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 2357.325062 ``` ## When it goes wrong One layer goes red, and the layer names the cause. `layers.mjs --drift` starts the stand-in with its list responses renamed from `next_cursor` to `nextCursor`, which is what a vendor's wire-format change looks like from the client. ```bash node layers.mjs --drift ``` ```text output unit pass 11 fail 0 skip 0 proves request building, error mapping, paging, retry integration pass 2 fail 1 skip 0 failed: the list is paged and the client sees every order once live pass 0 fail 0 skip 2 skipped: ORDERS_LIVE_URL and ORDERS_LIVE_KEY are not set red: integration ``` The unit layer stays green because its scripted fetch still says `next_cursor`, the integration layer fails on the one test that pages, and the live layer is untouched. The vendor moved. Fix the client against the renamed field, then change the unit script to match. The live suite fails for everyone. `pitfall.mjs` runs a live test with no guard, the way such a test is usually first written, beside the guarded one. ```bash node pitfall.mjs ``` ```text output unguarded live suite, no credentials: pass 0 fail 1 exit code 1 error: baseUrl is required guarded live suite, no credentials: pass 0 fail 0 skip 2 exit code 0 skipped because: ORDERS_LIVE_URL and ORDERS_LIVE_KEY are not set ``` Without the guard the file fails at import, so `npm test` is red on every laptop that lacks the key, and every contributor ends up holding a credential to make it pass. GitHub adds the second half of the problem. With the exception of `GITHUB_TOKEN`, [secrets are not passed](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets) to the runner when a workflow is triggered from a forked repository.[^4] A live job on `pull_request` can therefore never pass for an outside contributor. The live suite burns the sandbox. A suite on `pull_request` runs on every push to every open branch, and each run spends the vendor's quota and leaves records in the account. Move it to the schedule, and give the write test a name it can leave behind, as `SKU-LIVE-CHECK` does here. ## When not to do this Do not build the pyramid under a generated client whose generator ships the bottom layer. A Stainless or Speakeasy SDK arrives with endpoint tests against a mock built from the document. sdkgen's `test` feature answers every operation from a seeded map with no server at all, which the [in-memory mock page](/howto/run-an-in-memory-mock-inside-sdk-tests) shows running. Write unit tests for the behavior you added around the generated code. Do not point the integration layer at production with a test key and call it live. The two layers exist because they fail for different reasons: a sandbox is yours to seed and reset, and the live layer is the one that notices the vendor deploying. Do not let the live layer skip silently. A skip with a reason is a decision a reader sees in the log. A scheduled job whose output carries `# SKIP` has reached nothing, so treat that line as red there and check the variable names. Do not run the live layer on every pull request, even with the secret available. The quota and the leftover records are per run, and a flaky vendor blocks merges that have nothing to do with the vendor. ## Related how-tos - [Run an in-memory mock of your API inside SDK tests](/howto/run-an-in-memory-mock-inside-sdk-tests) - [Mock a generated SDK in your application tests](/howto/mock-a-generated-sdk-in-application-tests) - [Choose consumer-driven or spec-driven contract tests](/howto/choose-consumer-driven-or-spec-driven-contract-tests) - [Seed a sandbox account for SDK integration tests](/howto/seed-a-sandbox-account-for-sdk-integration-tests) ## Last verified Verified 2026-10-21 against Node 22.22.0. Every output block is what the command preceding it printed. The live layer ran against no vendor: it is shown skipping, which is its behavior without credentials, and the sandbox in the integration layer is the committed stand-in. The sdkgen figures are read from the model documentation and test templates in the @voxgig/sdkgen 4.35.0 package, and the Speakeasy and Stainless rows from their documentation. None of the three generated suites was run. [^1]: Stainless published a fork of Prism, `@stainless-api/prism-cli`, which the registry lists at [5.16.1](https://registry.npmjs.org/@stainless-api/prism-cli), last modified in December 2025. Its [configuration reference](https://www.stainless.com/docs/reference/config/) names one supported mock server, steady, and the `scripts/mock` in a generated SDK such as the [Anthropic TypeScript SDK](https://github.com/anthropics/anthropic-sdk-typescript/blob/main/scripts/mock) runs `@stdy/cli` 0.22.2 on port 4010. [^2]: Martin Fowler's [entry](https://martinfowler.com/bliki/TestPyramid.html) of May 1, 2012 credits the pyramid to Mike Cohn, who described it in his 2009 book Succeeding with Agile after drawing it in 2003-4. Ham Vocke's [longer treatment](https://martinfowler.com/articles/practical-test-pyramid.html) on the same site advises readers not to become attached to the names of the layers, since Cohn's middle one was called service tests. Google's testing blog [suggested a 70/20/10 split](https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html) as a first guess in 2015, with the caveat that the exact mix differs for each team. [^3]: The model documentation in the @voxgig/sdkgen package gives `main.kit.test.live.strict` a default of `true`. Under it a live request that fails, or a live test missing an input it needs, fails the test; `false` skips it with the reason. Under either value, an account holding no record for a test to read skips that test, and each generated live test states the setting it was generated with in a comment. The [features page](https://voxgig.com/sdk/features) describes the `test` feature the offline half of the same suite runs on. [^4]: The sentence is GitHub's own, from its page on [using secrets](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets). With the exception of `GITHUB_TOKEN`, secrets are not passed to the runner when a workflow is triggered from a forked repository. A live job that needs a key and runs on `pull_request` is therefore a job outside contributors can only watch fail. The sample's live job runs on the schedule instead.