How-to › Ship a CLI or REPL

How to end-to-end test a CLI against a mock API#

Build the CLI binary, point it at a scripted mock through two environment variables, and assert on its output, its exit codes, and the requests the mock recorded.

Audience
API producer
Level
intermediate
Topic
Test, package and distribute a CLI
Languages
Go and TypeScript
Verified

Read first: Snapshot-test CLI output across operating systems

Your CLI’s tests call its command functions with a stubbed client. Nothing runs the binary you release, so a base URL read from the wrong environment variable ships, and a customer’s first command goes to the wrong host. The mock in the one binary test answers every request with the same example, which leaves the second page, the 404, and the 429 unexercised.

What you get

You will end up with a suite that runs your built CLI against a scripted mock on the loopback interface and asserts its output, exit codes, and requests. This is for you if you ship a CLI that wraps an HTTP API.

Short answer

Build the binary your users get, here the go-cli target that sdkgen generates, and run it as a child process against a mock API on 127.0.0.1. Inject the base URL and the key through environment variables, as FERRY_BASE and FERRY_APIKEY. Assert on stdout, stderr, the exit code, and the requests the mock recorded. Script stubs for a second page, a 404, and a 429 with Retry-After, because a mock that serves one example per operation never reaches those paths.

You will need

Node 22.18 or later, Go 1.24 or later, and an OpenAPI 3 document for the API. Verified 2026-11-19 against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9, and @voxgig/sdkgen 4.44.3. The document on this page, openapi.json, describes a ferry operator’s API behind a bearer token: a paged list of sailings, one sailing by id, and a PATCH that changes its seats. Normalizing a transcript before comparing it is covered in snapshot-test CLI output across operating systems.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
oapi-codegen strict serverA Go CLI on an oapi-codegen client, with a fake whose response types come from the documentEvery stub written in Go, and no request validation until you add the middleware for itThe CLI is not Go, or nobody will maintain a fake
orval MSW handlersA TypeScript CLI on an orval client, with handlers generated from the document’s examplesA second package to put the handlers on a port, and no check of a credential or a bodyThe tests need the mock to refuse a bad request
PrismResponses and request validation straight from the document, with no stub codeOne example per operation unless the request carries a selector, which a user’s command never doesThe test is about a second page or a retry
sdkgen test featureAn sdkgen CLI, when the test is about its words and its printingNothing is sent and the mock ignores the URL and headers, so a wrong base URL or missing key passesThe defect you fear is in the HTTP layer
WireMockScripted sequences and faults for a CLI in any language, with stubs a team can shareStub mappings you write or record, which name no document to check against, plus a JVM or container in CIYou want the document to stay the single source

Voxgig maintains sdkgen. It is one of five options here, not the recommendation.

The rows differ in who writes the stubs and what holds them to the document. Prism writes none and validates every request, but it serves one example per operation. WireMock serves whatever you script, including a response the document never declared. oapi-codegen puts the compiler between the fake and the document, and the price is a fake written in Go. The steps below script a node:http stand-in, the same kind of mock as WireMock with nothing to install.

Build the binary and hand it the mock’s address

The CLI under test is the go-cli target, a Go program that wraps the Go SDK generated beside it. The scaffold and the generation ran once, and their output is under generated/.

npx --yes @voxgig/create-sdkgen@0.31.9 ferry --def openapi.json --target go --target go-cli
(cd ferry-sdk/.sdk && npm run generate)

The generated main.go reads two environment variables, FERRY_APIKEY and FERRY_BASE, and has no flag for either. Left unset, FERRY_BASE falls back to the server in the document,1 so a test that forgets it calls the real host. The harness builds the binary with go build on every run, then starts it with a closed environment:

export function runCli(bin, args, env = {}) {
  return new Promise((resolve, reject) => {
    const systemRoot = process.env.SystemRoot ? { SystemRoot: process.env.SystemRoot } : {}
    const child = spawn(bin, args, { env: { ...systemRoot, ...env }, stdio: ['ignore', 'pipe', 'pipe'] })
    let stdout = ''
    let stderr = ''
    child.stdout.on('data', (chunk) => { stdout += chunk })
    child.stderr.on('data', (chunk) => { stderr += chunk })
    child.on('error', reject)
    child.on('close', (code, signal) => resolve({ code, signal, stdout: sortKeys(stdout), stderr }))
  })
}

The child gets only the variables the test passes, so a key exported in a developer’s shell cannot reach it. spawn returns at once, which leaves the event loop free to run a mock in the same process.

Script a second page, a 404, and a 429

The mock is a node:http server on port 0, which the operating system replaces with a free port. It checks the bearer token, records every request, and holds three stubs. The page number decides which rows come back:

// The stub for a second page: the page number decides which rows come back.
const page = Number(url.searchParams.get('page') ?? '1')
if (!Number.isInteger(page) || page < 1) return reply(400, problem('bad_request', 'page must be a positive integer'))
return reply(200, sailings.slice((page - 1) * pageSize, page * pageSize))

An id it does not hold gets a 404, and mock.once(tooManyRequests(1)) queues a 429 with Retry-After: 1 for the next request.2 e2e-demo.mjs runs six commands and prints the log:

node e2e-demo.mjs
ferry-cli list sailing
  exit 0  [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli list {page:2} sailing
  exit 0  [{id:'sl_103' route:'DUB-CHE' seats:8 vessel:'W. B. Yeats'}]
ferry-cli load {id:"sl_999"} sailing
  exit 1  error: FerrySDK: load: request: 404: Not Found
ferry-cli list sailing, after the mock is told to answer 429 once
  exit 1  error: FerrySDK: list: request: 429: Too Many Requests
ferry-cli update {id:"sl_101",seats:118} sailing
  exit 0  {id:'sl_101' route:'DUB-HOL' seats:118 vessel:'Ulysses'}
ferry-cli list sailing, with FERRY_APIKEY unset
  exit 1  error: FerrySDK: list: request: 401: Unauthorized

requests the mock recorded:
  GET   /v1/sailings             Bearer sk_test_ferry     200
  GET   /v1/sailings?page=2      Bearer sk_test_ferry     200
  GET   /v1/sailings/sl_999      Bearer sk_test_ferry     404
  GET   /v1/sailings             Bearer sk_test_ferry     429, Retry-After: 1
  PATCH /v1/sailings/sl_101      Bearer sk_test_ferry     200
  GET   /v1/sailings             no authorization header  401

The second page went out as ?page=2 and came back as sl_103 alone. The missing id and the 429 each exit 1 with the status on stderr. The log shows one request for the 429: the generated main.go turns no retry feature on, so the binary does not wait and try again. Every failure shares exit code 1, so a test that must tell them apart reads stderr.

Run the same commands against a one-example mock

Prism serves the same document with no stub written. prism-check.mjs starts it on a free port and repeats five of the commands:

node prism-check.mjs
Prism 5.14.2 serving openapi.json, with no stub written

ferry-cli list sailing
  exit 0  [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli list {page:2} sailing
  exit 0  [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli load {id:"sl_999"} sailing
  exit 0  {id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'}
ferry-cli update {id:"sl_101",seats:118} sailing
  exit 1  error: FerrySDK: update: request: 422: Unprocessable Entity
ferry-cli list sailing, with FERRY_APIKEY unset
  exit 1  error: FerrySDK: list: request: 401: Unauthorized

Page 2 is page 1, and the missing id exits 0 with the example record. Prism answers with the lowest 2XX response and its example unless the request selects another, through a Prefer header or a __code query parameter.3 A test can add {__code:404} to the command, but then it tests a command no user types.

Prism also caught what the scripted mock let through. The document reuses Sailing, all four fields required, as the PATCH body, and the CLI sends the id and the one field that changed. The fix belongs in the document: give the operation a body schema with no required fields.

Test the words in process when HTTP is not the question

The Go SDK under the CLI carries a test mode, an in-memory mock served from seed data. The built binary has no switch for it. The way in is a Go test in the go-cli package that hands the CLI’s own words a client from sdk.TestSDK, and a regeneration left that file in place.

// The seed has no pages. The mock compares only what the route requires, and page is optional.
code, ids, _ = eval(t, client, "list {page:2} sailing")
t.Logf("list {page:2} sailing      exit %d, ids %s", code, ids)
if code != 0 || ids != "sl_101 sl_102" {
	t.Fatalf("list {page:2} sailing: exit %d, ids %q", code, ids)
}
go test -C generated/go-cli -run TestWordsOverTestMode -v .
=== RUN   TestWordsOverTestMode
    words_inprocess_test.go:48: client mode: test
    words_inprocess_test.go:54: list sailing               exit 0, ids sl_101 sl_102
    words_inprocess_test.go:61: list {page:2} sailing      exit 0, ids sl_101 sl_102
    words_inprocess_test.go:67: load {id:"sl_999"} sailing exit 1, error: FerrySDK: load: request: 404: Not found

It needs no port and no binary. It also answers page 2 with both seeded records. The mock transport ignores the URL and the headers the client hands it, so a wrong base URL or a missing key passes.

Mock the API behind an oapi-codegen or orval client

oapi-codegen generates a typed Go client and no CLI, so the binary is your own main.go: pass the base URL to NewClientWithResponses and set the key in a RequestEditorFn. It generates no mock either. With std-http-server and strict-server set to true, it generates a server interface from the same document. A fake behind it returns generated response types, which exist only for the statuses the document declares:

if f.throttle > 0 {
	f.throttle--
	wait := 1
	return ferryapi.ListSailings429JSONResponse{TooManyRequestsJSONResponse: ferryapi.TooManyRequestsJSONResponse{
		Body:    ferryapi.Problem{Code: "rate_limited", Message: "Slow down"},
		Headers: ferryapi.TooManyRequestsResponseHeaders{RetryAfter: &wait},
	}}, nil
}

httptest.NewServer puts the fake on a loopback port and exec.Command runs the binary against it, as oapi/fake_test.go does. The strict server decodes a request body but checks it against no schema, and checks no credential. The README names a validation middleware for both.

orval’s mock option writes an MSW handler per operation, seeded from the document’s examples when useExamples is on. setupServer mocks the process it runs in, so a CLI in a child process never meets those handlers. createServer from @msw/serve puts them on a port:

const server = createServer(
  // A 429 with the header the generated 429 handler leaves out.
  http.get('*/sailings', () => {
    if (throttled > 0) {
      throttled--
      return HttpResponse.json(getListSailingsResponseMock429(), { status: 429, headers: { 'retry-after': '1' } })
    }
  }),
  // A second page, through the generated handler's override argument.
  getListSailingsMockHandler(({ request }) =>
    new URL(request.url).searchParams.get('page') === '2' ? PAGE_TWO : getListSailingsResponseMock()),
  // A missing record, with the body the document's 404 example gave the generated factory.
  http.get('*/sailings/sl_999', () => HttpResponse.json(getGetSailingResponseMock404(), { status: 404 })),
  ...getFerrylineAPIMock(),
)

MSW answers with the first handler that returns a response, so the scripted stubs sit ahead of the generated ones. The generated handlers check no credential.

Check it worked

Run the suite:

node --test e2e.test.mjs
1..14
# tests 14
# suites 0
# pass 14
# fail 0
# cancelled 0
# skipped 0
# todo 0

# pass 14 is the line to read. Nine tests drive the binary against the scripted mock, and one each covers the key sorter, Prism, the in-process Go test, the oapi-codegen fake, and the orval handlers.

When it goes wrong

Every test times out. The test called spawnSync while the mock ran in the same process, and a blocked event loop cannot accept the child’s connection. Use spawn, or run the mock in a process of its own.

node pitfall.mjs
spawnSync, with the mock in the same process:
  status null, signal SIGTERM, error ETIMEDOUT
  requests the mock had answered when it returned: 0
spawn, with a mock in the same process:
  exit 0, ids sl_101 sl_102
  requests the mock answered: 1

A stdout assertion passes on one run and fails on the next. The CLI prints each record as a Go map, and the Go specification leaves the iteration order of a map unspecified. Sort the keys inside each record before comparing, as sortKeys does.

A test fails with no such host. FERRY_BASE was unset, so the binary called the server in the document, a reserved name here and the real API in your project. Assert on the mock’s request log in every test.

When not to do this

Do not make every CLI test end to end. Each one costs a build and a process, and argument parsing needs no socket. Keep a dozen for what crosses the process boundary: the environment, the exit code, the wire.

Do not let sdkgen’s test mode stand in for the binary. It is the fastest check of the generated words, but its mock ignores the URL and headers, so a dropped header passes it. The sdkgen documentation describes no mock that listens on a port, so that one is yours to supply.

Do not hand-script a mock when a typed one is on offer. For a Go CLI on an oapi-codegen client, the strict server fake is the better choice. A stub for a status the document lacks has no type to return, so it fails to compile. For a TypeScript CLI on an orval client, the generated handlers start from the document’s examples, and you write only the stubs that differ.

Do not read a green run against Prism as coverage of the error paths.

Last verified

Verified 2026-11-19 against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, Prism 5.14.2, oapi-codegen v2.8.2, orval 8.48.0, MSW 3.0.7, and @msw/serve 0.11.0. Every output block is what the command preceding it printed. sdkgen, oapi-codegen, and orval each ran once on openapi.json; their output is committed, and the checks build and drive it without generating again. The harness sorts the keys inside each printed record. WireMock was not run; its row describes its documentation. No request left the loopback interface.

Footnotes

  1. The server in this document is api.ferryline.example, a name no resolver will find. RFC 2606 reserved four top-level names in June 1999 and recommends .example for use in documentation or as examples. A test that forgets the base URL fails at the resolver here, with no such host. With a real host in the document, the same forgetful test reaches production. ↩︎ Back to text

  2. The status code is younger than the habit of sending it. RFC 6585 defined 429 in April 2012 and says a response may include a Retry-After header, which makes the header optional. The same section declines to define how a server identifies the user or counts the requests. A client that waits as told is honoring a header the server never had to send. ↩︎ Back to text

  3. Prefer is a standard header put to a private use. RFC 7240 defined it in June 2014 for preferences such as return=minimal and respond-async, and the IANA registry it created lists those with wait, handling, and later entries such as safe. Prism’s code, example, and dynamic are not in it. None of the four preferences the RFC defines chooses a status code. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.