How-to › Ship a CLI or REPL

How to keep a CLI in sync with an API spec as it changes#

Fail CI when the OpenAPI document gains, loses or renames an operation your CLI does not reflect, using an operationId coverage test, oasdiff, and check-drift.

Audience
API producer
Level
advanced
Topic
Build a CLI over an API
Verified

Read first: Derive a noun-verb command tree from an OpenAPI document

A release renames the operationId getTicket to showTicket. Restish loads the document on its next sync, restish bo get-ticket stops existing, and every script that called it exits 2. Your hand-written CLI still maps tickets get to an id the document has dropped. The oasdiff job in CI rates the rename as information, so it passes.

What you get

You will end up with a test that fails when the document and the CLI disagree, and an oasdiff step that fails on a renamed operation. This is for you if your CLI ships on a different schedule from your API.

Short answer

Write a test that collects every operationId in the current OpenAPI document and fails unless each one maps to a registered command or sits in an ignore file with a reason. Give it the released CLI’s command names too, so a rename that drops a command fails. Run oasdiff breaking --fail-on ERR against the last released document, with api-operation-id-removed raised to an error. For an sdkgen CLI, run check-drift.

You will need

Node 22 or later, the current OpenAPI document and the one you last released, and Go 1.24 or later for oasdiff. oasdiff v1.11.7 is the last release that declares Go 1.24: v1.11.8 to v1.11.10 declare Go 1.25, and every release from v1.11.11 to v1.39.0 declares Go 1.26. Verified 2026-11-22 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, oasdiff v1.11.7, Restish v2.3.0, and OpenAPI Generator 7.27.0. The coverage test is JavaScript for Node; the leaf names no language, and the test reads only JSON and generated files. v1.json is the released Box Office document and v2.json the current one. Reading operationIds as commands is the subject of derive a noun-verb command tree from an OpenAPI document.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
Hand-written operationId coverage testA hand-written CLI whose command table you can importA table and an ignore file to keep, and a test you wrote yourselfThe CLI is generated, so its table is the document
oasdiff with —fail-onAny CLI, as the job that compares two documentsExits 0 without --fail-on, and rates an operationId rename as informationYou need to know what the CLI exposes, not only the document
OpenAPI Generator bash clientYou want a generated script per release with no account and no runtime serviceOne command per operationId, so a rename removes the command unless a name mapping restores itScripts depend on stable names across renames
Restish api syncYour own team, against a document the server publishesNothing to regenerate, and no review step before a user’s command changesCustomers script the CLI and need warning before a rename
sdkgen regeneration with check-driftA project that already generates a Go SDK and its go-cli targetA regeneration per document change, and three stock words that leave create and remove uncoveredYou need every operation as a command without editing fragments
Speakeasy CLI targetCustomers want a Cobra CLI regenerated by a pull requestA vendor account and API key, and versioning that ignores added or removed operationsYour build cannot depend on a vendor service

Voxgig maintains sdkgen. It is one of six options here, not the recommendation.

Restish builds its commands from the copy of the document that its last api sync fetched. Nothing needs regenerating, but a change reaches users without review. The coverage test and a regeneration both need someone to act on each change, and both put that change in a pull request first. oasdiff guards the document whatever builds the CLI, but only at the severity levels you give it.

Fail the test when the document and the command table disagree

coverage.mjs reads the document’s operationIds and a command table. It fails on an id with no command and no ignore entry, and on a command mapped to an id the document lacks. A stale ignore entry fails it too, and so does a released command name that is gone, a command whose method or path differs from its id’s, and a name registered twice. A command and an alias count as names alike. The command table of the hand-written CLI is a list of entries such as { command: 'tickets get', operationId: 'getTicket', ... }. Run it on the table from release 1.4.0 against the current document:

node coverage.mjs v2.json hand cli/boxoffice-1.4.mjs
v2.json against hand cli/boxoffice-1.4.mjs: 7 operations, 6 command mappings, 0 ignored
  stale command: "tickets get" maps to getTicket, which the document no longer has
  unmapped: showTicket (GET /tickets/{ticketId}) has no command
  unmapped: refundTicket (POST /tickets/{ticketId}/refund) has no command
3 problems

It exits 1, so a CI step fails. The fix is two decisions in reviewed files. tickets get keeps its name and maps to showTicket, with tickets show added as an alias. refundTicket goes in cli/ignore.json with a reason, because box office staff issue refunds:

[
  { "operationId": "refundTicket", "reason": "Refunds are issued by box office staff, so the public CLI leaves them out." }
]
node coverage.mjs v2.json hand cli/boxoffice.mjs --ignore cli/ignore.json --released cli/boxoffice-1.4.mjs
v2.json against hand cli/boxoffice.mjs: 7 operations, 6 command mappings, 1 ignored
in sync

--released compares command names with the 1.4.0 table, so a rename in the CLI itself fails until the former name survives as an alias. Parameters are outside the test. v2.json made limit required, tickets list sends none, and a test shows the stand-in answering 400, the change oasdiff reports below. Cobra gives a command the same escape through its Aliases field.

Run oasdiff against the last released document

oasdiff compares the documents, not the CLI. record.sh runs it from oasdiff/go.mod, which pins v1.11.7 as a Go tool, three ways:

cat recorded/oasdiff.txt
$ oasdiff breaking v1.json v2.json
1 changes: 1 error, 0 warning, 0 info
error	[request-parameter-became-required] at ../v2.json
	in API GET /tickets
		the 'query' request parameter 'limit' became required


exit 0
$ oasdiff breaking v1.json v2.json --fail-on ERR
exit 1
$ oasdiff breaking v1.json v2.json --fail-on ERR --severity-levels severity-levels.txt
2 changes: 2 error, 0 warning, 0 info
error	[request-parameter-became-required] at ../v2.json
	in API GET /tickets
		the 'query' request parameter 'limit' became required

error	[api-operation-id-removed] at ../v1.json
	in API GET /tickets/{ticketId}
		api operation id 'getTicket' removed and replaced with 'showTicket'

The first run found an error and exited 0. The second fails the step. By default the rename is INFO, below anything breaking prints. One line in oasdiff/severity-levels.txt, api-operation-id-removed err, raises it, as the severity levels section describes.1 For a CLI whose commands are operationIds, that line is the rename check. record.sh ran both documents through oasdiff v1.39.0 as well, on a Go 1.26 toolchain that Go downloaded. It lists the check as info, and the same two runs exit 0 and 1.

Check an sdkgen CLI after the document changes

The sdkgen go-cli target names its commands from the model’s entities, so load ticket came from the path and survived the rename. generate.sh scaffolds a project with go and go-cli targets from v1.json, regenerates it from v2.json, and keeps each word table and model. The checker reads those files the same way:

node coverage.mjs v2.json go-cli generated/sdkgen-v2 --released generated/sdkgen-v1
v2.json against go-cli generated/sdkgen-v2: 7 operations, 4 command mappings, 0 ignored
  unmapped: createTicket (POST /tickets) has no command
  unmapped: deleteTicket (DELETE /tickets/{ticketId}) has no command
  unmapped: refundTicket (POST /tickets/{ticketId}/refund) has no command
3 problems

No released name went missing. Three operations have no command, because the stock target registers list, load and update, and the model files the refund under create. Either add the words in the target’s fragments under .sdk/, or list the three in go-cli-ignore.json with that reason. A test checks the word list against the help text of the built binary.

The second check is whether the pull request regenerated at all. record.sh committed v2.json without regenerating, then ran both checks:

cat recorded/pr-4.44.3.txt
@voxgig/sdkgen 4.44.3, api.json changed and committed, nothing regenerated
voxgig-sdkgen doctor exit 0
  .sdk matches the scaffold (0 additive, 0 customized)
npm run check-drift exit 1
  check-drift: deleting and regenerating every target ...
  check-drift: DRIFT. These differ from what the generator produces:

doctor compares .sdk/ with the scaffold and passed the stale tree. The scaffold’s check-drift.sh regenerates every target and lists 14 changed files, go/entity/types.go among them, where Limit lost its pointer when the parameter became required. No file under go-cli/ changed. @voxgig/sdkgen 4.45.0 recorded the same exits and the same list.

Catch the same drift in OpenAPI Generator and Speakeasy

OpenAPI Generator’s bash generator writes a script whose commands are the operationIds. Regenerate it on every document change, and compare the dispatch table with the released one:

node coverage.mjs v2.json oag-bash generated/oag-v2 --released generated/oag-v1
v2.json against oag-bash generated/oag-v2: 7 operations, 7 command mappings, 0 ignored
  removed command: "getTicket" was in the released CLI, and scripts that call it break
1 problem

Coverage is complete by construction, so the released-name check carries the job. The bash generator’s option list has no alias, and the customization page has --operation-id-name-mappings. generate.sh passes it showTicket=getTicket from oag-name-mappings.txt, and the command keeps its released name:

node coverage.mjs v2.json oag-bash generated/oag-v2-mapped --released generated/oag-v1
v2.json against oag-bash generated/oag-v2-mapped: 7 operations, 7 command mappings, 0 ignored
in sync

The mapping renames rather than aliases, so the script offers getTicket and no showTicket.

Speakeasy’s CLI target writes a Go CLI on Cobra, with commands grouped by tag. Its GitHub setup runs regeneration in pr mode, which opens a pull request, the review step Restish lacks. Its versioning page says it does not analyze added or removed operations. “Only the info.version field and the overall document checksum are evaluated,” it adds. Its CLI pages name commands from tags and operation names, and document no alias setting. x-speakeasy-name-override pins a method name to an operation, and those pages do not say whether it reaches CLI commands. Speakeasy needs an account, so it was not run here.

Check it worked

Twenty-three tests cover the checker, each generated table, the CLI and the recorded runs:

node --test coverage.test.mjs
# tests 23
# suites 0
# pass 23
# fail 0
# cancelled 0
# skipped 0
# todo 0

The neighbors are tested too: a dropped command, a rename without an alias, a stale ignore entry, a moved route, a route the document dropped, and an operation with no id. An alias that repeats another entry’s name fails the check, and the CLI refuses that table. The command line refuses empty inputs, an unknown flag and a flag with no file, with exit 2.

When it goes wrong

A Restish user’s script keeps working after the server renames the operation, then fails. Restish keeps the operations it synced, and in the recorded run the former command answered until restish api sync:

cat recorded/restish.txt
after the sync
  restish bo get-ticket t1
  exit 2  unknown command "get-ticket" for "restish bo"; did you mean "get-venue"?; run "restish bo --help" to see generated operations
after the sync, the new name
  restish bo show-ticket t1
  exit 0  {"id":"t1","event":"Hamlet","seat":"B12","status":"issued"}

Add x-cli-aliases: [get-ticket] to the renamed operation, as v2-aliased.json does, and the former name answers 0 again after the next sync.

The coverage step passes while operations go missing. The table came from a loader that found nothing, or the document failed to parse. The checker refuses an empty table and a document with no operations, and exits 2 so the step fails.

When not to do this

Do not rename an operationId to tidy a name. Restish and the OpenAPI Generator script name their commands after it, and Speakeasy’s method names come from it. Each rename is a removed command for some user’s script, so pin the name with x-cli-name or an alias instead.

Do not rely on voxgig-sdkgen doctor to catch a document change. It reads .sdk/ and passed a tree that was never regenerated. Run check-drift, which deletes and regenerates every target and needs a clean git tree. Skip sdkgen for a CLI alone: the go-cli target covers three of five verbs until you edit its fragments.

Restish fits an internal team better than any generator here, because nothing needs regenerating. OpenAPI Generator suits a reader who wants a free, offline script per release. Speakeasy suits a team that wants each regeneration as a pull request and accepts a hosted vendor.

Last verified

Verified 2026-11-22 against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3 and 4.45.0, @voxgig/apidef 8.32.4, oasdiff v1.11.7 and v1.39.0, Restish v2.3.0 and OpenAPI Generator 7.27.0 on OpenJDK 21. Every output block is what the command preceding it printed. Generation ran in generate.sh and the tool runs in record.sh, with their results committed under generated/ and recorded/. The page’s checks need Node alone: oasdiff, Restish and the go-cli binary ran once, in record.sh, and the page shows what they printed. Speakeasy is described from its CLI, versioning and GitHub setup pages as read that day, without an account.

Footnotes

  1. oasdiff keeps its own version in one variable, Version in build/build.go, with the comment “to be injected at build time” and the value main. The release build injects it through -X in .goreleaser.yml. Built as a Go tool from the pinned module, nothing injects it, and the binary that ran here answers oasdiff version main. Which release that was is written in go.mod, as v1.11.7. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.