A release renames the operationId getTicket to showTicket. Restish loads the document on its
next sync, restish bo get-ticket stops existing, and every script that called it exits 2. Your
hand-written CLI still maps tickets get to an id the document has dropped. The oasdiff job in CI
rates the rename as information, so it passes.
What you get
You will end up with a test that fails when the document and the CLI disagree, and an oasdiff step that fails on a renamed operation. This is for you if your CLI ships on a different schedule from your API.
Short answer
Write a test that collects every operationId in the current OpenAPI document and fails unless each
one maps to a registered command or sits in an ignore file with a reason. Give it the released
CLI’s command names too, so a rename that drops a command fails. Run oasdiff breaking --fail-on ERR against the last released document, with api-operation-id-removed raised to an error. For an
sdkgen CLI, run check-drift.
You will need
Node 22 or later, the current OpenAPI document and the one you last released, and Go 1.24 or later
for oasdiff. oasdiff v1.11.7 is the last release that declares Go 1.24: v1.11.8 to v1.11.10
declare Go 1.25, and every release from v1.11.11 to v1.39.0 declares Go 1.26. Verified 2026-11-22
against Node 22.23.5, @voxgig/create-sdkgen 0.31.9,
@voxgig/sdkgen 4.44.3, oasdiff v1.11.7, Restish v2.3.0, and OpenAPI Generator 7.27.0. The
coverage test is JavaScript for Node; the leaf names no language, and the test reads only JSON and
generated files. v1.json is the released Box Office document and v2.json the current one.
Reading operationIds as commands is the subject of
derive a noun-verb command tree from an OpenAPI document.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| Hand-written operationId coverage test | A hand-written CLI whose command table you can import | A table and an ignore file to keep, and a test you wrote yourself | The CLI is generated, so its table is the document |
| oasdiff with —fail-on | Any CLI, as the job that compares two documents | Exits 0 without --fail-on, and rates an operationId rename as information | You need to know what the CLI exposes, not only the document |
| OpenAPI Generator bash client | You want a generated script per release with no account and no runtime service | One command per operationId, so a rename removes the command unless a name mapping restores it | Scripts depend on stable names across renames |
Restish api sync | Your own team, against a document the server publishes | Nothing to regenerate, and no review step before a user’s command changes | Customers script the CLI and need warning before a rename |
| sdkgen regeneration with check-drift | A project that already generates a Go SDK and its go-cli target | A regeneration per document change, and three stock words that leave create and remove uncovered | You need every operation as a command without editing fragments |
| Speakeasy CLI target | Customers want a Cobra CLI regenerated by a pull request | A vendor account and API key, and versioning that ignores added or removed operations | Your build cannot depend on a vendor service |
Voxgig maintains sdkgen. It is one of six options here, not the recommendation.
Restish builds its commands from the copy of the document that its last api sync fetched.
Nothing needs regenerating, but a change reaches users without review. The coverage test and a regeneration both need someone to
act on each change, and both put that change in a pull request first. oasdiff guards the document
whatever builds the CLI, but only at the severity levels you give it.
Fail the test when the document and the command table disagree
coverage.mjs reads the document’s operationIds and a command table. It fails on an id with no
command and no ignore entry, and on a command mapped to an id the document lacks. A stale ignore
entry fails it too, and so does a released command name that is gone, a command whose method or
path differs from its id’s, and a name registered twice. A command and an alias count as names
alike. The command table of the hand-written CLI is a list of
entries such as { command: 'tickets get', operationId: 'getTicket', ... }. Run it on the table
from release 1.4.0 against the current document:
node coverage.mjs v2.json hand cli/boxoffice-1.4.mjs
v2.json against hand cli/boxoffice-1.4.mjs: 7 operations, 6 command mappings, 0 ignored
stale command: "tickets get" maps to getTicket, which the document no longer has
unmapped: showTicket (GET /tickets/{ticketId}) has no command
unmapped: refundTicket (POST /tickets/{ticketId}/refund) has no command
3 problems
It exits 1, so a CI step fails. The fix is two decisions in reviewed files. tickets get keeps its
name and maps to showTicket, with tickets show added as an alias. refundTicket goes in
cli/ignore.json with a reason, because box office staff issue refunds:
[
{ "operationId": "refundTicket", "reason": "Refunds are issued by box office staff, so the public CLI leaves them out." }
]
node coverage.mjs v2.json hand cli/boxoffice.mjs --ignore cli/ignore.json --released cli/boxoffice-1.4.mjs
v2.json against hand cli/boxoffice.mjs: 7 operations, 6 command mappings, 1 ignored
in sync
--released compares command names with the 1.4.0 table, so a rename in the CLI itself fails
until the former name survives as an alias. Parameters are outside the test. v2.json made
limit required, tickets list sends none, and a test shows the stand-in answering 400, the
change oasdiff reports below. Cobra gives a command the same escape through its
Aliases field.
Run oasdiff against the last released document
oasdiff compares the documents, not the CLI. record.sh runs it from oasdiff/go.mod, which pins
v1.11.7 as a Go tool, three ways:
cat recorded/oasdiff.txt
$ oasdiff breaking v1.json v2.json
1 changes: 1 error, 0 warning, 0 info
error [request-parameter-became-required] at ../v2.json
in API GET /tickets
the 'query' request parameter 'limit' became required
exit 0
$ oasdiff breaking v1.json v2.json --fail-on ERR
exit 1
$ oasdiff breaking v1.json v2.json --fail-on ERR --severity-levels severity-levels.txt
2 changes: 2 error, 0 warning, 0 info
error [request-parameter-became-required] at ../v2.json
in API GET /tickets
the 'query' request parameter 'limit' became required
error [api-operation-id-removed] at ../v1.json
in API GET /tickets/{ticketId}
api operation id 'getTicket' removed and replaced with 'showTicket'
The first run found an error and exited 0. The second fails the step. By default the rename is
INFO, below anything breaking prints. One line in oasdiff/severity-levels.txt,
api-operation-id-removed err, raises it, as the
severity levels section
describes.1 For a CLI whose commands are operationIds, that line is the rename check.
record.sh ran both documents through oasdiff v1.39.0 as well, on a Go 1.26 toolchain that Go
downloaded. It lists the check as info, and the same two runs exit 0 and 1.
Check an sdkgen CLI after the document changes
The sdkgen go-cli target names its commands from the model’s entities, so load ticket came from
the path and survived the rename. generate.sh scaffolds a project with go and go-cli targets
from v1.json, regenerates it from v2.json, and keeps each word table and model. The checker
reads those files the same way:
node coverage.mjs v2.json go-cli generated/sdkgen-v2 --released generated/sdkgen-v1
v2.json against go-cli generated/sdkgen-v2: 7 operations, 4 command mappings, 0 ignored
unmapped: createTicket (POST /tickets) has no command
unmapped: deleteTicket (DELETE /tickets/{ticketId}) has no command
unmapped: refundTicket (POST /tickets/{ticketId}/refund) has no command
3 problems
No released name went missing. Three operations have no command, because the stock target
registers list, load and update, and the model files the refund under create. Either add the
words in the target’s fragments under .sdk/, or list the three in go-cli-ignore.json with that
reason. A test checks the word list against the help text of the built binary.
The second check is whether the pull request regenerated at all. record.sh committed v2.json
without regenerating, then ran both checks:
cat recorded/pr-4.44.3.txt
@voxgig/sdkgen 4.44.3, api.json changed and committed, nothing regenerated
voxgig-sdkgen doctor exit 0
.sdk matches the scaffold (0 additive, 0 customized)
npm run check-drift exit 1
check-drift: deleting and regenerating every target ...
check-drift: DRIFT. These differ from what the generator produces:
doctor compares .sdk/ with the scaffold and passed the stale tree. The scaffold’s
check-drift.sh
regenerates every target and lists 14 changed files, go/entity/types.go among them, where Limit
lost its pointer when the parameter became required. No file under go-cli/ changed.
@voxgig/sdkgen 4.45.0 recorded the same exits and the same list.
Catch the same drift in OpenAPI Generator and Speakeasy
OpenAPI Generator’s bash generator writes a script whose commands are the operationIds. Regenerate it on every document change, and compare the dispatch table with the released one:
node coverage.mjs v2.json oag-bash generated/oag-v2 --released generated/oag-v1
v2.json against oag-bash generated/oag-v2: 7 operations, 7 command mappings, 0 ignored
removed command: "getTicket" was in the released CLI, and scripts that call it break
1 problem
Coverage is complete by construction, so the released-name check carries the job. The bash
generator’s option list has no alias, and the
customization page has
--operation-id-name-mappings. generate.sh passes it showTicket=getTicket from
oag-name-mappings.txt, and the command keeps its released name:
node coverage.mjs v2.json oag-bash generated/oag-v2-mapped --released generated/oag-v1
v2.json against oag-bash generated/oag-v2-mapped: 7 operations, 7 command mappings, 0 ignored
in sync
The mapping renames rather than aliases, so the script offers getTicket and no showTicket.
Speakeasy’s CLI target writes a Go CLI
on Cobra, with commands grouped by tag. Its
GitHub setup runs regeneration in pr
mode, which opens a pull request, the review step Restish lacks. Its
versioning page says it does not analyze
added or removed operations. “Only the info.version field and the overall document checksum are
evaluated,” it adds. Its CLI pages name commands from tags and operation names, and document no alias
setting. x-speakeasy-name-override
pins a method name to an operation, and those pages do not say whether it reaches CLI commands.
Speakeasy needs an account, so it was not run here.
Check it worked
Twenty-three tests cover the checker, each generated table, the CLI and the recorded runs:
node --test coverage.test.mjs
# tests 23
# suites 0
# pass 23
# fail 0
# cancelled 0
# skipped 0
# todo 0
The neighbors are tested too: a dropped command, a rename without an alias, a stale ignore entry, a moved route, a route the document dropped, and an operation with no id. An alias that repeats another entry’s name fails the check, and the CLI refuses that table. The command line refuses empty inputs, an unknown flag and a flag with no file, with exit 2.
When it goes wrong
A Restish user’s script keeps working after the server renames the operation, then fails. Restish
keeps the operations it synced, and in the recorded run the former command answered until
restish api sync:
cat recorded/restish.txt
after the sync
restish bo get-ticket t1
exit 2 unknown command "get-ticket" for "restish bo"; did you mean "get-venue"?; run "restish bo --help" to see generated operations
after the sync, the new name
restish bo show-ticket t1
exit 0 {"id":"t1","event":"Hamlet","seat":"B12","status":"issued"}
Add x-cli-aliases: [get-ticket] to the renamed operation, as v2-aliased.json does, and the
former name answers 0 again after the next sync.
The coverage step passes while operations go missing. The table came from a loader that found nothing, or the document failed to parse. The checker refuses an empty table and a document with no operations, and exits 2 so the step fails.
When not to do this
Do not rename an operationId to tidy a name. Restish and the OpenAPI Generator script name their
commands after it, and Speakeasy’s method names come from it. Each rename is a removed command for
some user’s script, so pin the name with x-cli-name or an alias instead.
Do not rely on voxgig-sdkgen doctor to catch a document change. It reads .sdk/ and passed a
tree that was never regenerated. Run check-drift, which deletes and regenerates every target and
needs a clean git tree. Skip sdkgen for a CLI alone: the go-cli target covers three of five verbs
until you edit its fragments.
Restish fits an internal team better than any generator here, because nothing needs regenerating. OpenAPI Generator suits a reader who wants a free, offline script per release. Speakeasy suits a team that wants each regeneration as a pull request and accepts a hosted vendor.
Related how-tos
Last verified
Verified 2026-11-22 against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9,
@voxgig/sdkgen 4.44.3 and 4.45.0, @voxgig/apidef 8.32.4, oasdiff v1.11.7 and v1.39.0, Restish
v2.3.0 and OpenAPI Generator 7.27.0 on OpenJDK 21. Every output block is what the command
preceding it printed. Generation ran in generate.sh and the tool runs in record.sh, with their
results committed under generated/ and recorded/. The page’s checks need Node alone: oasdiff,
Restish and the go-cli binary ran once, in record.sh, and the page shows what they printed. Speakeasy is described from its CLI, versioning and
GitHub setup pages as read that day, without an account.
Footnotes
-
oasdiff keeps its own version in one variable,
Versioninbuild/build.go, with the comment “to be injected at build time” and the valuemain. The release build injects it through-Xin.goreleaser.yml. Built as a Go tool from the pinned module, nothing injects it, and the binary that ran here answersoasdiff version main. Which release that was is written ingo.mod, as v1.11.7. ↩︎ Back to text