How to detect a reference site drifting from its OpenAPI spec#

A release removes an endpoint from the spec and the pipeline goes green, but the published reference still shows that endpoint with a Try it button. A week later a customer files a ticket against a page that describes nothing on the server. The spec linter and the deploy passed without anything comparing the site to the spec it was built from.

What you get

You will end up with a CI job that fails when the built reference carries an operation the spec no longer has. It also fails when the site lacks an operation the spec added, or shows a deprecated operation as current. A lint step in front of it fails a malformed spec before it renders. This is for you if you publish a reference site from an OpenAPI document and ship releases that change it.

Short answer

Extract every operationId and tag from the OpenAPI document and every operation and tag anchor from the built HTML, and diff the two lists in CI. Fail on an operation that is on one side only, and on one the spec deprecates that the page shows as current. Lint the spec with Redocly CLI or Spectral before rendering, and run oasdiff between releases so you know what the site had to change.

You will need

Node 22 or later, an OpenAPI 3 document, and a reference built from it by Redoc, whose anchors this page reads. Verified 2026-09-25 against Node 22.22.2, @redocly/cli 2.54.3, @stoplight/spectral-cli 6.16.3, and yaml 2.9.1. oasdiff is a Go binary and is not run here. The change list below comes from a Node script that covers the operation level only, and the page says where it stops short.

Approaches compared

Only one of the five can see the HTML. oasdiff, the two linters, and the checks built into the renderer all read the spec. A spec-only check passes when the site was never rebuilt at all, which is the failure this page is about. The coverage script sees the page and nothing else, and it costs you a script that knows how the renderer names its anchors and stops working when that changes.

Lint the spec before anything renders

A description that is an object instead of a string, and a reference to a schema that does not exist, are cheap to find in the spec. Both are expensive to find on the page. spec/broken.yaml carries one of each.

npx redocly lint spec/broken.yaml --format=stylish

spec/broken.yaml:
  45:9   error    struct              Expected type `string` but got `object`.
  60:19  error    no-unresolved-refs  Can't resolve $ref

Spectral finds the same two under its own names, with the JSON path beside each.

npx spectral lint spec/broken.yaml

 44:19  error  oas3-schema  "description" property must be string.      paths./pets/search.get.description
 60:25  error  invalid-ref  '#/components/schemas/Pett' does not exist  paths./pets/search.get.responses[200].content.application/json.schema.items.$ref

✖ 2 problems (2 errors, 0 warnings, 0 infos, 0 hints)

Both exit 1, so either one fails the job. What the renderer does with the same file is the reason to run one of them first: redocly build-docs spec/broken.yaml stops with (_.description || "").concat is not a function and no line number. The build refused the file, which is better than a blank section, and worse than either message the linters printed.

List what changed between the releases

The published site was built from release 1.0.0. Release 1.1.0 removes getPet, adds searchPets, and marks deletePet deprecated. In CI that list comes from oasdiff changelog, which exits 1 on a breaking change and takes --fail-on to say which level stops the job.¹ The script here reads the two documents and reports the same three kinds of change at the operation level.

node changes.mjs spec/previous.yaml spec/openapi.yaml

1.0.0 -> 1.1.0
  added       searchPets   GET /pets/search
  deprecated  deletePet    DELETE /pets/{petId}
  removed     getPet       GET /pets/{petId}
  1 removed: every one must be gone from the site or redirected

The removal is what matters. An added operation that never reached the site is undocumented, which is embarrassing. A removed operation that stayed on the site is a page describing nothing, which is the ticket in the opening paragraph.

Read the anchors the site carries

Redoc writes an anchor per operation, id="operation/<operationId>", and a section per tag, id="tag/<name>". The operation heading carries a Deprecated badge when the spec sets the flag. Those three things are all the script reads.

export function anchorsOf(html) {
  const operations = new Map()
  // The id attribute itself, not data-section-id, which Redoc writes on the same element.
  const re = /(?<![-\w])id="operation\/([^"]+)"/g
  let m
  while ((m = re.exec(html))) {
    const from = m.index
    const rest = html.slice(from)
    const heading = rest.slice(0, rest.search(/<\/h2>/) + 5)
    operations.set(m[1], { deprecated: />\s*Deprecated\s*</.test(heading) })
  }
  const tags = new Set([...html.matchAll(/id="tag\/([^"/]+)"/g)].map((t) => t[1]))
  return { operations, tags }
}

The operationId is the only name an operation has on both sides.² The spec requires it to be unique and nothing more, so the script refuses a document where one is missing or used twice, because the anchor for that operation cannot be checked.

Run the check with the shipped spec against the site as published, which is site/index.html in the sample, built from spec/previous.yaml by redocly build-docs and committed as the stand-in for the live page.

node coverage.mjs spec/openapi.yaml site/index.html

spec/openapi.yaml 1.1.0 against site/index.html
  deprecated-as-current  deletePet    deprecated in the spec, rendered without the badge
  stale                  getPet       on the site and not in the spec
  undocumented           searchPets   GET /pets/search is in the spec and not on the site
  3 findings, the site does not match the spec

The check exits 1 with three findings, each naming its operation. The first one is the pitfall. Redoc reads the deprecated flag and draws the badge, so nothing is wrong with the renderer. The site was built before the flag was set, and the page shows a current operation that the spec has retired. No spec-only check can see that, because the spec is correct.

Rebuild from the shipped spec and the same command passes. --build renders the spec into a temporary file with redocly build-docs and checks that.

node coverage.mjs spec/openapi.yaml --build

spec/openapi.yaml 1.1.0 against a fresh build
  site matches spec: 4 operations, 1 deprecated, 2 tags

A removed operation whose old anchor is redirected is recorded rather than failed. Put the pairs in a file, one from to per line, and pass --redirects. That case exists for a site with a page per operation; on a single Redoc page the fragment never reaches the server, so gone means gone.

Check it worked

Eleven tests pin the behavior, and two carry the page. The site built from 1.0.0 matches the 1.0.0 spec and fails the 1.1.0 spec on exactly deprecated-as-current deletePet, stale getPet, and undocumented searchPets. And a page with no operation anchors at all is an error rather than a clean run with nothing to compare.

node --test coverage.test.mjs

1..11
# tests 11
# suites 0
# pass 11
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 176.567449

To prove the job fails on a stale page, delete an operation from the spec and run the check against the site you have. The finding reads stale, the exit code is 1, and the deploy that would have carried the stale page stops.

When it goes wrong

The check reports no operation anchors and exits 1. The renderer changed its anchor format, or you pointed the script at a page that loads the spec in the browser and has no prerendered content. Open the HTML and look for id="operation/. Redoc 3 gives each operation its own route rather than a position on one page, so the pattern will need updating when you move to it.³

A tag with a space in its name is reported twice: undocumented under its name, stale under its slug. Redoc turns the name into a slug for the anchor and the script compares the raw name. Add the same slug rule to tagsOf, and keep it beside the anchor pattern, since both belong to the renderer.

The check passes, but a customer still sees the old page. The job compared the fresh build, not the deployed site. Fetch the published HTML in the job and compare that, and treat --build as the proof that the renderer still emits the anchors, not as proof of what is live.

A deprecated operation passes and the page shows no badge. The renderer ignores the flag. The script can only read what the page carries, so this one is a renderer bug to file. Until it is fixed, the finding to expect from a renderer that does read the flag is deprecated-as-current.

When not to do this

Do not build this check for a reference that renders the spec on request, in the browser, from the same URL the spec is served at. There is no built page to drift, and the check would compare the spec with itself. Spend the effort on the spec’s URL instead.

Do not let the coverage script replace oasdiff. It sees anchors and badges and nothing about parameters, schemas, or responses, so a changed request body reaches the site with a green check. oasdiff exists for that, and it runs on the spec alone.

Do not treat a passing --build as proof the site is up to date. It proves the renderer emits the anchors the script expects, from the spec you handed it, on the machine running the job. The live page is a separate fetch.

Do not skip the lint because the coverage check exists. A spec that fails to render produces no anchors, which the coverage check reports as a broken renderer, one step later, and with less information than the linter had.

Related how-tos

• Publish an OpenAPI document at a stable URL for agents
• Enforce a naming convention across 40 microservice specs

Last verified

Verified 2026-09-25 against Node 22.22.2, @redocly/cli 2.54.3, @stoplight/spectral-cli 6.16.3, and yaml 2.9.1. Every output block is what the command preceding it printed. The check reads a site built by redocly build-docs from the previous release and committed to the sample, not a deployed one. oasdiff was not run; the change list is the sample’s own script.