# How to detect a reference site drifting from its OpenAPI spec > Fail CI when the built reference carries an operation the spec removed, lacks one it added, or shows a deprecated operation as current, and lint the spec first. Source: https://voxgig.com/howto/check-a-reference-site-in-ci - Audience: platform-team - Level: advanced - Verified: 2026-09-25 - Published: 2026-09-25 ## Short answer Extract every operationId and tag from the OpenAPI document and every operation and tag anchor from the built HTML, and diff the two lists in CI. Fail on an operation that is on one side only, and on one the spec deprecates that the page shows as current. Lint the spec with Redocly CLI or Spectral before rendering, and run oasdiff between releases so you know what the site had to change. --- ## You will need Node 22 or later, an OpenAPI 3 document, and a reference built from it by [Redoc](https://redocly.com/docs/cli/commands/build-docs), whose anchors this page reads. Verified 2026-09-25 against Node 22.22.2, `@redocly/cli` 2.54.3, `@stoplight/spectral-cli` 6.16.3, and `yaml` 2.9.1. oasdiff is a Go binary and is not run here. The change list below comes from a Node script that covers the operation level only, and the page says where it stops short. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [A coverage script over the built site](https://spec.openapis.org/oas/v3.1.0.html#operation-object) | You publish a rendered site and need proof that this release reached it | Yours to maintain, and it breaks the day the renderer changes its anchor format | The reference is rendered on request from the spec, so there is no built page to drift | | [oasdiff](https://www.oasdiff.com/docs/getting-started) | You need the list of what changed between two releases, and which changes break a client | A Go binary in CI and a change list that says nothing about the site | You only need to know whether the site matches, not what moved | | [Redocly CLI lint](https://redocly.com/docs/cli/commands/lint) | The site is built by Redoc and you want the same rules the renderer will trip on | A ruleset to configure, and a check that ends at the spec's edge | Your renderer is not Redoc and its failure modes differ | | [Spectral](https://docs.stoplight.io/docs/spectral/b8391e051b7d8-overview) | You already keep a style ruleset and want the lint to carry naming rules too | A ruleset to write, a second linter to keep in step, and no view of the HTML | Redocly already lints the spec in the same pipeline | | [Validation built into the renderer](https://redocly.com/docs/cli/commands/build-docs) | You want no extra step, and a build that refuses a spec it cannot render | An error with no line number, found at render time rather than at review time | Anyone needs to know which line to fix | Only one of the five can see the HTML. oasdiff, the two linters, and the checks built into the renderer all read the spec. A spec-only check passes when the site was never rebuilt at all, which is the failure this page is about. The coverage script sees the page and nothing else, and it costs you a script that knows how the renderer names its anchors and stops working when that changes. ## Lint the spec before anything renders A description that is an object instead of a string, and a reference to a schema that does not exist, are cheap to find in the spec. Both are expensive to find on the page. `spec/broken.yaml` carries one of each. ```bash npx redocly lint spec/broken.yaml --format=stylish ``` ```text output spec/broken.yaml: 45:9 error struct Expected type `string` but got `object`. 60:19 error no-unresolved-refs Can't resolve $ref ``` Spectral finds the same two under its own names, with the JSON path beside each. ```bash npx spectral lint spec/broken.yaml ``` ```text output 44:19 error oas3-schema "description" property must be string. paths./pets/search.get.description 60:25 error invalid-ref '#/components/schemas/Pett' does not exist paths./pets/search.get.responses[200].content.application/json.schema.items.$ref ✖ 2 problems (2 errors, 0 warnings, 0 infos, 0 hints) ``` Both exit 1, so either one fails the job. What the renderer does with the same file is the reason to run one of them first: `redocly build-docs spec/broken.yaml` stops with `(_.description || "").concat is not a function` and no line number. The build refused the file, which is better than a blank section, and worse than either message the linters printed. ## List what changed between the releases The published site was built from release 1.0.0. Release 1.1.0 removes `getPet`, adds `searchPets`, and marks `deletePet` deprecated. In CI that list comes from [`oasdiff changelog`](https://www.oasdiff.com/docs/getting-started), which exits 1 on a breaking change and takes `--fail-on` to say which level stops the job.[^1] The script here reads the two documents and reports the same three kinds of change at the operation level. ```bash node changes.mjs spec/previous.yaml spec/openapi.yaml ``` ```text output 1.0.0 -> 1.1.0 added searchPets GET /pets/search deprecated deletePet DELETE /pets/{petId} removed getPet GET /pets/{petId} 1 removed: every one must be gone from the site or redirected ``` The removal is what matters. An added operation that never reached the site is undocumented, which is embarrassing. A removed operation that stayed on the site is a page describing nothing, which is the ticket in the opening paragraph. ## Read the anchors the site carries Redoc writes an anchor per operation, `id="operation/"`, and a section per tag, `id="tag/"`. The operation heading carries a Deprecated badge when the spec sets the flag. Those three things are all the script reads. ```js title="coverage.mjs" export function anchorsOf(html) { const operations = new Map() // The id attribute itself, not data-section-id, which Redoc writes on the same element. const re = /(?/) + 5) operations.set(m[1], { deprecated: />\s*Deprecated\s* t[1])) return { operations, tags } } ``` The operationId is the only name an operation has on both sides.[^2] The spec requires it to be unique and nothing more, so the script refuses a document where one is missing or used twice, because the anchor for that operation cannot be checked. Run the check with the shipped spec against the site as published, which is `site/index.html` in the sample, built from `spec/previous.yaml` by `redocly build-docs` and committed as the stand-in for the live page. ```bash node coverage.mjs spec/openapi.yaml site/index.html ``` ```text output spec/openapi.yaml 1.1.0 against site/index.html deprecated-as-current deletePet deprecated in the spec, rendered without the badge stale getPet on the site and not in the spec undocumented searchPets GET /pets/search is in the spec and not on the site 3 findings, the site does not match the spec ``` The check exits 1 with three findings, each naming its operation. The first one is the pitfall. Redoc reads the `deprecated` flag and draws the badge, so nothing is wrong with the renderer. The site was built before the flag was set, and the page shows a current operation that the spec has retired. No spec-only check can see that, because the spec is correct. Rebuild from the shipped spec and the same command passes. `--build` renders the spec into a temporary file with `redocly build-docs` and checks that. ```bash node coverage.mjs spec/openapi.yaml --build ``` ```text output spec/openapi.yaml 1.1.0 against a fresh build site matches spec: 4 operations, 1 deprecated, 2 tags ``` A removed operation whose old anchor is redirected is recorded rather than failed. Put the pairs in a file, one `from to` per line, and pass `--redirects`. That case exists for a site with a page per operation; on a single Redoc page the fragment never reaches the server, so gone means gone. ## Check it worked Eleven tests pin the behavior, and two carry the page. The site built from 1.0.0 matches the 1.0.0 spec and fails the 1.1.0 spec on exactly `deprecated-as-current deletePet`, `stale getPet`, and `undocumented searchPets`. And a page with no operation anchors at all is an error rather than a clean run with nothing to compare. ```bash node --test coverage.test.mjs ``` ```text output 1..11 # tests 11 # suites 0 # pass 11 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 176.567449 ``` To prove the job fails on a stale page, delete an operation from the spec and run the check against the site you have. The finding reads `stale`, the exit code is 1, and the deploy that would have carried the stale page stops. ## When it goes wrong The check reports no operation anchors and exits 1. The renderer changed its anchor format, or you pointed the script at a page that loads the spec in the browser and has no prerendered content. Open the HTML and look for `id="operation/`. Redoc 3 gives each operation its own route rather than a position on one page, so the pattern will need updating when you move to it.[^3] A tag with a space in its name is reported twice: undocumented under its name, stale under its slug. Redoc turns the name into a slug for the anchor and the script compares the raw name. Add the same slug rule to `tagsOf`, and keep it beside the anchor pattern, since both belong to the renderer. The check passes, but a customer still sees the old page. The job compared the fresh build, not the deployed site. Fetch the published HTML in the job and compare that, and treat `--build` as the proof that the renderer still emits the anchors, not as proof of what is live. A deprecated operation passes and the page shows no badge. The renderer ignores the flag. The script can only read what the page carries, so this one is a renderer bug to file. Until it is fixed, the finding to expect from a renderer that does read the flag is `deprecated-as-current`. ## When not to do this Do not build this check for a reference that renders the spec on request, in the browser, from the same URL the spec is served at. There is no built page to drift, and the check would compare the spec with itself. Spend the effort on the spec's URL instead. Do not let the coverage script replace oasdiff. It sees anchors and badges and nothing about parameters, schemas, or responses, so a changed request body reaches the site with a green check. oasdiff exists for that, and it runs on the spec alone. Do not treat a passing `--build` as proof the site is up to date. It proves the renderer emits the anchors the script expects, from the spec you handed it, on the machine running the job. The live page is a separate fetch. Do not skip the lint because the coverage check exists. A spec that fails to render produces no anchors, which the coverage check reports as a broken renderer, one step later, and with less information than the linter had. ## Related how-tos - [Publish an OpenAPI document at a stable URL for agents](/howto/publish-openapi-at-a-stable-url) - [Enforce a naming convention across 40 microservice specs](/howto/enforce-naming-conventions-across-many-specs) ## Last verified Verified 2026-09-25 against Node 22.22.2, `@redocly/cli` 2.54.3, `@stoplight/spectral-cli` 6.16.3, and `yaml` 2.9.1. Every output block is what the command preceding it printed. The check reads a site built by `redocly build-docs` from the previous release and committed to the sample, not a deployed one. oasdiff was not run; the change list is the sample's own script. [^1]: oasdiff's [own description of itself](https://www.oasdiff.com/) says that every field of the OpenAPI specification, and every way each one can change, has been enumerated, at more than 15,000 possible edits. Each is either covered by one of its 755 checks or excluded with a written reason, and its build fails when a new field appears that nobody has classified. A project that counts the ways a document can change, and then writes down why it ignores some of them, has done the paperwork the specification left to others. [^2]: The [OpenAPI Specification](https://spec.openapis.org/oas/v3.1.0.html#operation-object) gives the `operationId` one rule and one hint. It MUST be unique among all operations, and because tools and libraries MAY use it to identify an operation, it is RECOMMENDED to follow common programming naming conventions. Redoc puts it in a URL fragment, Spectral warns when it is missing, and SDK generators turn it into a method name. This page's script treats it as the only name an operation has. The specification calls all of that MAY. [^3]: Redocly's [announcement of Redoc 3](https://redocly.com/blog/redoc-3-whats-new) puts linking in its comparison table. Redoc 2 has scroll positions on one long page, and Redoc 3 has a separate route per operation, tool, and schema, with hash or history routing. Dark mode is built in, and the nested JavaScript theme object is replaced by CSS custom properties. The anchors this page reads are the scroll positions, so the script has a shelf life, and the announcement is the notice.