How to detect breaking changes between two OpenAPI versions in CI#

A pull request removes status from the Order schema. The reviewer reads a two-line diff in schemas/Order.yaml, the checks pass, and it merges. After the deploy, every mobile client that reads order.status fails, because no step in CI compared the spec on the branch with the spec the clients were built against.

What you get

You will end up with a GitHub Actions job that bundles both versions of the spec, runs oasdiff, comments the verdict on the pull request, and fails on a breaking change. This is for you if clients you do not control are built from your OpenAPI document.

Short answer

Check out the base branch beside the pull request, bundle both OpenAPI documents with the same Redocly version, and run oasdiff breaking base.yaml head.yaml --fail-on ERR. Post the markdown report as a pull request comment, and treat any exit status other than 0 or 1 as a failed check, never a clean one. pb33f openapi-changes and OpenAPITools openapi-diff do the same job with different rules.

You will need

Node 22 or later, an OpenAPI 3 document tracked in git, and Docker on the runner, which GitHub’s ubuntu-latest image provides. Verified against Node 22.22.0. The page on making additive API changes without a new version covers what to do once this job blocks a change.

Approaches compared

oasdiff publishes its rules:¹ its documentation lists 339 breaking kinds of change, so a verdict can be checked against a page. openapi-changes hands that decision to you and costs you a file of rules to keep current. OpenAPITools openapi-diff runs from a Docker image with --fail-on-incompatible, at the price of a JVM and a 3.0-only claim. If Optic’s optic diff is already in your pipeline, plan to replace it: the CLI is archived.²

Bundle both sides with one tool

Diff what the clients are built from, and build both sides the same way. A spec split across files reaches the differ as a different shape from the same spec bundled, and the oasdiff action refuses external $refs by default. This script runs one Redocly bundle over each side.

export function bundleBoth(baseDir, headDir, outDir, entry = 'openapi.yaml') {
  // A stale bundle from an earlier run must never stand in for this one.
  for (const side of ['base', 'head']) rmSync(join(outDir, `${side}.yaml`), { force: true })
  const head = join(headDir, entry)
  if (!existsSync(head)) {
    throw new Error(`${head} does not exist: a deleted or moved spec breaks every client`)
  }
  const base = join(baseDir, entry)
  if (!existsSync(base)) return { skipped: `${base} does not exist, so no client depends on it yet` }

  mkdirSync(outDir, { recursive: true })
  const sides = {}
  for (const [side, file] of [['base', base], ['head', head]]) {
    const doc = canonical(bundle(file))
    const refs = externalRefs(doc)
    if (refs.length) throw new Error(`${file} still has external refs after bundling: ${refs.join(', ')}`)
    const text = YAML.stringify(doc)
    writeFileSync(join(outDir, `${side}.yaml`), text)

A missing head fails, because deleting the spec breaks every client at once. A missing base is skipped, because nothing can depend on a spec that does not exist yet. canonical sorts keys, so moving a path within the file does not show up in the text a reviewer reads.

node bundle-both.mjs base head .diff

base  base/openapi.yaml -> .diff/base.yaml  paths 2, schemas 1
head  head/openapi.yaml -> .diff/head.yaml  paths 2, schemas 1
bundled with @redocly/cli 2.54.3 on both sides
the bundles differ: run oasdiff

The two bundles differ only where the pull request changed something.

node bundle-both.mjs base head .diff > /dev/null && diff -u --label base --label head .diff/base.yaml .diff/head.yaml

--- base
+++ head
@@ -15,17 +15,10 @@
           type:
             - string
             - "null"
-        status:
-          enum:
-            - pending
-            - paid
-            - shipped
-          type: string
         total:
           type: number
       required:
         - id
-        - status
         - total
       type: object

Run oasdiff and comment on the pull request

Put the oasdiff settings in the repository, where the CLI and the action both find them.

fail-on: ERR
format: markdown
allow-external-refs: false

The job checks the base branch out with git worktree, bundles both sides, and runs oasdiff 1.33.0 from its Docker image. The github.base_ref context names the target branch.

      - name: Bundle both sides with the same Redocly
        run: node bundle-both.mjs .base/api api .diff
      - name: Compare with oasdiff
        id: oasdiff
        if: hashFiles('.diff/base.yaml') != ''
        run: |
          status=0
          docker run --rm -v "$PWD:/w" -w /w tufin/oasdiff:v1.33.0 \
            breaking .diff/base.yaml .diff/head.yaml > .diff/report.md || status=$?
          echo "status=$status" >> "$GITHUB_OUTPUT"
      - name: Comment, then pass or fail
        if: steps.oasdiff.outcome == 'success'
        env:
          GH_TOKEN: ${{ github.token }}
          STATUS: ${{ steps.oasdiff.outputs.status }}
          PR: ${{ github.event.pull_request.number }}
        run: |
          verdict=0
          node comment.mjs "$STATUS" .diff/report.md > .diff/comment.md || verdict=1
          gh pr comment "$PR" --edit-last --create-if-none --body-file .diff/comment.md \
            || echo "::warning::could not comment; a fork's token is read-only"
          exit "$verdict"

The workflow itself was not run for this page. bundle-both.mjs and comment.mjs ran as shown on this page, and oasdiff 1.33.0 ran locally from a go install build rather than the image.

oasdiff exits 0 for no breaking change and 1 when --fail-on matches, as its breaking-changes guide says. Run locally on a spec it could not load, it exited 102 with an empty report. comment.mjs turns an empty report into “no breaking changes” only when the status is 0. It refuses a status that is not a number, because a misspelled step output expands to an empty string, which JavaScript’s Number reads as 0.

The report below is a recording, not a live run. oasdiff 1.33.0 produced it locally from the two bundles built earlier, and it is committed as fixtures/oasdiff-breaking.md. The captured command runs comment.mjs over that file, so CI never runs oasdiff.

node comment.mjs 1 fixtures/oasdiff-breaking.md

### oasdiff: breaking changes

# API Changelog 2.4.0 vs. 2.4.0


## API Changes

### GET /orders
- :warning: removed the required property `items/status` from the response with the `200` status


### GET /orders/{orderId}
- :warning: removed the required property `status` from the response with the `200` status

oasdiff reports a change once for each endpoint it reaches, so one schema change produced two findings. The header compares 2.4.0 with 2.4.0, because the pull request left info.version alone.³

Check it worked

Open a pull request that removes a required property from a response schema, as head/ does here. Run locally against the two bundles, oasdiff printed two response-required-property-removed errors and exited 1. The job comments the report and goes red.

Then confirm the job cannot go green by crashing.

node comment.mjs 102 .diff/report.md

### oasdiff did not run

It exited 102 before comparing anything, so this pull request is unchecked. See the job log.

node --test ci.test.mjs

1..9
# tests 9
# suites 0
# pass 9
# fail 0

The tests cover a reordered copy of the base bundling to the same bytes, a $ref that does not resolve, a deleted spec, the crash just shown, and an empty status.

When it goes wrong

The action fails on every pull request with encountered disallowed external reference. Its allow-external-refs input defaults to false, which blocks a split spec. Bundle first, as bundle-both.mjs does, and keep the default, which guards against SSRF on untrusted pull requests.

openapi-changes reports a breaking change on a pull request that changed nothing. Run locally on a bundled base and the same spec unbundled, it reported “Removed schemas” as breaking, and oasdiff’s changelog reported api-schema-removed. The documents were equivalent but not the same shape. Bundle both sides with the same tool and version.

The job overwrites another workflow’s comment. gh pr comment --edit-last edits the last comment by the current user, per the GitHub CLI manual, and every workflow using GITHUB_TOKEN comments as the same bot. Start the body with a marker line and find the comment by it with gh api instead.

An encrypted copy of your spec leaves CI. The action’s review input defaults to true and uploads the comparison to oasdiff.com when it finds changes. Set review: false to keep the spec in CI.

When not to do this

Do not gate a spec that no external client builds from. An internal API whose callers deploy with it gets more from contract tests than from a rule list written for strangers.

Do not let the job become the versioning policy. oasdiff says a change breaks clients. A person decides whether that means a new major version, a deprecation window, or a refused merge, and records it where a reviewer can find it.

Do not silence findings with err-ignore to get a release out. Each ignored line is a break you shipped on purpose, so write down who agreed to it.

Do not compare against the branch tip when clients are built from releases. If consumers use tagged versions, diff the pull request against the last release tag instead of main.

Related how-tos

• Split and bundle a large OpenAPI document

• Detect a reference site drifting from its OpenAPI spec

• Evolve an error contract without breaking clients

• Publish an OpenAPI 3.0 copy beside your 3.1 document

Last verified

Verified 2026-10-15 against Node 22.22.0, @redocly/cli 2.54.3 and yaml 2.9.1. Every output block is what the preceding command printed in the page’s code directory after npm ci. oasdiff 1.33.0 and openapi-changes 0.2.13 were built with go install and run locally, outside the captured runs. The workflow was not run on GitHub.