How to snapshot-test CLI output across operating systems#

Your snapshot test fails on the Windows runner after passing on your Mac. The diff shows the same words: a path with backslashes where the snapshot has slashes, a temp directory with a different name, and a carriage return at the end of every line. You rerun the job and it fails the same way, so the team talks about dropping Windows from the matrix.

What you get

You will end up with golden-file tests that run your real binary, a normalizer that makes one transcript true on three operating systems, and a matrix job that reports each one. This is for you if your CLI’s tests pass only where they were recorded.

Short answer

Run the built CLI as a child process with spawnSync under node:test, and capture stdout, stderr, and the exit code into one transcript. Normalize it before comparing: CRLF to LF, ANSI escapes stripped, the scratch directory replaced by <tmp>, and backslashes under it turned into slashes. Compare with a committed golden file, rewrite it only behind an explicit UPDATE_GOLDEN=1, and run the same test on Linux, macOS, and Windows in a matrix with fail-fast off.

You will need

Node 22 or later and a CLI that builds to one binary or runs from one entry file. Verified 2026-10-21 against Node 22.22.0, with the testscript, Jest, and syrupy behavior read from their documentation rather than run. The sample CLI is two commands over an in-memory list. Its output carries a path, a color code, and a progress line on purpose, because those are the three things that differ between machines.

Approaches compared

The framework rows and the hand-rolled row disagree about where the noise goes. Jest and syrupy record what the command printed and leave you to strip the machine out of it afterwards, with a property matcher or a serializer. testscript substitutes $WORK and environment variables before it compares,¹ and the hand-rolled runner normalizes the transcript on the way in. Framework snapshots are quicker to adopt, because the first run writes them, but the noise that run recorded fails the first run on a second operating system.

Capture all three streams in one transcript

A CLI test that reads stdout alone misses the two things a script branches on, the exit code and the error text. The sample CLI exports a CSV and reports where it put it, which drags a path, a progress line on stderr, and an optional color code into its output on purpose.

  if (command === 'export') {
    const at = rest.indexOf('--out')
    if (at < 0 || !rest[at + 1]) {
      console.error('meters: export needs --out DIR')
      return 2
    }
    const dir = resolve(rest[at + 1])
    mkdirSync(dir, { recursive: true })
    const file = join(dir, 'meters.csv')
    const rows = METERS.map((m) => `${m.id},${m.serial},${m.state}`)
    writeFileSync(file, ['id,serial,state', ...rows].join('\n') + '\n')
    console.error(`meters: exporting ${METERS.length} meters`)
    console.log(`${green('ok')} wrote ${METERS.length} meters to ${file}`)
    return 0
  }

The last line of the file sets process.exitCode instead of calling process.exit(), because process.exit can end the process before a write to a pipe has drained. A test that captures stdout through a pipe then records a truncated transcript on one platform and a full one on another.

node cli.mjs export

meters: export needs --out DIR

The transcript puts the three streams under markers, so a failing diff names which one changed:

// The transcript a golden file holds: the exit code, then each stream under a
// marker, so a diff names which of the three changed.
export function transcript({ status, stdout, stderr }) {
  return `exit ${status}\n--- stdout ---\n${stdout}--- stderr ---\n${stderr}`

Normalize the machine out before you compare

Four things differ between a Linux, a macOS, and a Windows run of the same command, and the normalizer removes each one before the compare. Line endings become LF, because a tool writing in text mode on Windows emits CRLF.² Color goes, through util.stripVTControlCharacters, because a CI runner may or may not look like a terminal. The scratch directory becomes <tmp>, because os.tmpdir answers differently on every machine, and backslashes under it become slashes, because Windows paths spell the separator the other way.

// Make a transcript the same on every operating system: one line ending, no
// colour, one spelling for the scratch directory, and forward slashes under it.
export function normalize(text, { tmp } = {}) {
  let out = stripVTControlCharacters(text).replaceAll('\r\n', '\n')
  if (tmp) {
    const spellings = new Set([tmp, tmp.replaceAll('\\', '/'), tmp.replaceAll('/', '\\')])
    for (const spelling of spellings) out = out.split(spelling).join('<tmp>')
    out = out.replace(/<tmp>[^\s"'`]*/g, (path) => path.replaceAll('\\', '/'))
  }
  return out
}

windows.mjs holds a transcript with all four kinds of noise, as a Windows runner would record it. It compares that with the golden a Linux run committed, first byte for byte and then through normalize:

node windows.mjs

raw Windows transcript against the committed golden: 5 of 5 lines differ
  line 1
    got:  "exit 0\r"
    want: "exit 0"
  line 2
    got:  "--- stdout ---\r"
    want: "--- stdout ---"
  line 3
    got:  "\u001b[32mok\u001b[0m wrote 2 meters to C:\\Users\\ci\\AppData\\Local\\Temp\\meters-k3f9x1\\out\\meters.csv\r"
    want: "ok wrote 2 meters to <tmp>/out/meters.csv"

the same transcript after normalize(): equal

Line 3 is the whole page in one line. An escape code, a backslash path under a temp directory with a random suffix, and a carriage return all sit in the output of a command that did the right thing.

Record a golden, then make updating deliberate

The test runs the CLI through spawnSync with a fresh scratch directory per case. It hands that directory to the normalizer, so the substitution is exact rather than a guess at what a temp path looks like.

// Run the built CLI as a subprocess in a scratch directory of its own, and
// return the normalised transcript of what it did.
function run(args, env = {}) {
  const tmp = mkdtempSync(join(tmpdir(), 'meters-'))
  try {
    const result = spawnSync(process.execPath, [cli, ...args.map((a) => a.replaceAll('<tmp>', tmp))], {
      encoding: 'utf8',
      env: { ...process.env, FORCE_COLOR: undefined, NO_COLOR: undefined, ...env },
    })
    return normalize(transcript(result), { tmp })
  } finally {
    rmSync(tmp, { recursive: true, force: true })
  }
}

Updating is a separate, visible act. UPDATE_GOLDEN=1 rewrites the files; without it a missing golden fails with the instruction to record one, and a changed one fails with the diff. Jest keeps the same discipline behind --ci, which refuses to write a new snapshot and asks for --updateSnapshot, and syrupy behind --snapshot-update.

// One golden per case, shared by every operating system. A case whose output
// has to differ per platform gets a <name>.<platform>.golden beside it, and
// that file wins when it exists.
function matchGolden(name, actual) {
  const perPlatform = join(goldenDir, `${name}.${process.platform}.golden`)
  const file = existsSync(perPlatform) ? perPlatform : join(goldenDir, `${name}.golden`)
  if (UPDATE) writeFileSync(file, actual)
  assert.ok(existsSync(file), `${file} is missing; record it with UPDATE_GOLDEN=1`)
  // The golden goes through normalize() too, so a checkout that rewrote its
  // line endings cannot fail the test by itself.
  assert.equal(actual, normalize(readFileSync(file, 'utf8')))
}

UPDATE_GOLDEN=1 node --test golden.test.mjs

The recorded file is small enough to review in a pull request, which is the point of committing it:

cat testdata/export-writes-csv.golden

exit 0
--- stdout ---
ok wrote 2 meters to <tmp>/out/meters.csv
--- stderr ---
meters: exporting 2 meters

Two tests share that golden: the plain export and the same export with FORCE_COLOR=1. One file for both is the proof that the normalizer, not the recording machine, decides what is compared. A case whose output has to differ per platform gets a <name>.<platform>.golden beside it, and matchGolden prefers that file when it exists; the sample needs none.

Run the same test on three runners

The matrix is three copies of one job, with fail-fast off so that a Windows failure does not cancel the macOS run before it reports. The workflow syntax documents that cancellation as the default.

    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: node --test golden.test.mjs

The checkout itself is the fourth source of noise. Windows checkouts commonly run with core.autocrlf set to true, the setting GitHub’s line-ending guide recommends there. The Git configuration reference defines it as the text attribute set to auto on every file with core.eol set to crlf, so a committed golden arrives with CRLF endings on that runner.³ One line of .gitattributes beside the tests pins the golden files to LF on every checkout:

*.golden text eol=lf

Check it worked

Run the suite:

node --test golden.test.mjs

1..5
# tests 5
# suites 0
# pass 5
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 407.641091

The summary line to read is # pass 5. The fifth test feeds a hand-written Windows transcript through the normalizer and asserts it equals the golden the Linux run recorded, which is the cross-platform claim checked without a Windows machine.

When it goes wrong

The golden files fail on every line, on Windows only, before any normalizing. The checkout rewrote them: with core.autocrlf on and no .gitattributes, every LF in the committed file became CRLF on the way to disk. crlf-checkout.mjs applies that conversion to the committed golden and shows both defenses, the attribute that stops the conversion and the normalizer applied to the golden on read:

node crlf-checkout.mjs

golden as committed:           equal to the transcript
golden after autocrlf:         differs from the transcript
  first line as read:          "exit 0\r"
golden normalized on read:     equal to the transcript
golden kept LF by .gitattributes: never converted, so the compare sees what was committed

A snapshot passes locally, but the CI job fails with a new-snapshot error. The framework wrote the file on the laptop and was told not to write on the runner, by Jest’s --ci or by the hand-rolled runner’s missing-file assertion. Commit the recorded file; the failure is the gate working.

The transcript shows exit null. spawnSync reports status as null when a signal ended the child, with the signal’s name in signal. A timeout, or the kernel ending the process for memory, then looks like an exit code that is missing. Put signal in the transcript beside the status, and give the slow runner a longer timeout option rather than a looser assertion.

When not to do this

Do not snapshot output you have not made deterministic. A progress bar, a timestamp, a duration, or a sorted-by-hash listing changes on every run, and Jest’s own guidance says the test must be deterministic before the snapshot can mean anything. Normalize those fields the way this page normalizes paths, or test a --json document with named fields instead of the text people read.

Do not record one golden per operating system as the default. Three files that each pass on their own runner prove only that the runner has not changed since the recording, and a behavior change has to be reviewed three times. Reach for the per-platform file when the output has to differ, such as a path printed for a person to open.

Do not let the update step happen implicitly. A runner that writes a new snapshot when none exists turns the first run after a refactor into a green run with nothing checked. Gate the write behind a flag or an environment variable, and keep the assertion that fails when the file is absent.

Do not test terminal behavior this way. A pipe is not a terminal, so a snapshot taken through spawnSync never sees the prompt, the spinner, or the colored path that a person sees. Asserting on those needs the stream-injection approach of the terminal-behavior page instead.

Related how-tos

• Test interactive prompts and terminal behavior in a CLI

• Design a CLI a coding agent can drive

• Write CLI help that people and agents can both use

Last verified

Verified 2026-10-21 against Node 22.22.0. Every output block is what the command preceding it printed, on Linux. The Windows transcript in windows.mjs and the CRLF conversion in crlf-checkout.mjs are modeled on the documented behavior of Python’s text mode and of core.autocrlf, not captured from a Windows runner. matrix.yml was not run on one. The testscript, Jest, and syrupy rows describe their documentation.