How to re-index docs on release without serving stale answers#

Your docs assistant tells a customer to pass an option that was removed two releases ago. The documentation site is correct, but the assistant answers from an index built in March because nothing in the pipeline noticed it had fallen behind the site. The answer sounded confident and cited a real page, but it was wrong.

What you get

You will end up with an index per release, a promotion that swaps every reader at once, and a check that fails when the live index is older than the published docs. This is for you if you answer questions over documentation that changes.

Short answer

Build the index under the release version and promote it with an alias, so readers always answer from one complete index and never from a half-built one. Compare the promoted version with the released docs version on every deploy, and treat a mismatch as a failure rather than a notification. A rollback is then one write, and it is visibly stale until the docs follow.

You will need

Node 22 or later, and docs that ship with a version number. The alias pattern is the one search engines use for exactly this. It is called an index alias in Elasticsearch and a collection alias in Qdrant, and the reasoning transfers to a file on disk.¹

Approaches compared

The difference is what a reader sees during the rebuild and what you can do afterwards. Rebuilding in place is simplest and has a window where the index is half empty, which produces confident answers from whatever happened to be loaded.² An alias has no such window: readers see the old index until one write makes them all see the new one.

The rollback argument settles it for most teams. When an index turns out to be wrong, the fix with an alias is one write and takes a second. Without one, the fix is another full rebuild from the previous docs, which you may no longer have checked out.

Build under the version, promote by name

Of the two operations, only the second is visible to readers.

build(version, chunks) {
  // Refuse to rebuild what the alias points at. Overwriting it swaps the
  // content under live readers during the build, which is the one thing
  // this whole arrangement exists to prevent. Build a new version instead.
  if (version === alias) {
    throw new Error(`refusing to rebuild ${version}: it is live, build a new version and promote it`)
  }
  indexes.set(version, { version, chunks, builtAt: version })
  return version
},
promote(version) {
  if (!indexes.has(version)) throw new Error(`cannot promote ${version}: it was never built`)
  alias = version
  return alias
},

Refusing to rebuild the live version is the other half of the guarantee. Reruns happen: a release job is retried, or somebody triggers it twice with the same version. Writing the index in place then swaps the content under readers during the build, before anybody promoted anything, and the alias was never the protection people assumed. Build a new version and promote that.

Refusing to promote something that was never built is worth the line. The usual failure is a pipeline where the build step is skipped, or fails without stopping the job, and the promote step runs anyway. The alias then points at nothing.

Report which index answered, on every response. A support conversation that starts with the index version is a different conversation from one that starts with a screenshot of a wrong answer. The first one ends in a lookup.

Fail the deploy on a mismatch

The check is a comparison, and it belongs where a red result stops something.

export function stale(store, docsVersion) {
  const index = store.current()
  if (!index) return { stale: true, reason: 'nothing is promoted' }
  if (index.version !== docsVersion) {
    return { stale: true, reason: `docs are ${docsVersion}, the live index is ${index.version}` }
  }
  return { stale: false }
}

An alert is not enough here. A stale index gives wrong answers that look right, so the failure mode is silent and the notification competes with everything else in a channel. Make it a check that fails. A gate that blocks a deploy gets fixed on the day it fires, and an alert about documentation quality does not.

Keep the previous index rather than dropping it on promote. It costs storage and it is the entire rollback plan.

Check it worked

Walk a release from indexed, through a docs release, to a rollback.

node demo.mjs

docs 2.0.0 released and indexed      serving 2.0.0   answer nothing matched fresh
docs 2.1.0 released, index not built serving 2.0.0   answer nothing matched STALE: docs are 2.1.0, the live index is 2.0.0
2.1.0 built, not yet promoted        serving 2.0.0   answer nothing matched STALE: docs are 2.1.0, the live index is 2.0.0
2.1.0 promoted                       serving 2.1.0   answer webhooks       fresh
rolled back while docs stay 2.1.0    serving 2.0.0   answer nothing matched STALE: docs are 2.1.0, the live index is 2.0.0

Lines two and three are the same for readers and different for you. Answers changed only at the promote on line four, after the docs had moved and the index had been built. That is the property an in-place rebuild cannot offer. During its build window a reader gets whatever share of the corpus happens to be loaded, while the assistant sounds just as certain.

The last line is a rollback that reads as stale, which is correct. Rolling back is a decision to serve older answers on purpose, and the check should keep saying so until somebody resolves it in one direction or the other.

node --test index.test.mjs

1..7
# tests 7
# suites 0
# pass 7
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 112.858884

When it goes wrong

Answers cite pages that no longer exist. The index was updated incrementally and the deletion was missed. Rebuild whole, or test deletions specifically, by asking for a page you removed and asserting nothing comes back. Deletions are the operation incremental indexing gets wrong most often.

The index is fresh and the answers are old. Something between the index and the reader is caching. Include the index version in the cache key. A cache that outlives a promote is indistinguishable from an index that never moved.

The build succeeds but the promote never runs. A pipeline step after the build failed unnoticed. Make the check part of the same job, so a missing promote fails the deploy that caused it.

Storage grows without limit. Every release keeps an index forever. Keep the last few, and refuse to drop the live one, which the sample enforces. Two is enough for a rollback and four is enough to answer a question about last month.

When not to do this

Do not version the index by a build timestamp. Two releases on one day become indistinguishable, and the check cannot compare a timestamp with a released version.

Do not promote automatically on build for a corpus you have not sampled. A bad chunking change reaches every reader in one write, which is the cost of the same property that makes rollback easy.

Do not run the assistant against an index nobody can name. If a response cannot say which index answered, every wrong answer becomes an investigation rather than a lookup, and the investigation starts by guessing.

Related how-tos

• Chunk an OpenAPI document for retrieval

• Serve your OpenAPI examples as mock responses

Last verified

Verified 2026-09-14 against Node 22.22.2. Both output blocks are what the preceding command printed.