How to make a Docusaurus site agent-ready#

Your Docusaurus site serves an llms.txt from static/. A writer adds a webhooks guide, the site deploys, and the guide is missing from llms.txt and has no markdown copy, so an agent reading the index never learns it exists. The build passes, because Docusaurus copies static/ as it is without comparing it with the pages it built.

What you get

You will end up with a plugin that writes llms.txt, markdown twins, AGENTS.md, and the OpenAPI document from the build itself, and fails the build when they disagree. This is for you if you run a Docusaurus site that agents read.

Short answer

Write a small Docusaurus plugin. Its postBuild hook reads the docs plugin’s loaded content from the plugins list it is handed, then writes llms.txt in sidebar order, a markdown twin per page, AGENTS.md, and the OpenAPI document. Leave out older versions and unlisted pages, then compare the build’s HTML pages with llms.txt and throw on any difference, so the build fails instead of shipping a stale index.

You will need

Node 22 or later and a Docusaurus 3 site. Verified 2026-10-15 against Node 22.22.0, Docusaurus 3.10.2 with @docusaurus/faster, React 19.2.0, and @signalwire/docusaurus-plugin-llms-txt 1.2.2. The sample is a real site, built by every command on this page. What goes in the site-root AGENTS.md is a separate task; this page only publishes the file.

Approaches compared

A plugin from npm is the least code, and its upgrades arrive on its maintainers’ schedule. Version 1.2.2 groups pages by URL and lists versioned docs unless told otherwise, which is measured below. The custom plugin is code you own, and it reads Docusaurus’s own sidebar. Static files cost nothing to set up, but they drift the first time a page is added without an edit to them.

Write the files from the plugin system

The sidebars and the doc metadata belong to the docs plugin. The lifecycle page lists a plugins array among the props postBuild receives, and in 3.10.2 the docs plugin’s entry carries its loaded content. The page does not describe that content’s shape, so pin the version and treat an upgrade as a change to test. The 3.10.2 type definitions also declare an allContentLoaded hook that the lifecycle page leaves out, which this plugin does not need.

    async postBuild({ outDir, plugins }) {
      const docsPlugin = plugins.find((p) => p.name === 'docusaurus-plugin-content-docs' && p.options.id === 'default')
      if (!docsPlugin) throw new Error('agent-surfaces needs the default instance of the docs plugin')
      const versions = docsPlugin.content.loadedVersions
      const current = versions.find((v) => v.versionName === 'current')
      const sections = sidebarSections(current)
      const write = async (path, text) => {
        await mkdir(dirname(join(outDir, path)), { recursive: true })
        await writeFile(join(outDir, path), text)
      }

      for (const doc of sections.flatMap((s) => s.docs)) {
        const source = await readFile(join(siteDir, doc.source.replace(/^@site\//, '')), 'utf8')
        await write(twinPath(doc.permalink), markdownTwin(source, doc.title))
      }

A twin is the page’s source with the frontmatter and MDX imports removed, where an import inside a code fence is part of a sample and stays. That keeps what the writer wrote, and it keeps a JSX component as JSX. The npm plugin converts the rendered HTML instead, which resolves components and loses the source. The twin sits at the page’s URL with .md appended, where the llms.txt proposal puts it.¹

The order comes from the sidebar. sidebarSections walks it, turns each top-level category into an llms.txt section, and drops unlisted docs. It puts any doc no sidebar names in an “Other pages” section, because an agent can land on it from search all the same:

  const walk = (items, section) => {
    for (const item of items) {
      if (item.type === 'doc' || item.type === 'ref') take(item.id, section)
      if (item.type === 'category') {
        // A top-level category becomes a section; a nested one stays in its parent's.
        const inner = section === sections[0] ? { title: item.label, docs: [] } : section
        if (inner !== section) sections.push(inner)
        if (item.link?.type === 'doc') take(item.link.id, inner)
        walk(item.items, inner)
      }
    }
  }

The sample site has a current version, a frozen 1.0 under /1.0/, and one page marked unlisted: true, which Docusaurus builds with a noindex robots tag. Build it:

npm run build

[agent-surfaces] llms.txt: 5 pages in 2 sections, sidebar order
[agent-surfaces] markdown twins: 5, AGENTS.md, openapi.yaml
[agent-surfaces] left out: /1.0/ and 1 unlisted
[agent-surfaces] check: llms.txt and the build agree

The index it wrote follows the sidebar, Paginate before Authenticate:

cat build/llms.txt

# Meterco API

> The Meterco energy metering API.

## Docs

- [Meterco API](https://docs.meterco.example/index.md): What the Meterco API does and where to start.
- [Quickstart](https://docs.meterco.example/quickstart.md): Install the client, authenticate, and read one meter.

## Guides

- [Paginate](https://docs.meterco.example/guides/paginate.md): Follow the next cursor until it is absent.
- [Authenticate](https://docs.meterco.example/guides/authenticate.md): Send an API key as a bearer token, and rotate it without downtime.
- [Errors](https://docs.meterco.example/guides/errors.md): Problem details on every error, and which ones to retry.

## Optional

- [OpenAPI document](https://docs.meterco.example/openapi.yaml): every operation and schema

Compare it with the plugin from npm

The same site, built with @signalwire/docusaurus-plugin-llms-txt and its includeVersionedDocs option off, then put through the same check:

SURFACES=signalwire npx docusaurus build --out-dir build-signalwire > /dev/null 2>&1 && cat build-signalwire/llms.txt && node check-surfaces.mjs build-signalwire --exclude /1.0/ --require llms.txt

# Meterco API

> The Meterco energy metering API.

- [Meterco API](/index.md)

## guides

- [Authenticate](/guides/authenticate.md): Send an API key as a bearer token, and rotate it without downtime.
- [Errors](/guides/errors.md): Problem details on every error, and which ones to retry.
- [Legacy session tokens](/guides/legacy-tokens.md): The session tokens removed in version 2.
- [Paginate](/guides/paginate.md): Follow the next cursor until it is absent.

## quickstart

- [Quickstart](/quickstart.md): Install the client, authenticate, and read one meter.
build-signalwire: 1 disagreement
  llms.txt lists /guides/legacy-tokens.md, which is not a page the site offers

The npm plugin’s index differs in three ways, and each can be configured away. Sections are URL path segments, sorted, so Authenticate comes before Paginate. The unlisted page is listed. With the option left at its default, a separate run listed both 1.0 pages under a ## 1.0 section as well. Its README offers includeOrder and excludeRoutes for the first two, which is a glob list to keep in step with the sidebar by hand.

Fail the build when the index and the pages disagree

The check reads only the build directory. Every HTML page that is not a 404, not under an excluded version path, and not noindex must be in llms.txt and have a twin. Every llms.txt entry must be such a page. It knows nothing of the plugin, which is why the same function checks all three routes. The plugin calls it last and throws on any finding, which fails the build.

add-page.mjs copies the site, adds a webhooks guide to the sidebar, and builds it twice: with the plugin, and with yesterday’s hand-written files in static/. Then it adds a pricing page outside the docs:

node add-page.mjs

added docs/guides/webhooks.md to the Guides category

plugin route
  llms.txt entry: - [Webhooks](https://docs.meterco.example/guides/webhooks.md): Receive a signed event when a reading arrives.
  markdown twin:  guides/webhooks.md
  check: agrees

static route
  llms.txt entry: none
  markdown twin:  none
  check: 2 disagreements
    /guides/webhooks is built but not in llms.txt
    /guides/webhooks has no markdown twin at /guides/webhooks.md

plugin route, after adding src/pages/pricing.md
  docusaurus build exited 1
  agent surfaces disagree with the build:
  /pricing is built but not in llms.txt
  /pricing has no markdown twin at /pricing.md

The static build passed, so only a check run after the build can catch the missing page. For a static route, that check is a CI step after the build: node check-surfaces.mjs build --exclude /1.0/. The pricing page fails the plugin’s build because the docs plugin does not own it. List it, or name it in the plugin’s exclude option if agents should not have it.²

Check it worked

The tests build small sites on disk, so they cover cases the sample lacks, such as a nested category and a doc no sidebar names. They also cover a link to another site, an exclude of /pricing that must not swallow /pricing-faq, and an import line inside a code sample.

npm test

# tests 10
# suites 0
# pass 10
# fail 0

When it goes wrong

The build fails with require.resolveWeak is not a function. The site’s package.json has "type": "module", which broke this sample’s 3.10.2 build with future.faster on and with it off. Remove the field and keep ESM in .mjs files, as the plugin here does.

The build is too slow to run on every pull request. future.faster with @docusaurus/faster cut a cold build of this sample from 26 seconds to 7 in the container the page was checked in. Measure your own site before you put a full build in front of every pull request.

When not to do this

Do not write a custom plugin for a site with a handful of pages and one version. Install the npm plugin, set includeVersionedDocs: false, name unlisted pages in excludeRoutes, and run the check script in CI. That covers the drift without code of your own to upgrade.

Do not keep the agent files in static/ on a site that changes weekly. They are right on the day they are written, and this page’s check is the only thing that will say when they stop being so.

Do not read the twins as a rendering of the page. They are the source, so an MDX component reaches the agent as JSX. If your pages lean on components for their content, convert the built HTML instead, which is what the npm plugin does.

Related how-tos

• Publish an OpenAPI document at a stable URL for agents

• Set robots.txt rules for each AI crawler

• Detect a reference site drifting from its OpenAPI spec

• Install an API skill and check a coding agent uses it

• Organize a large OpenAPI reference by tags and tag groups

Last verified

Verified 2026-10-15 against Node 22.22.0, @docusaurus/core 3.10.2, @docusaurus/faster 3.10.2, React 19.2.0, and @signalwire/docusaurus-plugin-llms-txt 1.2.2. Every output block is what the command preceding it printed, from real Docusaurus builds of the sample site. The build times are from one machine and are not captured.