How to chunk an OpenAPI document for retrieval#

Somebody asks your docs assistant how to list branches, and it answers with half a schema and no parameters. The description was fed to a text splitter that cut it every 600 characters. The chunk holding the summary does not contain the parameter table, and the chunk holding the parameters never says which endpoint they belong to.

What you get

You will end up with one chunk per operation, each carrying the method, the path, the parameters, the body and the responses, with references resolved. You also get a check that says whether a chunk could be acted on alone. This is for you if you run retrieval over an API description.

Short answer

Make one chunk per operation, and put the method, the path, the summary, every parameter, the request body and each response into it. Resolve local references inline so the chunk stands alone, and cap that inlining at two levels so a shared or recursive schema cannot make one chunk larger than the document. Never cut the serialized document by character count.

You will need

An OpenAPI 3 description, and Node 22 or later. The structure being walked is the Paths Object, and the references being resolved are the local Reference Objects that a bundled document carries.¹

Voxgig maintains apidef. This page compares it with swagger-parser, Redocly CLI, and a generic text splitter.

Approaches compared

The real division is between cutting text and walking structure. A splitter treats the description as prose, and an API description is not prose: it is a tree where the useful unit is an operation and the boundaries are known exactly. Anything that walks the tree gets those boundaries for free. What differs after that is grouping, and whether the tool decides it or you do. Grouping is worth having when your API divides into resources, and worth refusing when it does not.

Emit one chunk per operation

The unit is the operation, and everything a reader needs goes in it.

export function operationChunks(doc, { maxDepth = 2 } = {}) {
  const chunks = []
  for (const [path, item] of Object.entries(doc.paths)) {
    for (const verb of METHODS) {
      const operation = item[verb]
      if (!operation) continue
      chunks.push({
        id: operation.operationId,
        title: `${verb.toUpperCase()} ${path} - ${operation.summary ?? operation.operationId}`,
        text: render(doc, path, verb, withPathParams(item, operation), maxDepth),
      })
    }
  }
  return chunks
}

Walk the methods, not the keys. A Path Item holds its operations under get, post and the rest, and beside them it may hold summary, description, servers and parameters.² Iterating the object treats those as operations, and the first one reaching the renderer throws on a missing responses member. Those path-level parameters apply to every operation underneath, so merge them in: a chunk that omits the path parameter describes a call the reader cannot make.

Include the failure responses. A question about listing branches is often really a question about what happens when the repository does not exist, and a chunk that stops at the 200 cannot answer it.

Keep the operation id as the chunk id. Retrieval that cites an id a reader can find in the description is checkable, and one that cites a character range is not.

Sort the chunks by path rather than by insertion order. It costs nothing, and it makes two builds of the same description produce the same index, which is what lets you diff one against the next.

Cap the inlining

Full dereferencing is the trap, and it appears only on a large description.

export function inline(doc, node, depth) {
  if (!node || typeof node !== 'object') return node
  if (typeof node.$ref === 'string') {
    const name = node.$ref.split('/').pop()
    if (depth <= 0) return { schema: name }
    return inline(doc, doc.components.schemas[name], depth - 1)
  }

Two levels covers the shape a reader needs: the envelope and the thing inside it. Past that, the name is more useful than the body, and a recursive component that references itself would otherwise expand until the process stops.³ Leaving the name behind also gives a follow-up question something to retrieve. A reader who asks what a commit looks like gets the component chunk, and the operation chunk stays the size of an operation.

Check it worked

Ask one real question against both chunkings.

node demo.mjs

one chunk per operation
  chunks     3, longest 1137 chars
  top hit    GET /branches - List the branches of a repository
  usable     yes
  parameters 3 named in the hit
one chunk per 600 characters
  chunks     9, longest 600 chars
  top hit    characters 0 to 600
  usable     no: no method and path, no parameter list, no response section
  parameters 0 named in the hit

The second block is not a bad score, it is a broken answer. The top hit is the opening of the document, it contains no operation at all, and an assistant handed that chunk will invent the parameters or apologize. The first block returns three named query parameters with their types and descriptions, plus both response shapes, and the failure case.

node --test chunk.test.mjs

1..7
# tests 7
# suites 0
# pass 7
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 116.013385

When it goes wrong

Chunks are enormous and retrieval is slow. A shared schema is being inlined everywhere it appears. Lower the depth cap, and check the longest chunk after every description change. A chunk longer than the context you allot per retrieval is one the model never sees in full.

The assistant cites the wrong operation. Two operations share a summary, and the path is buried. Put the method and path first in the chunk text, where a term match will weight it. Summaries repeat across a resource far more than paths do.

A question about a field returns nothing. Field names live inside a schema that was named rather than inlined. Emit a second chunk per component schema, and let both be retrievable. The operation chunk and the schema chunk answer different questions, and both are cheap to build.

Retrieval works and answers are out of date. Chunks were built once. Rebuild them in the same job that publishes the description, and key them on the description version. An index that cannot say which version it was built from cannot be trusted after the first release.

When not to do this

Do not chunk a description that changes hourly without rebuilding the index in the same pipeline. A stale index answers with an endpoint you removed, which is worse than no assistant.

Do not use apidef’s entity grouping without reading what it produced for your odd endpoints. A search operation, a bulk import, and a webhook registration are not operations on an entity, and grouping them under one distorts what a reader is told.

Do not put the whole description in one chunk to be safe. Retrieval then returns everything for every question, and the model does the filtering the index should have done. It will do it badly on a large description, and the cost per question is the whole document every time.

Related how-tos

• Split and bundle a large OpenAPI document

• Serve your OpenAPI examples as mock responses

Last verified

Verified 2026-09-14 against Node 22.22.2. Both output blocks are what the preceding command printed.