How to host docs question answering on Cloudflare Workers#

Your docs assistant answers a question about version 2 with a flag that version 2 removed, and its citation points at a page that says the opposite. The index holds both versions and the query cannot tell them apart. The Node service that hosts it is one more process to keep alive for a page of documentation.

What you get

You will end up with a Worker that embeds a question, queries a Vectorize index scoped to one docs version, and streams the answer with its sources as Server-Sent Events. An ingestion job fills the index from CI. This is for you if you run a platform on Cloudflare and want no server behind the docs.

Short answer

Run the retrieval step as a Worker with an AI binding and a Vectorize binding. Embed the question with the bge-base-en-v1.5 model, query the index with a metadata filter on version and product, and stream a sources event followed by the model’s frames as Server-Sent Events. Create the metadata indexes before the first upsert, or the filter cannot see what you wrote.

You will need

Node 22 or later, a Cloudflare account with Workers AI and Vectorize enabled, and docs in markdown. Verified 2026-09-25 against Node 22.22.2. The Worker here ran under Node with stand-in bindings, because neither binding has a local simulation and the sample makes no network calls. The embedding model is bge-base-en-v1.5, which returns 768 dimensions and reads at most 512 tokens per input.

Approaches compared

AI Search removes the ingestion code and takes the chunking with it. Workers plus Vectorize keeps the chunking and the metadata in your hands and gives you no query language. The Node route over pgvector gives you SQL and hybrid search, which Vectorize does not have, and a service to run, which the other two do not need.

Index the chunks with their version and product

Every heading becomes a chunk that carries the metadata the Worker filters on and cites. The chunk text goes into the metadata too, within the 10KiB a vector may carry, so the Worker never needs a second store.

    const id = createHash('sha256').update(`${product}|${version}|${url}|${current.anchor}`).digest('hex').slice(0, 32)
    out.push({
      id,
      metadata: {
        url: current.anchor ? `${url}#${current.anchor}` : url,
        title: current.anchor ? `${pageTitle}: ${current.heading}` : pageTitle,
        version,
        product,
        text,
      },
    })

The id is a hash of the product, the version, the URL, and the anchor, so a rebuild writes the same ids and an upsert replaces rather than duplicates. A vector id is at most 64 bytes, which rules out the URL itself. A heading repeated on one page gets a numbered anchor, as GitHub gives it, so two sections never share an id, and a # inside a code block is not a heading.

node ingest.mjs docs/v2 --version 2 --product orders-sdk --base-url https://docs.orders.example/v2 --out embeddings.ndjson --stand-in

5 chunks from docs/v2, orders-sdk version 2, 768 dimensions each
  33fa00feeb395e30d66b5d8c15f8c5aa  https://docs.orders.example/v2/errors#read-an-error
  a191569d57b3704e0fd8c428dc15fcd3  https://docs.orders.example/v2/errors#rate-limits
  6ecb2a07f65b3e5182613b99c34b9359  https://docs.orders.example/v2/quickstart#install-the-sdk
  26cd36e10e2fb0148d57de2895380f67  https://docs.orders.example/v2/quickstart#create-an-order
  c8bbe1a45a6e6c01700f55656f037dec  https://docs.orders.example/v2/quickstart#retry-a-failed-order
wrote embeddings.ndjson, 11387 bytes, one vector per line

Without --stand-in the job asks Workers AI for the embeddings over its REST endpoint and writes the same file. The file is NDJSON, one vector per line, which is what wrangler uploads. The index and its metadata indexes come first, and the metadata indexes come before any upsert.

npx wrangler vectorize create docs-index --dimensions=768 --metric=cosine
npx wrangler vectorize create-metadata-index docs-index --property-name=version --type=string
npx wrangler vectorize create-metadata-index docs-index --property-name=product --type=string
npx wrangler vectorize upsert docs-index --file=embeddings.ndjson

Those four commands need an account and were not run here. The wrangler reference lists their flags, and upsert takes a batch size that defaults to 5000.

Answer from Vectorize and send the sources first

The Worker embeds the question with the AI binding, queries the Vectorize binding with a filter, and returns a stream. Everything it uses is a web-standard API, so the same file runs under workerd and under Node.

export async function retrieve(env, { question, version, product, topK = TOP_K }) {
  const vector = await embed(env, question)
  const result = await env.VECTORIZE.query(vector, {
    topK,
    returnMetadata: 'all',
    filter: { version, product },
  })
  return result.matches.map((m) => ({ id: m.id, score: m.score, ...m.metadata }))
}

returnMetadata: 'all' is what brings the chunk text back with the match, and the client API caps topK at 50 when it is set. The filter is an implicit equality on two properties, which metadata filtering reads as an AND.¹

The response is built as a stream. The sources go out first, in their own event, so a client can show citations before the first token arrives, and the model’s frames follow unchanged.

    const stream = new ReadableStream({
      async start(controller) {
        controller.enqueue(encoder.encode(frame('sources', sources)))
        if (!chunks.length) {
          controller.enqueue(encoder.encode(frame('done', { sources: 0, reason: `nothing indexed for ${product} version ${version}` })))
          controller.close()
          return
        }
        // The last two characters forwarded. An event of our own goes out only after a blank
        // line, or a client reads it as more of the model's last frame.
        let tail = '\n\n'
        const endFrame = () => { if (tail !== '\n\n') controller.enqueue(encoder.encode('\n\n')) }
        try {
          const model = await env.AI.run(ANSWER_MODEL, { messages: buildMessages(question, chunks), stream: true })

With stream: true the AI binding returns the model’s output as Server-Sent Events, and the Worker forwards those bytes as they arrive. A Response with a ReadableStream body streams to the client without waiting for the end. The framing is the event stream format: a blank line ends each event. The Worker checks that the model’s last frame was terminated before it appends an event of its own, done or error.²

Run the handler with stand-in bindings

The demo ingests both docs versions into an in-memory index, creates the metadata indexes first, and asks the same question three times.

node demo.mjs

node 22.22.2, 10 vectors, metadata indexes on version and product

POST /ask {"question":"How do I retry a failed order?","version":"2","product":"orders-sdk"}
200 text/event-stream
event: sources
data: [{"url":"https://docs.orders.example/v2/quickstart#retry-a-failed-order","title":"Quickstart: Retry a failed order","version":"2"},{"url":"https://docs.orders.example/v2/errors#read-an-error","title":"Errors: Read an error","version":"2"},{"url":"https://docs.orders.example/v2/errors#rate-limits","title":"Errors: Rate limits","version":"2"}]

data: {"response":"Stand-in answer, not a model: "}

data: {"response":"3 sources were retrieved for \"How do I retry a failed order?\" "}

data: {"response":"and would be cited here as [1] and [2]."}

event: done
data: {"sources":3}

POST /ask {"question":"How do I retry a failed order?","version":"1","product":"orders-sdk"}
200 text/event-stream
event: sources
data: [{"url":"https://docs.orders.example/v1/quickstart#retry-a-failed-order","title":"Quickstart: Retry a failed order","version":"1"},{"url":"https://docs.orders.example/v1/errors#read-an-error","title":"Errors: Read an error","version":"1"},{"url":"https://docs.orders.example/v1/errors#rate-limits","title":"Errors: Rate limits","version":"1"}]

data: {"response":"Stand-in answer, not a model: "}

data: {"response":"3 sources were retrieved for \"How do I retry a failed order?\" "}

data: {"response":"and would be cited here as [1] and [2]."}

event: done
data: {"sources":3}

POST /ask {"question":"How do I retry a failed order?","version":"3","product":"orders-sdk"}
200 text/event-stream
event: sources
data: []

event: done
data: {"sources":0,"reason":"nothing indexed for orders-sdk version 3"}

The sources for version 2 come from version 2 pages only, and version 1 gets its own. Version 3 has nothing indexed, so the Worker sends an empty sources event and a done event with the reason, and never calls the model. The answer frames are the stand-in’s, and they say so; a real deployment gets the model’s frames in the same place.

Under wrangler the request is a curl. Both bindings need remote: true in the configuration, because local development has no simulation for Workers AI or for Vectorize.

npx wrangler dev
curl -N http://127.0.0.1:8787/ask -H 'content-type: application/json' -d '{"question":"How do I retry a failed order?","version":"2","product":"orders-sdk"}'

Create the metadata index before the first upsert

Vectorize filters on a property only through a metadata index on it, and a vector carries in that index only what it held when it was upserted. Vectors written before the index existed are not in it. The stand-in keeps that rule, so the wrong order can be shown.

node pitfall.mjs

upserted 5 vectors, then created the metadata index on version
  filter version = 2                   0 matches
  after upserting the same vectors again 3 matches

The vectors are there, the index is there, and the filter finds nothing, because the index was created after the writes. Upserting the same vectors again is the fix, and it is the fix the documentation gives. Put the two create-metadata-index commands in the job that creates the index, ahead of the first upsert, and the order cannot go wrong on a later run.

Check it worked

Twenty-four tests pin the Worker and the stand-ins. Three carry the page. The first event is sources and every source is from the requested version. A version with nothing indexed gets a done event and no model call. The model’s frames sit between sources and done, unchanged.

node --test worker.test.mjs

1..24
# tests 24
# suites 0
# pass 24
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 250.07615

Against a deployment, send the curl from the previous section and read the first line of the response. It is event: sources, and the URLs in the data line that follows all carry the version you asked for.

When it goes wrong

The filter returns nothing and the vectors are there. The metadata index was created after the upsert. Upsert the vectors again, and move the index creation ahead of the upsert in the job.

The query fails with a limit on topK. returnMetadata: 'all' caps it at 50. Lower topK, or return indexed metadata and fetch the text from somewhere else.

The client shows one event that never ends. The model’s last frame had no blank line after it, so the done or error event was read as more of the same frame. The Worker appends the blank line when it is missing, before either event, and a test covers each case.

wrangler dev answers every request with an error about the binding. The bindings are running against the local simulation, which does not exist for these two. Set remote: true on both, or run wrangler dev --remote.

When not to do this

Do not build the ingestion job if AI Search’s chunking and five custom fields are enough for your docs. It reads an R2 bucket, chunks and embeds the content, and re-indexes on change, and the job in this page is the code it exists to remove.

Do not put a whole page into a chunk’s metadata. The cap is 10KiB per vector, and a chunk that long is retrieved for everything and answers nothing.

Do not skip the version filter to get more matches. An answer assembled from two versions of the same page is the failure in the opening paragraph, with citations.

Do not judge retrieval quality from the stand-in. It embeds a bag of words, and it ranks the retry section first here because the question repeats the heading. The model’s embeddings do better and differently, and the tests assert the plumbing, not the ranking.

Do not stream the model before the sources. A client that shows tokens and then citations has to redraw, and an agent reading the stream has to buffer the whole answer to find them.

Related how-tos

• Chunk an OpenAPI document for retrieval
• Re-index docs on release without serving stale answers

Last verified

Verified 2026-09-25 against Node 22.22.2. Every output block is what the command preceding it printed. The Worker ran under Node with the stand-in bindings in stand-ins.mjs, which kept every request away from Workers AI, Vectorize, and a deployed Worker. The wrangler and curl commands are shown and were not run.