# How to host docs question answering on Cloudflare Workers > Embed the question with Workers AI, query a Vectorize index filtered to one docs version, and stream the answer with its sources over Server-Sent Events. Source: https://voxgig.com/howto/host-docs-question-answering-on-cloudflare-workers - Audience: platform-team - Level: intermediate - Languages: typescript - Verified: 2026-09-25 - Published: 2026-09-25 ## Short answer Run the retrieval step as a Worker with an AI binding and a Vectorize binding. Embed the question with the bge-base-en-v1.5 model, query the index with a metadata filter on version and product, and stream a sources event followed by the model's frames as Server-Sent Events. Create the metadata indexes before the first upsert, or the filter cannot see what you wrote. --- ## You will need Node 22 or later, a Cloudflare account with Workers AI and Vectorize enabled, and docs in markdown. Verified 2026-09-25 against Node 22.22.2. The Worker here ran under Node with stand-in bindings, because neither binding has a local simulation and the sample makes no network calls. The embedding model is [bge-base-en-v1.5](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/), which returns 768 dimensions and reads at most 512 tokens per input. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [Cloudflare AI Search](https://developers.cloudflare.com/ai-search/concepts/how-ai-search-works/) | An R2 bucket or a site to index, and no wish to write an ingestion job | Chunking you steer only by size and overlap, and filters on at most five custom fields you declare in a schema | You need your own chunk boundaries, or a filter on a field it does not extract | | [Cloudflare Workers plus Vectorize](https://developers.cloudflare.com/vectorize/get-started/embeddings/) | Your own chunking and metadata at the edge, with no server to run | The ingestion job is yours, ten metadata indexes at most, and no keyword or SQL query over the store | You need full-text search beside the vectors, or a join | | [pgvector on Neon or Supabase behind a Node service](https://neon.com/docs/extensions/pgvector) | Postgres is already there and you want SQL, full-text search and vectors in one query | A Node service to host and scale, and HNSW or IVFFlat index tuning that is yours to get right | No Postgres, and no appetite for operating a service | | [podmind, a Node service over OpenSearch on AWS](https://github.com/voxgig/podmind) | Reading a whole pipeline of this shape, from ingestion to chat widget, before writing yours | A podcast transcript system on Lambda, DynamoDB and OpenSearch, so the parts you want come entangled with parts you do not | You want a product with support rather than a reference implementation | AI Search removes the ingestion code and takes the chunking with it. Workers plus Vectorize keeps the chunking and the metadata in your hands and gives you no query language. The Node route over pgvector gives you SQL and [hybrid search](https://supabase.com/docs/guides/ai/hybrid-search), which Vectorize does not have, and a service to run, which the other two do not need. ## Index the chunks with their version and product Every heading becomes a chunk that carries the metadata the Worker filters on and cites. The chunk text goes into the metadata too, within the 10KiB a vector may carry, so the Worker never needs a second store. ```js title="ingest.mjs" const id = createHash('sha256').update(`${product}|${version}|${url}|${current.anchor}`).digest('hex').slice(0, 32) out.push({ id, metadata: { url: current.anchor ? `${url}#${current.anchor}` : url, title: current.anchor ? `${pageTitle}: ${current.heading}` : pageTitle, version, product, text, }, }) ``` The id is a hash of the product, the version, the URL, and the anchor, so a rebuild writes the same ids and an upsert replaces rather than duplicates. A vector id is at most 64 bytes, which rules out the URL itself. A heading repeated on one page gets a numbered anchor, as GitHub gives it, so two sections never share an id, and a `#` inside a code block is not a heading. ```bash node ingest.mjs docs/v2 --version 2 --product orders-sdk --base-url https://docs.orders.example/v2 --out embeddings.ndjson --stand-in ``` ```text output 5 chunks from docs/v2, orders-sdk version 2, 768 dimensions each 33fa00feeb395e30d66b5d8c15f8c5aa https://docs.orders.example/v2/errors#read-an-error a191569d57b3704e0fd8c428dc15fcd3 https://docs.orders.example/v2/errors#rate-limits 6ecb2a07f65b3e5182613b99c34b9359 https://docs.orders.example/v2/quickstart#install-the-sdk 26cd36e10e2fb0148d57de2895380f67 https://docs.orders.example/v2/quickstart#create-an-order c8bbe1a45a6e6c01700f55656f037dec https://docs.orders.example/v2/quickstart#retry-a-failed-order wrote embeddings.ndjson, 11387 bytes, one vector per line ``` Without `--stand-in` the job asks Workers AI for the embeddings over [its REST endpoint](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/) and writes the same file. The file is [NDJSON](https://developers.cloudflare.com/vectorize/best-practices/insert-vectors/), one vector per line, which is what wrangler uploads. The index and its metadata indexes come first, and the metadata indexes come before any upsert. ```bash npx wrangler vectorize create docs-index --dimensions=768 --metric=cosine npx wrangler vectorize create-metadata-index docs-index --property-name=version --type=string npx wrangler vectorize create-metadata-index docs-index --property-name=product --type=string npx wrangler vectorize upsert docs-index --file=embeddings.ndjson ``` Those four commands need an account and were not run here. The [wrangler reference](https://developers.cloudflare.com/vectorize/reference/wrangler-commands/) lists their flags, and upsert takes a batch size that defaults to 5000. ## Answer from Vectorize and send the sources first The Worker embeds the question with the AI binding, queries the Vectorize binding with a filter, and returns a stream. Everything it uses is a web-standard API, so the same file runs under workerd and under Node. ```js title="worker.mjs" export async function retrieve(env, { question, version, product, topK = TOP_K }) { const vector = await embed(env, question) const result = await env.VECTORIZE.query(vector, { topK, returnMetadata: 'all', filter: { version, product }, }) return result.matches.map((m) => ({ id: m.id, score: m.score, ...m.metadata })) } ``` `returnMetadata: 'all'` is what brings the chunk text back with the match, and the [client API](https://developers.cloudflare.com/vectorize/reference/client-api/) caps `topK` at 50 when it is set. The filter is an implicit equality on two properties, which [metadata filtering](https://developers.cloudflare.com/vectorize/reference/metadata-filtering/) reads as an AND.[^1] The response is built as a stream. The sources go out first, in their own event, so a client can show citations before the first token arrives, and the model's frames follow unchanged. ```js title="worker.mjs" const stream = new ReadableStream({ async start(controller) { controller.enqueue(encoder.encode(frame('sources', sources))) if (!chunks.length) { controller.enqueue(encoder.encode(frame('done', { sources: 0, reason: `nothing indexed for ${product} version ${version}` }))) controller.close() return } // The last two characters forwarded. An event of our own goes out only after a blank // line, or a client reads it as more of the model's last frame. let tail = '\n\n' const endFrame = () => { if (tail !== '\n\n') controller.enqueue(encoder.encode('\n\n')) } try { const model = await env.AI.run(ANSWER_MODEL, { messages: buildMessages(question, chunks), stream: true }) ``` With `stream: true` the [AI binding](https://developers.cloudflare.com/workers-ai/configuration/bindings/) returns the model's output as Server-Sent Events, and the Worker forwards those bytes as they arrive. A [Response](https://developers.cloudflare.com/workers/runtime-apis/streams/) with a `ReadableStream` body streams to the client without waiting for the end. The framing is the [event stream format](https://html.spec.whatwg.org/multipage/server-sent-events.html): a blank line ends each event. The Worker checks that the model's last frame was terminated before it appends an event of its own, `done` or `error`.[^2] ## Run the handler with stand-in bindings The demo ingests both docs versions into an in-memory index, creates the metadata indexes first, and asks the same question three times. ```bash node demo.mjs ``` ```text output node 22.22.2, 10 vectors, metadata indexes on version and product POST /ask {"question":"How do I retry a failed order?","version":"2","product":"orders-sdk"} 200 text/event-stream event: sources data: [{"url":"https://docs.orders.example/v2/quickstart#retry-a-failed-order","title":"Quickstart: Retry a failed order","version":"2"},{"url":"https://docs.orders.example/v2/errors#read-an-error","title":"Errors: Read an error","version":"2"},{"url":"https://docs.orders.example/v2/errors#rate-limits","title":"Errors: Rate limits","version":"2"}] data: {"response":"Stand-in answer, not a model: "} data: {"response":"3 sources were retrieved for \"How do I retry a failed order?\" "} data: {"response":"and would be cited here as [1] and [2]."} event: done data: {"sources":3} POST /ask {"question":"How do I retry a failed order?","version":"1","product":"orders-sdk"} 200 text/event-stream event: sources data: [{"url":"https://docs.orders.example/v1/quickstart#retry-a-failed-order","title":"Quickstart: Retry a failed order","version":"1"},{"url":"https://docs.orders.example/v1/errors#read-an-error","title":"Errors: Read an error","version":"1"},{"url":"https://docs.orders.example/v1/errors#rate-limits","title":"Errors: Rate limits","version":"1"}] data: {"response":"Stand-in answer, not a model: "} data: {"response":"3 sources were retrieved for \"How do I retry a failed order?\" "} data: {"response":"and would be cited here as [1] and [2]."} event: done data: {"sources":3} POST /ask {"question":"How do I retry a failed order?","version":"3","product":"orders-sdk"} 200 text/event-stream event: sources data: [] event: done data: {"sources":0,"reason":"nothing indexed for orders-sdk version 3"} ``` The sources for version 2 come from version 2 pages only, and version 1 gets its own. Version 3 has nothing indexed, so the Worker sends an empty `sources` event and a `done` event with the reason, and never calls the model. The answer frames are the stand-in's, and they say so; a real deployment gets the model's frames in the same place. Under wrangler the request is a curl. Both bindings need `remote: true` in the configuration, because [local development](https://developers.cloudflare.com/workers/local-development/) has no simulation for Workers AI or for Vectorize. ```bash npx wrangler dev curl -N http://127.0.0.1:8787/ask -H 'content-type: application/json' -d '{"question":"How do I retry a failed order?","version":"2","product":"orders-sdk"}' ``` ## Create the metadata index before the first upsert Vectorize filters on a property only through a metadata index on it, and a vector carries in that index only what it held when it was upserted. Vectors written before the index existed are not in it. The stand-in keeps that rule, so the wrong order can be shown. ```bash node pitfall.mjs ``` ```text output upserted 5 vectors, then created the metadata index on version filter version = 2 0 matches after upserting the same vectors again 3 matches ``` The vectors are there, the index is there, and the filter finds nothing, because the index was created after the writes. Upserting the same vectors again is the fix, and it is the fix the [documentation](https://developers.cloudflare.com/vectorize/reference/metadata-filtering/) gives. Put the two `create-metadata-index` commands in the job that creates the index, ahead of the first upsert, and the order cannot go wrong on a later run. ## Check it worked Twenty-four tests pin the Worker and the stand-ins. Three carry the page. The first event is `sources` and every source is from the requested version. A version with nothing indexed gets a `done` event and no model call. The model's frames sit between `sources` and `done`, unchanged. ```bash node --test worker.test.mjs ``` ```text output 1..24 # tests 24 # suites 0 # pass 24 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 250.07615 ``` Against a deployment, send the curl from the previous section and read the first line of the response. It is `event: sources`, and the URLs in the `data` line that follows all carry the version you asked for. ## When it goes wrong The filter returns nothing and the vectors are there. The metadata index was created after the upsert. Upsert the vectors again, and move the index creation ahead of the upsert in the job. The query fails with a limit on `topK`. `returnMetadata: 'all'` caps it at 50. Lower `topK`, or return `indexed` metadata and fetch the text from somewhere else. The client shows one event that never ends. The model's last frame had no blank line after it, so the `done` or `error` event was read as more of the same frame. The Worker appends the blank line when it is missing, before either event, and a test covers each case. `wrangler dev` answers every request with an error about the binding. The bindings are running against the local simulation, which does not exist for these two. Set `remote: true` on both, or run `wrangler dev --remote`. ## When not to do this Do not build the ingestion job if AI Search's chunking and five custom fields are enough for your docs. It reads an R2 bucket, chunks and embeds the content, and re-indexes on change, and the job in this page is the code it exists to remove. Do not put a whole page into a chunk's metadata. The cap is 10KiB per vector, and a chunk that long is retrieved for everything and answers nothing. Do not skip the version filter to get more matches. An answer assembled from two versions of the same page is the failure in the opening paragraph, with citations. Do not judge retrieval quality from the stand-in. It embeds a bag of words, and it ranks the retry section first here because the question repeats the heading. The model's embeddings do better and differently, and the tests assert the plumbing, not the ranking. Do not stream the model before the sources. A client that shows tokens and then citations has to redraw, and an agent reading the stream has to buffer the whole answer to find them. ## Related how-tos - [Chunk an OpenAPI document for retrieval](/howto/chunk-an-openapi-document-for-retrieval) - [Re-index docs on release without serving stale answers](/howto/reindex-docs-on-every-release) ## Last verified Verified 2026-09-25 against Node 22.22.2. Every output block is what the command preceding it printed. The Worker ran under Node with the stand-in bindings in `stand-ins.mjs`, which kept every request away from Workers AI, Vectorize, and a deployed Worker. The `wrangler` and `curl` commands are shown and were not run. [^1]: A string metadata index covers the first 64 bytes of the value, truncated on a UTF-8 boundary, and the [filtering reference](https://developers.cloudflare.com/vectorize/reference/metadata-filtering/) says so in one sentence. A version string fits in 64 bytes with room to spare and a URL does not, which is why the filter on this page is on `version` and not on `url`. The [limits page](https://developers.cloudflare.com/vectorize/platform/limits/) lists the same 64 bytes under maximum indexed data per metadata index per vector, beside a limit of ten such indexes. A vector may be filterable on 640 bytes of itself and no more. [^2]: The event stream format lives inside the [HTML Standard](https://html.spec.whatwg.org/multipage/server-sent-events.html), in its chapter on communication, between the `MessageEvent` interface and cross-document messaging. Streams must be UTF-8, and the standard notes that there is no way to specify another character encoding. A line beginning with a colon is ignored, which is the format's only comment syntax and also its heartbeat. A server that sends a colon and a newline every so often has said nothing, and has said it on time.