How-to › Use AI to do the integration

How to compare an AI-written API client with generated clients#

Score an agent-written client and three generated ones on the same calls, type probes, and mock, then measure what one schema change costs each.

Audience
API producer
Level
advanced
Topic
Write an integration with a coding agent
Languages
TypeScript and Python
Verified

Read first: Measure drift between two AI-written clients

An agent wrote your TypeScript client in one pass, and all five operations pass against a Prism mock. The API then changes published from an integer year to a date string. Nothing in the client reads the document again, so its types still say number, the mock run stays green, and the first sign of the change is a caller that computes NaN.

What you get

You will end up with a scorecard that runs one agent-written TypeScript client and three generated ones through the same calls, type probes, mock, and schema change. This is for you if an agent wrote a client you now maintain.

Short answer

Generate clients from the same OpenAPI document with Swagger Codegen, openapi-typescript, and sdkgen. Run each, beside the agent-written client, through one script: every operation against a stand-in, type probes compiled with tsc, a Prism mock, a 503, and a .. id. Then change one schema and measure what each takes to agree with it again, because only the generated clients have a step that does it.

You will need

Node 22 or later, an OpenAPI 3 document, and the client an agent wrote from it. Swagger Codegen needs Java 11 or later. The document here, api.v1.json, is a Bookshelf API with five operations and cursor pages. This page measures TypeScript clients only, so it says nothing about Python output. An AI coding agent, in a Claude Code session, wrote agent/client.ts in one pass with api.v1.json as its only input, before any generator ran. It is one specimen, not edited after the measurements, and agent/PROMPT.md holds the prompt.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
Agent-written clientYour codebase has conventions a generator ignores, or the document describes an endpoint badlyEvery line is yours to keep in step with the spec, and nothing reports when it falls behindThe API changes often, or several languages need clients
openapi-typescript with openapi-fetchA TypeScript codebase that wants exact types over a thin fetch wrapperNo retry, no paging, and path ids sent as given, so each is code you addYou want the client to carry behavior, not only types
sdkgenRetry, paging, and one behavior across several languages matter more than sizeAbout 7,300 lines of compiled JavaScript, the slowest regeneration here, and no page or error typeOne TypeScript client where types are all you need
Swagger Codegen typescript-fetchSeveral languages from one tool, with Java already in your buildA Java runtime, types that drop nullable, and a regeneration that rewrites the spec version in four file headersYou want retry or paging without writing a fetch wrapper

Voxgig maintains sdkgen, one of the four clients measured here, and the page does not recommend it.

The agent-written client is the shortest and the only one with nothing to regenerate, which is also why it has no way back to the spec. openapi-fetch costs a single types file and regenerates in under a second, and leaves retry and paging to you. Swagger Codegen brings Java and one types defect. sdkgen carries the most behavior and the most code.

Score the four clients on the same axes

score.mjs runs each client through the same script against stand-in.mjs, a local stand-in held to the document: unknown paths get 404, other methods 405, and bodies are checked against the schemas. It compiles six type probes per client with TypeScript, such as a book whose published is a string, which the document forbids. It also reads what Prism returned when generate.sh ran every client against it.

node score.mjs
agent-written
  operations that work, of 5              5
  getBook('..')                           sent /v1/
  one 503, then a 200                     failed
  books reached by its own paging, of 3   3
  type probes that agree with v1          6 of 6
  operations ok against Prism, v1         5 of 5
  lines written by hand                   94
  compiled JavaScript lines               70
Swagger Codegen
  operations that work, of 5              5
  getBook('..')                           sent /v1/
  one 503, then a 200                     failed
  books reached by its own paging, of 3   2, one page
  type probes that agree with v1          5 of 6
  operations ok against Prism, v1         5 of 5
  lines written by hand                   0
  compiled JavaScript lines               526
openapi-fetch
  operations that work, of 5              5
  getBook('..')                           sent /v1/
  one 503, then a 200                     failed
  books reached by its own paging, of 3   2, one page
  type probes that agree with v1          6 of 6
  operations ok against Prism, v1         5 of 5
  lines written by hand                   0
  compiled JavaScript lines               0, types only
sdkgen
  operations that work, of 5              5
  getBook('..')                           refused, nothing sent
  one 503, then a 200                     failed; ok with retry on
  books reached by its own paging, of 3   3, paging on
  type probes that agree with v1          4 of 4, 2 with no type
  operations ok against Prism, v1         5 of 5
  lines written by hand                   0
  compiled JavaScript lines               7294

Coverage does not separate them: every client covers all five operations and passes Prism. Two of the three generated clients are thin endpoint wrappers, because Swagger Codegen and openapi-fetch neither retry nor page. Both send a .. id through, which the URL resolves to /v1/.1 sdkgen does not retry by default either, but its retry and paging features, added at scaffold time and switched on per client, do both, and it refused the .. id.

The agent-written client pages through listAllBooks, which the prompt did not ask for, and it sent .. through too. Swagger Codegen’s one failed probe is a last page whose nextCursor is null: the document marks the field nullable, and the generated type is a plain string. sdkgen has no type for BookPage or Error, so two probes have nothing to compile against.

Change one schema and measure the way back

api.v2.json changes published from an integer to a string with format: date in three schemas. generate.sh timed each generator on it and recorded what changed. For the agent-written client, the change is agent/client.v2.ts, an edit made by hand. regen.mjs compiles the v2 probes against each client before and after.

node regen.mjs
api.v2.json: published is a date string, not an integer year
agent-written
  back in line by        edit client.ts by hand, not timed
  files changed          1, 6 lines
  v2 probes, before      4 of 6
  v2 probes, after       6 of 6
Swagger Codegen
  back in line by        generate again, 1.6 s
  files changed          4, 20 lines
  v2 probes, before      3 of 6
  v2 probes, after       5 of 6
openapi-fetch
  back in line by        openapi-typescript again, 0.7 s
  files changed          1, 13 lines
  v2 probes, before      4 of 6
  v2 probes, after       6 of 6
sdkgen
  back in line by        npm run generate, 3.6 s
  files changed          2, 9 lines
  v2 probes, before      2 of 4, 2 with no type
  v2 probes, after       4 of 4, 2 with no type
openapi-typescript --check, v1 output against api.v2.json: exit 1

Every client was stale before the change was applied, the generated ones included. The difference is the step back. Each generator took under four seconds, and openapi-typescript’s --check exits 1 when its output no longer matches the document, so CI can see the drift. The hand edit is six lines, made only because someone read the change. Eight of Swagger Codegen’s 20 lines are the spec version in four file headers, and sdkgen’s count covers ts/src.

Add retry to the two thinner generated clients

Neither generator offers retry or paging as an option. Swagger Codegen’s config-help -l typescript-fetch lists ten options, none of them for either, and the API page, middleware page, and examples page of openapi-fetch describe neither. Both clients take a fetch function, so the substitute goes there. Swagger Codegen’s generated base class:

export class BaseAPI {
    protected configuration: Configuration;

    constructor(configuration?: Configuration, protected basePath: string = BASE_PATH, protected fetch: FetchAPI = isomorphicFetch) {
        if (configuration) {
            this.configuration = configuration;
            this.basePath = configuration.basePath || this.basePath;
        }
    }
}

openapi-fetch takes the same function as the fetch option of createClient. retry-fetch.mjs retries a 429 or 503 twice and sends a request with a body once, so a retried create cannot write twice.

node substitute.mjs
Swagger Codegen
  its default fetch:   failed, status 503
  retryingFetch():     3 books, 2 requests
openapi-fetch
  its default fetch:   status 503
  retryingFetch():     3 books, 2 requests

Paging takes the same kind of loop the agent-written client already has, written once per client.

Check it worked

The tests hold the stand-in to the document, then assert each scorecard row, the probes that disagree, the ids sdkgen refuses, the stale caller, the recorded Prism runs, and the retrying fetch.

node --test compare.test.mjs
# tests 18
# suites 0
# pass 18
# fail 0
# cancelled 0
# skipped 0
# todo 0

When it goes wrong

The day-one score is clean and the second version breaks a caller. The agent-written client has no path back to the spec, so after the change its types still say number. Everything around it stays green, and the arithmetic returns NaN.

node pitfall.mjs
the agent-written client, not edited, against api.v2.json
  tsc of a caller, with the client's own types: 0 errors
  tsc of the same caller, with openapi-typescript's v2 types: 1 error, TS2363
  getBook('b1').published at run time: "1965-01-01", a string
  2026 - published in the caller: NaN
  Prism mock of v2, the five operations: 5 of 5 ok
  Prism mock of v2, a create body with published: 1965: failed: 400

The Prism run passes because Prism validates the request that arrives, not the types that produced it. Only a caller still sending an integer gets the 400. Score a second version of the document, and compile your callers against types regenerated from it.

The .. id reaches another endpoint. Three of the four clients sent getBook('..') to /v1/, because escaping leaves the dots in place. Refuse an empty id, ., and .. before the call. sdkgen refused . and .., but sent an empty id as /v1/books/.

When not to do this

Do not score one agent-written client and call the result a verdict on agents. This page measured one specimen from one prompt, and a second run from the same prompt is a different client.

Do not pick sdkgen for one TypeScript client whose callers need only types. Its 7,294 compiled lines, its features, and a 3.6 second regeneration buy retry, paging, and the same behavior in other languages, which that client does not use.

Swagger Codegen is the stronger option when a team already runs Java and wants Go, Python, and TypeScript clients from one tool. openapi-fetch suits a TypeScript codebase that wants types to track the document exactly and keeps its own fetch wrapper.

Do not keep an agent-written client for an API that changes, unless a check like --check or a regenerated type file tells you when it has fallen behind.

Last verified

Verified 2026-11-21 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, Swagger Codegen 3.0.85 on OpenJDK 21.0.13, openapi-typescript 7.13.0, openapi-fetch 0.17.0, Prism 5.14.2, and TypeScript 5.9.3. Generation, its timings, and both Prism runs happened once in generate.sh, with network access, and their output is committed under generated/. The gate re-runs the commands on this page against the committed clients and a local stand-in.

Footnotes

  1. RFC 3986 gives the algorithm, remove_dot_segments, that turns /v1/books/.. into /v1/. encodeURIComponent leaves a dot alone, because the dot is one of the characters it never escapes, so the escaped id is still a dot segment. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.