Your agent merges and deletes support tickets, and you learn what it decided by reading the tickets afterwards. A delete is not undone by asking the model again. You want every write it intends, with its arguments, in front of you before any of them happens, but the tools have no switch for that.
What you get
You will end up with write tools that take a dry-run flag from the runner, three backends for the dry run, and a diff against the audit log. This is for you if an agent writes to an API you care about.
Short answer
Give every tool that creates, updates or deletes a dry-run flag that the runner sets and the model never sees, and default it to on. In a dry run the tool records its call and answers from the API’s own dry-run parameter, an in-memory fake, or a placeholder. Run the task dry, review the recorded calls, run it for real, and diff the two against the API’s audit log.
You will need
Node 22 or later, Python 3.7 or later for the Python twin, and an agent with at least one tool that writes to an API. Verified 2026-10-15 against Node 22.22.0, Python 3.13.16, @voxgig/create-sdkgen 0.30.7 and @voxgig/sdkgen 4.34.7. A scripted model stub drives every run, so no provider is called and no key is set. The ticket API is a local server each command starts.
Voxgig maintains sdkgen. This page compares its test feature with a server-side dry run, nock or MSW, and a recorded plan.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| nock or MSW around any client | A hand-written or vendor client, where a fake at the transport is enough | Handlers per endpoint, and a server rule the fake never runs passes the dry run | The API documents a dry-run parameter of its own |
| Plan, then execute | A person approves every write before any happens, from a file they can read | Two passes and twice the model calls, with placeholders wherever a write produces a value | The second pass may choose calls the approved plan never listed |
| sdkgen test feature | A client sdkgen generated, on a task that chains writes such as create then update | Mints ids at random and skips the server’s rules, so a refused delete reads as fine, even of an id the seed never held | You need to know the server will accept each call |
| Server-side dry run | APIs that have one: Kubernetes dryRun=All, Google Ads validate_only | Few APIs offer it, and chained writes fail because nothing earlier was kept | The task chains writes that depend on each other |
Only the server-side dry run knows the server’s rules. It judges each call against the state before the run, so a chain of writes fails where the real run succeeds. The test feature carries state from one call to the next but runs none of the server’s checks. nock and MSW sit at the same layer for any client. A recorded plan validates nothing, but it is the easiest of the four to review. Stripe’s sandbox keys are a fifth shape: the calls really run, against data kept apart from live mode.
Keep the flag where the model cannot reach it
The flag belongs to the runner. It is an argument to makeTools, never a property in a tool’s
schema, so neither the model nor a document the model reads can turn a dry run off.
// dryRun defaults to true, so a runner that forgets to set it records instead of writing.
// `via` picks what makes the dry run answer: 'server' sends the API's own dryRun parameter,
// 'fake' expects a client built with DesklineSDK.test(), 'record' answers with a placeholder.
export function makeTools(sdk, { dryRun = true, via = 'record' } = {}) {
// A name matching none of the three would fall through to a real write, so a typo stops here.
if (dryRun && !['server', 'fake', 'record'].includes(via)) {
throw new Error(`no dry-run backend named ${via}`)
}
if (dryRun && via === 'fake' && sdk._mode !== 'test') {
throw new Error('a fake dry run needs a test-mode client, and this one is live')
}
const calls = []
async function write(tool, op, raw) {
// Only what the schema declares reaches the client, so an argument the model adds, a
// dryRun of its own included, is dropped here rather than sent in the body.
const args = pick(raw, TOOL_SCHEMAS[tool].parameters.properties)
const call = { tool, args }
calls.push(call)
if (dryRun && via === 'record') {
call.outcome = 'recorded'
// The value only a real write can produce, marked as such, so the plan shows the
// dependency instead of hiding it.
return op === 'create' ? { ...args, id: `<id from call ${calls.length}>` } : { recorded: true }
}
const payload = dryRun && via === 'server' ? { ...args, dry_run: 'All' } : args
const explain = {}
try {
const entity = await sdk.Ticket()[op](payload, { explain })
call.outcome = 'ok'
call.result = entity.data()
return call.result
} catch (err) {
// The SDK keeps the status and the parsed error body; the model needs the reason.
call.outcome = `${err.status ?? 'error'} ${err.result?.body?.error ?? err.message}`
return { error: call.outcome }
} finally {
call.request = requestOf(explain.fetchdef)
}
}The default is a dry run, so the dangerous mode is the one you have to ask for, and a misspelled
backend name throws instead of writing. pick drops any
argument the schema does not declare, which stops a ticket body that tells the model to pass
dryRun: 'None' from reaching the client. And the fake path refuses a live client, because a
test-mode client and a live one look the same from outside.
The client is the TypeScript SDK sdkgen generated from api.json, which declares the
Kubernetes-style dryRun query parameter on its three writes. The generated client takes it as
dry_run and puts it in the query string, not the body. In sdkgen 4.34.7 the test feature
replaces the transport, not the request, so ctrl.explain hands back the request the client
built in test mode as well as live. The tool records that.
Run the same task four ways
The scripted model opens one ticket, closes t1 and t2 into it, and deletes t3, which is
still open. The server refuses to delete an open ticket and refuses a merge into a ticket that
does not exist.
node demo.mjscall server dryRun sdkgen test mode record only real run
create ok, id tk-4 ok, id minted recorded ok, id tk-4
update t1 422 ok recorded ok
update t2 422 ok recorded ok
delete t3 409 ok recorded 409
write requests 4 0 0 4
writes kept 0 0 0 3
422 mergedInto names no ticket: tk-4
409 close the ticket before deleting itAll three dry runs left the server as it was, as the last row shows.
Only the server-side dry run saw the 409, which the real run then hit. The test feature and the plan both passed the delete of an open ticket.
The server-side dry run also reported two 422 responses that the real run never hit, because the
dry-run create was not stored and the updates merged into a tk-4 that did not exist.
Kubernetes documents the same behavior: a dry run is never persisted, and generated values such
as a name will likely differ from the real request’s.1 The test feature carried the created
ticket into both updates, under a 16-character id it minted. The plan carried <id from call 1>, which
shows the dependency without inventing a value.2
Hold the flag per run in Python
The Python twin keeps the flag in a ContextVar, which asyncio copies into every task it
creates, so two runs in one process each see their own value.
DRY_RUN = contextvars.ContextVar("dry_run", default=True) # forget to set it, get a dry run
PLAN = contextvars.ContextVar("plan")
def write_tool(fn):
"""Wrap a tool that changes data. In a dry run it records the call and returns a placeholder."""
@functools.wraps(fn)
async def wrapper(api, **args):
if DRY_RUN.get():
PLAN.get().append({"tool": fn.__name__, "args": args})
return {"recorded": True}
return await fn(api, **args)
return wrapperThe script runs a dry run and a real run at once, then the same pair with a module-level flag.
python3 dry_run.pyContextVar flag, a dry run and a real run at once
plan [{"tool": "close_ticket", "args": {"id": "t1"}}, {"tool": "close_ticket", "args": {"id": "t2"}}]
dry api {"t1": "open", "t2": "open"}
real {"t1": "closed", "t2": "closed"}
module-level flag, the same two runs
dry api {"t1": "closed", "t2": "closed"}
real {"t1": "closed", "t2": "closed"}With the module-level flag, the real run set it to false while the dry run was between calls,
and the dry run closed both of its tickets. The
asyncio documentation states the rule the
first half relies on: a task runs in a copy of the current context.3
A run awaited directly gets no copy, so run_agent puts the old values back when it ends.
async def run_agent(api, dry_run):
"""The scripted model: close two tickets, yielding to the event loop between calls."""
# Put the old values back when the run ends, so a real run awaited directly does not leave
# the flag off for whatever this task runs next.
flag, plan = DRY_RUN.set(dry_run), PLAN.set([])
try:
for ticket in ("t1", "t2"):
await asyncio.sleep(0)
await close_ticket(api, id=ticket)
return PLAN.get()
finally:
DRY_RUN.reset(flag)
PLAN.reset(plan)Three Python tests pin both halves and the reset.
python3 test_dry_run.pyok a_dry_run_and_a_real_run_at_once_keep_their_own_flags
ok a_real_run_leaves_the_default_dry_run_behind_it
ok a_module_level_flag_lets_the_dry_run_writeCheck it worked
Run the task dry with the test feature, run it for real, and line the recorded requests up against the server’s audit log of the real run.
node verify.mjsPOST /v1/tickets same
PATCH /v1/tickets/t1 mergedInto differs
PATCH /v1/tickets/t2 mergedInto differs
DELETE /v1/tickets/t3 dry run ok, real 409
1 of 4 calls match the audit log exactlySame calls, same order, same paths. The two body differences are the id the create returned, which the dry run could not know. The fourth line is the one to read before you trust a client-side dry run again: the server refused a delete that the dry run had passed.
Eleven tests pin that behavior, including that a dry run keeps nothing and that neither an argument the model adds nor a misspelled backend reaches the API.
node --test dryrun.test.mjs1..11
# tests 11
# suites 0
# pass 11
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 377.899612When it goes wrong
The dry run kept three writes. The API ignores dryRun, as many HTTP servers ignore a query
parameter they do not know. Before trusting the flag, run a dry-run create against a sandbox and
list the collection.
Every chained write fails in the server-side dry run. Each call is checked against stored state, which the earlier dry-run calls never changed. Use the fake for chains, seeded with the records the plan expects, or accept the 422 responses as noise and read only the single-call rules from that run.
A dry run closed tickets. Two runs shared a module-level flag. Keep it in a ContextVar in
Python or in the tool factory’s closure in TypeScript, never in a global.
The real run made a call the approved plan did not list. The second pass is a fresh sample from the model. Diff each real call against the plan before sending it, and stop at the first mismatch.
When not to do this
Do not read a clean pass from sdkgen’s test feature as proof the real run will succeed. The mock
answered ok to deleting an open ticket the server refuses, because a client-side fake knows the
request’s shape and none of the server’s rules. In 4.34.7 it also passes a delete of an id it
never held. Use it to review intent, and the server-side flag
or a sandbox to test validity.
Do not send a dry-run parameter to an API that does not document one. An unknown query parameter is usually dropped, and the write goes through for real.
Do not give a dry-run agent credentials you would not give the real one, and do not expect a server-side dry run to let you narrow them. Kubernetes requires the same authorization for a dry run as for the real request,4 so the dry-run agent holds keys that can write.
Do not wrap read tools. Running a read for real changes nothing on the server, and it is how the model learns the ids it plans with.
Related how-tos
Last verified
Verified 2026-10-15 against Node 22.22.0, Python 3.13.16 and 3.12.3, @voxgig/create-sdkgen 0.30.7
and @voxgig/sdkgen 4.34.7. Every output block is what the command preceding it printed. The model is
a scripted stub and the API is a local server, so no provider and no hosted API was called. The
generated SDK’s compiled JavaScript is committed under generated/, and README.md beside it
names the commands that produced it.
Footnotes
-
The Kubernetes API concepts page lists some of the fields a dry run fills in differently from the real request. They include a name derived from
generateName, the timestamps, the UID, the resource version, and anything a mutating admission controller sets. The page asks the reader not to rely on any of them. The rehearsal runs every stage but storage, and the values it cannot rehearse are the ones that say which object was made. ↩︎ Back to text -
Terraform’s machine-readable plan faces the same gap and answers it with a second object. Its JSON output format pairs each planned value with
after_unknown, which marks the leaves that will only be known after apply, and the example the documentation chose is{ "id": true }. Of all the attributes a resource can carry, the one no plan can predict is the one everything else refers to. ↩︎ Back to text -
The module arrived in Python 3.7 through PEP 567, which describes itself as a simplified version of PEP 550. The earlier proposal, from the same author and a second, also tried to keep values straight inside generators, and it is filed as withdrawn. The half of the idea that keeps a dry run apart from a real one is the half that survived review. ↩︎ Back to text
-
The Kubernetes page says that authorization for dry-run and non-dry-run requests is identical, so a dry-run patch to a Deployment needs permission to patch Deployments. Google’s
validate_onlytakes the other shortcut: the request is validated but not executed, and only errors are returned, not results. A successful dry run of a create there returns no id at all, which avoids a wrong id by supplying none. ↩︎ Back to text