How to account for tokens and cost per agent task#

The provider invoice is one line per model per month. Three agents and a nightly batch job share the key across two tenants, and a customer asks what their tenant cost last month. The number exists nowhere, because every response’s usage object was read for the answer text beside it and dropped.

What you get

You will end up with a ledger that prices every model response against its task and tenant from a price table you own, in two languages that agree to the last digit. This is for you if one provider key serves several tasks or tenants.

Short answer

Record every model response against the task id your agent loop carries, and read its usage fields into exclusive buckets. On the Responses API, subtract cached and written tokens from input_tokens and reasoning tokens from output_tokens; on the Messages API the fields are already separate. Price each bucket from a table you maintain, with cache reads at their discount and reasoning at the output rate, then sum per task and per tenant.

You will need

Node 22 or later, Python 3.11 or later, responses saved from your own model calls, and the pricing page of each provider you call. Verified 2026-10-21 against Node 22.22.0 and Python 3.13.16. The sample uses the standard library of both languages and sends nothing to a provider. The usage objects in calls.json are shaped as the Responses API and the Messages API document them, with token counts chosen for the example. The prices in prices.json are placeholders in round figures that keep the providers’ documented ratios between a plain input token, a cache read, and a cache write.

Approaches compared

A proxy meters every caller that goes through it, which is its attraction and its limit. A request that skips it, or arrives without the tag, is spend with no owner, and the proxy’s price table is one more thing to keep in step with the invoice. In-process accounting is exact for the loop you own and blind to everything else. The sample below takes the in-process route, and the other three need the same two decisions it makes: which bucket a token falls into, and what it costs.

Record every response against the task that made it

The loop that calls the model is the only place that knows which task and tenant a response belongs to, so that is where the ledger is written. calls.json holds five responses recorded from two providers, each tagged with the task id and the tenant the loop carried.

  {
    "task": "T-1001", "tenant": "acme", "provider": "openai",
    "response": {
      "id": "resp_q7vzk2p9x", "object": "response", "model": "openai-example",
      "usage": {
        "input_tokens": 12400,
        "input_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 12288 },
        "output_tokens": 900,
        "output_tokens_details": { "reasoning_tokens": 600 },
        "total_tokens": 13300
      }
    }
  },

Ledger.record takes that tuple, reads the usage, prices it, and keeps the row.

  // Call this with every provider response, in the loop that made the call: the loop is the only
  // place that knows which task and tenant the response belongs to.
  record({ task, tenant, provider, response }) {
    const usage = usageOf(provider, response)
    const nano = priceOf(usage, response.model, this.prices)
    const row = { task, tenant, model: response.model, ...usage, nano }
    this.rows.push(row)
    return row
  }

A proxy needs the same id, and it can only get it from a header or a request field. Helicone reads Helicone-Session-Id and any Helicone-Property- header you add, LiteLLM reads metadata.tags and the user field, and a call that leaves either out is counted against the key alone.

Read the usage fields into buckets that do not overlap

The two APIs report the same five quantities and disagree about which fields contain which. The Responses API’s input_tokens includes the cached and written tokens that input_tokens_details breaks down, and its output_tokens includes the reasoning tokens.¹ The Messages API’s input_tokens counts only the tokens after the last cache breakpoint, with the cache reads and writes reported beside it.² usageOf turns both into buckets where each token appears once, and checked throws on a usage object of another shape rather than pricing it as NaN.

export function usageOf(provider, response) {
  const u = response.usage
  if (provider === 'openai') {
    // Responses API: input_tokens INCLUDES the cached and written tokens it breaks down, and
    // output_tokens INCLUDES the reasoning tokens. Subtract, or the same token is paid twice.
    const details = u.input_tokens_details ?? {}
    const cached = details.cached_tokens ?? 0
    const written = details.cache_write_tokens ?? 0
    const reasoning = u.output_tokens_details?.reasoning_tokens ?? 0
    return checked(provider, {
      input: u.input_tokens - cached - written,
      cached_input: cached,
      cache_write: written,
      output: u.output_tokens - reasoning,
      reasoning,
    })
  }
  if (provider === 'anthropic') {
    // Messages API: input_tokens is already only the tokens after the last cache breakpoint. The
    // cache fields sit beside it and are typed nullable, so a null counts as zero.
    return checked(provider, {
      input: u.input_tokens,
      cached_input: u.cache_read_input_tokens ?? 0,
      cache_write: u.cache_creation_input_tokens ?? 0,
      output: u.output_tokens,
      reasoning: 0,
    })
  }
  throw new Error(`no usage reader for provider ${provider}`)
}

// Every bucket is a whole count of zero or more. A response in another shape (Chat Completions
// reports prompt_tokens, not input_tokens) or a field whose meaning moved fails here, rather than
// pricing as NaN or as a negative number of tokens.
function checked(provider, buckets) {
  for (const [name, n] of Object.entries(buckets)) {
    if (!Number.isInteger(n) || n < 0) throw new Error(`${provider} usage gives ${name} = ${n}, which this reader cannot price`)
  }
  return buckets
}

The reasoning bucket exists for the report, because a task that spends most of its output on tokens nobody reads is worth knowing about. It is priced at the output rate, which is how OpenAI bills it.³

Price from a table you keep, in whole numbers

The table is yours, keyed by the model string the response reports, in dollars per million tokens. Each model carries four rates. The two providers’ documentation gives the shape: a cache read at a fraction of the input price, a cache write at a quarter more, and output at several times either.

{
  "unit": "USD per million tokens",
  "models": {
    "openai-example": { "input": 2.0, "cached_input": 0.2, "cache_write": 2.5, "output": 10.0 },
    "anthropic-example": { "input": 3.0, "cached_input": 0.3, "cache_write": 3.75, "output": 15.0 }
  }
}

ratesOf converts each rate to an integer count of billionths of a dollar per token. A call’s cost is then an integer, a roll-up is an exact sum, and the Python ledger below agrees with this one to the last digit. A model missing from the table throws rather than pricing at zero. So does a rate finer than a billionth of a dollar per token, which the two languages would round apart.

export function ratesOf(model, prices) {
  const rate = prices.models[model]
  if (!rate) throw new Error(`no price for model ${model}: add it to prices.json`)
  const nano = (perMillion) => {
    const n = Math.round(perMillion * 1000)
    if (!(Math.abs(perMillion * 1000 - n) < 1e-6)) throw new Error(`${model}: ${perMillion} per million is finer than a nanodollar per token`)
    return n
  }
  return {
    input: nano(rate.input),
    cached_input: nano(rate.cached_input),
    cache_write: nano(rate.cache_write),
    output: nano(rate.output),
    reasoning: nano(rate.output),
  }
}

export function priceOf(usage, model, prices) {
  const rates = ratesOf(model, prices)
  let nano = 0
  for (const b of BUCKETS) nano += usage[b] * rates[b]
  return nano
}

node run.mjs

task    model              input  cached  write   output  reasoning  usd
T-1001  openai-example       112       0  12288      300        600  0.039944
T-1001  openai-example       812   12288      0      300          0  0.007082
T-1001  anthropic-example    410       0  11900      220          0  0.049155
T-1001  anthropic-example    650   11900      0      180          0  0.008220
T-1002  openai-example      4000       0      0      120          0  0.009200

per task
  T-1001   4 calls   51960 tokens  0.104401
  T-1002   1 calls    4120 tokens  0.009200

per tenant
  acme     4 calls   51960 tokens  0.104401
  globex   1 calls    4120 tokens  0.009200

Read the first two rows together. The first call writes 12,288 tokens to the cache and the second reads them back, so the second call carries 13,100 input tokens and costs less than a fifth of the first. The token column in the roll-up adds every bucket, and it is the one number in the table that means little. A cached token and a reasoning token are both tokens, and they differ in price by a factor of fifty.

meter.py is the same ledger in Python, and run.py prints the same table from the same files.

def price_of(usage, model, prices):
    rates = rates_of(model, prices)
    return sum(usage[b] * rates[b] for b in BUCKETS)


def usd(nano):
    return f"{nano / 1e9:.6f}"

python3 run.py

per task
  T-1001   4 calls   51960 tokens  0.104401
  T-1002   1 calls    4120 tokens  0.009200

per tenant
  acme     4 calls   51960 tokens  0.104401
  globex   1 calls    4120 tokens  0.009200

Check it worked

Run both suites. The Node tests pin the bucket arithmetic, the reasoning rate, the cache-read ratio, and the roll-up as an exact sum. They also expect an error for an unknown model, a misread usage object, and a sub-nanodollar rate.

node --test meter.test.mjs

1..8
# tests 8
# suites 0
# pass 8
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 92.139844

python3 -m unittest -v test_meter.py 2>&1

test_anthropic_buckets_are_already_exclusive (test_meter.MeterTest.test_anthropic_buckets_are_already_exclusive) ... ok
test_openai_buckets_are_exclusive (test_meter.MeterTest.test_openai_buckets_are_exclusive) ... ok
test_rate_finer_than_a_nanodollar_is_refused (test_meter.MeterTest.test_rate_finer_than_a_nanodollar_is_refused) ... ok
test_reasoning_tokens_take_the_output_rate (test_meter.MeterTest.test_reasoning_tokens_take_the_output_rate) ... ok
test_rollups_are_exact_sums_of_their_rows (test_meter.MeterTest.test_rollups_are_exact_sums_of_their_rows) ... ok
test_unknown_model_is_an_error (test_meter.MeterTest.test_unknown_model_is_an_error) ... ok
test_usage_the_reader_does_not_fit_is_an_error (test_meter.MeterTest.test_usage_the_reader_does_not_fit_is_an_error) ... ok

# pass 8 and seven ok lines. The roll-up test is the one that fails first when money is kept in floating point. As dollars, the four calls of T-1001 sum to 0.10440060000000001 in call order and 0.1044006 under Python’s sum(), which is why the rates are integers.

When it goes wrong

Every token was counted, but the bill is off by a third. pitfall.mjs prices the same task three wrong ways against the ledger.

node pitfall.mjs

task T-1001, correct: 0.104401 USD
  cached tokens at the plain input rate    0.143580 USD  +37.5%
  reasoning tokens at the input rate       0.099601 USD  -4.6%
  cached tokens counted twice              0.153553 USD  +47.1%

The first row charges cache reads and writes at the plain input rate and overstates the task by more than a third, because this task reads most of its input from cache. The second prices reasoning tokens as input and understates it, by a margin that grows with the share of output spent thinking. The third adds cached_tokens on top of a Responses API input_tokens that already contains them, which is the mistake Langfuse’s documentation warns about when it asks for exclusive buckets.

A model id arrives that the table does not know. A provider snapshot changes the model string a response reports, and the ledger throws on the first call rather than pricing it at zero. LiteLLM takes the other position for a known model with no rate: it writes the row with spend zero and flags it. Either is better than a silent zero, and either needs a person to add the row.

The ledger and the invoice disagree at month end. Prices move, and a table edited by hand lags them; a proxy’s map lags them too,⁴ and Langfuse applies a price change only to generations ingested after it. Reconcile the roll-up against the provider’s own usage export once a month, and treat a gap as a stale row in prices.json until shown otherwise.

When not to do this

Do not build this if a gateway already fronts every call and your question is spend per key or per team. LiteLLM and Helicone answer that from their own tables, and a second ledger in the process is a second number to reconcile.

Do not take the ledger for the invoice. It prices what your loop saw at the rates you typed, and the provider bills what it metered at the rates it publishes. The invoice settles disputes; the ledger explains them.

Do not add up tokens across models and call the total usage. A tokenizer change between model generations moves the count for the same text, and a cached token is a different purchase from a reasoning token. Dollars add; tokens from two models do not.

Do not price from memory. Copy each rate from the provider’s pricing page into the table when a model is added, with the page it came from in a comment, so the next person can check it.

Related how-tos

• Cut agent cost with provider prompt caching

• Cache tool results across the steps of one agent run

• Measure the token cost of an agent task against your API

• Compare agent task success rates across model providers

Last verified

Verified 2026-10-21 against Node 22.22.0 and Python 3.13.16, with no packages installed. Every output block is what the command preceding it printed. No request reached a provider: the usage objects are shaped as the two APIs document them, and the prices are placeholders, so the dollar figures show the arithmetic and not a bill.