# How to account for tokens and cost per agent task

> Read each provider's usage fields into exclusive token buckets, price them from a table you keep, and roll the money up per task and per tenant.

Source: https://voxgig.com/howto/account-for-tokens-and-cost-per-task

- Audience: platform-team
- Level: intermediate
- Languages: typescript, javascript, python
- Verified: 2026-10-21
- Published: 2026-10-19

## Short answer

Record every model response against the task id your agent loop carries, and read its usage fields into exclusive buckets. On the Responses API, subtract cached and written tokens from `input_tokens` and reasoning tokens from `output_tokens`; on the Messages API the fields are already separate. Price each bucket from a table you maintain, with cache reads at their discount and reasoning at the output rate, then sum per task and per tenant.

---
## You will need

Node 22 or later, Python 3.11 or later, responses saved from your own model calls, and the pricing
page of each provider you call. Verified 2026-10-21 against Node 22.22.0 and Python 3.13.16. The
sample uses the standard library of both languages and sends nothing to a provider. The `usage`
objects in `calls.json` are shaped as the
[Responses API](https://developers.openai.com/api/reference/resources/responses) and the
[Messages API](https://platform.claude.com/docs/en/api/messages) document them, with token counts
chosen for the example. The prices in `prices.json` are placeholders in round figures that keep the
providers' documented ratios between a plain input token, a cache read, and a cache write.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [Helicone](https://docs.helicone.ai/features/advanced-usage/custom-properties) | Several services call the provider and none of them should carry accounting code | A hop before every call, a price table that is not yours, and a request without the header lands in nobody's bucket | You need the number inside the process, to stop a task at a budget |
| [Langfuse cost tracking](https://langfuse.com/docs/observability/features/token-and-cost-tracking) | You already trace runs there and want cost beside the spans | Usage must arrive as exclusive buckets, and a price table change reaches only new generations | Nothing else in the stack is traced |
| [LiteLLM proxy spend tracking](https://docs.litellm.ai/docs/proxy/cost_tracking) | Several providers behind one OpenAI-compatible endpoint, with spend wanted per key, team, or end user | A proxy to run, a cost map to keep current, and spend by tag only for callers that send the tag | One provider and one process, where a proxy is a second deployment for one table |
| [Provider usage fields read in-process](https://developers.openai.com/api/docs/guides/prompt-caching) | You own the loop and want the exact bill for what it did, as it happens | Your own price table and a reader per provider, and blindness to every call made outside the loop | The calls come from code you do not control |

A proxy meters every caller that goes through it, which is its attraction and its limit. A request
that skips it, or arrives without the tag, is spend with no owner, and the proxy's price table is
one more thing to keep in step with the invoice. In-process accounting is exact for the loop you
own and blind to everything else. The sample below takes the in-process route, and the other three
need the same two decisions it makes: which bucket a token falls into, and what it costs.

## Record every response against the task that made it

The loop that calls the model is the only place that knows which task and tenant a response
belongs to, so that is where the ledger is written. `calls.json` holds five responses recorded
from two providers, each tagged with the task id and the tenant the loop carried.

```json title="calls.json"
  {
    "task": "T-1001", "tenant": "acme", "provider": "openai",
    "response": {
      "id": "resp_q7vzk2p9x", "object": "response", "model": "openai-example",
      "usage": {
        "input_tokens": 12400,
        "input_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 12288 },
        "output_tokens": 900,
        "output_tokens_details": { "reasoning_tokens": 600 },
        "total_tokens": 13300
      }
    }
  },
```

`Ledger.record` takes that tuple, reads the usage, prices it, and keeps the row.

```js title="meter.mjs"
  // Call this with every provider response, in the loop that made the call: the loop is the only
  // place that knows which task and tenant the response belongs to.
  record({ task, tenant, provider, response }) {
    const usage = usageOf(provider, response)
    const nano = priceOf(usage, response.model, this.prices)
    const row = { task, tenant, model: response.model, ...usage, nano }
    this.rows.push(row)
    return row
  }
```

A proxy needs the same id, and it can only get it from a header or a request field. Helicone reads
[`Helicone-Session-Id`](https://docs.helicone.ai/features/sessions) and any `Helicone-Property-`
header you add, LiteLLM reads `metadata.tags` and the `user` field, and a call that leaves either
out is counted against the key alone.

## Read the usage fields into buckets that do not overlap

The two APIs report the same five quantities and disagree about which fields contain which. The
Responses API's `input_tokens` includes the cached and written tokens that `input_tokens_details`
breaks down, and its `output_tokens` includes the reasoning tokens.[^1] The Messages API's
`input_tokens` counts only the tokens after the last cache breakpoint, with the cache reads and
writes reported beside it.[^2] `usageOf` turns both into buckets where each token appears once,
and `checked` throws on a usage object of another shape rather than pricing it as `NaN`.

```js title="meter.mjs"
export function usageOf(provider, response) {
  const u = response.usage
  if (provider === 'openai') {
    // Responses API: input_tokens INCLUDES the cached and written tokens it breaks down, and
    // output_tokens INCLUDES the reasoning tokens. Subtract, or the same token is paid twice.
    const details = u.input_tokens_details ?? {}
    const cached = details.cached_tokens ?? 0
    const written = details.cache_write_tokens ?? 0
    const reasoning = u.output_tokens_details?.reasoning_tokens ?? 0
    return checked(provider, {
      input: u.input_tokens - cached - written,
      cached_input: cached,
      cache_write: written,
      output: u.output_tokens - reasoning,
      reasoning,
    })
  }
  if (provider === 'anthropic') {
    // Messages API: input_tokens is already only the tokens after the last cache breakpoint. The
    // cache fields sit beside it and are typed nullable, so a null counts as zero.
    return checked(provider, {
      input: u.input_tokens,
      cached_input: u.cache_read_input_tokens ?? 0,
      cache_write: u.cache_creation_input_tokens ?? 0,
      output: u.output_tokens,
      reasoning: 0,
    })
  }
  throw new Error(`no usage reader for provider ${provider}`)
}

// Every bucket is a whole count of zero or more. A response in another shape (Chat Completions
// reports prompt_tokens, not input_tokens) or a field whose meaning moved fails here, rather than
// pricing as NaN or as a negative number of tokens.
function checked(provider, buckets) {
  for (const [name, n] of Object.entries(buckets)) {
    if (!Number.isInteger(n) || n < 0) throw new Error(`${provider} usage gives ${name} = ${n}, which this reader cannot price`)
  }
  return buckets
}
```

The reasoning bucket exists for the report, because a task that spends most of its output on
tokens nobody reads is worth knowing about. It is priced at the output rate, which is how OpenAI
bills it.[^3]

## Price from a table you keep, in whole numbers

The table is yours, keyed by the `model` string the response reports, in dollars per million
tokens. Each model carries four rates. The two providers' documentation gives the shape: a cache
read at a fraction of the input price, a cache write at a quarter more, and output at several
times either.

```json title="prices.json"
{
  "unit": "USD per million tokens",
  "models": {
    "openai-example": { "input": 2.0, "cached_input": 0.2, "cache_write": 2.5, "output": 10.0 },
    "anthropic-example": { "input": 3.0, "cached_input": 0.3, "cache_write": 3.75, "output": 15.0 }
  }
}
```

`ratesOf` converts each rate to an integer count of billionths of a dollar per token. A call's
cost is then an integer, a roll-up is an exact sum, and the Python ledger below agrees with this
one to the last digit. A model missing from the table throws rather than pricing at zero. So does
a rate finer than a billionth of a dollar per token, which the two languages would round apart.

```js title="meter.mjs"
export function ratesOf(model, prices) {
  const rate = prices.models[model]
  if (!rate) throw new Error(`no price for model ${model}: add it to prices.json`)
  const nano = (perMillion) => {
    const n = Math.round(perMillion * 1000)
    if (!(Math.abs(perMillion * 1000 - n) < 1e-6)) throw new Error(`${model}: ${perMillion} per million is finer than a nanodollar per token`)
    return n
  }
  return {
    input: nano(rate.input),
    cached_input: nano(rate.cached_input),
    cache_write: nano(rate.cache_write),
    output: nano(rate.output),
    reasoning: nano(rate.output),
  }
}

export function priceOf(usage, model, prices) {
  const rates = ratesOf(model, prices)
  let nano = 0
  for (const b of BUCKETS) nano += usage[b] * rates[b]
  return nano
}
```

```bash
node run.mjs
```

```text output
task    model              input  cached  write   output  reasoning  usd
T-1001  openai-example       112       0  12288      300        600  0.039944
T-1001  openai-example       812   12288      0      300          0  0.007082
T-1001  anthropic-example    410       0  11900      220          0  0.049155
T-1001  anthropic-example    650   11900      0      180          0  0.008220
T-1002  openai-example      4000       0      0      120          0  0.009200

per task
  T-1001   4 calls   51960 tokens  0.104401
  T-1002   1 calls    4120 tokens  0.009200

per tenant
  acme     4 calls   51960 tokens  0.104401
  globex   1 calls    4120 tokens  0.009200
```

Read the first two rows together. The first call writes 12,288 tokens to the cache and the second
reads them back, so the second call carries 13,100 input tokens and costs less than a fifth of
the first.
The token column in the roll-up adds every bucket, and it is the one number in the table that
means little. A cached token and a reasoning token are both tokens, and they differ in price by a
factor of fifty.

`meter.py` is the same ledger in Python, and `run.py` prints the same table from the same files.

```python title="meter.py"
def price_of(usage, model, prices):
    rates = rates_of(model, prices)
    return sum(usage[b] * rates[b] for b in BUCKETS)


def usd(nano):
    return f"{nano / 1e9:.6f}"
```

```bash
python3 run.py
```

```text output
per task
  T-1001   4 calls   51960 tokens  0.104401
  T-1002   1 calls    4120 tokens  0.009200

per tenant
  acme     4 calls   51960 tokens  0.104401
  globex   1 calls    4120 tokens  0.009200
```

## Check it worked

Run both suites. The Node tests pin the bucket arithmetic, the reasoning rate, the cache-read
ratio, and the roll-up as an exact sum. They also expect an error for an unknown model, a misread
usage object, and a sub-nanodollar rate.

```bash
node --test meter.test.mjs
```

```text output
1..8
# tests 8
# suites 0
# pass 8
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 92.139844
```

```bash
python3 -m unittest -v test_meter.py 2>&1
```

```text output
test_anthropic_buckets_are_already_exclusive (test_meter.MeterTest.test_anthropic_buckets_are_already_exclusive) ... ok
test_openai_buckets_are_exclusive (test_meter.MeterTest.test_openai_buckets_are_exclusive) ... ok
test_rate_finer_than_a_nanodollar_is_refused (test_meter.MeterTest.test_rate_finer_than_a_nanodollar_is_refused) ... ok
test_reasoning_tokens_take_the_output_rate (test_meter.MeterTest.test_reasoning_tokens_take_the_output_rate) ... ok
test_rollups_are_exact_sums_of_their_rows (test_meter.MeterTest.test_rollups_are_exact_sums_of_their_rows) ... ok
test_unknown_model_is_an_error (test_meter.MeterTest.test_unknown_model_is_an_error) ... ok
test_usage_the_reader_does_not_fit_is_an_error (test_meter.MeterTest.test_usage_the_reader_does_not_fit_is_an_error) ... ok
```

`# pass 8` and seven `ok` lines. The roll-up test is the one that fails first when money is kept
in floating point. As dollars, the four calls of `T-1001` sum to `0.10440060000000001` in call
order and `0.1044006` under Python's `sum()`, which is why the rates are integers.

## When it goes wrong

Every token was counted, but the bill is off by a third. `pitfall.mjs` prices the same task three
wrong ways against the ledger.

```bash
node pitfall.mjs
```

```text output
task T-1001, correct: 0.104401 USD
  cached tokens at the plain input rate    0.143580 USD  +37.5%
  reasoning tokens at the input rate       0.099601 USD  -4.6%
  cached tokens counted twice              0.153553 USD  +47.1%
```

The first row charges cache reads and writes at the plain input rate and overstates the task by
more than a third, because this task reads most of its input from cache. The second prices
reasoning tokens as input and understates it, by a margin that grows with the share of output
spent thinking. The third adds `cached_tokens` on top of a Responses API `input_tokens` that
already contains them, which is the mistake Langfuse's documentation warns about when it asks for
exclusive buckets.

A model id arrives that the table does not know. A provider snapshot changes the `model` string a
response reports, and the ledger throws on the first call rather than pricing it at zero. LiteLLM
takes the other position for a known model with no rate: it writes the row with `spend` zero and
flags it. Either is better than a silent zero, and either needs a person to add the row.

The ledger and the invoice disagree at month end. Prices move, and a table edited by hand lags
them; a proxy's map lags them too,[^4] and Langfuse applies a price change only to generations
ingested after it. Reconcile the roll-up against the provider's own usage export once a month, and
treat a gap as a stale row in `prices.json` until shown otherwise.

## When not to do this

Do not build this if a gateway already fronts every call and your question is spend per key or
per team. LiteLLM and Helicone answer that from their own tables, and a second ledger in the
process is a second number to reconcile.

Do not take the ledger for the invoice. It prices what your loop saw at the rates you typed, and
the provider bills what it metered at the rates it publishes. The invoice settles disputes; the
ledger explains them.

Do not add up tokens across models and call the total usage. A tokenizer change between model
generations moves the count for the same text, and a cached token is a different purchase from a
reasoning token. Dollars add; tokens from two models do not.

Do not price from memory. Copy each rate from the provider's pricing page into the table when a
model is added, with the page it came from in a comment, so the next person can check it.

## Related how-tos

- [Cut agent cost with provider prompt caching](/howto/cut-agent-cost-with-prompt-caching)

- [Cache tool results across the steps of one agent run](/howto/cache-tool-results-across-agent-steps)

- [Measure the token cost of an agent task against your API](/howto/measure-token-cost-per-agent-task)

- [Compare agent task success rates across model providers](/howto/measure-task-success-across-model-providers)

## Last verified

Verified 2026-10-21 against Node 22.22.0 and Python 3.13.16, with no packages installed. Every
output block is what the command preceding it printed. No request reached a provider: the usage
objects are shaped as the two APIs document them, and the prices are placeholders, so the dollar
figures show the arithmetic and not a bill.

[^1]: The Langfuse documentation on
[token and cost tracking](https://langfuse.com/docs/observability/features/token-and-cost-tracking)
says every key in `usage_details` must be an exclusive bucket. Its worked case: a response
reporting 17,903 prompt tokens, of which 17,817 were cache hits, is stored as 86 input tokens and
17,817 cached ones. It adds that overlapping buckets double the inferred cost, which
is a thing a tracing vendor learns from its own dashboards.

[^2]: The Claude [prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
documentation defines `input_tokens` as the tokens after the last cache breakpoint and gives the
total as the sum of three fields. The Responses API
[reference](https://developers.openai.com/api/reference/resources/responses) defines its
`input_tokens` as the number of input tokens, with cached and written tokens as a breakdown of it.
The same field name carries two different totals, so each API needs a reader of its own.

[^3]: OpenAI's [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) says
reasoning tokens occupy the context window and are billed as output tokens, and shows a usage
object with 1,186 output tokens of which 1,024 were reasoning. The tokens are never returned to
the caller. They are the only line on the bill for something the customer did not receive and
could not have read.

[^4]: Each tool in the table keeps a price list of its own. LiteLLM's is
[one JSON file](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
of a few megabytes. It has a key for every tier a provider has invented, down to the cost of a
cache read past 272 thousand tokens in a fast mode. Langfuse runs a daily automated audit of its
definitions against the providers' pages and opens a pull request when a price moves. Each list is a transcription of
pages that change without notice, which is also what your `prices.json` is.