AI Token Efficiency: Measure Cost per Accepted Result

Published · Updated • articles
Inference EfficiencyPricingBenchmarksAI Agents

Compare the total expense of producing accepted results. A lower token count can reduce a bill, but a workload with more retries or costly mistakes can lose that saving.

Correction, 21 October 2026: The earlier version mixed Opus versions and prices, assigned a benchmark delta to the wrong Sonnet comparison and called an accuracy loss broadly acceptable. It also attributed a GPT model's economics to distillation without establishing that mechanism. Those comparisons are withdrawn. This revision separates reported token observations from our illustrative cost method.

Distinguish rate changes from usage changes

Anthropic's Sonnet 5.5 announcement lists $2 per million input tokens and $10 per million output tokens, the same as Sonnet 5, while describing lower token use. It separately lists cheaper cache reads. Those are different mechanisms: fewer billed tokens, a changed cache rate and ordinary input/output pricing.

A customer observation in a launch announcement can help identify a workload to test. It does not establish that the same reduction applies to every task. Compare the same input set and acceptance criteria, and retain the model version and effort configuration.

Read each token denominator

Fireworks' Ember-1 release includes a coding A/B table: output tokens fall from 49.3K to 29.9K while task scores move from 0.751 to 0.753. It labels total-token reduction as 39% and reasoning-token reduction as 71.3%.

The displayed output counts imply about 39.4% fewer output tokens, using rounded inputs. That arithmetic is distinct from the release's separately labeled total-token measurement. Do not treat output, reasoning and total tokens as interchangeable. The reported test also does not establish equivalent performance on every kind of reasoning task.

ThinkingCap's model card reports a 37.2% equal-weight mean of per-benchmark thinking-token reductions. Its AIME row changes from 98.13 to 94.27 accuracy with 30.2% fewer thinking tokens: a 3.86-percentage-point decline. The headline mean is not a promise of the same saving on every request.

The practical consequence is to retain the distribution. A large saving on many easy requests can coexist with a quality loss on a rare task that matters to the application.

Calculate savings without assuming quality

Suppose two hypothetical systems charge the same $10 per million output tokens. One emits 50,000 output tokens per attempt and the other emits 30,000. Output-only cost is $0.50 versus $0.30, or a 40% reduction.

These are invented example inputs, not another vendor result. They omit input, caching, tool calls and operations. If the first system produces 100 accepted results from 100 attempts, its output cost per accepted result is $0.50. If the second needs 150 attempts to produce those 100 accepted results, its output cost per accepted result becomes $0.45.

A different retry rate could erase the saving. Human repair and external tool costs can change the result further.

RecordWhy it matters
Input, output and cached tokens separatelyThey can carry different prices and vary independently
All attempts, including failed runsCompleted-only measurements can hide waste
Accepted results under fixed criteriaThe denominator must represent useful work
Tool and repair expenseModel billing is only part of delivery cost
Task-level quality and latencyAn overall mean can hide an unacceptable subgroup

Set the acceptable trade-off first

Define which errors can be repaired, which require rejection and which should stop automation. Compare cost only after applying those rules consistently.

Do not infer a training mechanism from a billing chart or assume a model tier establishes the best choice. Our benchmark-reading framework explains how to retain the evaluation conditions when using reported results to plan a local test.

Frequently asked questions

Does fewer output tokens always mean a cheaper completed task?

No. Include input, cached input, tool use, retries and human repair, then divide by accepted results. Token savings alone do not establish that quality stayed sufficient.

Which Sonnet comparison was wrong in the earlier article?

It assigned the wrong comparator to a benchmark delta and mixed Opus versions in its price comparison. The revised article avoids those mixed-version comparisons and cites Anthropic's actual Sonnet announcement.

What does Fireworks report for Ember-1?

Its published coding A/B table shows output tokens falling from 49.3K to 29.9K and task scores of 0.751 versus 0.753. Fireworks labels total-token reduction 39 percent. These are vendor-reported workload results.

How large is ThinkingCap's AIME 2026 accuracy change?

The card shows 98.13 to 94.27, a decline of 3.86 percentage points, with 30.2 percent fewer thinking tokens. Acceptability depends on the application; this article does not declare that loss acceptable for enterprise use.

Free interactive tools for the decisions this piece raises.

Related Reading