Why Does Cheaper AI Keep Producing Bigger AI Bills?
From the Tokenomics: The Economics of AI collection
A language model asked how many times the letter r appears in "strawberry" can get the answer wrong, and the reason is not a lack of intelligence. The model never saw the letters. Before any reading happened, a tokenizer cut the word into sub-word fragments and replaced each fragment with an integer. A legal brief, a spreadsheet, and a line of code pass through the same machine.
Those fragments, called tokens, are what a language model reads, what it writes, and what its provider charges for. That makes the token more than a technical detail. The token is the smallest billable unit of machine cognition, and every prompt runs a meter counted in it.
Most discussion of AI treats it as a capability: what a model can do, how well, and how soon. The meter gets far less scrutiny, even as AI spending becomes a visible line in operating budgets. A familiar pattern follows. Per-token prices fall, usage climbs faster, and nobody can say which tokens produced anything worth having.
The way out is to read AI as an economic system rather than a magic interface. Four ideas do most of the work: how cost is actually generated, why cost and value diverge, where in the industry value is kept, and what physical limits sit beneath an apparently endless supply.
The asymmetry inside the meter
A language model does two different kinds of work on every request. It reads the prompt in a single parallel pass, often called prefill, and then writes the answer one token at a time, each new token depending on everything before it. Reading parallelizes well on modern chips; writing does not. That is why providers commonly publish separate input and output prices, with output priced higher.
The consequences run against intuition. A short question that triggers a long answer can cost more than a long document that is merely read. Models that reason before responding generate intermediate tokens the user never sees, and those tokens are billed too. A large context window is not free research space; it is working memory rented on some of the most expensive hardware in use.
Add the levers that surround the published price and the picture gets stranger. Many providers discount input that repeats across requests and has been cached, and many offer a cheaper batch lane for work that can wait hours rather than seconds. Model choice, context length, retry loops, and reasoning settings each move the bill. The effective cost is never the sticker price, and two teams paying identical rates can receive very different invoices.
Why cheaper does not mean smaller
In The Coal Question, published in 1865, William Stanley Jevons argued that more efficient steam engines would increase Britain's total coal consumption rather than reduce it. Cheaper power made new uses economical, and the new uses outgrew the savings. The argument survives today as the Jevons paradox.
Falling per-token prices invite the same dynamic. Longer contexts become affordable, so they get used. Reasoning modes become routine, so answers carry a hidden layer of generation. Autonomous agents that loop through tools and retries can consume many times the tokens of a single exchange, often without a matching improvement in results.
Whether total spending rises or falls depends on how much new demand cheaper cognition creates, which makes the price trend a poor guide to the bill. The useful measure is not cost per call but cost per successful outcome: what it took, in tokens, to produce a result someone actually used. Measured that way, experimentation and production spending separate cleanly. Routing easy tasks to cheap models and hard tasks to capable ones becomes a margin decision, and guardrails on agent loops stop looking like bureaucracy.
Cost is mechanical, value is not
Token cost follows rules. It rises with length, model tier, and repetition, and it can be forecast. Token value follows no such rules. The same spend can produce generic filler or replace hours of skilled analysis, depending on whether the question deserved asking and whether the answer feeds a decision.
This asymmetry is the central fact of the token economy. Implementation has become cheap; judgment about what to implement has not. Two people with the same subscription and the same budget can produce radically different value, and the difference lies almost entirely in problem selection and workflow design. As execution gets cheaper, choosing the right objective becomes the scarce skill.
Where the value settles
The same logic explains the industry's shape. At the base, owners of chips, memory, data centers, and power collect a rent on every token generated, whoever the end customer is. At the top, applications woven deeply into a customer's workflow hold the relationship, the data, and the trust. Between them sits a layer of undifferentiated tools that resell model output behind a thin interface.
That middle is exposed from both sides. Each model release can absorb features a wrapper product once sold as its advantage, while infrastructure costs set a floor under what it pays. The result is a barbell economy, heavy at the extremes and thin in the middle. Vertical depth and workflow integration, not access to a good model, decide whether an AI product keeps its margin.
Scarcity beneath abundance
Cheap cognition rests on expensive, slow-moving physical capacity. Every token passes through advanced chips, high-bandwidth memory, specialized packaging, networking, data centers, electricity, cooling, and water. These inputs expand on the timescale of construction permits and grid connections, not software releases.
Governments have noticed. Since October 2022 the United States has restricted exports of advanced AI chips and chipmaking equipment to China, which turns chip policy into a form of token policy: it shapes which countries produce frontier capability and which import it. Compute is becoming strategic infrastructure, much as energy did in the last century. The paradox is sharp: output grows cheaper while the capacity to produce it stays concentrated and contested.
The meter and the measure
There is one more place where the meter misleads, and it is the national accounts. When a service becomes cheaper and better at once, spending on it can fall while the work delivered rises, and gains that stay inside firms often carry no price at all. In 1987 Robert Solow remarked that the computer age could be seen everywhere but in the productivity statistics. The economic historian Paul David later pointed to factory electrification, where the large gains arrived only after plants were redesigned around electric motors, decades after the motors themselves appeared.
Whether AI follows that path is a live debate, not a settled finding. But the electrification story carries a lesson that does not depend on the macro outcome. Metering a technology is easy. The value arrives when work is reorganized around it.
That lesson reduces to three questions every organization using AI has to answer. Access: can it obtain the capability and capacity it needs, on terms it can live with? Direction: are its tokens aimed at work with real friction, real value, and a measurable result? Capture: does the value created stay with the organization, or leak to suppliers and to the next vendor its customers find?
Tokens are cheap and getting cheaper. Good direction is scarce and getting scarcer. An economy built on metered cognition will not reward those who consume the most tokens. It will reward those who know what each one costs, point it at a problem worth solving, and keep what it produces.