Every AI answer has a meter running behind it. The unit on that meter is called a token, and once you understand tokens, you understand what AI actually costs, why the price keeps dropping, and why the chip stocks I follow care so much.
What a token actually is
An AI model doesn't read words the way you do. It reads text chopped into small pieces: tokens. A token is roughly a piece of a word: on average, in English, one token is about four characters, so 1,000 tokens is roughly 750 words, about a page and a half of text. French usually needs a few more tokens to say the same thing, which is one of those small details that quietly matters for costs.
Everything an AI does is tokens in, tokens out. Your question goes in as tokens. The answer comes out as tokens. A long document you ask it to summarize? Tokens. Code it writes? Tokens. The token is the litre at the gas station: the basic unit the whole industry is billed in.
How the price works: input and output
The big AI labs (OpenAI, Anthropic, Google and their peers) publish their own price lists, and the prices are quoted per million tokens. There are always two prices:
- Input tokens (what you send the model): your question, your document, the conversation so far. Cheaper.
- Output tokens (what the model writes back, generated piece by piece). This is the expensive direction, usually several times the input price, because producing the answer is where the heavy computing happens.
What does that look like in money? Take round numbers, purely as an example: a flagship model might list at $5 per million input tokens and $25 per million output tokens. Now ask it to summarize a long report (say 10,000 tokens in, 1,000 tokens of summary out). The input costs five cents. The output costs two and a half cents. Total: about eight cents for the whole job. Smaller, faster models list at a small fraction of those example prices, which is exactly why so much everyday AI feels free.
Those figures are illustrations, not quotes. Prices change constantly (labs cut them, launch cheaper tiers, discount repeated context), so the only honest way to state an AI price is with a date on it, straight from the provider's own published pricing page.
Why the price keeps falling
Token prices have been falling fast, and for boring, durable reasons:
- Competition. Several labs sell roughly comparable models, plus open models anyone can run. Nobody gets to sit on a fat margin for long.
- Efficiency. Better chips, better software, and smaller specialized models that do one job well instead of everything at maximum size.
- Caching. If the same context is sent over and over, providers discount it heavily. Reading the same page twice shouldn't cost full price twice.
The result: the same answer costs a fraction of what it cost a short while ago, while the models keep getting better. That almost never happens in other industries: imagine cars getting faster and cheaper every few months.
What it means for the stocks
Here is the part that matters for the names I follow. When the price per answer falls, people don't pocket the savings. They ask more questions. Usage explodes. And every one of those billions of extra answers has to be computed somewhere: on accelerators, fed by exactly the high-bandwidth memory the chip names on my watchlist make. Cheaper tokens, paradoxically, have meant more total demand for compute and memory, not less.
For the labs themselves, falling token prices squeeze the per-answer margin: the business becomes a volume game, won by whoever serves the most tokens at the lowest cost. For investors, the direction of token prices is a demand gauge hiding in plain sight: prices falling while usage booms is the healthiest combination this industry can print.
That's why the weekly recap now carries a 60-second AI Price Watch: one number from the providers' published list prices, the date it was checked, and one line on what it means for the stocks. One number, honestly labelled, beats a feed of guesses.