Tokens, Prices and Why Long Inputs Cost More
Models are priced and timed in tokens, not words. Knowing how text becomes tokens helps you estimate cost and avoid surprises.
On this page
Hosted models are priced and timed in tokens, which explains why so many cost estimates built on word counts come in low or high. A token is a piece of text chosen by the tokenizer rather than a dictionary word, and the same page can yield a noticeably different count depending on its language, its content and the formatting around it. Once the unit is clear, the bill becomes far easier to predict.
What a token is
A token is a piece from the model's vocabulary, and the rules for splitting text into tokens were fixed before the model was trained. The vocabulary differs between model families, so the same text can produce different counts on different providers. In everyday English prose a token is often a whole word or a fragment of one, with the leading space usually attached to the word that follows. Punctuation, numbers and symbols are split too, sometimes one token per character.
The practical consequence is that a word count gives only a rough guess. Two texts with the same number of words can have quite different token counts, and a per-word estimate will drift as the content changes.
Why the same text costs different amounts
Several features push the count up. Text in languages the tokenizer saw less of often splits into more pieces. Source code, with its brackets, indentation and long identifiers, is frequently heavier than prose of the same length. Structured formats add overhead: a JSON object carries quotation marks, braces and repeated keys, and each of those characters is a token the model must read. Hashes, random strings and long identifiers are among the least efficient inputs of all.
The language effect is easy to measure, so measure it. Take the same paragraph in each language the product serves and count the tokens for each. The differences are often large enough to change a budget, and they stay invisible if all testing happens in one language.
Repetition adds cost in a different way. A system prompt is part of every request, so a paragraph of boilerplate is paid for on each call, not once. An instruction that runs to three sentences may be worth shortening if it is sent thousands of times a day.
Input, output and repeated calls
Most providers charge for input and output tokens separately, and output is usually the more expensive of the two per token. A feature that writes long answers therefore has a different cost profile from one that reads long documents and returns a short label.
Conversations carry a trap. If an application re-sends the entire history on every turn, the tokens processed grow with each exchange, so the total for a long conversation rises faster than the number of turns. Agents that call tools in a loop multiply the effect, because each step re-reads the growing context. The fix is structural: keep the history short, summarize earlier turns, and pass only the state a step needs.
Caching repeated context
If many requests begin with the same long prefix, such as a policy document or a fixed set of instructions, some providers offer a reduced price for input that has been processed before. The details differ. Caching usually requires the prefix to match exactly, the cache expires after a period, and the reduced rate applies to the cached portion only. Read the current terms before designing around caching, because they change.
The design lesson holds regardless of terms. Put stable content first and variable content last, so the shared prefix is as long as possible. Changing one word near the top of a prompt can invalidate everything after it.
Estimating a batch before running it
A reliable estimate for a batch job comes from a sample rather than a guess:
- Draw a sample of real inputs large enough to include the long and short cases, and run each one through the provider's token counter.
- Compute the average and spread of input tokens. Estimate output tokens from a sample of real outputs, since they are harder to predict.
- Multiply by the number of items, then add room for retries, for truncated outputs and for the longest cases in the tail.
- Convert the totals to cost using the current price sheet, and check the result against a small live run before scaling up.
- Repeat the measurement whenever the prompt, the model or the input source changes.
Each step is simple. The discipline is in keeping the measurement current, since an estimate made before a template change says little about the template after it.
Averages can mislead. Suppose a sample has a modest average input size but includes a handful of very long documents. The average hides them, while the batch will pay for every one in full. Estimate the tail separately, and treat the unusually long items as their own group, perhaps with a different model, a different batch size or a truncation rule agreed in advance. The same logic applies to output. A feature that usually returns a short label but occasionally returns an essay has a cost distribution, not a single cost.
A counting exercise
Take the prompt that runs most often in your own work and count its tokens with the provider's tool. Remove whatever the model does not need and count again. Then split the remaining prompt into its parts (instructions, examples, input data and the expected output format) and count each part. Find the largest, and ask whether the model needs all of it or only a short version that preserves what the output depends on. Test the shorter version against the same inputs. If quality holds, the result is a cheaper prompt and a method that works for the next one.
