Why LLM APIs bill in tokens instead of words, official English conversion rules of thumb, worked examples, and when to use Exact tokenization instead of a heuristic.
Try it in TokenCALC
Paste real text into the tokenizer for Exact OpenAI counts, or open Tokens ↔ words when you only have a word or character estimate.
Tokens are not words
A word count from Google Docs or Microsoft Word is not what OpenAI, Anthropic, Google, or other LLM APIs bill. Models split text into tokens: pieces that may be whole words, parts of words, spaces, or punctuation. The same sentence can become a different number of tokens on different providers because each family uses its own vocabulary.
Official English rules of thumb
OpenAI’s public guidance for common English text is the industry baseline most teams start from: about 4 characters per token, and about ¾ of a word per token (so 100 tokens ≈ 75 words). That is a planning heuristic, not a billing guarantee. Always treat it as an estimate until you run a real tokenizer on your prompt.
- 1 token ≈ 4 characters of English text
- 1 token ≈ 0.75 words (¾ of a word)
- 100 tokens ≈ 75 words
- 1,000 tokens ≈ 750 words
Quick conversion formulas
Use these only for English prose ballparks. Round up when the budget matters.
- Words → tokens: multiply words by about 1.33
- Tokens → words: multiply tokens by about 0.75
- Characters → tokens: divide characters by about 4
- Tokens → characters: multiply tokens by about 4
Worked examples
These examples use the OpenAI English rule of thumb so you can sanity-check spreadsheets and product briefs.
- 100 words ≈ 133 tokens
- 750 words ≈ 1,000 tokens
- 1,000 words ≈ 1,333 tokens
- A short chat message (1 to 2 sentences) is often on the order of tens of tokens, not hundreds
- A double-spaced page (~500 words) is often roughly 650 to 700 tokens
When the heuristic fails
Code, URLs, markdown tables, IDs, and many non-English scripts often use more tokens per word or per character than plain English prose. Rare or long words also tend to split into more subword pieces. Empirical work across tokenizer families shows the familiar 0.75 words-per-token rule can over- or under-estimate depending on the model family and the text mix. If your workload is code-heavy or multilingual, do not trust a Word count for production budgets.
Tokens vs words vs characters
Words are for humans. Characters are a rough length signal. Tokens are what models read and what APIs price. For budgeting, convert words or characters to tokens first, then apply input and output rates. Never multiply a word count directly by a “price per million words.” That unit does not exist on major LLM price sheets.
Exact counts vs rough conversion
Use the Tokens ↔ words converter when you only have a word or character estimate and need a quick Approx budget. Use the main tokenizer when you have the real prompt: OpenAI encodings can be Exact in your browser via gpt-tokenizer; other providers stay labeled Approx when their official tokenizer is not available client-side. That honesty matters more than a false sense of precision.
From word count to API cost
After you have an input token estimate, remember output tokens are usually priced higher than input. A short prompt with a long answer can dominate the bill. Paste a representative prompt into the cost calculator, set expected output size and monthly volume, and compare models on the same workload. Do not invent rates in a spreadsheet when TokenCALC already stores curated per-model prices with lastVerified dates.
Context windows in words (approx only)
People often ask whether a document “fits” by converting the context window back into words (tokens × 0.75). That is fine for a napkin check, but the window must also hold system prompts, tools, history, and output. Use the tokenizer context meter for a real fit check, and read the context windows guide before you assume a 128K model will hold a full book plus a long reply.
Common mistakes
These errors show up constantly in early LLM budgeting.
- Treating Microsoft Word’s word count as the API bill
- Ignoring output tokens when estimating cost
- Assuming Claude or Gemini match OpenAI token counts on the same text
- Using English heuristics on code or non-English corpora
- Forgetting chat/system/tool overhead beyond the visible user message
Best practices
Start from tokens, not words, whenever the real text exists. Keep one representative prompt for comparisons. Label every estimate Exact or Approx. Re-check monthly forecasts after you have invoice data. Link product docs to the converter for rough planning and to the tokenizer for shipping decisions.
Real-world scenarios
A product manager receives a fifty page specification as a Word attachment and asks whether it fits in a 128K context model. Word reports 25,000 words. Multiplying by 1.33 suggests about 33,000 tokens, but tables, headings, and legal numbering often push real counts higher. Tokenization is the gate, not Word.
A developer estimates API cost from GitHub line counts on a repository. Lines are not words or tokens. Source files tokenize densely. The heuristic fails and the pilot budget breaks in week one.
Marketing copy promises users they can upload “ten pages” of content. Without locale and formatting constraints, ten pages of dense PDF text versus ten pages of bullet slides produce wildly different token totals on the same model.
Step-by-step in TokenCALC
If you only have a word or character estimate, open the Tokens ↔ words tool linked from this guide. Enter the estimate and read the Approx token output for planning.
When the real text exists, skip conversion and paste into the tokenizer. Pick the target model and read Exact or Approx counts directly.
Move to the cost calculator with the same model. Apply input tokens from counting and set output tokens separately. Compare models on identical pasted text, not identical word counts.
Related concepts
What is a token defines the billing unit. Tokenization explains why heuristics diverge from reality on code and multilingual text. Context windows shows why converted word estimates still need output headroom.
Exact vs Approx labeling applies when you move from heuristic to counted text. Provider bridge guides cover official count APIs when Approx is not enough.
Expert notes
Spreadsheets often hardcode 1.33 as a universal multiplier. Maintain separate factors per locale or content type when you must plan without raw text, and document that they are Approx.
Character based estimates (divide by four) behave similarly to word based ones for English prose but also break on base64, hex dumps, and minified JSON.
When communicating with non technical stakeholders, you may show words in UI while engineering enforces token caps internally. Translate between the two using counted samples from real user content, not theory alone.
Warnings
Do not multiply word counts by published per million token rates as if words were tokens. That math is wrong by construction.
Do not assume Claude or Gemini match OpenAI heuristic conversions. Each family needs its own count path for shipping decisions.
Frequently asked questions
Are tokens the same as words?
No. Tokens can be whole words, subwords, spaces, or punctuation. APIs bill in tokens, not words.
How many words is 1,000 tokens?
For common English text, OpenAI’s rule of thumb is about 750 words per 1,000 tokens. Treat that as an estimate until you tokenize real content.
How many tokens is 1,000 words?
About 1,333 tokens using the ×1.33 English heuristic. Verify with a tokenizer on your real text before you commit budget.
Do Claude and Gemini use the same tokens as OpenAI?
No. Each provider uses its own tokenizer vocabulary. Counts can differ on identical text even when heuristic word conversions match.
Should I budget from word counts?
Only for early ballparks. For shipping budgets, count tokens with Exact OpenAI tooling or provider count APIs, and label everything else Approx.
What is the difference between the guide and the Tokens ↔ words tool?
This guide explains the concepts and formulas. The Tokens ↔ words tool is an interactive Approx converter. The tokenizer page counts real text for a selected model.
When is character division by four accurate enough?
For rough English prose planning when you lack word count tools. It fails on code, URLs, and many non-English scripts. Prefer tokenizing real text when possible.
Next steps
Use the calculator links above for Exact or Approx counts on your own prompts, then browse related guides and model pages to compare pricing assumptions.