Tokens to words converter

Models read text as tokens: whole words, pieces of words, punctuation and spaces. The popular rule of thumb, from OpenAI, is that a token is about four characters or three-quarters of an English word. Most converters apply that one ratio to everything. It’s in the right range for plain English: with OpenAI’s o200k_base tokenizer we measured 0.87 words per token on formal prose and 0.70 on technical writing. But it can be badly wrong for other languages, for code and data, and for models whose tokenizers split text differently. So this converter uses a measured ratio for each combination.

Why the ratio changes

The tokenizer. Each model family learns its own vocabulary. Larger, newer vocabularies hold more whole words, especially outside English: OpenAI’s older cl100k_base (GPT-4) needed 3.3× as many tokens as o200k_base for the Hindi declaration, while for English the two were almost level (2,016 and 2,017 tokens). Anthropic says its tokenizer for Claude Opus 4.7 and later uses up to about 35 percent more tokens for the same text than earlier Claude models, varying by content.

The language. The same declaration took from 1.1× the English token count (Chinese (Simplified)) to 1.8× (Japanese) with o200k_base. Words per token also depends on how long a language’s words are: German and Turkish build long words from several parts, so they show few words per token even though the same text costs only 1.3× and 1.5× the English token count.

The kind of text. Code, Markdown and JSON are full of symbols, indentation and short fragments. With o200k_base, English prose came to 0.87 words per token, this site’s technical guides to 0.70, Markdown to 0.58, TypeScript to 0.47 and pretty-printed JSON to 0.25. Minified JSON has almost no spaces, so its “words” are meaningless: convert it by characters instead.

How we measured

For languages we used the Universal Declaration of Human Rights in 19 languages from the UDHR in Unicode project: the title, preamble and all 30 articles (1,747 words in English), without markup or notes. Because every version says the same thing, the token counts compare like with like. It is formal, legal language, so everyday text can come out a little different. For kinds of text we used English from this site: the reader-facing text of our 17 guides, 57 TypeScript source files, 10 Markdown documents, and our model price file as pretty-printed and minified JSON.

Words are counted the way word processors count them, by splitting on spaces. Chinese and Japanese are written without spaces between words, and splitting them into words needs a dictionary that varies between tools, so for those we convert by characters. Characters are Unicode characters without spaces, after normalising the text to the standard composed form (NFC) that almost all web text uses. Tokens were counted with OpenAI’s o200k_base and cl100k_base (through gpt-tokenizer, which matches OpenAI’s tiktoken) and with the official open-weight tokenizers for DeepSeek V4, Qwen 3.8 and Mistral Small 4, the same files our token counter uses. DeepSeek V4 Pro and V4.1 Flash gave identical counts on every sample, so they share a row. The measurement ran on 2026-10-11; the script is in the site’s source code, so anyone can repeat it.

Claude’s tokenizer isn’t public, so we can’t measure it. Instead the converter uses Anthropic’s published figure, that 1 million tokens is roughly 555,000 words on current models and 750,000 words on models before Claude Opus 4.7. It is labelled as Anthropic’s figure and only offered for English prose, because Anthropic gives no figures for other languages or for code. Gemini was counted with Google’s official countTokens API (gemini-3.8-flash).

Pages

A page isn’t a fixed unit. The converter uses a common convention for 12-point text with 1-inch margins: about 500 words single-spaced or 250 double-spaced. Printed books often hold fewer words per page, and slides far fewer, so treat page counts as rough. To see how many tokens fit a model, use the context window checker; to price them, use the LLM cost calculator.