Every LLM price, context limit and output cap is measured in tokens, so it pays to count them properly. Here is the short version, with the numbers I measured.

What a token is

A token is a chunk of text: a short word, part of a longer word, a few digits or a symbol. OpenAI's rule of thumb is about 4 characters or 0.75 words per token in English (OpenAI API concepts). With tiktoken's o200k_base, "tokenization" is 2 tokens, "2026" splits into 202 + 6, and the German compound "Straßenbahnhaltestelle" splits into 7. The vocabulary comes from byte-pair encoding, applied to subwords as in Sennrich, Haddow and Birch (2015).

Count exactly (OpenAI) 

import tiktoken enc = tiktoken.get_encoding("o200k_base") print(len(enc.encode("Hello world"))) # 2

tiktoken is OpenAI's open-source tokenizer. If you would rather not run code, this browser token counter reproduces it; I compared it with tiktoken on more than 1,600 strings in both encodings and the ids matched every time. Nothing you paste is uploaded.

Claude and Gemini do not publish downloadable tokenizers, so third-party counts for them are estimates; use each provider's token-counting endpoint for billing. Anthropic says its newer tokenizer (Claude 4.7 and later) produces roughly 30% more tokens for the same text than earlier Claude models (Anthropic models overview).

Words and tokens

About 750 English words per 1,000 tokens. Code, JSON, long numbers and URLs split into many small tokens, so fewer words fit. A tokens to words converter gives the quick table from 1,000 to 1M tokens.

Languages are not equal

Two sentences in 15 languages with OpenAI's exact tokenizers: English 31 tokens, German 37, Russian 43, Hindi 47 on o200k_base but 153 on the older cl100k_base; Chinese used 25. It is a small sample with my own translations, so treat it as direction, not a benchmark. Count a sample of your own text before pricing a non-English workload.

Context windows

As of 13 November 2026, the current Claude 5 models list 1M tokens and 128K max output, GPT-6 lists 1.05M, and Gemini 4 and DeepSeek V4 list 1M, per each provider's model page. A bigger window is not free: you pay for every input token on every request.