Token Counter
A language model does not count in words but in tokens, and the difference is larger than people expect: text in languages other than English costs more tokens per word, and code, JSON and emoji behave differently again. This tool counts using the actual tokenizer OpenAI uses — cl100k_base for GPT-3.5 and GPT-4, o200k_base for GPT-4o and newer — rather than the common rule of thumb of four characters per token. You see not only the number but also how your text breaks into individual token pieces, which is often surprisingly instructive: that is exactly where you see why a long compound term costs more than you thought. Useful for checking whether a prompt fits the context window, weighing two phrasings against each other, or estimating cost up front — models are billed per thousand tokens, after all. One thing to note: unlike most tools in this toolbox, this one does not process locally. Your text goes to our own server to be counted, because the tokenizer runs there. It does not go to an AI provider and is not stored. Use is free and unlimited, including commercially — no account, no licence, no cap on how often.
What people use this for
- Checking whether a system prompt fits the context window
- Comparing two phrasings on token cost
- Estimating up front what a batch of a thousand documents will cost
What others think of this tool
No reviews for this tool yet. Yours would be the first.
Do you work with Westcube?
Leave a review on Google tooFrequently asked questions
Is this the exact token count for my AI model?
For OpenAI models using the selected encoding: yes, exactly. This is the same tokenizer library OpenAI publishes itself, not an approximation. Choose cl100k_base for GPT-3.5 and GPT-4, and o200k_base for GPT-4o and newer models. For other providers it is an indication: Anthropic and Google each use their own tokenizer, which arrives at a different number for the same text. In practice, for ordinary prose they land within roughly ten percent of each other, which is ample for a cost estimate, but do not settle accounts to the token with it.
Roughly how many tokens is a word?
The rule of thumb you read everywhere — about four characters per token, or three quarters of a word per token — holds reasonably for English prose and much less well for anything else. Languages that build long compound words are more expensive, because a word like the Dutch "verzekeringsmaatschappij" is chopped into several pieces where English uses a single token. Code, JSON and URLs also fare badly because of all the punctuation. Emoji and non-Latin scripts can cost several tokens per character. That is precisely why counting beats estimating, especially with a tight context window or a large volume.
Is my text sent to an AI provider?
No. Your text does go to our own server — the tokenizer runs there, because it needs the full BPE tables, which are too large to load in a browser. It is not passed on to OpenAI, Anthropic or any other provider, is not stored and is not logged; once counted, it is gone. This is a difference from most other tools in this toolbox, which run entirely in your browser. If you are working with text that genuinely must not leave the building, keep that distinction in mind.
Is there a maximum text length?
Yes, fifty thousand characters at a time. That is generous: roughly a document of fifteen to twenty pages, and more than enough for virtually any prompt. The limit exists to stop a single request occupying the server for minutes. For a larger document, split it and add the counts — tokenizing is close to additive, with at most a few tokens of difference at the split points. There is no daily cap and no paid tier with a higher limit.
Do input and output tokens both count towards the cost?
Yes, and that is often forgotten. Providers bill input and output separately, with output usually considerably more expensive per token — a factor of four or five with some models. This tool counts what you put in. For a full cost estimate, add the expected response, and for a multi-turn conversation also the entire history that is resent each time. That last one is the most underestimated cost in a chat application.