What is a token in AI
The piece of text an AI model works with, and the unit used to charge for its use.
A model does not read words but tokens: fragments that can be a short word, part of a long word or a punctuation mark.
Prices are quoted per million input tokens, what you send, and output tokens, what it writes. The limit on how much it can read or write at once is also measured in tokens.
