Home  >   FAQs   >  What are tokens in Chat GPT?
FAQs in this section

What are tokens in Chat GPT?

Table of Contents
Tokens are chunks of text into which a given input or output is divided. Chat GPT uses Natural Language Processing and Machine Learning languages, where a token can be as short as one character or as long as one word.

For example, ChatGPT is Great! can be tokenized into the following tokens:
  1. Chat
  2. G
  3. PT
  4. is
  5. Great
  6. !
Each token corresponds to a meaningful unit in the text. Spaces are usually tokens themselves, and punctuation marks can be separate tokens or part of adjacent words.
Was this FAQ helpful?
10  out of  15  found this helpful.
Comments 0
Be the first to comment
© Copyright 2023 Vtiger. All rights reserved.