Types (categories) of tokens such as “number”, “symbol” and “word”.
Description
The TokenType class defines different types (categories) of tokens, such as “number”, “symbol” and “word”.
Enumeration members
The codes are as follows:
- Unknown = iota
- Eof
- Eol
- Float
- Integer
- HexDecimal
- Number
- Symbol
- Quoted
- Word
- Keyword
- Whitespace
- Comment
- Special