Defines an interface for tokenizer state that processes delimiters.
Implements: ITokenizerState
Description
The ISymbolState interface is used for tokenizer states that process delimiters.
Instance methods
add
Add a multi-character symbol.
add(value: string, tokenType: TokenType): void
- value: string - symbol to add, such as "=:=".
- tokenType: TokenType - token type.