Implements a symbol state to tokenize delimiters in CSV streams.
Extends: GenericSymbolState
Description
The CsvSymbolState implements a symbol state to tokenize delimiters in CSV streams.
Constructors
Constructs this object with specified parameters.
CsvSymbolState()
Instance methods
nextToken
Gets the next token from the stream started from the character linked to this state.
@override
Token nextToken(IScanner scanner, ITokenizer? tokenizer)
- scanner: IScanner - text string to be tokenized.
- tokenizer: ITokenizer? - tokenizer class that controls the process.
- returns: Token - next token from the top of the stream.