CsvSymbolState

Implements a symbol state to tokenize delimiters in CSV streams.

Extends: GenericSymbolState

Description

The CsvSymbolState implements a symbol state to tokenize delimiters in CSV streams.

Constructors

Constructs this object with specified parameters.

public constructor()

Instance methods

nextToken

Gets the next token from the stream started from the character linked to this state.

public nextToken(scanner: IScanner, tokenizer: ITokenizer): Token

  • scanner: IScanner - text string to be tokenized.
  • tokenizer: ITokenizer - tokenizer class that controls the process.
  • returns: Token - next token from the top of the stream.