This state will either delegate to a comment-handling state, or return a token with just a slash in it.
Description
The CCommentState class allows you to create a state that will either delegate to a comment-handling state, or return a token with just a slash in it.
Instance methods
next_token
Either delegates to a comment-handling state, or returns a token with just a slash in it.
next_token(scanner: IScanner, tokenizer: ITokenizer): Token
- scanner: IScanner - text string to be tokenized.
- tokenizer: ITokenizer - tokenizer class that controls the process.
- returns: Token - next token from the top of the stream.