Defines an interface for tokenizer states that processes words, identificators or keywords
Inherits: ITokenizerState
Description
The IWordState interface is used by tokenizer states that process words, identificators or keywords.
Instance methods
ClearWordChars
Clears definitions of word chars.
void ClearWordChars()
SetWordChars
Establish characters in the given range as valid characters for part of a word after the first character. Note that the tokenizer must determine which characters are valid as the beginning character of a word.
void SetWordChars(char fromSymbol, char toSymbol, char enable)
- fromSymbol: char - first character index of the interval.
- toSymbol: char - last character index of the interval.
- enable: char - true if this state should use characters in the given range.