GenericTokenizer

Implements a default tokenizer class.

Implements: AbstractTokenizer

Description

The GenericTokenizer allows you to implement a default tokenizer class.

Constructors

Creates an instance of GenericTokenizer

GenericTokenizer()