Plucene::Analysis::TokenFilter - base class for token filters

This is an abstract base class for token filters.

A TokenFilter is a TokenStream whose input is another token stream.

This must be defined in a subclass

Does nothing.