Plucene::Analysis::Tokenizer - base class for tokenizers

Plucene documentation Contained in the Plucene distribution.

Index

Code Index:

NAME

Top

Plucene::Analysis::Tokenizer - base class for tokenizers

SYNOPSIS

Top

	my $tokenizer = Plucene::Analysis::Tokenizer::Subclass
		->new({ reader => $reader });

DESCRIPTION

Top

This is an abstract base class for tokenizers.

A Tokenizer is a TokenStream whose input is a Reader.

METHODS

Top

new

	my $tokenizer = Plucene::Analysis::Tokenizer::Subclass
		->new({ reader => $reader });

This will create a new tokenizer.

next

This must be defined in a subclass

close

Close the input reader.

Plucene documentation Contained in the Plucene distribution. 
package Plucene::Analysis::Tokenizer;


use strict;
use warnings;

use base qw/Class::Accessor::Fast/;
__PACKAGE__->mk_accessors(qw[ reader ]);


sub next { die "next must define this in a subclass" }


sub close {
	my $self = shift;
	$self->{reader}->close if $self->{reader};
	return 1;
}

1;