NAME
Pipe - Framework to create pipes using iterators
SYNOPSIS
use Pipe;
my @input = Pipe->cat("t/data/file1", "t/data/file2")->run;
my @lines = Pipe->cat("t/data/file1", "t/data/file2")->chomp->run;
my @uniqs = Pipe->cat("t/data/file1", "t/data/file2")->chomp->uniq->run;
my $pipe = Pipe->cat("t/data/file1", "t/data/file2")->uniq->print("t/data/out");
$pipe->run;
WARNING
This is Alpha version. The user API might still change
DESCRIPTION
Building an iterating pipe with prebuilt and home made tubes.
Methods
logger
Method to print something to the log file, especially for debugging
This method is here to be use by Tube authors
$self->logger("log messages");
run The method that actually executes the whole pipe.
my $pipe = Pipe->cat("file"); $pipe->run;
Tubes
Tubes available in this distibution:
cat Read in the lines of one or more file.
chomp
Remove trailing newlines from each line.
find
Pipe->find(".")
Returns every file, directory, etc. under the directory tree passed
to it.
for Pipe->for(@array)
Iterates over the elements of an array. Basically the same as the
for or foreach loop of Perl.
glob
Implements the Perl glob function.
grep
Selectively pass on values.
Can be used either with a regex:
->grep( qr/regex/ )
Or with a sub:
->grep( sub { length($_[0]) > 12 } )
Very similar to the built-in grep command of Perl but instead of
regex you have to pass a compiled regex using qr// and instead of a
block you have to pass an anonymous sub {}
map Similar to the Perl map construct, except that instead of a block
you pass an anonymous function sub {}.
->map( sub { length $_[0] } );
TODO Not implemented yet
Prints out its input. By default it prints to STDOUT but the user
can supply a filename or a filehandle.
Pipe->cat("t/data/file1", "t/data/file2")->print;
Pipe->cat("t/data/file1", "t/data/file2")->print("out.txt");
Pipe->cat("t/data/file1", "t/data/file2")->print(':a', "out.txt");
say It is the same as print but adds a newline at the end of each line.
The name is Perl6 native.
sort
Similar to the built in sort function of Perl. As sort needs to have
all the data in the memory, once you use sort in the Pipe it stops
being an iterator for the rest of the pipe.
By default it sorts based on ascii table but you can provide your
own sorting function. The two values to be compared are passed to
this function.
Pipe->cat("t/data/numbers1")->chomp->sort( sub { $_[0] <=> $_[1] } );
split
Given a regex (or a simple string), will split all the incoming
strings and return an array reference for each row.
Param: string or regex using qr//
Input: string(s)
Output: array reference(s)
tuple
Given one or more array references, on every iteration it will
return an n-tuple (n is the number of arrays), one value from each
source array.
my @a = qw(foo bar baz moo);
my @b = qw(23 37 77 42);
my @one_tuple = Pipe->tuple(\@a);
# @one_tuple is ['foo'], ['bar'], ['baz'], ['moo']
my @two_tuple = Pipe->tuple(\@a, \@b);
# @two_tuple is ['foo', 23], ['bar', 37], ['baz', 77], ['moo', 42]
Input: disregards any input so it can be used as a starting element
of a Pipe
Ouput: array refs of n elements
uniq
Similary to the unix uniq command eliminate duplicate consecutive
values.
23, 23, 19, 23 becomes 23, 19, 23
Warning: as you can see from the example this method does not give
real unique values, it only eliminates consecutive duplicates.
Building your own tube
If you would like to build a tube called "thing" create a module called Pipe::Tube::Thing that inherits from Pipe::Tube, our abstract Tube.
Implement one or more of these methods in your subclass as you please.
init
Will be called once when initializing the pipeline. It will get
($self, @args) where $self is the Pipe::Tube::Thing object and @args
are the values given as parameters to the ->thing(@args) call in the
pipeline.
run Will be called every time the previous tube in the pipe returns one
or more values. It can return a list of values that will be passed
on to the next tube. If based on the current state of Thing there is
nothing to do you should call return; with no parameters.
finish
Will be called once when the Pipe Manager notices that this Thing
should be finished. This happens when Thing is the first active
element in the pipe (all the previous tubes have already finshed)
and its run() method returns an empty list.
The finish() method should return a list of values that will be
passed on to the next tube in the pipe. This is especially useful
for Tubes such as sort that can to their thing only after they have
received all the input.
Debugging your tube
You can call $self->logger("some message") from your tube. It will be
printed to pipe.log if someone sets $Pipe::DEBUG = 1;
Examples
A few examples of UNIX Shell commands combined with pipelines
cat file1 file2 > filenew
open my $out, ">", "filenew" or die $!;
while (<>) {
print $out $_;
}
Perl with Pipe:
perl -MPipe 'Pipe->cat(@ARG)->print("filenew")'
grep REGEX file* | uniq
my $last;
while (<>) {
next if not /REGEX/;
if (not defined $last) {
$last = $_;
print;
next;
}
next if $last eq $_;
$last = $_;
print;
}
Perl with Pipe:
one of these will work, we hope:
Pipe->grep(qr/REGEX/, <file>)->uniq->print
Pipe->cat(<file>)->grep(qr/REGEX/)->uniq->print
Pipe->files("file*")->cat->grep(qr/REGEX/)->uniq->print
find / -name filename -print
Perl with Pipe:
perl -MPipe -e'Pipe->find("/")->grep(qr/filename/)->print'
find . -name CVS | xargs rm -rf
find . -name CVS -type d -exec rm -rf '{}' \;
find2perl . -name CVS -type d -exec rm -rf '{}' \; > rm-cvs.pl
perl rm-cvs.pl
Perl with Pipe:
perl -MPipe -e'Pipe->find(".")->grep(qr/^CVS$/)->rmtree;
BUGS
Probably plenty but nothing I know of. Please report them to the author.
Development
The Subversion repository is here:
http://svn1.hostlocal.com/szabgab/trunk/Pipe/
Thanks
to Gaal Yahas
AUTHOR
Gabor Szabo <gabor@pti.co.il>
COPYRIGHT
Copyright 2006 by Gabor Szabo <gabor@pti.co.il>.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
See http://www.perl.com/perl/misc/Artistic.html
See Also
Shell::Autobox and File::Tools