Net::FTP::Recursive - Recursive FTP Client class

    use Net::FTP::Recursive;

    $ftp = Net::FTP::Recursive->new("some.host.name", Debug => 0);
    $ftp->login("anonymous",'me@here.there');
    $ftp->cwd('/pub');
    $ftp->rget( ParseSub => \&yoursub );
    $ftp->quit;

Net::FTP::Recursive is a class built on top of the Net::FTP package that implements recursive get and put methods for the retrieval and sending of entire directory structures.

This module's default behavior is such that the remote ftp server should understand the "dir" command and return UNIX-style directory listings. If you'd like to provide your own function for parsing the data retrieved from this command (in case the ftp server does not understand the "dir" command), all you need do is provide a function to one of the Recursive method calls. This function will take the output from the "dir" command (as a list of lines) and should return a list of Net::FTP::Recursive::File objects. This module is described below.

All of the methods also take an optional KeepFirstLine argument which is passed on to the default parsing routine. This argument supresses the discarding of the first line of output from the dir command. wuftpd servers provide a total line, the default behavior is to throw that total line away. If yours does not provide the total line, KeepFirstLine is for you. This argument is used like the others, you provide the argument as the key in a key value pair where the value is true (ie, KeepFirstLine => 1).

When the Debug flag is used with the Net::FTP object, the Recursive package will print some messages to STDERR.

All of the methods should return false ('') if they are successful, and a true value if unsuccessful. The true value will be a string of the concatenations of all of the error messages (with newlines). Note that this might be the opposite of the more intuitive return code.

new (HOST [,OPTIONS])

A call to the new method to create a new Net::FTP::Recursive object just calls the Net::FTP new method. Please refer to the Net::FTP documentation for more information.

rget ( [ParseSub =>\&yoursub] [,FlattenTree => 1] [,RemoveRemoteFiles => 1] )

The recursive get method call. This will recursively retrieve the ftp object's current working directory and its contents into the local current working directory.

This will also take an optional argument that will control what happens when a symbolic link is encountered on the ftp server. The default is to ignore the symlink, but you can control the behavior by passing one of these arguments to the rget call (ie, $ftp->rget(SymlinkIgnore => 1)):

SymlinkIgnore - disregards symlinks (default)

SymlinkCopy - copies the link target from the server to the client (if accessible). Works on files other than a directory. For directories, see the SymlinkFollow option.

SymlinkFollow - will recurse into a symlink if it points to a directory. This option may be given along with one of the others above.

SymlinkLink - creates the link on the client. This is superceded by each of the previous options.

The SymlinkFollow option, as of v1.6, does more sophisticated handling of symlinks. It will detect and avoid cycles, on all client platforms. Also, if on a UNIX (tm) platform, if it detects a cycle, it will create a symlink to the location where it downloaded the directory (or will download it subsequently, if it is in the subtree under where the recursing started). On Windows, it will call symlink just as on UNIX (tm), but that's probably not gonna do much for you. :)

The FlattenTree optional argument will retrieve all of the files from the remote directory structure and place them in the current local directory. This option will resolve filename conflicts by retrieving files with the same name and renaming them in a "$filename.$i" fashion, where $i is the number of times it has retrieved a file with that name.

The optional RemoveRemoteFiles argument to the function will allow the client to delete files from the server after it retrieves them. The default behavior is to leave all files and directories intact. The default behavior for this is to check the return code from the FTP GET call. If that is successful, it will delete the file. CheckSizes is an additional argument that will check the filesize of the local file against the file size of the remote file, and only if they are the same will it delete the file. You must l provide the RemoveRemoteFiles option in order for option to affect the behavior of the code. This check will only be performed for regular files, not directories or symlinks.

For the v1.6 release, I have also added some additional functionality that will allow the client to be more specific in choosing those files that are retrieved. All of these options take a regex object (made using the qr operator) as their value. You may choose to use one or more of these options, they are applied in the order that they are listed. They are:

MatchAll - Will process file that matches this regex, regardless of whether it is a plainish file, directory, or a symlink. This behavior can be overridden with the previous options.

OmitAll - Do not process file that matches this regex. Also may be overridden with the previous options.

MatchFiles - Only transfer plainish (not a directory or a symlink) files that match this pattern.

OmitFiles - Omit those plainish files that match this pattern.

MatchDirs - Only recurse into directories that match this pattern.

OmitDirs - Do not recurse into directories that match this pattern.

MatchLinks - Only deal with those links that match this pattern (based on your symlink option, above).

OmitLinks - Do not deal with links that match this pattern.

Currently, some of the added functionality given to the rget method is not implemented for the rput method.

rput ( [FlattenTree => 1] [,RemoveLocalFiles => 1] )

The recursive put method call. This will recursively send the local current working directory and its contents to the ftp object's current working directory.

This will take an optional argument that will control what happens when a symbolic link is encountered on the ftp server. The default is to ignore the symlink, but you can control the behavior by passing one of these arguments to the rput call (ie, $ftp->rput(SymlinkIgnore => 1)):

SymlinkIgnore - disregards symlinks

SymlinkCopy - will copy the link target from the client to the server.

SymLinkFollow - will recurse into a symlink if it points to a directory. This does not do cycle checking, use with EXTREME caution. This option may be given along with one of the others above.

The FlattenTree optional argument will send all of the files from the local directory structure and place them in the current remote directory. This option will resolve filename conflicts by sending files with the same name and renaming them in a "$filename.$i" fashion, where $i is the number of times it has retrieved a file with that name.

The optional RemoveLocalFiles argument to the function will allow the client to delete files from the client after it sends them. The default behavior is to leave all files and directories intact. This option is very unintelligent, it does a delete no matter what.

As of v1.11, there is a CheckSizes option that can be used in conjunction with the RemoveLocalFiles that will check the filesize of the file locally against the remote filesize and only delete if the two are the same. This option only affects regular files, not symlinks or directories. This option does not affect the normal behavior of RemoveRemoteFiles option (ie, it will try to delete symlinks and directories no matter what).

rdir ( Filehandle => $fh [, FilenameOnly => 1 [, PrintType => 1] ] [, ParseSub => \&yoursub ] )

The recursive dir method call. This will recursively retrieve directory contents from the server in a breadth-first fashion.

The method needs to be passed a filehandle to print to. The method call just does a print $fh, so as long as this call can succeed with whatever you pass to this function, it'll work.

The second, optional argument, is to retrieve only the filenames (including path information). The default is to display all of the information returned from the $ftp-dir call.

This method WILL follow symlinks. It has the same basic cycle-checking code that is in rget, so it should not infinitely loop.

The PrintType argument will print either an 's', an 'f', or a 'd' after the filename when printed, to tell you what kind of file it thinks it is. This argument must be given along with the FilenameOnly argument. (Submitted by Arturas Slajus).

rls ( Filehandle => $fh [, PrintType => 1 ] [, ParseSub => \&yoursub] )

The recursive ls method call. This will recursively retrieve directory contents from the server in a breadth-first fashion. This is equivalent to calling $ftp-rdir( Filehandle => $fh, FilenameOnly => 1 )>.

There is also a new argument to this, the PrintType referenced in the rdir part of the documentation. For rls, the FilenameOnly argument is automatically passed.

rdelete ( [ ParseSub => \&yoursub ] )

The recursive delete method call. This will recursively delete everything in the directory structure. This disregards the SymlinkFollow option and does not recurse into symlinks that refer to directories.

This is a helper class that encapsulates the data representing one file in a directory listing.

new ( )

This method creates the File object. It should be passed several parameters. It should always be passed:

OriginalLine => $line

Fields => \@fields

And it should also be passed at least one (but only one a true value) of:

IsPlainFile => 1

IsDirectory => 1

IsSymlink => 1

OriginalLine should provide the original line from the output of a directory listing.

Fields should provide an 8 element list that supplies information about the file. The fields, in order, should be:

Permissions

Link Count

User Owner

Group Owner

Size

Last Modification Date/Time

Filename

Link Target

The IsPlainFile, IsDirectory, and IsSymlink fields need to be supplied so that for the output on your particular system, your code (in the ParseSub) can determine which type of file it is so that the Recursive calls can take the appropriate action for that file. Only one of these three fields should be set to a "true" value.

Allow for formats to be given for output on rdir/rls.

When reporting bugs, please provide as much information as possible. A script that exhibits the bug would also be helpful, as well as output with the "Debug => 1" flag turned on in the FTP object.

Jeremiah Lee <texasjdl_AT_yahoo.com>

Net::FTP

Net::Cmd

ftp(1), ftpd(8), RFC 959

Thanks to everyone who has submitted bugs over the years.

Copyright (c) 2009 Jeremiah Lee.

This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself.