Net::Download::Queue - Download files with one or more workers.

Net-Download-Queue documentation Contained in the Net-Download-Queue distribution.

Index

Code Index:

NAME

Top

Net::Download::Queue - Download files with one or more workers.

SYNOPSIS

Top

  # *** In the app which needs to download files
  use Net::Download::Queue;
  my $oQueue = Net::Download::Queue->new() or die;
  my $oDownload = $oQueue->oDownloadAdd(
      "http://www.darserman.com/Perl/TexQL/texql.pl",
      "./downloads",
      "texql.pl.txt",
      $urlReferer,      #Optional
  ) or die;
  #The url is now queued




  # *** Or using the command line
  download_queue --url=http://www.darserman.com/Perl/TexQL/texql.pl \
      --dir=./downloads --file=texql.pl.txt --
  #The url is now queued







  # *** On another command line (you can have many of these)
  download_queue --process
  #Urls are downloaded as they appear in the queue

DESCRIPTION

Top

Download files asynchronously in a queued fashion.

Your application (or a CLI script) can add files to the queue, and other workes will process the queue and actually download the files.

METHODS

Top

new()

Create new queue.

oDownloadAdd($url, $dirDownload, $file, [$urlReferer = ""])

Add $url to the queue, to be downloaded in $dirDownload/$file.

Return the new Download object on success, else die.

oDownloadDequeue()

If there is any pending download in the queue, return a Download object, or return undef if there are no pending downloads.

The Download object is now in "downloading" state.

Die on errors.

requeueDownloading()

Set any downloads in status "downloading" back into status "queued".

Die on errors.

percentComplete()

Return the percent 0..100 how complete the download of all current downloads are.

If none is current, that's 100% complete.

Die on errors.

CLASS METHODS

Top

rebuildDatabase()

Empty and rebuild the SQLite database.

Return 1 on success, else die on errors.

ensureDatabase()

Rebuild the SQLite database if it's not present.

Return 1 on success, else die on errors.

SEE ALSO

Top

I haven't found any modules that perform the downloads asynchronously.

KNOWN BUGS

Top

Ther is a race condition when dequeueing a Download from the queue. Should run in a transaction.

AUTHOR

Top

Johan Lindstrom, <johanl@cpan.org>

BUGS

Top

Please report any bugs or feature requests to bug-net-download-queue@rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Net-Download-Queue. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

COPYRIGHT & LICENSE

Top

Copyright 2005 Johan Lindstrom, All Rights Reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

Net-Download-Queue documentation Contained in the Net-Download-Queue distribution. 




package Net::Download::Queue;

use warnings;
use strict;

use File::Basename;

use LWP::UserAgent;
use HTTP::Response;
use HTTP::Request::Common qw(HEAD);

use Net::Download::Queue::DBI;
use Net::Download::Queue::Download;
use Net::Download::Queue::DownloadStatus;






our $VERSION = '0.04';




sub new {
    my $self = bless {}, shift;

    return($self);
}






sub oDownloadAdd {
    my $self = shift;
    my ($url, $dirDownload, $file, $urlReferer) = @_;
    $urlReferer ||= "";

    return( Net::Download::Queue::Download->create({
        url => $url,
        dirDownload => $dirDownload,
        fileDownload => $file,
        urlReferer => $urlReferer,
    }) or die );

}






sub oDownloadDequeue {
    my $self = shift;

    #Race condition, ignore that for now.
    my $oDownload = Net::Download::Queue::Download->search_first({
        download_status_id => Net::Download::Queue::Download->oDownloadStatus("queued"),
    }) or return(undef);

    $oDownload->setDownloading();
    #End race condition

    return($oDownload);
}






sub requeueDownloading {
    my $self = shift;

    $_->setQueued() for(Net::Download::Queue::Download->retrieve_downloading);

    return(1);
}






sub percentComplete {
    my $self = shift;

    my $currentCount = Net::Download::Queue::Download->sql_bytesSumCurrent->select_val or return(100);
    my $downloadingCount = Net::Download::Queue::Download->sql_bytesSumDownloading->select_val;
#    my $currentCount = Net::Download::Queue::Download->retrieve_current or return(100);
#    my $downloadingCount = Net::Download::Queue::Download->retrieve_downloading;
    
    my $percent = $downloadingCount / ($currentCount + $downloadingCount) * 100;

    return($percent);
}






sub rebuildDatabase {
    Net::Download::Queue::DBI->rebuildDatabase();
}






sub ensureDatabase {
    Net::Download::Queue::DBI->ensureDatabase();
}





1;





__END__