Skip Menu |
 

This queue is for tickets about the libwww-perl CPAN distribution.

Report information
The Basics
Id: 6487
Status: resolved
Priority: 0/
Queue: libwww-perl

People
Owner: Nobody in particular
Requestors: bob [...] starlabs.net
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 5.65
Fixed in: (no value)



Subject: FTP problems under LWP
Download q.txt
text/plain 10.5k
Hello I am trying to retrieve a directory listing from an FTP server using LWP. I think that the site may well be sending me non-ascii characters? However, Net::FTP can handle it, and from reading LWP::Protocol::ftp.pm, there is an implication that Net::FTP is used by LWP? LWP does act as I expect if I use it to get a directory listing of the parent directory of the one I need to list. I could bypass the problem by using Net::FTP for this one ftp server, however, since the rest of my code uses LWP, I'd sooner get to the bottom of the problem. I have listed the programs here, followed by their output collected via script. I have tried to reduce the programs to their essential statements. I have changed the ftp address, username and password because I do not know if I have permission to distribute them. However, the outputs are those garnered before this change. The code is ordered: netftp.pl netftp.op lwpbad.pl lwpbad.op lwpgood.pl lwpgood.op I suspect that the fact that 'content-type' => 'application/octet-stream' is set in the HTTP::Header of the lwpbad output is wrong. It is set to 'content-type' => 'text/ftp-dir-listing' in the lwpgood output. I think that this is probably due to the fact that LWP::MediaTypes is interpreting the non-ascii character? Is it possible for me to override the assignation given. I do set the content_type of my request object to be "text/ftp-dir-listing". Bob ---------------------------------------------------------------------------------------------- #!/usr/bin/perl -w use strict; use Net::FTP; use Data::Dumper; my $ftp = Net::FTP->new("partner.abcde.ru", Debug => 0) or die "Cannot connect to partner.abcde.ru: $@"; $ftp->login("xyz",'123') or die "Cannot login ", $ftp->message; $ftp->cwd("/bases") or die "Cannot change working directory ", $ftp->message; my $dir = $ftp->dir or die "dir failed ", $ftp->message; warn "DIR is ",Dumper($dir); $ftp->quit; ---------------------------------------------------------------------------------------------- Script started on Wed Jun 2 12:25:13 2004 ]0;bob@snoogen:~/work/accurev/VirusSignatures[bob@snoogen VirusSignatures]$ perl netftp.pl DIR is $VAR1 = [ 'total 3870', '-rw-r--r-- 1 2133 4000 15297 May 20 12:39 advware.avc', '-rw-r--r-- 1 2133 4000 962750 May 20 12:39 avp0404.avc', '-rw-r--r-- 1 2133 4000 286326 May 20 12:39 backdoor.avc', '-rw-r--r-- 1 2133 4000 53719 May 20 12:39 ca.avc', '-rw-r--r-- 1 2133 4000 32075 Jun 2 14:54 daily.avc', '-rw-r--r-- 1 2133 4000 1843 May 20 12:39 eicar.avc', '-rw-r--r-- 1 2133 4000 19046 May 20 12:39 extr-cab.avc', '-rw-r--r-- 1 2133 4000 211150 May 20 12:39 extract.avc', '-rw-r--r-- 1 2133 4000 8401 May 20 12:39 kernel.avc', '-rw-r--r-- 1 2133 4000 5275 May 20 12:39 krndos.avc', '-rw-r--r-- 1 2133 4000 23515 May 20 12:39 krnengn.avc', '-rw-r--r-- 1 2133 4000 64482 May 20 12:39 krnexe.avc', '-rw-r--r-- 1 2133 4000 37999 May 20 12:39 krnjava.avc', '-rw-r--r-- 1 2133 4000 76929 May 20 12:39 krnmacro.avc', '-rw-r--r-- 1 2133 4000 55548 May 20 12:39 krnunp.avc', '-rw-r--r-- 1 2133 4000 141049 May 20 12:39 macro.avc', '-rw-r--r-- 1 2133 4000 14114 May 20 12:39 mail.avc', '-rw-r--r-- 1 2133 4000 115922 May 20 12:39 malware.avc', '-rw-r--r-- 1 2133 4000 375390 May 20 12:39 newexe.avc', '-rw-r--r-- 1 2133 4000 14920 May 20 12:39 newexeg.avc', '-rw-r--r-- 1 2133 4000 3162 May 20 12:39 ocr.avc', '-rw-r--r-- 1 2133 4000 27541 May 20 12:39 pornware.avc', '-rw-r--r-- 1 2133 4000 28492 May 20 12:39 riskware.avc', '-rw-r--r-- 1 2133 4000 99749 May 20 12:39 script.avc', '-rw-r--r-- 1 2133 4000 5366 May 20 12:39 smart.avc', '-rw-r--r-- 1 2133 4000 356774 May 20 12:39 trojan.avc', '-rw-r--r-- 1 2133 4000 569565 May 20 12:39 unpack.avc', '-rw-r--r-- 1 2133 4000 15450 May 20 12:39 up040402.avc', '-rw-r--r-- 1 2133 4000 18563 May 20 12:39 up040409.avc', '-rw-r--r-- 1 2133 4000 23479 May 20 12:39 up040416.avc', '-rw-r--r-- 1 2133 4000 40902 May 20 12:39 up040423.avc', '-rw-r--r-- 1 2133 4000 27378 May 20 12:39 up040430.avc', '-rw-r--r-- 1 2133 4000 19431 May 20 12:39 up040507.avc', '-rw-r--r-- 1 2133 4000 35446 May 20 12:39 up040514.avc', '-rw-r--r-- 1 2133 4000 23532 May 20 12:39 x-files.avc', '-rw-r--r-- 1 2133 4000 2040 Jun 2 14:54 xdaily.avc' ]; ]0;bob@snoogen:~/work/accurev/VirusSignatures[bob@snoogen VirusSignatures]$ Script done on Wed Jun 2 12:25:22 2004 ---------------------------------------------------------------------------------------------- #!/usr/bin/perl -w use strict; use LWP::UserAgent; my $url = "ftp://partner.abcde.ru/bases"; my $user = "xyz"; my $pass = "123"; my $content_type = "text/ftp-dir-listing"; my $ua = LWP::UserAgent->new; my $req = HTTP::Request->new('GET', $url); $req->authorization_basic($user, $pass); $req->content_type($content_type); my $res = $ua->request($req); use Data::Dumper; warn "RES is ",Dumper($res); ---------------------------------------------------------------------------------------------- Script started on Wed Jun 2 12:34:13 2004 ]0;bob@snoogen:~/work/accurev/VirusSignatures[bob@snoogen VirusSignatures]$ perl lwpbad.pl RES is $VAR1 = bless( { '_content' => ', . * .., servers.lst, advware.avc, avp0404.avc, backdoor.avc³+À,ca.avcǐ, daily.avcä, eicar.avcä, extr-cab.avc³+À , extract.avc!, kernel.avcä., krndos.avcä/, krnengn.avc0, krnexe.avcä1, krnjava.avc2, krnmacro.avc³+À3, krnunp.avcä4, macro.avcä5,mail.avc™ä6, malware.avc7, newexe.avcä8, newexeg.avc9,ocr.avc:, pornware.avc³+À<, script.avcä;, riskware.avc³+À=, smart.avcä>, trojan.avcä?, unpack.avcä@, up040402.avc³+ÀA, up040409.avc³+ÀB, up040416.avc³+ÀC, up040423.avc³+ÀD, up040430.avc³+ÀE, up040507.avc³+ÀF, up040514.avc³+ÀG, x-files.avcH,ð xdaily.avcä', '_rc' => 200, '_headers' => bless( { 'client-date' => 'Wed, 02 Jun 2004 11:34:20 GMT', 'content-type' => 'application/octet-stream', 'server' => ' ###############################################################', 'client-request-num' => 1 }, 'HTTP::Headers' ), '_msg' => 'OK', '_request' => bless( { '_content' => '', '_uri' => bless( do{\(my $o = 'ftp://partner.abcde.ru/bases')}, 'URI::ftp' ), '_headers' => bless( { 'user-agent' => 'libwww-perl/5.65', 'content-type' => 'text/ftp-dir-listing', 'authorization' => 'Basic S0FWREI6UG5RR0c4ZThXM2lKWlVTeQ==' }, 'HTTP::Headers' ), '_method' => 'GET' }, 'HTTP::Request' ) }, 'HTTP::Response' ); ]0;bob@snoogen:~/work/accurev/VirusSignatures[bob@snoogen VirusSignatures]$ Script done on Wed Jun 2 12:34:29 2004 ---------------------------------------------------------------------------------------------- #!/usr/bin/perl -w use strict; use LWP::UserAgent; my $url = "ftp://partner.abcde.ru/"; my $user = "xyz"; my $pass = "123"; my $content_type = "text/ftp-dir-listing"; my $ua = LWP::UserAgent->new; my $req = HTTP::Request->new('GET', $url); $req->authorization_basic($user, $pass); $req->content_type($content_type); my $res = $ua->request($req); use Data::Dumper; warn "RES is ",Dumper($res); ---------------------------------------------------------------------------------------------- Script started on Wed Jun 2 12:33:32 2004 ]0;bob@snoogen:~/work/accurev/VirusSignatures[bob@snoogen VirusSignatures]$ perl lwpgood.pl RES is $VAR1 = bless( { '_content' => 'total 2 drwxr-xr-x 2 2133 4000 1024 May 20 12:39 bases ', '_rc' => 200, '_headers' => bless( { 'client-date' => 'Wed, 02 Jun 2004 11:33:40 GMT', 'content-type' => 'text/ftp-dir-listing', 'content-length' => 58, 'server' => ' ###############################################################', 'client-request-num' => 1 }, 'HTTP::Headers' ), '_msg' => 'OK', '_request' => bless( { '_content' => '', '_uri' => bless( do{\(my $o = 'ftp://partner.abcde.ru/')}, 'URI::ftp' ), '_headers' => bless( { 'user-agent' => 'libwww-perl/5.65', 'content-type' => 'text/ftp-dir-listing', 'authorization' => 'Basic S0FWREI6UG5RR0c4ZThXM2lKWlVTeQ==' }, 'HTTP::Headers' ), '_method' => 'GET' }, 'HTTP::Request' ) }, 'HTTP::Response' ); ]0;bob@snoogen:~/work/accurev/VirusSignatures[bob@snoogen VirusSignatures]$ Script done on Wed Jun 2 12:33:45 2004 ----------------------------------------------------------------------------------------------
Download (untitled) / with headers
text/plain 119b
On Wed Jun 02 12:52:42 2004, guest wrote: Show quoted text
>
Looks like this is no longer an issue. We're unable to reproduce. Olaf


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.