Skip Menu |
 

This queue is for tickets about the POE-Component-Client-HTTP CPAN distribution.

Report information
The Basics
Id: 12683
Status: resolved
Priority: 0/
Queue: POE-Component-Client-HTTP

People
Owner: Nobody in particular
Requestors: James_Armstrong [...] cox.net
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



From: "James" <James_Armstrong [...] cox.net>
To: <bug-POE-Component-Client-HTTP [...] rt.cpan.org>
CC: "'Jeff Bisbee'" <jbisbee [...] cpan.org>
Subject: Many ( > 5) concurrent HTTP exchanges cause all to fail
Date: Thu, 5 May 2005 00:31:16 -0700
Download (untitled) / with headers
text/plain 4.9k
--> You are now talking on #PoE --- Topic for #PoE is 'POE Operates Efficiently' | 'Power Over Everything' | billn: I think I just outsmarted myself. | PoCo::Lightspeed is on CPAN, play with it! --- Topic for #PoE set by Apocalypse!~apoc@c-24-6-201-170.hsd1.ca.comcast.net at Sun Apr 24 11:11:11 2005 <jda2000> Hello, If I want to have concurrent HTTP requests, must I call POE::Component::Client::HTTP->spawn() for each desired concurrent request? <LotR> jda2000: no <jda2000> LotR, Then it would be a bug if an error in one request causes all concurrent requests to fail? <LotR> jda2000: sounds like it, yes <dngor> I would suggest running a test case against LotR's version of Client::HTTP before reporting it as a bug, though. It may already be fixed. And the version in my repository's currently bloxed. * LotR would say report a bug if you have a nice small script that exhibits the problem <jda2000> LotR, I was using Version 0.65 from CPAN. <LotR> jda2000: *nod* <LotR> jda2000: so try to reproduce the problem with as few lines of code as possible, and send that to bug-POE-Component-Client-HTTP@rt.cpan.org <jda2000> LotR, Ok Jeff, *** I am resending this message to the address indicated by LotR. *** It is plausible that the error is in Client::HTTP. Thanks for the quick turn-around on the XML::RSS::Feed issue, I'll pick up the fix as soon as I figure out how to get CPAN to do an update :-). I thought I would play around a bit more to get familiar with the code. The included code fails in an interesting way. It seems to connect to all web servers, have a problem somewhere and then report that all of the downloads failed. The 26 "[!!] Failed to fetch" messages all come out together. This seems very un-POE like. I would expect the downloads to be more concurrent. Also, I would expect that a download that works by itself would still work even if another download should fail. Is there a limit I need to adjust somewhere to accommodate more sockets? I think I saw some sort of factory object being used in one of your examples. Should I be using that? I'd be glad to do another bug report but I'm not sure at this point which component to write it against. Thanks, JDA ------------------------------------------------%<-------------------------- -------------------- #!/usr/bin/perl -w use strict; use POE; use POE::Component::RSSAggregator; my @feeds; sub LoadFeeds { my $Input; my %names; # open ($Input, "</home/jda2000/bashpodder/bp.conf") or die "Can't open bp.conf"; while (<DATA>) { if (m/^#/) { next; } # permit commenting out URLs my $Item = {}; chomp; my $url = $_; my $name = 'whatever'; print "URL = " . $url . "\n"; $Item->{'url'} = $url; if ($url=~m/\/$/) { $url=~m/([^\/]+)\/$/; $name = $1; } else { $url=~m/([^\/]+)$/; $name = $1; } $name =~ s/\..*//; while (exists($names{$name})) { $name .= 'I'; } $names{$name} = 1; $Item->{'name'} = $name; print "Name = " . $name . "\n"; $Item->{'delay'} = 60; push (@feeds, $Item); } # close($Input); } LoadFeeds; POE::Session->create( inline_states => { _start => \&init_session, handle_feed => \&handle_feed, } ); $poe_kernel->run(); sub init_session { my ($kernel, $heap, $session) = @_[KERNEL, HEAP, SESSION]; $heap->{rssagg} = POE::Component::RSSAggregator->new( alias => 'rssagg', debug => 1, callback => $session->postback("handle_feed"), tmpdir => '/tmp', # optional caching ); $kernel->post('rssagg','add_feed',$_) for @feeds; } sub handle_feed { my ($kernel,$feed) = ($_[KERNEL], $_[ARG1]->[0]); for my $headline ($feed->late_breaking_news) { # do stuff with the XML::RSS::Headline object print $headline->headline . "\n"; } } __END__ http://rasterweb.net/raster/feeds/rwaudio.rss http://radio.weblogs.com/0001014/categories/dailySourceCode/rss.xml http://radio.weblogs.com/0001014/categories/qotd/rss.xml http://secrets.scripting.com/xml/rss.xml http://ruk.ca/rss/formosa.xml http://www.scripting.com/rss.xml http://feeds.feedburner.com/career http://www.evilgeniuschronicles.org/static/index.rss http://feeds.feedburner.com/esc http://feeds.feedburner.com/ITConversations-EverythingMP3 http://pentdego.com/feeds/freeflow.xml http://www.blogosphereradio.com/rss.aspx http://feeds.feedburner.com/Techaura http://thelinuxlink.net/tllts/tllts.rss http://radio.linuxquestions.org/syndicate/lq.php http://www.lugradio.org/episodes.rss http://linux.quicksurf.com/wp-rss2.php http://www.binrev.com/radio/podcast/ http://www.infonomicon.org/info.xml http://feeds.feedburner.com/sliceofscifi http://feeds.feedburner.com/YeastRadio.xml http://leoville.tv/tlr/tlr.php http://thelinuxbox.org/show/tlbs.rss http://www.dawnanddrew.com/rss2.xml http://www.sciencecast.net/podcast.php http://feeds.feedburner.com/ScienceCast
Download (untitled) / with headers
text/plain 325b
Hi, I didn't exactly suggested you send your complete app in the bug report :) I asked for a minimal piece of code that demonstrates the problem. I tried running the code, and it seems to work just fine here. So, until you give me a Client::HTTP specific test case that I can reproduce, there isn't much I can do about this.
Download (untitled) / with headers
text/plain 685b
[James_Armstrong@cox.net - Thu May 5 03:38:34 2005]: Show quoted text
> Thanks for the quick turn-around on the XML::RSS::Feed issue, I'll pick > up the fix as soon as I figure out how to get CPAN to do an update :-). > I thought I would play around a bit more to get familiar with the code. > The included code fails in an interesting way. It seems to connect to > all web servers, have a problem somewhere and then report that all of > the downloads
James, I cleaned up your script a bit, tested and ran it without any problems. I have a feeling something else is going on. I'm 'biz' in #poe on irc.perl.org. Let's see if we can work this out. Jeff P.S. I've attached the cleaned up script
Download test-rss.pl
text/x-perl 2.3k
#!/usr/bin/perl use strict; use warnings; use POE; use POE::Component::RSSAggregator; POE::Session->create( inline_states => { _start => \&init_session, handle_feed => \&handle_feed, }, ); $poe_kernel->run(); sub init_session { my @feeds = load_feeds(); my ($kernel, $heap, $session) = @_[KERNEL, HEAP, SESSION]; $heap->{rssagg} = POE::Component::RSSAggregator->new( alias => 'rssagg', debug => 1, callback => $session->postback("handle_feed"), tmpdir => '/tmp', # optional caching ); $kernel->post('rssagg','add_feed',$_) for @feeds; } sub handle_feed { my ($kernel,$feed) = ($_[KERNEL], $_[ARG1]->[0]); for my $headline ($feed->late_breaking_news) { # do stuff with the XML::RSS::Headline object print $headline->headline . "\n"; } } sub load_feeds { my @feeds = (); my %names = (); while (my $url = <DATA>) { next if $url =~ /^#/; chomp $url; my %feed = (url => $url, delay => 60); ##### # DETERMINE NAME START my $name = 'whatever'; if ($url=~m/\/$/) { $url=~m/([^\/]+)\/$/; $name = $1; } else { $url=~m/([^\/]+)$/; $name = $1; } $name =~ s/\..*//; while (exists($names{$name})) { $name .= 'I'; } # DETERMINE NAME END ##### $names{$name} = 1; $feed{name} = $name; print "Name: $name\n"; push (@feeds,\%feed); } return @feeds; } __END__ http://rasterweb.net/raster/feeds/rwaudio.rss http://radio.weblogs.com/0001014/categories/dailySourceCode/rss.xml http://radio.weblogs.com/0001014/categories/qotd/rss.xml http://secrets.scripting.com/xml/rss.xml http://ruk.ca/rss/formosa.xml http://www.scripting.com/rss.xml http://feeds.feedburner.com/career http://www.evilgeniuschronicles.org/static/index.rss http://feeds.feedburner.com/esc http://feeds.feedburner.com/ITConversations-EverythingMP3 http://pentdego.com/feeds/freeflow.xml http://www.blogosphereradio.com/rss.aspx http://feeds.feedburner.com/Techaura http://thelinuxlink.net/tllts/tllts.rss http://radio.linuxquestions.org/syndicate/lq.php http://www.lugradio.org/episodes.rss http://linux.quicksurf.com/wp-rss2.php http://www.binrev.com/radio/podcast/ http://www.infonomicon.org/info.xml http://feeds.feedburner.com/sliceofscifi http://feeds.feedburner.com/YeastRadio.xml http://leoville.tv/tlr/tlr.php http://thelinuxbox.org/show/tlbs.rss http://www.dawnanddrew.com/rss2.xml http://www.sciencecast.net/podcast.php http://feeds.feedburner.com/ScienceCast


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.