Skip Menu |
 

This queue is for tickets about the POE-Component-Client-HTTP CPAN distribution.

Report information
The Basics
Id: 26874
Status: resolved
Priority: 0/
Queue: POE-Component-Client-HTTP

People
Owner: Nobody in particular
Requestors: KAZUHO [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.82
Fixed in: (no value)



Subject: "incomplete response b" error when using MaxSize option
Download (untitled) / with headers
text/plain 314b
It seems that POE::Component::Client::HTTP::Request::add_eof erronously generates a 400 "incomplete response b" error when all of the following conditions are met. 1) MaxSize options is used 2) response includes a Content-Length header 3) Content-Length > MaxSize For detail, please see the attached patch.
Subject: PoCoCl-HTTP-Request.pm.MaxSize.patch
--- /usr/local/lib/perl5/site_perl/5.8.8/POE/Component/Client/HTTP/Request.pm Sun Feb 4 14:25:29 2007 +++ POE/Component/Client/HTTP/Request.pm Wed May 2 05:50:50 2007 @@ -12,7 +12,7 @@ # Unique request ID, independent of wheel and timer IDs. my $request_seq = 0; -use constant DEBUG => 0; +use constant DEBUG => 1; use constant REQ_ID => 0; use constant REQ_POSTBACK => 1; @@ -218,11 +218,16 @@ # ignore content for HEAD requests. This may thwart keep-alive, # however. + my $expected = $self->[REQ_FACTORY]->max_response_size; + $expected = $self->[REQ_RESPONSE]->content_length + if + not defined $expected or + $self->[REQ_RESPONSE]->content_length < $expected; if ( $self->[REQ_REQUEST]->method() ne "HEAD" and defined $self->[REQ_RESPONSE]->content_length and not defined $self->[REQ_RESPONSE]->header("Transfer-Encoding") and - $self->[REQ_OCTETS_GOT] < $self->[REQ_RESPONSE]->content_length + $self->[REQ_OCTETS_GOT] < $expected ) { DEBUG and warn( "got " . $self->[REQ_OCTETS_GOT] . " of " .
Download (untitled) / with headers
text/plain 603b
On Tue May 01 17:05:52 2007, KAZUHO wrote: Show quoted text
> It seems that POE::Component::Client::HTTP::Request::add_eof erronously > generates a 400 "incomplete response b" error when all of the following > conditions are met. > > 1) MaxSize options is used > 2) response includes a Content-Length header > 3) Content-Length > MaxSize > > For detail, please see the attached patch.
Hi, KAZUHO. I appreciate the test case. It definitely fails for me here. And I also appreciate the patch, but applying it causes several tests to fail. Can you reformulate your patch so that "make test" passes? Thank you.
Download (untitled) / with headers
text/plain 121b
Another issue, this error seems to hold precedence over more important server error (400s). Which is why i'm failing t/01
From: ian [...] dal-acm.ca
Download (untitled) / with headers
text/plain 943b
On Thu Feb 19 01:51:52 2009, RCAPUTO wrote: Show quoted text
> On Tue May 01 17:05:52 2007, KAZUHO wrote:
> > It seems that POE::Component::Client::HTTP::Request::add_eof erronously > > generates a 400 "incomplete response b" error when all of the following > > conditions are met. > > > > 1) MaxSize options is used > > 2) response includes a Content-Length header > > 3) Content-Length > MaxSize > > > > For detail, please see the attached patch.
> > Hi, KAZUHO. I appreciate the test case. It definitely fails for me > here. And I also appreciate the patch, but applying it causes several > tests to fail. > > Can you reformulate your patch so that "make test" passes? Thank you.
There are some issues with the provided patch, but is the test it's failing (#9 for me) even correct? Perhaps I'm reading it wrong, but it sets a content length of 300, then gives a body of 300... and expects a 400 error because the content length is too long.
From: bitcard [...] ew.ewheeler.org
Download (untitled) / with headers
text/plain 897b
Show quoted text
> > On Tue May 01 17:05:52 2007, KAZUHO wrote:
> > > It seems that POE::Component::Client::HTTP::Request::add_eof
> erronously
> > > generates a 400 "incomplete response b" error when all of the
> following
> > > conditions are met. > > > > > > 1) MaxSize options is used > > > 2) response includes a Content-Length header > > > 3) Content-Length > MaxSize
I've found the same problem shipping to UPS's shipping estimation interface. Attached is a short example script that reliably regenerates the example. Also attached is the actual data being posted in case you need it (see $postdata in the script). I tried applying KAZUHO's patch, and this did not fix my problem. In my case, this POST works with HTTP, but not HTTPS (w/ SSLify installed). Note that the response from UPS does not end with a trailing carriage return. Is something buffered that should be flushed that isn't?
#!/usr/bin/perl # My code manifested the same error as this bug report: # http://rt.cpan.org/Public/Bug/Display.html?id=26874 # This is an attempt to supply a test case for the bug report. use warnings; use strict; use lib qw(. ../../libs ../../libs/lib/perl5); use POE qw/ Component::Client::HTTP Wheel::ListenAccept/; use HTTP::Request; use constant TIMEOUT => 15; POE::Session->create( inline_states => { _start => sub { POE::Component::Client::HTTP->spawn( Agent => 'SpiffCrawler/0.90', # defaults to something long Alias => 'UserAgent', # defaults to 'weeble' Timeout => TIMEOUT, # defaults to 180 seconds MaxSize => 1024*1024, # defaults to entire response ); # Retrieve the data to post to UPS. my $postdata = `wget -O- http://code.ewheeler.org/ssi/ups-request.xml`; # This problem only manifests with an https connection. # The http connection works fine. It acts as if not all data # is flushed in the response. Note that UPS does not feed us a CR/LF. #my $req = HTTP::Request->new('POST', "http://wwwcie.ups.com//ups.app/xml/Rate"); my $req = HTTP::Request->new('POST', "https://wwwcie.ups.com//ups.app/xml/Rate"); $req->content($postdata); $_[KERNEL]->post( 'UserAgent' => 'request', 'UserAgent_response', $req, ); $_[KERNEL]->post( 'UserAgent' => 'request', 'UserAgent_response', $req, ); print "Data posted...waiting for response or timeout " . TIMEOUT . "s\n" }, UserAgent_response => sub { my ($request_packet, $response_packet, $arg2, $arg3) = @_[ARG0, ARG1]; my $req = $request_packet->[0]; my $tag = $request_packet->[1]; my $response = $response_packet->[0]; print("tag=$tag\n" . "*" x 78, "\n", "*** my request:\n", "-" x 78, "\n", $req->as_string(), "*" x 78, "\n", "*** their response:\n", "-" x 78, "\n", $response->as_string(), ); } } ); POE::Kernel->run(); exit 0;
Download ups-request.xml
text/xml 3.8k
<?xml version='1.0' standalone='yes'?> <AccessRequest> <AccessLicenseNumber>1C44B1E71BB78164</AccessLicenseNumber> <Password>anypass</Password> <UserId>anyuser</UserId> </AccessRequest> <?xml version='1.0' standalone='yes'?> <RatingServiceSelectionRequest> <PickupType> <Code>06</Code> </PickupType> <Request> <RequestAction>Rate</RequestAction> <RequestOption>shop</RequestOption> <TransactionReference> <CustomerContext>Net::UPS</CustomerContext> <XpciVersion>1.0001</XpciVersion> </TransactionReference> </Request> <Shipment> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Package> <DimensionalWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> </DimensionalWeight> <Dimensions> <Height>10</Height> <Length>10</Length> <UnitOfMeasurement> <Code>IN</Code> </UnitOfMeasurement> <Width>10</Width> </Dimensions> <PackageWeight> <UnitOfMeasurement> <Code>LBS</Code> </UnitOfMeasurement> <Weight>27</Weight> </PackageWeight> <PackagingType> <Code>02</Code> </PackagingType> </Package> <Service> <Code>03</Code> </Service> <ShipTo> <Address> <CountryCode>US</CountryCode> <PostalCode>97007</PostalCode> </Address> </ShipTo> <Shipper> <Address> <CountryCode>US</CountryCode> <PostalCode>91789</PostalCode> </Address> </Shipper> </Shipment> </RatingServiceSelectionRequest>
Download (untitled) / with headers
text/plain 1.1k
On Mon Jul 13 20:55:44 2009, theg33kb0y wrote: Show quoted text
> > > On Tue May 01 17:05:52 2007, KAZUHO wrote:
> > > > It seems that POE::Component::Client::HTTP::Request::add_eof
> > erronously
> > > > generates a 400 "incomplete response b" error when all of the
> > following
> > > > conditions are met. > > > > > > > > 1) MaxSize options is used > > > > 2) response includes a Content-Length header > > > > 3) Content-Length > MaxSize
> > I've found the same problem shipping to UPS's shipping estimation > interface. Attached is a short example script that reliably regenerates > the example. Also attached is the actual data being posted in case you > need it (see $postdata in the script). > > I tried applying KAZUHO's patch, and this did not fix my problem. In my > case, this POST works with HTTP, but not HTTPS (w/ SSLify installed). > > Note that the response from UPS does not end with a trailing carriage > return. Is something buffered that should be flushed that isn't?
It appears that something has changed in the past fortnight. This test program and data don't trigger the bug. :( I'll try to craft a test case that doesn't require an external web server.
Download (untitled) / with headers
text/plain 401b
I have resolved this in revision 365: If the Content-Length is greater than MaxSize, or the length of the content is (in the case where there is no Content-Length header), then we return a 406 Not Acceptable error rather than truncated data. If you prefer the truncated data behavior, please use a range request instead of MaxSize. The range request will also let you complete the file if you want.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.