Skip Menu |
 
rt.cpan.org will be shut down on March 1st, 2021.

This queue is for tickets about the POE-Component-SSLify CPAN distribution.

Report information
The Basics
Id: 58243
Status: patched
Priority: 0/
Queue: POE-Component-SSLify

People
Owner: Nobody in particular
Requestors: johan [...] headweb.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Problem with large SSL responses && keepalive
Date: Tue, 8 Jun 2010 17:05:41 +0200
To: bug-POE-Component-SSLify [...] rt.cpan.org
From: Johan Ström <johan [...] headweb.com>
Download (untitled) / with headers
text/plain 1.2k
I'm running POE::Component::Client::HTTP 0.895 with a POE::Component::Client::Keepalive 0.262, and POE::Component::SSLify 0.20 with Net::SSLeay 1.36 on openssl 0.9.8e (RHEL5) My code makes a basic HTTPS request to another server, and so far it has been working fine. However, today I made a call which returned a larger than average response (Content-Length: 42643), and now it reads the first thousands byte of response (calls to Net::SSLeay::read), until the last "block", where it hangs for exactly 60 seconds and then another call comes through to Net::SSLeay::read, returning 2928 which is the rest of the response. After this, the requests returns and the response is fine! Enabling POE_TRACE_FILES gave me that select returned read for the fd all times until right befroe the last read, where it only got timeouts until after 60 seconds where a read came in, and the rest of the response was found. I was able to "fix" the problem by adding a Connection: close header to the request right before sending it of: $req->header('Connection'=> 'close') Now i get the full response immediately, but of course, no connection pooling. Otherwise, thanks for a nice lib, POE as well as POCO::SSLify and all others! Best Regards Johan Ström johan@headweb.com
Download (untitled) / with headers
text/plain 1.8k
Hello, Thanks again for reporting this! I apologize for the delay in getting back to you. I've just released v1.009 to CPAN with some more tests that try to track down large packet size issues. I wasn't able to reproduce your exact problem but another person reported in RT#95071 hit the same problem. I think it's a buffer size limitation somewhere in the stack and I'll try to fix it. Can you please test and let me know if the problem persists? Thanks! On Tue Jun 08 11:05:56 2010, johan@headweb.com wrote: Show quoted text
> I'm running POE::Component::Client::HTTP 0.895 with a > POE::Component::Client::Keepalive 0.262, and POE::Component::SSLify > 0.20 with Net::SSLeay 1.36 on openssl 0.9.8e (RHEL5) > > My code makes a basic HTTPS request to another server, and so far it > has been working fine. However, today I made a call which returned a > larger than average response (Content-Length: 42643), and now it reads > the first thousands byte of response (calls to Net::SSLeay::read), > until the last "block", where it hangs for exactly 60 seconds and then > another call comes through to Net::SSLeay::read, returning 2928 which > is the rest of the response. After this, the requests returns and the > response is fine! > > Enabling POE_TRACE_FILES gave me that select returned read for the fd > all times until right befroe the last read, where it only got timeouts > until after 60 seconds where a read came in, and the rest of the > response was found. > > > I was able to "fix" the problem by adding a Connection: close header > to the request right before sending it of: > > $req->header('Connection'=> 'close') > > Now i get the full response immediately, but of course, no connection > pooling. > > Otherwise, thanks for a nice lib, POE as well as POCO::SSLify and all > others! > > Best Regards > Johan Ström > johan@headweb.com
-- ~Apocalypse
Download (untitled) / with headers
text/plain 1.8k
Hey, I had some luck and found some information that pointed towards a 16K limitation in the TLS packet sizes. It seems like on certain conditions the write() call will lock up because the underlying buffers was not flushed. I haven't found the root cause for this but with my modification by clamping the writes the superbig tests passed! I've just released v1.010 to CPAN with the change, can you please test it to see if it helps you? If not, then I'll have to attack it from a different angle :) On Tue Jun 08 11:05:56 2010, johan@headweb.com wrote: Show quoted text
> I'm running POE::Component::Client::HTTP 0.895 with a > POE::Component::Client::Keepalive 0.262, and POE::Component::SSLify > 0.20 with Net::SSLeay 1.36 on openssl 0.9.8e (RHEL5) > > My code makes a basic HTTPS request to another server, and so far it > has been working fine. However, today I made a call which returned a > larger than average response (Content-Length: 42643), and now it reads > the first thousands byte of response (calls to Net::SSLeay::read), > until the last "block", where it hangs for exactly 60 seconds and then > another call comes through to Net::SSLeay::read, returning 2928 which > is the rest of the response. After this, the requests returns and the > response is fine! > > Enabling POE_TRACE_FILES gave me that select returned read for the fd > all times until right befroe the last read, where it only got timeouts > until after 60 seconds where a read came in, and the rest of the > response was found. > > > I was able to "fix" the problem by adding a Connection: close header > to the request right before sending it of: > > $req->header('Connection'=> 'close') > > Now i get the full response immediately, but of course, no connection > pooling. > > Otherwise, thanks for a nice lib, POE as well as POCO::SSLify and all > others! > > Best Regards > Johan Ström > johan@headweb.com
-- ~Apocalypse
CC: perl [...] miggy.org
Subject: Re: [rt.cpan.org #58243] Problem with large SSL responses && keepalive
Date: Tue, 26 Jan 2016 09:08:22 +0000
To: Bugs in POE-Component-SSLify via RT <bug-POE-Component-SSLify [...] rt.cpan.org>
From: Athanasius <perl [...] miggy.org>
Download (untitled) / with headers
text/plain 3.7k
Hi, I ran into just this issue as recently as yesterday, using POE::Component::SSLify 1.012 (in fact I just had cpanp update all of "POE::Component::SSLify POE::Component::SSLify::ClientHandle POE::Component::SSLify::NonBlock POE::Component::SSLify::NonBlock::ServerHandle POE::Component::SSLify::ServerHandle" to be sure). The symptom is that retrieving: <https://community.elitedangerous.com/galnet/uid/56a60d089657ba197a730a88> takes approximately 1 minute, this being the HTTP 1.1 keep-alive timeout of that server. I initially thought this to be a bug in POE::Component::Client::HTTP and thus reported it at: <https://rt.cpan.org/Ticket/Display.html?id=111425> but now I see that the issue is that SSLify doesn't install a custom driver for retrieving data. What happens is that POE's loop uses CORE::select() to test if there is data to be read, but the SSLification of the connection means that sysread() isn't reading directly from the socket, instead it gets the plaintext after the SSLification has decrypted it. In the case of the URL above the returned data is large enough to not fit in a single 4096 byte read AND the encrypted data is smaller than the plaintext. So initial sysread() causes the socket to be drained, and thus CORE::select() to not indicate the file handle as ready for read, yet not all of the plaintext has actually made it out yet. This is evidenced by a mixture of activating DEBUG statements in the POE modules (with some extra code for ISO8601 timestamps) and running it all under strace to see the actual calls. strace shows read(2) grabbing over 8302 bytes in the penultimate call on the filehandle BEFORE the POE code starts hanging pending the server side keep-alive timeout. The only further read(2) on the socket after this returns zero bytes as CORE::select() is indicating EOF. The program then enters a state where the event's $self->[DRIVER_BOTH]->get() method could still read more, but it isn't told to because of CORE::select() not saying it's ready to. Once the connection closes the filehandle is marked as ready for sysread() and the remainder of the plaintext data is read. Note that the sysread() calls are triggered by the 'select read' event being set in POE::Wheel::ReadWrite's _define_read_state, which calls the event's $self->[DRIVER_BOTH]'s get() method, which in this case is POE::Driver::SysRW's get(). I'd suggest the real fix is to have SSLify set a custom DRIVER_BOTH with a get() method that correctly drains the available plaintext as indicated by CORE::select(). That way the loop in _define_read_state() will have all available input to process (in this case ultimately by POE::Filter::HTTPCunk). I'll see if I can come up with a patch, but not being familiar with the code I'm not sure I'm capable, or how long it will take. A workaround for now is to send the header "Connection: close" as part of the request as this causes the server to not do keep-alive, instead closing the socket immediately it has sent all of the response. In my use this is acceptable, but anyone using POE for SSL where they'd like to make use of keep-alive to reduce overheads will not have this option. Oh, and I further suspect that this kind of issue is why POE::Component::Client::HTTP had transparent content decoding (for compression) disabled as of version 0.84, as the size mismatch between compressed on-the-wire and plaintext will likely cause the same problem. So, if a solution can be found for THIS issue then perhaps it's worth notifying other POE authors. -- - Athanasius = Athanasius(at)miggy.org / http://www.miggy.org/ Finger athan(at)fysh.org for PGP key "And it's me who is my enemy. Me who beats me up. Me who makes the monsters. Me who strips my confidence." Paula Cole - ME


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.