Skip Menu |

This queue is for tickets about the libwww-perl CPAN distribution.

Report information
The Basics
Id: 32356
Status: resolved
Priority: 0/
Queue: libwww-perl

Owner: Nobody in particular
Requestors: alexander.berger [...]

Bug Information
Severity: Important
Broken in:
  • 5.805
  • 5.806
  • 5.807
  • 5.808
Fixed in: (no value)

Subject: Wrong use of sysread - EAGAIN and EINTR not handled on Unix systems
Download (untitled) / with headers
text/plain 4.2k
LWP makes heavy use of sysread (Unix read) while reading the HTTP response from the corresponding socket. When sysread "fails" (it returns undef) this does not necessarily mean that there was an error. This is because sysread might fail reading from a non blocking file descriptor (socket), which has no data available yet but is still valid (and open). The reason for the failure is available using perl's $! variable. In order to avoid errors, LWP should check for the reason of a sysread failure (using $!) and retry reading if the failure was due to on of the errors EAGAIN or EINTR (Unix error codes returned by read). The current version of LWP will close the connection to the server after sysread failed, without checking for the reason. Therefore the response is never received and an error is signaled to the user agent, even though the repsonse could have been received if sysread was used correctly. This misbehavior of LWP might have to do with the fact that on Unix calls to select or poll can signal that a file descriptor (socket) is ready for reading while there actually is no data available on that file descriptor in that very moment. Therefore any call to read (sysread) and select on such a descriptor should handle EAGAIN. And to make things even more stable EINTR (call was Interrupted e.g. by a signal) should also be handled the same way. So in case of the http/https schema the problem is located in the file LWP/Protocol/ in the package LWP::Protocol::http::SocketMethods and the subroutines involved are: sysread and can_read. Here is a non portable (Unix/POSIX only) example solution to this Problem: ... package LWP::Protocol::http::SocketMethods; use Errno qw(EINTR EAGAIN); # This is in fact a blocking read which will die with the message # 'read timeout' when the so_socket_timout expired. So the name # sysread is misleading but this is because of the design of LWP # therefore just remember this is a very special sysread. sub sysread { my $self = shift; my $timeout = ${*$self}{io_socket_timeout}; my $stime = time if $timeout; # This loop exists only to support error recovery upon system call # failure caused by EAGAIN and EINTR. It will be left when either # the timeout expired, a not recoverable error occured or sysread # succeeded without any failure. while ( ! $timeout || $timeout > 0 ) { # wait for socket to become ready # on read timeout this call will die (see can_read below). my $ready = $self->can_read($timeout); # In case of an error act like the real sysread an return undef, # details about the error are available in $!. return undef if ! defined $ready; # If socket is not ready (there is no data available yet) then # we just continue an try sysread. $! = 0; my $bytesRead = sysread($self, $_[0], $_[1], $_[2] || 0); if ( ! defined $bytesRead && ( $! == EAGAIN || $! == EINTR) ) { $timeout = $timeout - (time - $stime) if $timeout; } else { return $bytesRead; } } die "read timeout"; } sub can_read { my($self, $timeout) = @_; my $nfound = 0; # This loop exists only to support error recovery upon system call # failure caused by EAGAIN and EINTR. It will be left when either # the timeout expired, a not recoverable error occured or select # succeeded without any failure. while ( ! $timeout || $timeout > 0 ) { my $fbits = ''; vec($fbits, fileno($self), 1) = 1; $! = 0; my $stime = time if $timeout; $nfound = select($fbits, undef, undef, $timeout); if ( ! defined $nfound && ( $! == EAGAIN || $! == EINTR) ) { # If select failed because of EAGAIN or EINTR we restart the # call. If a timeout is set then we first have to adjust it. $timeout = $timeout - (time - $stime) if $timeout; } elsif ( ! defined $nfound ) { # any other failure (error code) is signaled to the caller by # returning undef. The caller can then inspect $! for details # about the failure. return undef; } else { return $nfound > 0; } } # If wo got to this far then the read timeout expired and we have # to signal this to the caller using an exception (die). die('read timeout'); } ...
I've now applied a patch to my sources that test for these errors. It will appear in LWP-5.811.
i found out an issue to the patch applied. This is a snippet of code of the can_read function: SELECT: { my $before; $before = time if $timeout; my $nfound = select($fbits, undef, undef, $timeout); unless (defined $nfound) { if ($!{EINTR} || $!{EAGAIN}) { # don't really think EAGAIN can happen here if ($timeout) { $timeout -= time - $before; $timeout = 0 if $timeout < 0; } redo SELECT; } die "select failed: $!"; } return $nfound > 0; } the check of definedness of the $nfound var is not always correct because, in the case i experienced, the 'select' was being interrupted (EINTR) and it was returning a -1, so this case was being treated as a request timeout. The solution is quite simple, just change the check for definedness to check for the value as well: - unless (defined $nfound) { + if (not defined $nfound || $nfound < 0) {
Download (untitled) / with headers
text/plain 273b
Reading the documentation for select once more and taking a look at the sources there does not appear any case where select returns undef, so these now only tests for $nfound < 0: perl/mainline/commit/1c9ba0091b405c1be68a1b71f7e49da18df635d3

This service is sponsored and maintained by Best Practical Solutions and runs on infrastructure.

Please report any issues with to