Skip Menu |
 

This queue is for tickets about the libwww-perl CPAN distribution.

Report information
The Basics
Id: 70494
Status: resolved
Priority: 0/
Queue: libwww-perl

People
Owner: Nobody in particular
Requestors: rjray [...] blackperl.com
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 6.02
Fixed in: (no value)



Subject: LWP::UserAgent::request leaks memory when server is unreachable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
X-RT-Original-Encoding: utf-8
Content-Type: multipart/mixed; boundary="----------=_1314322980-20563-8"
Content-Length: 0
Content-Type: text/plain; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: binary
Content-Length: 1262
Download (untitled) / with headers
text/plain 1.2k
The attached sample script shows that memory grows consistently when requests are made to a server that is not reachable. It uses BSD::Resource to report the memory usage (each output line reports the current MAXRSS usage and the change in mem since the previous iteration), and RPC::XML::Client to manage the requests. If the server does answer the requests, memory does not grow. Using RPC::XML::Server, you can prove this by running a simple server as a one-liner: perl -MRPC::XML::Server -e 'RPC::XML::Server->new(port => 6666)->server_loop' (This is the same port that the attached script tries to connect to, and the server by default will provide the remote-called method that the script tries to invoke.) If the server is running, the script shows only memory growth on the first 2 iterations, when protocol code is first loaded, etc. If the server is not running, almost every iteration will show memory growth. This would appear to be somewhere in the code that deals with failure to connect to the server, so in theory the leak could be as deep as the socket code itself. But I've ruled out the RPC::XML code by running this through the debugger looking for leaks with Devel::Cycle. Randy -- Randy J. Ray rjray@blackperl.com randy.j.ray@gmail.com
Subject: testmem.pl
MIME-Version: 1.0
Content-Type: text/x-perl-script; name="testmem.pl"
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline; filename="testmem.pl"
Content-Transfer-Encoding: binary
Content-Length: 438
Download testmem.pl
text/x-perl 438b
#!/usr/bin/env perl use strict; use warnings; use BSD::Resource; use RPC::XML::Client; my ($maxrss, $oldmax, $diff); my $count = 20; my $cli = RPC::XML::Client->new('http://localhost:6666'); ($oldmax) = (getrusage())[2]; while ($count--) { my $foo = $cli->send_request('system.identity'); ($maxrss) = (getrusage())[2]; $diff = $maxrss - $oldmax; printf "%-8d - %-8d\n", $maxrss, $diff; $oldmax = $maxrss; } exit;
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-1392-1344846276-1239.70494-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
From: mattilinnanvuori [...] yahoo.com
X-RT-Original-Encoding: utf-8
Content-Length: 7265
I could not find any memory leak with Perl 5.10.0 x86_64-linux-thread-multi on SLES11 SP1 with a script that tries to post data. sub post { $SIG{'TERM'} = \&termination_flag; my $ua = LWP::UserAgent->new; $ua->timeout( $configuration->{'timeout'} ); while ($run) { my %post; { syslog LOG_DEBUG, "Locking to check for new location GeoJSON data"; lock %location; until ( %location or not $run ) { syslog LOG_DEBUG, "Waiting for location GeoJSON data"; cond_wait %location; } %post = %location; } if (%post) { my %feature; while ( my ( $id, $feature ) = each %post ) { $feature{$id} = encode_json $feature; } my $content = '{"type":"FeatureCollection","features":[' . join( ',', values %feature ) . ']}'; syslog LOG_INFO, "Posting location data $content"; my $response = $ua->post( $configuration->{'url'}, 'Content-Type' => 'application/json', 'Content' => $content ); if ( $response->is_success ) { syslog LOG_INFO, $response->as_string; syslog LOG_DEBUG, "Locking to delete the posted location data"; lock %location; while ( my ( $id, $feature ) = each %location ) { my $feature_json = $feature{$id}; if ( defined $feature_json and $feature_json eq encode_json $feature ) { delete $location{$id}; } } } else { syslog LOG_CRIT, $response->as_string; sleep $run * $configuration->{'retry'}; } } } if (%location) { syslog LOG_ERR, "Unposted location data: %s", encode_json \%location; } syslog LOG_INFO, "The post subroutine terminating"; } ==12274== Memcheck, a memory error detector ==12274== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==12274== Using Valgrind-3.8.0 and LibVEX; rerun with -h for copyright info ==12274== Command: /opt/avl/current/sbin/aplicom.pl ==12274== aplicom[12274]: Started the program with configuration {"url":"http://localhost:9210/external","port":60000} aplicom[12274]: Locking to check for new location GeoJSON data aplicom[12274]: Receiving a message on the socket aplicom[12274]: Waiting for location GeoJSON data aplicom[12274]: 127.0.0.1:39397: 44c200000000000001004000ffff6d0180501914a25017d0e20395d491017bac6305000088 8000000000000000002ece0000000004e7000004e70000000000000000000300120000000 000000000 aplicom[12274]: Data validity 128 at position 16 aplicom[12274]: Digital input status at position 37 aplicom[12274]: Voltage of analog inputs at position 38 aplicom[12274]: Voltage of main power in millivolts. 16bit unsigned integer at position 46 aplicom[12274]: Distance travelled 1 in meters since trip meter was reset. 32bit unsigned integer. aplicom[12274]: Distance travelled 2 in meters since trip meter was reset. 32bit unsigned integer. aplicom[12274]: Status of outputs at position 58 aplicom[12274]: iButton key ID at position 59 aplicom[12274]: Driver Log Keypad button at position 65 aplicom[12274]: Altitude at position 66: 0003 aplicom[12274]: {"geometry":{"coordinates": [24.882275,60.150929,3],"type":"Point"},"type":"Feature","id":1,"properties": {"maximum_velocity":0,"event_information":1,"iButton":"000000000000","travelled2":1255,"nu mber_of_satellites":5,"net_time":"20120813111619","time":1343820962,"analog_input_4":0,"f lag":"0000000000000000","snapshot_counter":18,"analog_input_3":0,"velocity":0,"direction":2 72,"external_battery":0,"maximum_velocity_error":0,"analog_input_1":0,"main_power":11982," additional_data":129,"locationgw":"aplicom","output":"Output 1 is off. Output 2 is off","travelled1":1255,"event":"ALARM_ACTIVE","lip_result_code":201,"reason_code":129,"posit ion_error":-1,"velocity_error":0,"gps_time":1343738082,"analog_input_2":0,"keypad":0,"digital _input":"IGN on. DIN1 off. DIN2 off. DIN3 off. DIN4 off. DIN5 off. DIN6 off."}} aplicom[12274]: Locking to signal new location data aplicom[12274]: Signaling new location data aplicom[12274]: Receiving a message on the socket aplicom[12274]: Posting location data {"type":"FeatureCollection","features":[{"geometry": {"coordinates":[24.882275,60.150929,3],"type":"Point"},"type":"Feature","id":1,"properties": {"maximum_velocity":0,"event_information":1,"iButton":"000000000000","travelled2":1255,"nu mber_of_satellites":5,"net_time":"20120813111619","time":1343820962,"analog_input_4":0,"f lag":"0000000000000000","snapshot_counter":18,"analog_input_3":0,"velocity":0,"direction":2 72,"external_battery":0,"maximum_velocity_error":0,"analog_input_1":0,"main_power":11982," additional_data":129,"locationgw":"aplicom","output":"Output 1 is off. Output 2 is off","travelled1":1255,"event":"ALARM_ACTIVE","lip_result_code":201,"reason_code":129,"posit ion_error":-1,"velocity_error":0,"gps_time":1343738082,"analog_input_2":0,"keypad":0,"digital _input":"IGN on. DIN1 off. DIN2 off. DIN3 off. DIN4 off. DIN5 off. DIN6 off."}}]} aplicom[12274]: 500 Can't connect to localhost:9210 (connect: Connection refused) Content-Type: text/plain Client-Date: Mon, 13 Aug 2012 08:16:20 GMT Client-Warning: Internal response ... aplicom[12274]: HTTP/1.1 200 HTTPResponse Connection: close Content-Length: 4 Content-Type: text/plain; charset=iso-8859-1 Client-Date: Mon, 13 Aug 2012 08:16:32 GMT Client-Peer: 127.0.0.1:9210 Client-Response-Num: 1 OK aplicom[12274]: Locking to delete the posted location data aplicom[12274]: Locking to check for new location GeoJSON data aplicom[12274]: Waiting for location GeoJSON data aplicom[12274]: Setting termination flag aplicom[12274]: Set termination flag aplicom[12274]: Termination signal caught aplicom[12274]: Failed to receive data from the socket: Interrupted system call aplicom[12274]: Closing the socket aplicom[12274]: Killing the post thread aplicom[12274]: Locking to signal termination aplicom[12274]: Signaling the post thread to terminate aplicom[12274]: Setting termination flag aplicom[12274]: Set termination flag aplicom[12274]: The post subroutine terminating aplicom[12274]: Joining the post thread aplicom[12274]: Program terminating ==12274== ==12274== HEAP SUMMARY: ==12274== in use at exit: 118,290 bytes in 104 blocks ==12274== total heap usage: 334,932 allocs, 334,828 frees, 38,350,543 bytes allocated ==12274== ==12274== LEAK SUMMARY: ==12274== definitely lost: 0 bytes in 0 blocks ==12274== indirectly lost: 0 bytes in 0 blocks ==12274== possibly lost: 0 bytes in 0 blocks ==12274== still reachable: 118,290 bytes in 104 blocks ==12274== suppressed: 0 bytes in 0 blocks ==12274== Reachable blocks (those to which a pointer was found) are not shown. ==12274== To see them, rerun with: --leak-check=full --show-reachable=yes ==12274== ==12274== For counts of detected and suppressed errors, rerun with: -v ==12274== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
X-RT-Interface: REST
MIME-Version: 1.0
X-Mailer: MIME-tools 5.504 (Entity 5.504)
RT-Message-ID: <rt-4.0.18-16263-1490987036-1487.70494-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: binary
Content-Length: 82


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.