Skip Menu |
 

This queue is for tickets about the List-MoreUtils CPAN distribution.

Report information
The Basics
Id: 15638
Status: resolved
Priority: 0/
Queue: List-MoreUtils

People
Owner: Nobody in particular
Requestors: tlowery [...] cc3.com
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 0.10
Fixed in: (no value)



Subject: Memory Leak in repeated usage of all any none
Download (untitled) / with headers
text/plain 2.7k
There appears to be a memory leak using the List::MoreUtils XS functions all, any, and none. Test on perl 5.6.1 and 5.8.7 Test script: use strict; use warnings; use List::MoreUtils qw{all any none}; my @checkme = qw{HI FL AK}; my @states = qw{AZ CA NV MI DE PA}; my $misses = 0; while ( 1 == 1 ) { foreach my $to_check (@checkme) { # if (all { $to_check eq $_ } @states) { # $misses++; # } # if (any { $to_check eq $_ } @states) { # $misses++; # } if (none { $to_check eq $_ } @states) { $misses++; } } } The kernel kill the process afer a minute or so. dmesg shows "Out of Memory". The problem doesn't happen when using the pure perl version (LIST_MOREUTILS_PP=1). $ perl -V Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=linux, osvers=2.4.18-4smp, archname=i686-linux uname='linux rip 2.4.18-4smp #1 smp thu may 2 18:32:34 edt 2002 i686 unknown ' config_args='-Dprefix=/u01/app/perl' hint=previous, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.3 2.96-112)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/u01/app/perl/lib' Characteristics of this binary (from libperl): Compile-time options: USE_LARGE_FILES Built under linux Compiled at Aug 7 2002 18:40:37 %ENV: PERL5LIB="/lf/21st/devel/lib:/lf/utils/devel/lib" @INC: /lf/21st/devel/lib /lf/utils/devel/lib/i686-linux /lf/utils/devel/lib /u01/app/perl/lib/perl5/5.6.1/i686-linux /u01/app/perl/lib/perl5/5.6.1 /u01/app/perl/lib/perl5/site_perl/5.6.1/i686-linux /u01/app/perl/rip/lib/perl5/site_perl/5.6.1 /u01/app/perl/rip/lib/perl5/site_perl .
Date: Wed, 09 Nov 2005 08:54:23 +0100
From: Tassilo von Parseval <tassilo.von.parseval [...] rwth-aachen.de>
Subject: Re: [cpan #15638] Memory Leak in repeated usage of all any none
To: Guest via RT <bug-List-MoreUtils [...] rt.cpan.org>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 1.6k
On Tue, Nov 08, 2005 at 08:24:45PM -0500 Guest via RT wrote: Show quoted text
> > This message about List-MoreUtils was sent to you by guest <> via rt.cpan.org > > Full context and any attached attachments can be found at: > <URL: https://rt.cpan.org/Ticket/Display.html?id=15638 > > > There appears to be a memory leak using the List::MoreUtils XS functions all, any, and none. > > Test on perl 5.6.1 and 5.8.7 > > Test script: > use strict; > use warnings; > > use List::MoreUtils qw{all any none}; > > my @checkme = qw{HI FL AK}; > my @states = qw{AZ CA NV MI DE PA}; > > my $misses = 0; > > while ( 1 == 1 ) { > > foreach my $to_check (@checkme) { > # if (all { $to_check eq $_ } @states) { > # $misses++; > # } > # if (any { $to_check eq $_ } @states) { > # $misses++; > # } > if (none { $to_check eq $_ } @states) { > $misses++; > } > } > > } > > The kernel kill the process afer a minute or so. dmesg shows "Out of Memory".
Gee, that is indeed one nice memory leak. My suspicion is that this is a memory leak in perl. List::Util::first() shows the same behaviour and my module does it in the same way for its iterative functions. Skimming through the source I can find no explanation for the leak and valgrind isn't a lot more helpful. I'll raise that issue on the perl-porters list to see whether the problem is with perl or List::Util. In the latter case I just have to wait for Graham Barr to come up with a fix and incorporate it into my module. :-) Show quoted text
> The problem doesn't happen when using the pure perl version (LIST_MOREUTILS_PP=1).
Cheers, Tassilo -- use bigint; $n=71423350343770280161397026330337371139054411854220053437565440; $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
Download (untitled) / with headers
text/plain 290b
[guest - Tue Nov 8 20:24:45 2005]: Show quoted text
> There appears to be a memory leak using the List::MoreUtils XS > functions all, any, and none.
And those are now fixed in 0.13. I think the leak was actually in every function taking a CODE-block as first argument. Thanks again for spotting! Tassilo


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.