Skip Menu |
 

This queue is for tickets about the List-MoreUtils CPAN distribution.

Report information
The Basics
Id: 41494
Status: resolved
Priority: 0/
Queue: List-MoreUtils

People
Owner: Nobody in particular
Requestors: user42 [...] zip.com.au
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: (no value)
Fixed in: 0.400_003



MIME-Version: 1.0
X-Spam-Status: No, hits=0.0 required=8.0 tests=
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
Content-Type: multipart/mixed; boundary="=-=-="
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id F403019B81EC for <bug-List-MoreUtils [...] rt.cpan.org>; Sat, 6 Dec 2008 17:39:16 -0500 (EST)
Received: (qmail 10978 invoked by uid 103); 6 Dec 2008 22:39:15 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 6 Dec 2008 22:39:15 -0000
Received: from mailout1-3.pacific.net.au (HELO mailout1.pacific.net.au) (61.8.2.210) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Sat, 06 Dec 2008 14:39:13 -0800
Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout1.pacific.net.au (Postfix) with ESMTP id 6ADE13FA155 for <bug-List-MoreUtils [...] rt.cpan.org>; Sun, 7 Dec 2008 09:39:06 +1100 (EST)
Received: from blah.blah (ppp2007.dyn.pacific.net.au [61.8.32.7]) by mailproxy1.pacific.net.au (Postfix) with ESMTP id D11FB8C0B for <bug-List-MoreUtils [...] rt.cpan.org>; Sun, 7 Dec 2008 09:39:05 +1100 (EST)
Received: from gg by blah.blah with local (Exim 4.69) (envelope-from <gg [...] zip.com.au>) id 1L95nb-0000n6-P0 for bug-List-MoreUtils [...] rt.cpan.org; Sun, 07 Dec 2008 09:38:55 +1100
Delivered-To: cpan-bug+List-MoreUtils [...] diesel.bestpractical.com
Subject: memory leak in indexes() [XS]
User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/22.2 (gnu/linux)
Return-Path: <gg [...] zip.com.au>
X-Original-To: bug-List-MoreUtils [...] rt.cpan.org
X-Spam-Check-BY: 16.mx.develooper.com
Date: Sun, 07 Dec 2008 09:38:55 +1100
X-Spam-Level: *
Message-Id: <871vwlornk.fsf [...] blah.blah>
To: bug-List-MoreUtils [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
Content-Length: 0
X-RT-Original-Encoding: ascii
content-type: text/plain; charset="utf-8"
Content-Length: 779
Download (untitled) / with headers
text/plain 779b
With the debian packaged MoreUtils 0.22 and perl 5.10.0 the memory used by the program below grows apparently forever. If the two lines at the start are uncommented to use the pure-perl MoreUtils there's no memory growth. I suspect the ref count on the return values from List::MoreUtils::indexes() may be keeping them alive forever. (Incidentally, I saw the comments in MoreUtils.xs firstval() and lastval() referring to doing the same inc as in indexes(), but I believe those former two don't leak, as far as I can see. I guess the typemap for an SV* return does an sv_2mortal on it. I don't think the same on the multiple returns from indexes() would be enough though, since an error throw in the callback would still leak the returns built up but not yet mortalized.)
Content-Type: text/x-perl
content-disposition: inline; filename="foo.pl"
Content-Length: 252
Download foo.pl
text/x-perl 252b
use strict; use warnings; # use Carp; # BEGIN { $ENV{LIST_MOREUTILS_PP} = 1; }; use List::MoreUtils; my @a = (10,11,12,13,14,15); print "odd numbers: ", (List::MoreUtils::indexes {$_&1} @a), "\n"; for (;;) { List::MoreUtils::indexes {$_&1} @a; }
MIME-Version: 1.0
In-Reply-To: <871vwlornk.fsf [...] blah.blah>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Charset: utf8
References: <871vwlornk.fsf [...] blah.blah>
Content-Type: text/plain
Message-ID: <rt-3.6.HEAD-25318-1247435090-293.41494-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 59
This is resolved in 0.23 which I just uploaded to the CPAN.
MIME-Version: 1.0
In-Reply-To: <rt-3.6.HEAD-25318-1247435090-293.41494-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <871vwlornk.fsf [...] blah.blah> <rt-3.6.HEAD-25318-1247435090-293.41494-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-2358-1291054893-732.41494-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 265
Download (untitled) / with headers
text/plain 265b
On Sun Jul 12 14:44:50 2009, VPARSEVAL wrote: Show quoted text
> This is resolved in 0.23 which I just uploaded to the CPAN.
Note that the 0.23 code line is no longer in the latest release, 0.26. Consequently this and a number of other tickets in this queue are no longer resolved.
From gg [...] zip.com.au Mon Nov 29 17: 24:49 2010
MIME-Version: 1.0
X-Spam-Status: No, score=-6.894 tagged_above=-99.9 required=10 tests=[AWL=-0.004, BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_TVD_MIME_NO_HEADERS=0.01] autolearn=ham
In-Reply-To: <rt-3.8.HEAD-2358-1291054894-439.41494-6-0 [...] rt.cpan.org> (Karen Etheridge via's message of "Mon, 29 Nov 2010 13:21:34 -0500")
X-Spam-Flag: NO
References: <RT-Ticket-41494 [...] rt.cpan.org> <871vwlornk.fsf [...] blah.blah> <rt-3.6.HEAD-25318-1247435090-293.41494-6-0 [...] rt.cpan.org> <rt-3.8.HEAD-2358-1291054894-439.41494-6-0 [...] rt.cpan.org>
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <87lj4b6a5d.fsf [...] blah.blah>
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -6.894
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id 6959224120A for <cpan-bug+List-MoreUtils [...] hipster.bestpractical.com>; Mon, 29 Nov 2010 17:24:49 -0500 (EST)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vw44-74b5y6W for <cpan-bug+List-MoreUtils [...] hipster.bestpractical.com>; Mon, 29 Nov 2010 17:24:47 -0500 (EST)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id C63092411FD for <bug-List-MoreUtils [...] rt.cpan.org>; Mon, 29 Nov 2010 17:24:46 -0500 (EST)
Received: (qmail 15425 invoked by uid 103); 29 Nov 2010 22:24:45 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 29 Nov 2010 22:24:45 -0000
Received: from mailout1-5.pacific.net.au (HELO mailout1.pacific.net.au) (61.8.2.212) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Mon, 29 Nov 2010 14:24:42 -0800
Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id DF6575E2EB3 for <bug-List-MoreUtils [...] rt.cpan.org>; Tue, 30 Nov 2010 09:24:37 +1100 (EST)
Received: from blah.blah (unknown [203.26.175.152]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 013D227423 for <bug-List-MoreUtils [...] rt.cpan.org>; Tue, 30 Nov 2010 09:24:37 +1100 (EST)
Received: from gg by blah.blah with local (Exim 4.72) (envelope-from <gg [...] zip.com.au>) id 1PNC98-00021Z-Kl for bug-List-MoreUtils [...] rt.cpan.org; Tue, 30 Nov 2010 09:24:30 +1100
Delivered-To: cpan-bug+List-MoreUtils [...] hipster.bestpractical.com
Subject: Re: [rt.cpan.org #41494] memory leak in indexes() [XS]
User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (gnu/linux)
Return-Path: <gg [...] zip.com.au>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: cpan-bug+List-MoreUtils [...] hipster.bestpractical.com
X-RT-Mail-Extension: list-moreutils
Date: Tue, 30 Nov 2010 09:24:30 +1100
X-Spam-Level:
To: bug-List-MoreUtils [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
RT-Message-ID: <rt-3.8.HEAD-2359-1291069489-1307.41494-0-0 [...] rt.cpan.org>
Content-Length: 0
Content-Type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
Content-Length: 921
Download (untitled) / with headers
text/plain 921b
"Karen Etheridge via RT" <bug-List-MoreUtils@rt.cpan.org> writes: Show quoted text
> > Note that the 0.23 code line is no longer in the latest release, 0.26. > Consequently this and a number of other tickets in this queue are no > longer resolved.
Yes, this has regressed, indexes() leaks again in 0.26. The program below prints "still alive" where I expected it to print "weakened away". The latter is what it did in 0.25_02. This check could go in the test suite if it's not already there (for new enough perl to have weaken()). I suppose some care must be taken to mortalize at the right position, so the multicalls don't cause a free (if that is in fact the motivation for the extra SvREFCNT_inc), but an error throw out of the called test func does free. Perhaps a mortalized temporary AV could hold onto the results being built and either push them on the stack or mortalize them individually at the end after all the calls.
Content-Type: text/x-perl
content-disposition: inline; filename="indexes-leak.pl"
Content-Length: 222
Download indexes-leak.pl
text/x-perl 222b
use strict; use warnings; use Scalar::Util; use List::MoreUtils 'indexes'; my $ref = \(indexes(sub{1}, 123)); Scalar::Util::weaken($ref); if (defined $ref) { print "still alive\n"; } else { print "weakened away\n"; }
MIME-Version: 1.0
In-Reply-To: <871vwlornk.fsf [...] blah.blah>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <871vwlornk.fsf [...] blah.blah>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-19313-1291600390-131.41494-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 100
Download (untitled) / with headers
text/plain 100b
I've disabled the XS version in 0.27_04 until C people have a chance to fix the XS version properly.
From gg [...] zip.com.au Fri Dec 10 17: 52:26 2010
MIME-Version: 1.0
X-Spam-Status: No, score=-6.899 tagged_above=-99.9 required=10 tests=[AWL=0.001, BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5] autolearn=ham
In-Reply-To: <rt-3.8.HEAD-19313-1291600391-41.41494-6-0 [...] rt.cpan.org> (Adam Kennedy via's message of "Sun, 5 Dec 2010 20:53:11 -0500")
X-Spam-Flag: NO
References: <RT-Ticket-41494 [...] rt.cpan.org> <871vwlornk.fsf [...] blah.blah> <rt-3.8.HEAD-19313-1291600391-41.41494-6-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <87oc8tkzrf.fsf [...] blah.blah>
Content-Type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
X-Spam-Score: -6.899
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id 6F3AB2410FC for <cpan-bug+List-MoreUtils [...] hipster.bestpractical.com>; Fri, 10 Dec 2010 17:52:26 -0500 (EST)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YcDa6yl-wJTl for <cpan-bug+List-MoreUtils [...] hipster.bestpractical.com>; Fri, 10 Dec 2010 17:52:25 -0500 (EST)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id DBE8524001F for <bug-List-MoreUtils [...] rt.cpan.org>; Fri, 10 Dec 2010 17:52:24 -0500 (EST)
Received: (qmail 10280 invoked by uid 103); 10 Dec 2010 22:52:24 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 10 Dec 2010 22:52:24 -0000
Received: from mailout1-5.pacific.net.au (HELO mailout1.pacific.net.au) (61.8.2.212) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Fri, 10 Dec 2010 14:52:23 -0800
Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id 845FD5E273D for <bug-List-MoreUtils [...] rt.cpan.org>; Sat, 11 Dec 2010 09:52:19 +1100 (EST)
Received: from blah.blah (unknown [203.26.175.116]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 2D3AE27409 for <bug-List-MoreUtils [...] rt.cpan.org>; Sat, 11 Dec 2010 09:52:14 +1100 (EST)
Received: from gg by blah.blah with local (Exim 4.72) (envelope-from <gg [...] zip.com.au>) id 1PRBoK-0001M4-9P for bug-List-MoreUtils [...] rt.cpan.org; Sat, 11 Dec 2010 09:51:32 +1100
Delivered-To: cpan-bug+List-MoreUtils [...] hipster.bestpractical.com
Subject: Re: [rt.cpan.org #41494] memory leak in indexes() [XS]
User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (gnu/linux)
Return-Path: <gg [...] zip.com.au>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: cpan-bug+List-MoreUtils [...] hipster.bestpractical.com
X-RT-Mail-Extension: list-moreutils
Date: Sat, 11 Dec 2010 09:51:32 +1100
X-Spam-Level:
To: bug-List-MoreUtils [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
RT-Message-ID: <rt-3.8.HEAD-17551-1292021546-1575.41494-0-0 [...] rt.cpan.org>
Content-Length: 542
Download (untitled) / with headers
text/plain 542b
"Adam Kennedy via RT" <bug-List-MoreUtils@rt.cpan.org> writes: Show quoted text
> > I've disabled the XS version in 0.27_04 until C people have a chance to > fix the XS version properly.
Don't leave it that way. I had the misfortune to use the plain perl bits for a while for past bugs and it's a noticeable slowdown on medium-large data. The temp refcounts aren't difficult (unlike the nonsense about the multicalling in different versions of perl or whatever it was), and the code before was actually almost close enough, only leaking on an error throw.
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-17551-1292021546-1575.41494-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <RT-Ticket-41494 [...] rt.cpan.org> <871vwlornk.fsf [...] blah.blah> <rt-3.8.HEAD-19313-1291600391-41.41494-6-0 [...] rt.cpan.org> <87oc8tkzrf.fsf [...] blah.blah> <rt-3.8.HEAD-17551-1292021546-1575.41494-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-19315-1292316324-1593.41494-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 926
Download (untitled) / with headers
text/plain 926b
On Fri Dec 10 17:52:26 2010, user42@zip.com.au wrote: Show quoted text
> "Adam Kennedy via RT" <bug-List-MoreUtils@rt.cpan.org> writes:
> > > > I've disabled the XS version in 0.27_04 until C people have a chance to > > fix the XS version properly.
> > Don't leave it that way. I had the misfortune to use the plain perl > bits for a while for past bugs and it's a noticeable slowdown on > medium-large data. The temp refcounts aren't difficult (unlike the > nonsense about the multicalling in different versions of perl or > whatever it was), and the code before was actually almost close enough, > only leaking on an error throw.
I care about that when I found a tuit (after fixing some other RT's on other modules). Please be patient. The current confusing forces Adam to walk this way to put L::MU on a clear base and have all RT's in a defined state so that I can pick the remaining tasks. Thank you very much, Adam and Kevin, Jens
MIME-Version: 1.0
In-Reply-To: <871vwlornk.fsf [...] blah.blah>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <871vwlornk.fsf [...] blah.blah>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-8840-1395211786-1499.41494-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 143
Download (untitled) / with headers
text/plain 143b
I added some more tests to current trunk and don't see any of the described leaking behavior anymore. I tend it has been fixed in the meantime.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.