Skip Menu |
 

This queue is for tickets about the XML-Twig CPAN distribution.

Report information
The Basics
Id: 71636
Status: resolved
Priority: 0/
Queue: XML-Twig

People
Owner: Nobody in particular
Requestors: TEAM [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 3.39
Fixed in: (no value)



Subject: Segfault with medium-sized document (30k elements)
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
X-RT-Original-Encoding: utf-8
Content-Type: multipart/mixed; boundary="----------=_1318450305-2604-47"
Content-Length: 0
Content-Type: text/plain; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: binary
Content-Length: 3542
Download (untitled) / with headers
text/plain 3.4k
Hi there, The attached code raises a segfault with perl-5.10.1 and perl-5.14.2. Seems fine for other XML parsers such as XML::LibXML::SAX and XML::TreeBuilder, and in XML::Twig for smaller documents (although significantly slower than the other parsers). Output of script on Linux/x64: $ perl xmlbench.pl Document is 1350062 chars Benchmark: timing 10 iterations of tree, twig... tree: 10.3706 wallclock secs (10.09 usr + 0.27 sys = 10.36 CPU) @ 0.97/s (n=10) Segmentation fault perl -V output: Summary of my perl5 (revision 5 version 14 subversion 2) configuration: Platform: osname=linux, osvers=2.6.35-30-generic, archname=x86_64-linux uname='linux roku 2.6.35-30-generic #56-ubuntu smp mon jul 11 20:01:08 utc 2011 x86_64 gnulinux ' config_args='-de -Dprefix=/home/tom/perl5/perlbrew/perls/perl-5.14.2' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.4.5', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64 /usr/lib64 libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.12.1.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.12.1' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector' Characteristics of this binary (from libperl): Compile-time options: PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 3 2011 02:24:00 %ENV: PERL5LIB="/home/tom/.cpan:/home/tom/.cpan/lib:/home/tom/.cpan/lib/perl5:/home/tom/.cpan/lib/perl5/site_perl:/home/tom/.cpan/lib/perl5/5.10.0/i486-linux-gnu-thread-multi:" PERLBREW_HOME="/home/tom/.perlbrew" PERLBREW_PATH="/home/tom/perl5/perlbrew/bin:/home/tom/perl5/perlbrew/perls/perl-5.14.2/bin" PERLBREW_PERL="perl-5.14.2" PERLBREW_ROOT="/home/tom/perl5/perlbrew" PERLBREW_VERSION="0.29" PERL_LWP_SSL_VERIFY_HOSTNAME="0" PERL_MM_USE_DEFAULT="1" @INC: /home/tom/.cpan /home/tom/.cpan/lib /home/tom/.cpan/lib/perl5 /home/tom/.cpan/lib/perl5/site_perl /home/tom/.cpan/lib/perl5/5.10.0/i486-linux-gnu-thread-multi /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2 /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2/x86_64-linux /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2 cheers, Tom
Subject: xmlbench.pl
MIME-Version: 1.0
Content-Type: application/x-perl; name="xmlbench.pl"
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline; filename="xmlbench.pl"
Content-Transfer-Encoding: base64
Content-Length: 431
Download xmlbench.pl
text/x-perl 431b
#!/usr/bin/perl use strict; use warnings; use XML::Twig; use XML::TreeBuilder; use Benchmark qw(:hireswallclock); my $xml = q{ <root> }; $xml .= q{ <child> <name>test name</name> </child> } for 0..30000; $xml .= q{ </root> }; warn "Document is " . length($xml) . " chars\n"; timethese(10, { twig => sub { my $parsed = XML::Twig->parse($xml); }, tree => sub { my $parsed = XML::TreeBuilder->new->parse($xml); } });
From xmltwig [...] gmail.com Thu Oct 13 04: 20:42 2011
MIME-Version: 1.0
X-Spam-Status: No, score=-6.669 tagged_above=-99.9 required=10 tests=[AWL=-0.460, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HK_RANDOM_ENVFROM=0.001, RCVD_IN_DNSWL_HI=-5, SPF_NEUTRAL=0.779, T_TO_NO_BRKTS_FREEMAIL=0.01] autolearn=ham
In-Reply-To: <rt-3.8.HEAD-2604-1318450306-1798.71636-4-0 [...] rt.cpan.org>
X-Spam-Flag: NO
References: <RT-Ticket-71636 [...] rt.cpan.org> <rt-3.8.HEAD-2604-1318450306-1798.71636-4-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <4E969F39.6010205 [...] gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
X-RT-Original-Encoding: utf-8
X-Spam-Score: -6.669
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] gmail.com
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id A25E2240B3A for <cpan-bug+XML-Twig [...] hipster.bestpractical.com>; Thu, 13 Oct 2011 04:20:42 -0400 (EDT)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a6iSpa1p7Vpo for <cpan-bug+XML-Twig [...] hipster.bestpractical.com>; Thu, 13 Oct 2011 04:20:38 -0400 (EDT)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id 7BA69240B45 for <bug-XML-Twig [...] rt.cpan.org>; Thu, 13 Oct 2011 04:20:36 -0400 (EDT)
Received: (qmail 27448 invoked by uid 103); 13 Oct 2011 08:20:35 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 13 Oct 2011 08:20:35 -0000
Received: from mail-wy0-f178.google.com (HELO mail-wy0-f178.google.com) (74.125.82.178) by 16.mx.develooper.com (qpsmtpd/0.80/v0.80-19-gf52d165) with ESMTP; Thu, 13 Oct 2011 01:20:32 -0700
Received: by wyf23 with SMTP id 23so1661336wyf.9 for <bug-XML-Twig [...] rt.cpan.org>; Thu, 13 Oct 2011 01:20:29 -0700 (PDT)
Received: by 10.216.14.40 with SMTP id c40mr817332wec.85.1318494029262; Thu, 13 Oct 2011 01:20:29 -0700 (PDT)
Received: from [192.168.2.200] (static-217-133-18-84.clienti.tiscali.it. [217.133.18.84]) by mx.google.com with ESMTPS id es10sm4593288wbb.4.2011.10.13.01.20.27 (version=SSLv3 cipher=OTHER); Thu, 13 Oct 2011 01:20:28 -0700 (PDT)
Delivered-To: cpan-bug+XML-Twig [...] hipster.bestpractical.com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15
Subject: Re: [rt.cpan.org #71636] Segfault with medium-sized document (30k elements)
Return-Path: <xmltwig [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=NXXARh2RIytNQHQyeKnRD0Mio6pBeSg5zuWqxGNXNqc=; b=wC08mtNQA367qs+wOIXaIW1+U/lkk5tWZLLSY2Mr8hEaQ0oyYBi9pNm9fBfH4hPtOE ELFmNjC35o/UpfzBnl5mDDlzMV1tHTqiF//qNMBL4eqsiE0fvw/I/mY4kqGOYTz2l1By CDUw/r+vktb7vSTRFqhsDy41V/fquv+/NYB7o=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: cpan-bug+XML-Twig [...] hipster.bestpractical.com
X-RT-Mail-Extension: xml-twig
Date: Thu, 13 Oct 2011 10:20:09 +0200
X-Spam-Level:
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: mirod <xmltwig [...] gmail.com>
RT-Message-ID: <rt-3.8.HEAD-2597-1318494043-1160.71636-0-0 [...] rt.cpan.org>
Content-Length: 5445
Download (untitled) / with headers
text/plain 5.3k
Interesting. The problem kicks in around 18055 elements for me (32-bit perl, gobs of memory on the machine). The trigger is not always the same BTW, 18054 may or may not cause it. The number of children seems to be the only important factor BTW, adding sub elements within the children does not change the trigger. You don't need a loop, the segfault happens during the DESTROY phase. The minimum test case I have found is below. #!/usr/bin/perl use strict; use warnings; use XML::Twig; my $xml = q{<root>} . q{<child><name>test name</name></child>} x 20000 . q{</root>}; warn "Document is " . length($xml) . " chars\n"; my $parsed = XML::Twig->new->parse($xml); my $t= XML::Twig->new->parse( '<d/>'); warn "done\n"; exit; The problem occurs _after_ "done" is printed. So it's in the DESTROY phase, and it doesn't happen unless the second parse is done. It also doesn't happen when I purge the twig as I create it (ie if I add a handler on child that does $t->purge. So at this point my guess is that either I messed up the cleanup of XML::Parser related objects, or its a bug in the layers below, either XML::Parser or perl itself. I'll keep on investigating and let you know. Thanks for the report. -- mirod On 10/12/2011 10:11 PM, Thomas Edward Alexander Molesworth via RT wrote: Show quoted text
> Wed Oct 12 16:11:46 2011: Request 71636 was acted upon. > Transaction: Ticket created by TEAM > Queue: XML-Twig > Subject: Segfault with medium-sized document (30k elements) > Broken in: 3.39 > Severity: (no value) > Owner: Nobody > Requestors: TEAM@cpan.org > Status: new > Ticket<URL: https://rt.cpan.org/Ticket/Display.html?id=71636> > > > Hi there, > > The attached code raises a segfault with perl-5.10.1 and perl-5.14.2. > > Seems fine for other XML parsers such as XML::LibXML::SAX and > XML::TreeBuilder, and in XML::Twig for smaller documents (although > significantly slower than the other parsers). > > Output of script on Linux/x64: > > $ perl xmlbench.pl > Document is 1350062 chars > Benchmark: timing 10 iterations of tree, twig... > tree: 10.3706 wallclock secs (10.09 usr + 0.27 sys = 10.36 CPU) @ > 0.97/s (n=10) > Segmentation fault > > perl -V output: > > Summary of my perl5 (revision 5 version 14 subversion 2) configuration: > > Platform: > osname=linux, osvers=2.6.35-30-generic, archname=x86_64-linux > uname='linux roku 2.6.35-30-generic #56-ubuntu smp mon jul 11 > 20:01:08 utc 2011 x86_64 gnulinux ' > config_args='-de -Dprefix=/home/tom/perl5/perlbrew/perls/perl-5.14.2' > hint=recommended, useposix=true, d_sigaction=define > useithreads=undef, usemultiplicity=undef > useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef > use64bitint=define, use64bitall=define, uselongdouble=undef > usemymalloc=n, bincompat5005=undef > Compiler: > cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector > -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', > optimize='-O2', > cppflags='-fno-strict-aliasing -pipe -fstack-protector > -I/usr/local/include' > ccversion='', gccversion='4.4.5', gccosandvers='' > intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 > d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 > ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', > lseeksize=8 > alignbytes=8, prototype=define > Linker and Libraries: > ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' > libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib > /usr/lib/x86_64-linux-gnu /lib64 /usr/lib64 > libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat > perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc > libc=/lib/libc-2.12.1.so, so=so, useshrplib=false, libperl=libperl.a > gnulibc_version='2.12.1' > Dynamic Linking: > dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' > cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib > -fstack-protector' > > > Characteristics of this binary (from libperl): > Compile-time options: PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP > PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT > USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF > Built under linux > Compiled at Oct 3 2011 02:24:00 > %ENV: > > PERL5LIB="/home/tom/.cpan:/home/tom/.cpan/lib:/home/tom/.cpan/lib/perl5:/home/tom/.cpan/lib/perl5/site_perl:/home/tom/.cpan/lib/perl5/5.10.0/i486-linux-gnu-thread-multi:" > PERLBREW_HOME="/home/tom/.perlbrew" > > PERLBREW_PATH="/home/tom/perl5/perlbrew/bin:/home/tom/perl5/perlbrew/perls/perl-5.14.2/bin" > PERLBREW_PERL="perl-5.14.2" > PERLBREW_ROOT="/home/tom/perl5/perlbrew" > PERLBREW_VERSION="0.29" > PERL_LWP_SSL_VERIFY_HOSTNAME="0" > PERL_MM_USE_DEFAULT="1" > @INC: > /home/tom/.cpan > /home/tom/.cpan/lib > /home/tom/.cpan/lib/perl5 > /home/tom/.cpan/lib/perl5/site_perl > /home/tom/.cpan/lib/perl5/5.10.0/i486-linux-gnu-thread-multi > > /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux > /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2 > /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2/x86_64-linux > /home/tom/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2 > > cheers, > > Tom
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-2597-1318494043-1160.71636-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <RT-Ticket-71636 [...] rt.cpan.org> <rt-3.8.HEAD-2604-1318450306-1798.71636-4-0 [...] rt.cpan.org> <4E969F39.6010205 [...] gmail.com> <rt-3.8.HEAD-2597-1318494043-1160.71636-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-6889-1323096332-38.71636-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
From: vcizek [...] suse.cz
X-RT-Original-Encoding: utf-8
Content-Length: 1493
Download (untitled) / with headers
text/plain 1.4k
On Thu Oct 13 04:20:43 2011, xmltwig@gmail.com wrote: Show quoted text
> Interesting. > > The problem kicks in around 18055 elements for me (32-bit perl, gobs of > memory on the machine). The trigger is not always the same BTW, 18054 > may or may not cause it. The number of children seems to be the only > important factor BTW, adding sub elements within the children does not > change the trigger. >
On my 64bit machine the treshold is around 18695. Show quoted text
> You don't need a loop, the segfault happens during the DESTROY phase. > > The minimum test case I have found is below. > > > #!/usr/bin/perl > use strict; > use warnings; > > use XML::Twig; > > my $xml = q{<root>} . q{<child><name>test name</name></child>} x 20000 . > q{</root>}; > warn "Document is " . length($xml) . " chars\n"; > > my $parsed = XML::Twig->new->parse($xml); > my $t= XML::Twig->new->parse( '<d/>'); > warn "done\n"; > exit; > > > The problem occurs _after_ "done" is printed. So it's in the DESTROY > phase, and it doesn't happen unless the second parse is done.
I can reproduce it without the second parse. Show quoted text
> It also doesn't happen when I purge the twig as I create it (ie if I add > a handler on child that does $t->purge. > > So at this point my guess is that either I messed up the cleanup of > XML::Parser related objects, or its a bug in the layers below, either > XML::Parser or perl itself. >
A workaround that works for me is forcing the module not to use weak references. Eg. setting weakrefs = 0.
From xmltwig [...] gmail.com Tue Dec 6 13: 34:23 2011
MIME-Version: 1.0
X-Spam-Status: No, score=-6.472 tagged_above=-99.9 required=10 tests=[AWL=-0.253, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HK_RANDOM_ENVFROM=0.001, RCVD_IN_DNSWL_HI=-5, SPF_NEUTRAL=0.779] autolearn=ham
In-Reply-To: <rt-3.8.HEAD-6889-1323096333-1558.71636-5-0 [...] rt.cpan.org>
X-Spam-Flag: NO
References: <RT-Ticket-71636 [...] rt.cpan.org> <rt-3.8.HEAD-2604-1318450306-1798.71636-4-0 [...] rt.cpan.org> <4E969F39.6010205 [...] gmail.com> <rt-3.8.HEAD-2597-1318494043-1160.71636-5-0 [...] rt.cpan.org> <rt-3.8.HEAD-6889-1323096333-1558.71636-5-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <4EDE6007.8090902 [...] gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
X-RT-Original-Encoding: utf-8
X-Spam-Score: -6.472
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] gmail.com
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id 730EC240EE0 for <cpan-bug+XML-Twig [...] hipster.bestpractical.com>; Tue, 6 Dec 2011 13:34:23 -0500 (EST)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TPTlagZVa3sQ for <cpan-bug+XML-Twig [...] hipster.bestpractical.com>; Tue, 6 Dec 2011 13:34:21 -0500 (EST)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id 53B3C240ECE for <bug-XML-Twig [...] rt.cpan.org>; Tue, 6 Dec 2011 13:34:21 -0500 (EST)
Received: (qmail 4916 invoked by uid 103); 6 Dec 2011 18:34:20 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 6 Dec 2011 18:34:20 -0000
Received: from mail-ee0-f50.google.com (HELO mail-ee0-f50.google.com) (74.125.83.50) by 16.mx.develooper.com (qpsmtpd/0.80/v0.80-19-gf52d165) with ESMTP; Tue, 06 Dec 2011 10:34:18 -0800
Received: by eeaq15 with SMTP id q15so2688425eea.9 for <bug-XML-Twig [...] rt.cpan.org>; Tue, 06 Dec 2011 10:34:15 -0800 (PST)
Received: by 10.14.1.2 with SMTP id 2mr2951238eec.21.1323196455080; Tue, 06 Dec 2011 10:34:15 -0800 (PST)
Received: from [192.168.2.200] (static-217-133-18-84.clienti.tiscali.it. [217.133.18.84]) by mx.google.com with ESMTPS id 17sm62686593eej.3.2011.12.06.10.34.11 (version=SSLv3 cipher=OTHER); Tue, 06 Dec 2011 10:34:12 -0800 (PST)
Delivered-To: cpan-bug+XML-Twig [...] hipster.bestpractical.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1
Subject: Re: [rt.cpan.org #71636] Segfault with medium-sized document (30k elements)
Return-Path: <xmltwig [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=fp9cOmfKtHX0rZNG3OSaCq3WiZreaKGgZKNx6Slt+3M=; b=L2R5nZJUJYxVaRRYQwR7/lFkEzyfBO+GpgLGoP0AeQoFn2rgn3EuOdd1S8MES20rLz xTp5LOo1YXLHVUzB4y8pSHJH74b8pdrqRp584dGrEPtd9cmSLvx/c0bKJo2vA10scBVY VxXpNmo24zTz7Zi7NwJBsGfyCnm40By1NeDs4=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: cpan-bug+XML-Twig [...] hipster.bestpractical.com
X-RT-Mail-Extension: xml-twig
Date: Tue, 06 Dec 2011 19:33:43 +0100
X-Spam-Level:
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: mirod <xmltwig [...] gmail.com>
RT-Message-ID: <rt-3.8.HEAD-6893-1323196464-1710.71636-0-0 [...] rt.cpan.org>
Content-Length: 1826
Download (untitled) / with headers
text/plain 1.7k
On 12/05/2011 03:45 PM, Vita Cizek via RT wrote: Show quoted text
> Queue: XML-Twig > Ticket<URL: https://rt.cpan.org/Ticket/Display.html?id=71636> > > On Thu Oct 13 04:20:43 2011, xmltwig@gmail.com wrote:
>> Interesting. >> >> The problem kicks in around 18055 elements for me (32-bit perl, gobs of >> memory on the machine). The trigger is not always the same BTW, 18054 >> may or may not cause it. The number of children seems to be the only >> important factor BTW, adding sub elements within the children does not >> change the trigger. >>
> > On my 64bit machine the treshold is around 18695. >
>> You don't need a loop, the segfault happens during the DESTROY phase. >> >> The minimum test case I have found is below. >> >> >> #!/usr/bin/perl >> use strict; >> use warnings; >> >> use XML::Twig; >> >> my $xml = q{<root>} . q{<child><name>test name</name></child>} x 20000 . >> q{</root>}; >> warn "Document is " . length($xml) . " chars\n"; >> >> my $parsed = XML::Twig->new->parse($xml); >> my $t= XML::Twig->new->parse( '<d/>'); >> warn "done\n"; >> exit; >> >> >> The problem occurs _after_ "done" is printed. So it's in the DESTROY >> phase, and it doesn't happen unless the second parse is done.
> > I can reproduce it without the second parse. >
>> It also doesn't happen when I purge the twig as I create it (ie if I add >> a handler on child that does $t->purge. >> >> So at this point my guess is that either I messed up the cleanup of >> XML::Parser related objects, or its a bug in the layers below, either >> XML::Parser or perl itself. >>
> > A workaround that works for me is forcing the module not to use > weak references. Eg. setting weakrefs = 0. >
Indeed this would work, since I believe the problem comes from a bug in weaken The bug actually seems to be fixed in 5.15.5, so there is hope. -- mirod
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-6893-1323196464-1710.71636-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <RT-Ticket-71636 [...] rt.cpan.org> <rt-3.8.HEAD-2604-1318450306-1798.71636-4-0 [...] rt.cpan.org> <4E969F39.6010205 [...] gmail.com> <rt-3.8.HEAD-2597-1318494043-1160.71636-5-0 [...] rt.cpan.org> <rt-3.8.HEAD-6889-1323096333-1558.71636-5-0 [...] rt.cpan.org> <4EDE6007.8090902 [...] gmail.com> <rt-3.8.HEAD-6893-1323196464-1710.71636-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-6890-1323209151-29.71636-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 2507
Download (untitled) / with headers
text/plain 2.4k
On Tue Dec 06 13:34:24 2011, xmltwig@gmail.com wrote: Show quoted text
> On 12/05/2011 03:45 PM, Vita Cizek via RT wrote:
> > Queue: XML-Twig > > Ticket<URL: https://rt.cpan.org/Ticket/Display.html?id=71636> > > > > On Thu Oct 13 04:20:43 2011, xmltwig@gmail.com wrote:
> >> Interesting. > >> > >> The problem kicks in around 18055 elements for me (32-bit perl, gobs of > >> memory on the machine). The trigger is not always the same BTW, 18054 > >> may or may not cause it. The number of children seems to be the only > >> important factor BTW, adding sub elements within the children does not > >> change the trigger. > >>
> > > > On my 64bit machine the treshold is around 18695. > >
> >> You don't need a loop, the segfault happens during the DESTROY phase. > >> > >> The minimum test case I have found is below. > >> > >> > >> #!/usr/bin/perl > >> use strict; > >> use warnings; > >> > >> use XML::Twig; > >> > >> my $xml = q{<root>} . q{<child><name>test name</name></child>} x 20000 . > >> q{</root>}; > >> warn "Document is " . length($xml) . " chars\n"; > >> > >> my $parsed = XML::Twig->new->parse($xml); > >> my $t= XML::Twig->new->parse( '<d/>'); > >> warn "done\n"; > >> exit; > >> > >> > >> The problem occurs _after_ "done" is printed. So it's in the DESTROY > >> phase, and it doesn't happen unless the second parse is done.
> > > > I can reproduce it without the second parse. > >
> >> It also doesn't happen when I purge the twig as I create it (ie if I add > >> a handler on child that does $t->purge. > >> > >> So at this point my guess is that either I messed up the cleanup of > >> XML::Parser related objects, or its a bug in the layers below, either > >> XML::Parser or perl itself. > >>
> > > > A workaround that works for me is forcing the module not to use > > weak references. Eg. setting weakrefs = 0. > >
> > Indeed this would work, since I believe the problem comes from a bug in > weaken > > The bug actually seems to be fixed in 5.15.5, so there is hope.
Do the nodes an an XML::Twig class have references to their siblings? If so, this may be related to <https://rt.perl.org/rt3/Ticket/Display.html?id=44225>, in which case turning off weak references only works by chance. If that is the case, you could create a DESTROY method to break links between sub-nodes or even free them one-by-one, starting with the innermost, to work around the problem in 5.14 and earlier. This is just a guess off the top of my head, without actually looking into the problem very far.
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-15875-1352803298-1032.71636-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 44
This is a Perl bug, fixed in 5.16+ __ mirod


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.