Skip Menu |
 

This queue is for tickets about the XML-Twig CPAN distribution.

Report information
The Basics
Id: 83037
Status: resolved
Priority: 0/
Queue: XML-Twig

People
Owner: Nobody in particular
Requestors: SREZIC [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 3.42
Fixed in: 3.45



Subject: Segfault while creating a large XML::Twig structure
Download (untitled) / with headers
text/plain 2.7k
The attached script segfaults in the destruction phase. The number of elements to be created before the segfault occurs varies, On some systems 20_000 were enough (FreeBSD 9.0/amd64), on some systems 200_000 were necessary (FreeBSD 8.0/i386). The segfault does not seem to appear with perl 5.16.0. Actually, Andreas Koenig did a bisect and he found out that the issue was fixed with the following perl commit (disregard the wording "first bad commit" below, git-bisect is optimized for finding regressions, not improvements :-) 104d7b699c0dfac5bf91d1c05f84b3822c2c3e88 is the first bad commit commit 104d7b699c0dfac5bf91d1c05f84b3822c2c3e88 Author: David Mitchell <davem@iabyn.com> Date: Wed May 11 12:07:14 2011 +0100 make hv freeing iterative rather than recursive make sv_clear() iteratively free the elements of a hash, rather than recursing. This stops freeing deeply nested hash of hash structures from blowing the stack. This commit is relatively straightfoward, now that a) the infrastruure is already in place in sv_clear to iteratively free AVs; b) the long sequence of commits leading up to this has provided us with the hfree_next_entry() function, which returns just the next sv in the hash that needs freeing. When starting to free a new hash, we have to save the old value of iter_sv somewhere; we do this by sticking it in the unused SvSTASH slot of the HV. This slot shouldn't get messed with, since, but this time we've already called the destructor on this object, and we should have a refcount of zero, so no destructor should be able to see us to rebless us. Ideally we'd like to be able to save the old index into HvARRAY when freeing a new HV, but I couldn't think of anywhere to hide it. So we get sub-optimal scanning of the parent's HvARRAY when freeing hashes of hashes. :100644 100644 0d296a4b9b096490f852acfd4acd3abd4f6fe04d 3bd3e6e89305384700d934418676871880fd54db M hv.c :100644 100644 8a663173bedc3088c7344ccab19a33084e637152 d78f7766e62143ce75023852712611d967ef6d99 M sv.c -------------------------------------------------------------------- This issue looks a little bit like the issue described in https://rt.cpan.org/Ticket/Display.html?id=71636 The significant difference is that here no parsing happens, but maybe under the hood similar things happen. As it looks to be a perl problem, you probably cannot do much about it, but maybe it's worth a sentence in the BUGS section of the documentation? After all, 5.16.x is quite new and many OS vendors have much older perls by default on their system (for example, RedHat 5 has even only perl 5.8.8 installed, and its EOL is in 2017). And you still mention perl 5.6.0 in the BUGS section :-) Regards, Slaven
Subject: xml_twig_segfault.pl
#!/usr/bin/perl -w use strict; use XML::Twig; { my $twig = XML::Twig->new; my $gpx = XML::Twig::Elt->new('gpx'); $twig->set_root($gpx); my $trkxml = XML::Twig::Elt->new("trk"); $trkxml->paste(last_child => $gpx); for (1..100_000) { my $trksegxml = XML::Twig::Elt->new("trkseg"); $trksegxml->paste(last_child => $trkxml); } } __END__
Subject: Re: [rt.cpan.org #83037] Segfault while creating a large XML::Twig structure
Date: Fri, 01 Feb 2013 14:50:39 +0100
To: bug-XML-Twig [...] rt.cpan.org
From: mirod <xmltwig [...] gmail.com>
Download (untitled) / with headers
text/plain 3.4k
Thanks Slaven, I know about this bug, and you're right, it's definitely worth a mention in the docs. I'll put it at the top of the list actually. -- mirod On 01/29/2013 09:44 PM, Slaven_Rezic via RT wrote: Show quoted text
> Tue Jan 29 15:44:23 2013: Request 83037 was acted upon. > Transaction: Ticket created by SREZIC > Queue: XML-Twig > Subject: Segfault while creating a large XML::Twig structure > Broken in: 3.42 > Severity: (no value) > Owner: Nobody > Requestors: SREZIC@cpan.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=83037 > > > > The attached script segfaults in the destruction phase. The number of > elements to be created before the segfault occurs varies, On some > systems 20_000 were enough (FreeBSD 9.0/amd64), on some systems 200_000 > were necessary (FreeBSD 8.0/i386). The segfault does not seem to appear > with perl 5.16.0. Actually, Andreas Koenig did a bisect and he found out > that the issue was fixed with the following perl commit (disregard the > wording "first bad commit" below, git-bisect is optimized for finding > regressions, not improvements :-) > > 104d7b699c0dfac5bf91d1c05f84b3822c2c3e88 is the first bad commit > commit 104d7b699c0dfac5bf91d1c05f84b3822c2c3e88 > Author: David Mitchell <davem@iabyn.com> > Date: Wed May 11 12:07:14 2011 +0100 > > make hv freeing iterative rather than recursive > > make sv_clear() iteratively free the elements of a hash, rather than > recursing. This stops freeing deeply nested hash of hash structures from > blowing the stack. > > This commit is relatively straightfoward, now that > a) the infrastruure is already in place in sv_clear to iteratively > free AVs; > b) the long sequence of commits leading up to this has provided us with > the hfree_next_entry() function, which returns just the next sv > in the > hash that needs freeing. > > When starting to free a new hash, we have to save the old value of > iter_sv > somewhere; we do this by sticking it in the unused SvSTASH slot of > the HV. > This slot shouldn't get messed with, since, but this time we've already > called the destructor on this object, and we should have a refcount of > zero, so no destructor should be able to see us to rebless us. > > Ideally we'd like to be able to save the old index into HvARRAY when > freeing a new HV, but I couldn't think of anywhere to hide it. > So we get sub-optimal scanning of the parent's HvARRAY when freeing > hashes > of hashes. > > :100644 100644 0d296a4b9b096490f852acfd4acd3abd4f6fe04d > 3bd3e6e89305384700d934418676871880fd54db M hv.c > :100644 100644 8a663173bedc3088c7344ccab19a33084e637152 > d78f7766e62143ce75023852712611d967ef6d99 M sv.c > > -------------------------------------------------------------------- > > This issue looks a little bit like the issue described in > https://rt.cpan.org/Ticket/Display.html?id=71636 > The significant difference is that here no parsing happens, but maybe > under the hood similar things happen. > > As it looks to be a perl problem, you probably cannot do much about it, > but maybe it's worth a sentence in the BUGS section of the > documentation? After all, 5.16.x is quite new and many OS vendors have > much older perls by default on their system (for example, RedHat 5 has > even only perl 5.8.8 installed, and its EOL is in 2017). And you still > mention perl 5.6.0 in the BUGS section :-) > > Regards, > Slaven >


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.