Skip Menu |
 

This queue is for tickets about the XML-Twig CPAN distribution.

Report information
The Basics
Id: 41594
Status: resolved
Priority: 0/
Queue: XML-Twig

People
Owner: Nobody in particular
Requestors: henridamien.laurent [...] biblibre.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 50F1C19B8186 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 11:03:37 -0500 (EST)
Received: (qmail 10724 invoked by uid 103); 10 Dec 2008 16:03:36 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 10 Dec 2008 16:03:36 -0000
Received: from mailhost-e3-p0.nerim-networks.com (HELO paquito.nerim.net) (195.5.209.17) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Wed, 10 Dec 2008 08:03:27 -0800
Received: from kraid.nerim.net (smtp-103-wednesday.nerim.net [62.4.16.103]) by paquito.nerim.net (Postfix) with ESMTP id 59F6FB701CC for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 15:48:16 +0100 (CET)
Received: from [192.168.1.66] (hdlaurent.pck.nerim.net [62.212.120.129]) by kraid.nerim.net (Postfix) with ESMTP id B9ABACFB3C for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 16:48:29 +0100 (CET)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
Subject: Segmentation fault at the very end of a script
MIME-Version: 1.0
User-Agent: Thunderbird 2.0.0.18 (X11/20081120)
X-Spam-Status: No, hits=0.0 required=8.0 tests=
Return-Path: <henridamien.laurent [...] biblibre.com>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Wed, 10 Dec 2008 16:48:29 +0100
X-Spam-Level: *
Message-Id: <493FE4CD.3060508 [...] biblibre.com>
Content-Type: text/plain; charset=UTF-8
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: "henridamien.laurent" <henridamien.laurent [...] biblibre.com>
X-RT-Original-Encoding: utf-8
Content-Length: 1993
Download (untitled) / with headers
text/plain 1.9k
Hi I am working on different Linux machines, XML::Twig is quite impressive, but I happen to have a seg fault problem with a script of mine : It occurs at the very end of the script, so process is OK, but it still is quite uncomfortable. And I would like to find out why it is so. XML file is quite big : 12Mo But I think it should be OK. is there something I should know ? Some thing I should do to provide you with enough information ? #!/usr/bin/perl use XML::Twig; use XML::Twig::XPath; use Data::Dumper; use Getopt::Long; use utf8; use strict; # buffers for holding text my %concepthash; my %mthash; my %BT; my %NT; my ($filename,$force); GetOptions( 'file:s' => \$filename, 'f' => \$force, ); # initialize parser with handlers for node processing my $twig = new XML::Twig::XPath( TwigHandlers => { "/th/langue/record" => \&concepthash, "/th/langue/mt" => \&mthash, }); # parse, handling nodes on the way $twig->parsefile( $filename ); my %mtdiff; my @nodeset = $twig->get_xpath('/th/langue[@lang-id="fre"]/record'); my (%modelem, %createelem, %newelem, %unmodifiedelem); CONCEPT :foreach my $elem (@nodeset) { # Construction de l'enregistrement candidat my $id= $elem->att( 'id' ); #process process process ..... #uses hashes } my %BT; my %NT; # initialize parser with handlers for node processing my @nodeset = $twig->get_xpath('/th/langue[@lang-id!="fre"]/record'); foreach my $elem (@nodeset){ my $id= $elem->att( 'id' ); # Process Process Process #uses hashes } $twig->dispose; # handle a concept element to build the concepts hash. sub concepthash { my( $tree, $elem ) = @_; # Process Process Process #creates a concept hash } sub mthash { my( $tree, $elem ) = @_; # Process Process Process #creates an mt hash } -- Henri-Damien LAURENT BibLibre SARL http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc tel : +33 4 67 65 75 50
MIME-Version: 1.0
X-Spam-Status: No, hits=0.0 required=8.0 tests=DK_SIGNED,SPF_PASS
In-Reply-To: <rt-3.6.HEAD-22019-1228925046-63.41594-4-0 [...] rt.cpan.org>
References: <RT-Ticket-41594 [...] rt.cpan.org> <493FE4CD.3060508 [...] biblibre.com> <rt-3.6.HEAD-22019-1228925046-63.41594-4-0 [...] rt.cpan.org>
Reply-To: xmltwig [...] gmail.com
Content-Type: text/plain; charset=UTF-8; format=flowed
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 9C6924D8015 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 11:34:26 -0500 (EST)
Received: (qmail 22464 invoked by uid 103); 10 Dec 2008 16:34:26 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 10 Dec 2008 16:34:26 -0000
Received: from nf-out-0910.google.com (HELO nf-out-0910.google.com) (64.233.182.190) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Wed, 10 Dec 2008 08:34:11 -0800
Received: by nf-out-0910.google.com with SMTP id h3so120672nfh.33 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 08:34:08 -0800 (PST)
Received: by 10.210.88.3 with SMTP id l3mr1866781ebb.88.1228926848174; Wed, 10 Dec 2008 08:34:08 -0800 (PST)
Received: from ?192.168.2.200? (static-217-133-3-216.clienti.tiscali.it [217.133.3.216]) by mx.google.com with ESMTPS id d26sm3934562nfh.42.2008.12.10.08.34.02 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 10 Dec 2008 08:34:06 -0800 (PST)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #41594] Segmentation fault at the very end of a script
User-Agent: Thunderbird 2.0.0.18 (X11/20081105)
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; b=nN4iNabqm89HAHDK4xV9FN3J7YQmoQMy18Hw6baHE9GxzbjY+2l/dXEHT4ZtzOPphQ d8jx2OjUlfSLGjb7th0TeDH+S9GN+oK2MCvzG0fz7lz8b2p0VDg9Ge4WCvIL1LC44vxI i7NGdDrPArgk6amQsY7ZOeNg/cy6cDjOMTqAg=
Return-Path: <xmltwig [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=FYrG5C1wsp9h1ZicuqnQgD/ObxdD+RSWmsSaMw6ZvWg=; b=SQ4taLgpdxgjuU2XbtMIbcdUwatnjFTFtZdgFnH53mjmY4EjIoasw0/VBW71iMVHDQ mskiNqGMLqM7vZDT9Kitr5Vp7+0lVrmjHMEHP8sAOfv3H3Y94UTy8vfIdPvgofUxc41H 5gWIAhdlJP83YtPh6EaGT3DfdDiB0CNoaTt3M=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Wed, 10 Dec 2008 17:34:01 +0100
X-Spam-Level: *
Message-Id: <493FEF79.30408 [...] gmail.com>
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: mirod <xmltwig [...] gmail.com>
RT-Message-ID: <rt-3.6.HEAD-22019-1228926898-1716.41594-0-0 [...] rt.cpan.org>
Content-Length: 502
Download (untitled) / with headers
text/plain 502b
A few things you could try: - not call dispose, it shouldn't be needed - try not using XML::Twig::XPath, I believe the XPath expression you use is supported by the native XPath-lite engine Which versions of perl and of XML::Twig are you using? Note that in quite a few cases dying is much faster than exiting the program properly (you don't spend time freeing lots of small bits of memory in an orderly fashion... just to free all of the process space as soon as you're done). Thanks -- mirod
MIME-Version: 1.0
X-Spam-Status: No, hits=0.0 required=8.0 tests=
In-Reply-To: <rt-3.6.HEAD-22019-1228926898-1716.41594-6-0 [...] rt.cpan.org>
References: <RT-Ticket-41594 [...] rt.cpan.org> <493FE4CD.3060508 [...] biblibre.com> <rt-3.6.HEAD-22019-1228925046-63.41594-4-0 [...] rt.cpan.org> <493FEF79.30408 [...] gmail.com> <rt-3.6.HEAD-22019-1228926898-1716.41594-6-0 [...] rt.cpan.org>
Content-Type: text/plain; charset=UTF-8
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 2622C23C0E7 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 11:51:05 -0500 (EST)
Received: (qmail 32688 invoked by uid 103); 10 Dec 2008 16:51:05 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 10 Dec 2008 16:51:05 -0000
Received: from smtp-103-wednesday.nerim.net (HELO kraid.nerim.net) (62.4.16.103) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Wed, 10 Dec 2008 08:50:22 -0800
Received: from [192.168.1.66] (hdlaurent.pck.nerim.net [62.212.120.129]) by kraid.nerim.net (Postfix) with ESMTP id 8B956CFED6 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 17:50:18 +0100 (CET)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #41594] Segmentation fault at the very end of a script
User-Agent: Thunderbird 2.0.0.18 (X11/20081120)
Return-Path: <henridamien.laurent [...] biblibre.com>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Wed, 10 Dec 2008 17:50:18 +0100
X-Spam-Level: *
Message-Id: <493FF34A.8030401 [...] biblibre.com>
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 8bit
From: "henridamien.laurent" <henridamien.laurent [...] biblibre.com>
RT-Message-ID: <rt-3.6.HEAD-22019-1228927880-1871.41594-0-0 [...] rt.cpan.org>
Content-Length: 939
Download (untitled) / with headers
text/plain 939b
xmltwig@gmail.com via RT a écrit : Show quoted text
> <URL: http://rt.cpan.org/Ticket/Display.html?id=41594 > > > A few things you could try: > > - not call dispose, it shouldn't be needed > - try not using XML::Twig::XPath, I believe the XPath expression you use is > supported by the native XPath-lite engine > > Which versions of perl and of XML::Twig are you using? > > Note that in quite a few cases dying is much faster than exiting the program > properly (you don't spend time freeing lots of small bits of memory in an > orderly fashion... just to free all of the process space as soon as you're done). > > Thanks > >
XML::Twig version is 3.32 Perl version is 5.10 but it occured with 5.8.8. I used dispose to get rid of this seg fault, without success. Can XML::Twig::XPath be the source of the problem ? -- Henri-Damien LAURENT BibLibre SARL http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc tel : +33 4 67 65 75 50
MIME-Version: 1.0
X-Spam-Status: No, hits=0.0 required=8.0 tests=DK_SIGNED,SPF_PASS
In-Reply-To: <rt-3.6.HEAD-22019-1228927880-1871.41594-5-0 [...] rt.cpan.org>
References: <RT-Ticket-41594 [...] rt.cpan.org> <493FE4CD.3060508 [...] biblibre.com> <rt-3.6.HEAD-22019-1228925046-63.41594-4-0 [...] rt.cpan.org> <493FEF79.30408 [...] gmail.com> <rt-3.6.HEAD-22019-1228926898-1716.41594-6-0 [...] rt.cpan.org> <493FF34A.8030401 [...] biblibre.com> <rt-3.6.HEAD-22019-1228927880-1871.41594-5-0 [...] rt.cpan.org>
Reply-To: xmltwig [...] gmail.com
Content-Type: text/plain; charset=UTF-8; format=flowed
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id CFD9A4D8015 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 12:54:06 -0500 (EST)
Received: (qmail 25509 invoked by uid 103); 10 Dec 2008 17:54:05 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 10 Dec 2008 17:54:05 -0000
Received: from mail-ew0-f16.google.com (HELO mail-ew0-f16.google.com) (209.85.219.16) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Wed, 10 Dec 2008 09:53:24 -0800
Received: by ewy9 with SMTP id 9so825720ewy.21 for <bug-XML-Twig [...] rt.cpan.org>; Wed, 10 Dec 2008 09:53:20 -0800 (PST)
Received: by 10.210.25.19 with SMTP id 19mr1963997eby.114.1228931599655; Wed, 10 Dec 2008 09:53:19 -0800 (PST)
Received: from ?192.168.2.200? (static-217-133-3-216.clienti.tiscali.it [217.133.3.216]) by mx.google.com with ESMTPS id i7sm2780413nfh.13.2008.12.10.09.53.17 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 10 Dec 2008 09:53:18 -0800 (PST)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #41594] Segmentation fault at the very end of a script
User-Agent: Thunderbird 2.0.0.18 (X11/20081105)
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; b=g7rqdilux26cFLwaan0hSsATcY35WeFqRvr1Ek9EGuieOXJh7x0VIav9ymAnHdAEZw kqSTSELXAxswHeEin6uBXtQr1dO48aVh+cfEuMMpP3EA7oQkcFgZRheX/FTdl0787HTZ NKBrEzJAIvLlJ4fYPzOdO4uujWdpQ1f+J1IeA=
Return-Path: <xmltwig [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=RSWShMkhGTFDziZTh1NBJXRWg6h/BmpDcUzAWoWd6LQ=; b=Qp6SlprcFnGRBLIVpcW7D3YZdcLSvj/xUxqbO7F2aOhnTzvmHGj6p0qzjkthiLm/bY tgFMV37CR83rPr7a8hQhPWZZZZeylEKUxvWF+K5FATgNKC3DMoO1K2tb69kVbtMClvEl L59trsGWO49mgtr4hF7MWPTLDCbNYFyd4H3MU=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Wed, 10 Dec 2008 18:53:11 +0100
X-Spam-Level: *
Message-Id: <49400207.9090908 [...] gmail.com>
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: mirod <xmltwig [...] gmail.com>
RT-Message-ID: <rt-3.6.HEAD-22009-1228931700-1856.41594-0-0 [...] rt.cpan.org>
Content-Length: 424
Download (untitled) / with headers
text/plain 424b
henridamien.laurent via RT wrote: Show quoted text
> XML::Twig version is 3.32 > Perl version is 5.10 but it occured with 5.8.8.
Show quoted text
> I used dispose to get rid of this seg fault, without success.
I kinda suspected that ;--( Show quoted text
> Can XML::Twig::XPath be the source of the problem ?
I don't know, I am just trying to narrow down the problem. OK, could you send me a (small!) file that shows the problem, that would help. Thanks. -- mirod
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id D52CB63C11E for <bug-XML-Twig [...] rt.cpan.org>; Fri, 16 Jan 2009 12:17:51 -0500 (EST)
Received: (qmail 4998 invoked by uid 103); 16 Jan 2009 17:17:50 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 16 Jan 2009 17:17:50 -0000
Received: from smtp2.tech.numericable.fr (HELO smtp2.tech.numericable.fr) (82.216.111.38) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Fri, 16 Jan 2009 09:17:49 -0800
Received: from [192.168.0.12] (abo-65-182-68.mtp.modulonet.fr [85.68.182.65]) by smtp2.tech.numericable.fr (Postfix) with ESMTP id 2FCD918D82E for <bug-XML-Twig [...] rt.cpan.org>; Fri, 16 Jan 2009 18:17:06 +0100 (CET)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
User-Agent: Thunderbird 2.0.0.19 (X11/20090105)
MIME-Version: 1.0
Subject: Re: [rt.cpan.org #41594] Segmentation fault at the very end of a script
Return-Path: <henridamien [...] koha-fr.org>
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Fri, 16 Jan 2009 18:17:08 +0100
Message-ID: <4970C114.7030902 [...] koha-fr.org>
Content-Type: multipart/mixed; boundary="------------040300000601040600040806"
To: bug-XML-Twig [...] rt.cpan.org
From: LAURENT Henri-Damien <henridamien [...] koha-fr.org>
RT-Message-ID: <rt-3.6.HEAD-29719-1232126289-280.41594-0-0 [...] rt.cpan.org>
Content-Length: 0
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: 7bit
X-RT-Original-Encoding: utf-8
Content-Length: 317
Download (untitled) / with headers
text/plain 317b
Data file is quite big. But otherwise : process as such : see int_test.pl I sent data along. launch perl int_test.pl data.xml Is there something I can doo about this segmentation fault problem ? It does not even process all the file data.xml.bz2 as I supposed. Thanks for you quick answer. -- Henri-Damien LAURENT
content-type: application/x-bzip; name="data.xml.bz2"
content-disposition: inline; filename="data.xml.bz2"
Content-Transfer-Encoding: base64
Content-Length: 620109
Download data.xml.bz2
application/x-bzip 605.5k

Message body not shown because it is not plain text.

content-type: application/x-perl; name="int_test.pl"
content-disposition: inline; filename="int_test.pl"
Content-Transfer-Encoding: 8bit
Content-Length: 3734
Download int_test.pl
text/x-perl 3.6k
#!/usr/bin/perl use XML::Twig; use XML::Twig::XPath; # use Unicode::String qw(utf8 latin1); use MARC::Record; use Data::Dumper; use Date::Manip; use Getopt::Long; use utf8; use strict; # buffers for holding text my %concepthash; my %mthash; my %BT; my %NT; my ($filename,$force); GetOptions( 'file:s' => \$filename, 'f' => \$force, ); # initialize parser with handlers for node processing my $twig = new XML::Twig::XPath( TwigHandlers => { "/thesaurusMultilingue/langue/thesaurus/concept" => \&concepthash, "/thesaurusMultilingue/langue/thesaurus/microthesaurus" => \&mthash, }); # parse, handling nodes on the way $twig->parsefile( $filename ); # use Data::Dumper; warn "base : ".$dbh->{Name}; #Construction du hachage de concept. #Il faut le faire AVANT de proceder a la reconnaissance des relations. my %mtdiff; my @nodeset = $twig->get_xpath('/thesaurusMultilingue/langue[@lang-id="fre"]/thesaurus/concept'); my (%modelem, %createelem, %newelem, %unmodifiedelem); CONCEPT :foreach my $elem (@nodeset) { my $id= $elem->att( 'id' ); my $create = $elem->trimmed_field( 'dateCreation' ); my $modify = $elem->trimmed_field( 'dateModification' ); $create =~s/-//g; $modify =~s/-//g; my $form= $elem->trimmed_field( 'term' ); my $note = $elem->trimmed_field( 'noteApplication' ); #search for an existing record in thesaurus my @relations = $elem->descendants('relation'); my $nok; my %tagfields=( "UF"=>'450', "RT"=>'550', "BT"=>'550', "NT"=>'550', ); my %relations=( "BT"=>"g", "NT"=>"h", ); my @fields; foreach my $relation (@relations){ if ($relation->att('type')=~/MT/){ $concepthash{$id}->{'MT'} = $relation->att('ref'); } elsif ($relation->att('type') =~/UF|RT|BT|NT/){ push @fields,($tagfields{$relation->att('type')}, "2"=>"".$mthash{$concepthash{$relation->att( 'ref' )}->{'MT'}}->{'fre'}, "3"=>$relation->att( 'ref' ), "a"=>$concepthash{$relation->att( 'ref' )}->{'fre'}); } elsif ($relation->att('type') =~ /USE/){ next CONCEPT; } } ################################### Fin Construction record autorit� # Ajouter l'arbre ici my $trees=GetParents($id); my $hierarchies; $hierarchies=join (';',map{join(',',@{$_})} @$trees); $modelem{"$id"}=$form; my $hierarchies; $hierarchies=join (';',map{join(',',@$_)} @$trees); print "$id:$hierarchies", join("\n\t",@fields),"\n\n"; } sub GetParents{ my ($id)= @_; if (defined $concepthash{$id}->{'parents'}){ my @parents=map{my $ancestors=GetParents($_); map{[@{$_},$id]} @$ancestors} keys %{$concepthash{$id}->{'parents'}}; return \@parents; } else { return [[$id]]; } } # handle a concept element to build the concepts hash. sub concepthash { my( $tree, $elem ) = @_; # utf8::decode($elem->trimmed_field( 'term' )); $concepthash{$elem->att( 'id' )}->{$elem->parent('langue')->att( 'lang-id' )}=$elem->trimmed_field( 'term' ); if ($elem->first_descendant('relation[@type="MT"]')){ $concepthash{$elem->att( 'id' )}->{'MT'} = $elem->first_descendant('relation[@type="MT"]')->att('ref') ; } else { warn $elem->att( 'id' )." Pas de microthésaurus attaché"; } map { $concepthash{$elem->att( 'id' )}->{'parents'}->{$_->att('ref')}=1} $elem->descendants('relation[@type="BT"]') if ($elem->descendants('relation[@type="BT"]')); } sub mthash { my( $tree, $elem ) = @_; # utf8::decode($elem->trimmed_field( 'mt-name' )); $mthash{$elem->att( 'mt-id' )}->{$elem->parent('langue')->att( 'lang-id' )}=$elem->trimmed_field( 'mt-name' ); }
MIME-Version: 1.0
X-Spam-Status: No, hits=0.0 required=8.0 tests=DK_SIGNED,SPF_PASS
In-Reply-To: <rt-3.6.HEAD-29719-1232126289-280.41594-5-0 [...] rt.cpan.org>
References: <RT-Ticket-41594 [...] rt.cpan.org> <4970C114.7030902 [...] koha-fr.org> <rt-3.6.HEAD-29719-1232126289-280.41594-5-0 [...] rt.cpan.org>
Message-ID: <4971AA3E.2040909 [...] gmail.com>
Reply-To: xmltwig [...] gmail.com
Content-Type: text/plain; charset=UTF-8; format=flowed
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 937764D806A for <bug-XML-Twig [...] rt.cpan.org>; Sat, 17 Jan 2009 04:52:05 -0500 (EST)
Received: (qmail 30702 invoked by uid 103); 17 Jan 2009 09:52:04 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 17 Jan 2009 09:52:04 -0000
Received: from mail-ew0-f16.google.com (HELO mail-ew0-f16.google.com) (209.85.219.16) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Sat, 17 Jan 2009 01:51:58 -0800
Received: by ewy9 with SMTP id 9so240953ewy.21 for <bug-XML-Twig [...] rt.cpan.org>; Sat, 17 Jan 2009 01:51:32 -0800 (PST)
Received: by 10.210.82.7 with SMTP id f7mr1148487ebb.93.1232185892547; Sat, 17 Jan 2009 01:51:32 -0800 (PST)
Received: from ?192.168.2.200? ([217.133.3.216]) by mx.google.com with ESMTPS id k5sm4205555nfh.12.2009.01.17.01.51.31 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 17 Jan 2009 01:51:32 -0800 (PST)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #41594] Segmentation fault at the very end of a script
User-Agent: Thunderbird 2.0.0.19 (X11/20081209)
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; b=jmNmdq7kTCY+8v1t3gkfLjk4zoYPYVJ5lzIERBPGcPfe5rUPbn9p/crgHiPnCRJVXQ a9KHKvdRyd7X9GnnmwNA/RNkZnp7R1zn1FwaV7c3byr0o1Uaytfx/qW+SiuK9ys3b0NP qAX+0RrWWI40ZFNfTLGl+3SG4ncSawEF6DJF8=
Return-Path: <xmltwig [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=+tgLvqN68wUqbGc3jwpY5B4Iz8a8v+NaLPZ8GLaix8Q=; b=LRPIZIk0uqPh/bHwPuy365GhPUpVf+ouQ+Gj/uudNrORBrUlgUGquznwuLIA0cEgqf nfXmQC6C1rwjSJ3uhdQuH8n8pGfL9eKExNDXg/guCpouypBVQ2ylIJdV40rS601MeZUN Nj7LoHr+1AUP+xp1xhKOlkf/qzJJobU0+gO1E=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Sat, 17 Jan 2009 10:51:58 +0100
X-Spam-Level: *
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: mirod <xmltwig [...] gmail.com>
RT-Message-ID: <rt-3.6.HEAD-29719-1232185933-1845.41594-0-0 [...] rt.cpan.org>
Content-Length: 841
Download (untitled) / with headers
text/plain 841b
LAURENT Henri-Damien via RT wrote: Show quoted text
> Queue: XML-Twig > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=41594 > > > Data file is quite big. > But otherwise : process as such : > see int_test.pl > I sent data along. > launch perl int_test.pl data.xml > Is there something I can doo about this segmentation fault problem ? > It does not even process all the file data.xml.bz2 as I supposed. > Thanks for you quick answer.
Well, the data is just too big. A simple test like perl -MXML::Twig -e'XML::Twig->new->parsefile( "data.xml");' would have shown it (and saved me some time, as I assumed you had tested that, and sent me the simplest test that generated the error). Do you need to load the entire tree in memory, or could you purge it, maybe after each concept, I can't tell from a quick look at your code? -- mirod
MIME-Version: 1.0
X-Spam-Status: No, hits=0.0 required=8.0 tests=
In-Reply-To: <rt-3.6.HEAD-29719-1232185933-1845.41594-6-0 [...] rt.cpan.org>
References: <RT-Ticket-41594 [...] rt.cpan.org> <4970C114.7030902 [...] koha-fr.org> <rt-3.6.HEAD-29719-1232126289-280.41594-5-0 [...] rt.cpan.org> <4971AA3E.2040909 [...] gmail.com> <rt-3.6.HEAD-29719-1232185933-1845.41594-6-0 [...] rt.cpan.org>
Message-ID: <497445FF.8060904 [...] biblibre.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 335C64D8040 for <bug-XML-Twig [...] rt.cpan.org>; Mon, 19 Jan 2009 04:21:18 -0500 (EST)
Received: (qmail 23621 invoked by uid 103); 19 Jan 2009 09:21:17 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 19 Jan 2009 09:21:17 -0000
Received: from smtp2.tech.numericable.fr (HELO smtp2.tech.numericable.fr) (82.216.111.38) by 16.mx.develooper.com (qpsmtpd/0.43rc1) with ESMTP; Mon, 19 Jan 2009 01:21:07 -0800
Received: from [192.168.0.12] (abo-65-182-68.mtp.modulonet.fr [85.68.182.65]) by smtp2.tech.numericable.fr (Postfix) with ESMTP id 6286D18D84C for <bug-XML-Twig [...] rt.cpan.org>; Mon, 19 Jan 2009 10:21:03 +0100 (CET)
Delivered-To: cpan-bug+XML-Twig [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #41594] Segmentation fault at the very end of a script
User-Agent: Thunderbird 2.0.0.19 (X11/20090105)
Return-Path: <henridamien.laurent [...] biblibre.com>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-XML-Twig [...] rt.cpan.org
Date: Mon, 19 Jan 2009 10:21:03 +0100
X-Spam-Level: *
To: bug-XML-Twig [...] rt.cpan.org
Content-Transfer-Encoding: 8bit
From: LAURENT Henri-Damien <henridamien.laurent [...] biblibre.com>
RT-Message-ID: <rt-3.6.HEAD-29719-1232356887-1515.41594-0-0 [...] rt.cpan.org>
Content-Length: 1697
Download (untitled) / with headers
text/plain 1.6k
xmltwig@gmail.com via RT a écrit : Show quoted text
> <URL: http://rt.cpan.org/Ticket/Display.html?id=41594 > > > LAURENT Henri-Damien via RT wrote: >
>> Queue: XML-Twig >> Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=41594 > >> >> Data file is quite big. >> But otherwise : process as such : >> see int_test.pl >> I sent data along. >> launch perl int_test.pl data.xml >> Is there something I can doo about this segmentation fault problem ? >> It does not even process all the file data.xml.bz2 as I supposed. >> Thanks for you quick answer. >>
> > Well, the data is just too big. A simple test like perl -MXML::Twig > -e'XML::Twig->new->parsefile( "data.xml");' would have shown it (and saved me > some time, as I assumed you had tested that, and sent me the simplest test that > generated the error). >
MMM... Sorry for not failing to test this. I just couldnot imagine it was failing parsing the file, since it partly achieved the job. And I believed XML::Twig was designed to process big files. It seems that XML::LibXML can parse this file. Show quoted text
> Do you need to load the entire tree in memory, or could you purge it, maybe > after each concept, I can't tell from a quick look at your code? >
In my opinion, if that file was big for a mail, it is not so big for process. I have to first read the whole file to find relations between nodes and to determine the type of node. The problem is that for each node, I need elements from related nodes and those nodes can be further in the data file, so not processed yet. If I found a solution to do this, would there be a simple way to adapt my code so that after memory is freed after each concept ?* -- Henri-Damien LAURENT


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.