Skip Menu |
 

This queue is for tickets about the PAR-Packer CPAN distribution.

Report information
The Basics
Id: 69848
Status: open
Priority: 0/
Queue: PAR-Packer

People
Owner: Nobody in particular
Requestors: frank.wang [...] validusresearch.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: pp created exe crash if useing parallel loops
Date: Wed, 27 Jul 2011 17:41:11 -0400
To: "'bug-par [...] rt.cpan.org'" <bug-par [...] rt.cpan.org>
From: "Wang, Frank" <frank.wang [...] validusresearch.com>
Download (untitled) / with headers
text/plain 1.6k
Hi Mr. Mueller, When using pp to create a stanalone executable in activeperl for x64, if the source code used Parallel::Loops, the executable will crash and the following is the problem detail: Problem signature: Problem Event Name: APPCRASH Application Name: pre-process-elt-x64.exe Application Version: 0.0.0.0 Application Timestamp: 4e3026dd Fault Module Name: perl512.dll Fault Module Version: 5.12.2.1203 Fault Module Timestamp: 4d009fe0 Exception Code: c0000005 Exception Offset: 000000000013eb35 OS Version: 6.0.6002.2.2.0.18.10 Locale ID: 1033 Additional Information 1: 98c4 Additional Information 2: adc3265ea6226a0345ffdbdc4025017e Additional Information 3: 360c Additional Information 4: f2486cf67a3bbd6d9170c007387206e1 Is this problem related to how fork in perl is handled in windows or is this problem only related to Parallel::Loops? But the perl code itself works without any problem. Thank you very much for your attention. Frank Wang Show quoted text
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Download (untitled) / with headers
text/plain 549b
On 2011-07-27 17:41:15, frank.wang@validusresearch.com wrote: Show quoted text
> When using pp to create a stanalone executable in activeperl for x64, > if the source code used Parallel::Loops, the executable will crash and > the following is the problem detail:
This information is useless. Please provide a minimal code example that exhibits the problem. Also include the output of "perl -V" and the versions of PAR and PAR::Packer used. Show quoted text
> But the perl code itself works without any problem.
By that you mean: the code works when not packed? Cheers, Roderich
Subject: RE: [rt.cpan.org #69848] pp created exe crash if useing parallel loops
Date: Thu, 28 Jul 2011 09:43:16 -0400
To: "'bug-PAR-Packer [...] rt.cpan.org'" <bug-PAR-Packer [...] rt.cpan.org>
From: "Wang, Frank" <frank.wang [...] validusresearch.com>
Download (untitled) / with headers
text/plain 5.1k
Hi Roderich, Yes the code work fine and fast when not packed. Only crash when run packed code. The following is a minimal code example that exhibits the problem: #!/usr/bin/perl use strict; use warnings; use diagnostics; $|=1; my $workers=@ARGV ? shift : 16; use Parallel::Loops; my @modelfiles=("ct_125_gulfstream_ws_26119a.txt"); my $pl = Parallel::Loops->new($workers); $pl->foreach(\@modelfiles, sub { my $mf = $_; print $mf; } ); And this is the output of perl-V: Set up gcc environment - 4.4.5 20101001 (release) [svn/rev.164871 - mingw-w64/oz] Summary of my perl5 (revision 5 version 12 subversion 2) configuration: Platform: osname=MSWin32, osvers=5.2, archname=MSWin32-x64-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags =' -DNDEBUG -O2 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DCONSERVATIVE -DUSE_SITECUSTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLI CIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX -DWIN64', optimize='-O2', cppflags='-DWIN32' ccversion='', gccversion='4.4.5 20101001 (release) [svn/rev.164871 - mingw-w64/oz]', gccosandvers='' intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='__int64', ivsize=8, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='g++', ldflags ='-s -L"C:\Perl64\lib\CORE"' libpth="C:\mingw64\lib" "C:\mingw64\x86_64-w64-mingw32\lib" libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lve rsion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt libc=-lmsvcrt, so=dll, useshrplib=true, libperl=libperl512.a gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-s -mdll -L"C:\Perl64\lib\CORE"' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF USE_SITECUSTOMIZE Locally applied patches: ActivePerl Build 1203 [294165] 1fd8fa4 Add Wolfram Humann to AUTHORS f120055 make string-append on win32 100 times faster a2a8d15 Define _USE_32BIT_TIME_T for VC6 and VC7 007cfe1 Don't pretend to support really old VC++ compilers 6d8f7c9 Get rid of obsolete PerlCRT.dll support d956618 Make Term::ReadLine::findConsole fall back to STDIN if /dev/tty can't be opened 321e50c Escape patch strings before embedding them in patchlevel.h Built under MSWin32 Compiled at Dec 9 2010 00:50:22 %ENV: PERL5OPT="-MConfig_w64" @INC: C:/Perl64/site/lib C:/Perl64/lib . The file " C:\Perl64\site\lib\PAR \ Packer .pm" has our $VERSION = '1.010'; Actually my previous version of Par::Packer is 1.008 and showed this problem so I updated it to 1.010 and hope this problem will go away but seems it is not. (Also if we do not use parallel loop the packed code run fine.) Thank you. Frank Show quoted text
-----Original Message----- From: Roderich Schupp via RT [mailto:bug-PAR-Packer@rt.cpan.org] Sent: Wednesday, July 27, 2011 6:04 PM To: Wang, Frank Subject: [rt.cpan.org #69848] pp created exe crash if useing parallel loops <URL: https://rt.cpan.org/Ticket/Display.html?id=69848 > On 2011-07-27 17:41:15, frank.wang@validusresearch.com wrote:
> When using pp to create a stanalone executable in activeperl for x64, > if the source code used Parallel::Loops, the executable will crash and > the following is the problem detail:
This information is useless. Please provide a minimal code example that exhibits the problem. Also include the output of "perl -V" and the versions of PAR and PAR::Packer used.
> But the perl code itself works without any problem.
By that you mean: the code works when not packed? Cheers, Roderich
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Download (untitled) / with headers
text/plain 5.8k
On 2011-07-28 09:43:16, frank.wang@validusresearch.com wrote: Show quoted text
> The following is a minimal code example that exhibits the problem:
Thanks. The good news is that I can easily reproduce the problem here (with e.g. ActiveState 5.10.1 on 32-bit Windows XP). The bad news is that I have no clue what's going on. Looks like the problem occurs in some END block or in the global destruction phase. Note that a packed executable actually runs in a custom Perl interpreter which might have some problem with the fork emulation on Windows. I keep looking into this, but don't hold your breadth. Cheers, Roderich Show quoted text
> > #!/usr/bin/perl > use strict; > use warnings; > use diagnostics; > $|=1; > my $workers=@ARGV ? shift : 16; > use Parallel::Loops; > my @modelfiles=("ct_125_gulfstream_ws_26119a.txt"); > my $pl = Parallel::Loops->new($workers); > $pl->foreach(\@modelfiles, > sub { > my $mf = $_; > print $mf; > } > ); > > And this is the output of perl-V: > > Set up gcc environment - 4.4.5 20101001 (release) [svn/rev.164871 - > mingw-w64/oz] > Summary of my perl5 (revision 5 version 12 subversion 2) > configuration: > > Platform: > osname=MSWin32, osvers=5.2, archname=MSWin32-x64-multi-thread > uname='' > config_args='undef' > hint=recommended, useposix=true, d_sigaction=undef > useithreads=define, usemultiplicity=define > useperlio=define, d_sfio=undef, uselargefiles=define, > usesocks=undef > use64bitint=define, use64bitall=undef, uselongdouble=undef > usemymalloc=n, bincompat5005=undef > Compiler: > cc='gcc', ccflags =' -DNDEBUG -O2 -DWIN32 -D_CONSOLE -DNO_STRICT > -DHAVE_DES_FCRYPT -DCONSERVATIVE -DUSE_SITECUSTOMIZE > -DPRIVLIB_LAST_IN_INC -DPERL_IMPLI > CIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX > -DWIN64', > optimize='-O2', > cppflags='-DWIN32' > ccversion='', gccversion='4.4.5 20101001 (release) [svn/rev.164871 > - mingw-w64/oz]', gccosandvers='' > intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678 > d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8 > ivtype='__int64', ivsize=8, nvtype='double', nvsize=8, > Off_t='__int64', lseeksize=8 > alignbytes=8, prototype=define > Linker and Libraries: > ld='g++', ldflags ='-s -L"C:\Perl64\lib\CORE"' > libpth="C:\mingw64\lib" "C:\mingw64\x86_64-w64-mingw32\lib" > libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 > -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 > -lmpr -lwinmm -lve > rsion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt > perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool > -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid > -lws2_32 -lmpr -lwinmm > -lversion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt > libc=-lmsvcrt, so=dll, useshrplib=true, libperl=libperl512.a > gnulibc_version='' > Dynamic Linking: > dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' > cccdlflags=' ', lddlflags='-s -mdll -L"C:\Perl64\lib\CORE"' > > > Characteristics of this binary (from libperl): > Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV > PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS > PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC > USE_64_BIT_INT > USE_ITHREADS USE_LARGE_FILES USE_PERLIO > USE_PERL_ATOF > USE_SITECUSTOMIZE > Locally applied patches: > ActivePerl Build 1203 [294165] > 1fd8fa4 Add Wolfram Humann to AUTHORS > f120055 make string-append on win32 100 times faster > a2a8d15 Define _USE_32BIT_TIME_T for VC6 and VC7 > 007cfe1 Don't pretend to support really old VC++ compilers > 6d8f7c9 Get rid of obsolete PerlCRT.dll support > d956618 Make Term::ReadLine::findConsole fall back to STDIN if > /dev/tty can't be opened > 321e50c Escape patch strings before embedding them in > patchlevel.h > Built under MSWin32 > Compiled at Dec 9 2010 00:50:22 > %ENV: > PERL5OPT="-MConfig_w64" > @INC: > C:/Perl64/site/lib > C:/Perl64/lib > . > > > The file " C:\Perl64\site\lib\PAR \ Packer .pm" has > our $VERSION = '1.010'; > > Actually my previous version of Par::Packer is 1.008 and showed this > problem so I updated it to 1.010 and hope this problem will go away > but seems it is not. > (Also if we do not use parallel loop the packed code run fine.) > > Thank you. > > Frank > > > -----Original Message----- > From: Roderich Schupp via RT [mailto:bug-PAR-Packer@rt.cpan.org] > Sent: Wednesday, July 27, 2011 6:04 PM > To: Wang, Frank > Subject: [rt.cpan.org #69848] pp created exe crash if useing parallel > loops > > <URL: https://rt.cpan.org/Ticket/Display.html?id=69848 > > > On 2011-07-27 17:41:15, frank.wang@validusresearch.com wrote:
> > When using pp to create a stanalone executable in activeperl for
> x64,
> > if the source code used Parallel::Loops, the executable will crash
> and
> > the following is the problem detail:
> > This information is useless. > > Please provide a minimal code example that exhibits the problem. > Also include the output of "perl -V" and the versions of PAR and > PAR::Packer used. >
> > But the perl code itself works without any problem.
> > By that you mean: the code works when not packed? > > Cheers, Roderich > > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________
Download (untitled) / with headers
text/plain 576b
On 2011-08-01 10:26:13, RSCHUPP wrote: Show quoted text
> I keep looking into this,
The actual problem is that a packed script actually runs inside a BEGIN block. If you simply put everything in your example script (except the "use" statements) into a BEGIN block, then it will blow up even unpacked. The perlfork man page explicitly states that fork (i.e. the emulation for Windows) might fail when invoked inside BEGIN. As to why PAR::Packer wraps a packed script into a BEGIN block: I don't know - it has always been the case (according to the history in Subversion). Cheers, Roderich
Subject: RE: [rt.cpan.org #69848] pp created exe crash if useing parallel loops
Date: Tue, 2 Aug 2011 17:55:57 -0400
To: "'bug-PAR-Packer [...] rt.cpan.org'" <bug-PAR-Packer [...] rt.cpan.org>
From: "Wang, Frank" <frank.wang [...] validusresearch.com>
Download (untitled) / with headers
text/plain 1.7k
Hi Rodrich, Thank you for inform me of this finding. Just wondering maybe it is possible to update PAR::Packer soon since wrap packed script into a BEGIN block or into BODY block seems do not need much change? and the pack function and parallel process function are very desirable feature of Perl. Regards, Frank Show quoted text
-----Original Message----- From: Roderich Schupp via RT [mailto:bug-PAR-Packer@rt.cpan.org] Sent: Tuesday, August 02, 2011 5:26 AM To: Wang, Frank Subject: [rt.cpan.org #69848] pp created exe crash if useing parallel loops <URL: https://rt.cpan.org/Ticket/Display.html?id=69848 > On 2011-08-01 10:26:13, RSCHUPP wrote:
> I keep looking into this,
The actual problem is that a packed script actually runs inside a BEGIN block. If you simply put everything in your example script (except the "use" statements) into a BEGIN block, then it will blow up even unpacked. The perlfork man page explicitly states that fork (i.e. the emulation for Windows) might fail when invoked inside BEGIN. As to why PAR::Packer wraps a packed script into a BEGIN block: I don't know - it has always been the case (according to the history in Subversion). Cheers, Roderich
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Download (untitled) / with headers
text/plain 621b
On 2011-08-02 17:55:58, frank.wang@validusresearch.com wrote: Show quoted text
> Just wondering maybe it is possible to > update PAR::Packer soon since wrap packed script into a BEGIN block or > into BODY block seems do not need much change?
Sorry, unlikely soon. There has to be a reason why practically all of script/par.pl (which eventually calls "do 'your_original_script'") is wrapped in BEGIN (and eval {} to boot), but I don't know it (mind you, I'm not the original author). Any change in this crucial piece of the PAR::Packer machinery needs consideration and extensive testing for which I don't have time ATM. Cheers, Roderich
Download (untitled) / with headers
text/plain 1.2k
On 2011-08-03 02:38:43, RSCHUPP wrote: Show quoted text
> piece of the PAR::Packer machinery needs consideration and > extensive testing for which I don't have time ATM.
Just as a reminder to myself... Things done to narrow down the problem - but still no solution: - reduced the span of the BEGIN block in script/par.pl - verified that the packed script is NOT executed from "inside" BEGIN or eval() any more - set PL_perl_destruct_level=1 in myldr/main.c for perl_construct() and perl_destruct() call (but to no avail) - problem occurs definitely AFTER the end of normal execution (but still in perl_run()) - using Devel::Symdump we see that the only END blocks registered are in File::Temp, PAR and par.pl - by instrumenting these END blocks we see that the problem occurs AFTER they have been run - the problem is not specific to Parallel::Loops or Parallel::ForkManager, the following script shows the same symptoms (with or without the wait()ing at the end) use strict; use warnings; for (1..5) { my $pid = fork(); die "fork failed: $!" unless defined $pid; if ($pid == 0) { # child print STDERR "child $$\n"; exit(0); } } print STDERR "waiting for children ...\n"; while ((my $pid = wait()) != -1) { print STDERR "reaped $pid\n"; }


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.