Skip Menu |
 

This queue is for tickets about the PAR-Packer CPAN distribution.

Report information
The Basics
Id: 42986
Status: resolved
Priority: 0/
Queue: PAR-Packer

People
Owner: Nobody in particular
Requestors: TJC [...] cpan.org
Cc: tobyc [...] strategicdata.com.au
AdminCc:

Bug Information
Severity: Important
Broken in: (no value)
Fixed in: (no value)



CC: tobyc [...] strategicdata.com.au
Subject: PAR-based modules use system XS modules over included modules
Download (untitled) / with headers
text/plain 1.1k
I have created a PAR file which contains my script, and the included modules include an XS module. However if I try to run the script, the XS module fails to load with DynaLoader, as the system-installed .so file has been loaded, rather than the one included with the PAR file. To give an example: test.pl contains: #!/usr/bin/perl use strict; use warnings; use Template; my $foo = Template->new({STASH => 'Template::Stash::XS'}); I create a par like so: pp -p -o test.par test.pl I verify that the contents of the PAR contain Template/Stash/XS.pm and Template/Stash/XS.so, and that they are version 2.20. Now I attempt to run the PAR on a machine with version 2.19 of the above installed, and get: $ parl test.par test.pl Failed to initialise Template: failed to create context: failed to create context: failed to load Template/Stash/XS.pm: Couldn't load Template::Stash::XS 2.20: Template::Stash::XS object version 2.19 does not match bootstrap parameter 2.20 at /usr/lib/perl/5.10.0/DynaLoader.pm line 219. I may be making a mistake here of course! This is tested on: Perl 5.10.0 PAR 0.984 PAR::Packer 0.982 AutoLoader 5.68 PAR::Dist 0.43
CC: par [...] perl.org, jesse+par [...] bestpractical.com
Subject: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules
Date: Fri, 20 Feb 2009 11:43:13 +0100
To: bug-PAR [...] rt.cpan.org
From: Steffen Mueller <wyp3rlx02 [...] sneakemail.com>
Download (untitled) / with headers
text/plain 3.4k
Hi Toby, long mail ahead. Sorry. Toby C via RT wrote: Show quoted text
> Tue Feb 03 00:35:06 2009: Request 42986 was acted upon. > Queue: PAR > Subject: PAR-based modules use system XS modules over included modules
Show quoted text
> I have created a PAR file which contains my script, and the included > modules include an XS module. > However if I try to run the script, the XS module fails to load with > DynaLoader, as the system-installed .so file has been loaded, rather > than the one included with the PAR file.
Very strange. I did essentially exactly as you say and I could not reproduce the problem. What happens when you try to run the .par with "par.pl test.par"? What's the architectures and perl and PAR::Packer versions that were used to create the parl's you're using to execute the test.par? I guess you can find out like this: mkdir tmp cd tmp cp `which parl` . # save extract_embedded.pl from this mail to the directory perl extract_embedded.pl parl . grep archname Config.pm # --> should print something like # archname => 'i686-linux-thread-multi', grep "version =>" Config.pm # version => '5.10.0', To find out the PAR (not PAR::Packer unfortunately) version, you can then do: grep "VERSION" PAR.pm With this information, you'll be able to verify or disprove the following guess of mine: "The architectures or perl versions of your target machine's pArl and your development machine's pErl aren't really compatible." Let me explain with some more detail: - When you use pp to generate .par or ".exe" files, it puts a small script into the archive as "script/main.pl". That script is executed when you try to run a script from the archive via "parl" or "par.pl" and also for stand-alone executables. - The contents of main.pl are different depending on context. Specifically, if generating a stand-alone executable or using the -B option to pp, then main.pl will include some code that clears out the @INC before running the script. After all what you're executing is supposed to be stand-alone. - If you're using the -p option and not -B, i.e. creating a dependent .par file that expects a full perl installation, then it *naturally* doesn't clear @INC before running the user code. - Your case is the second one. - Since (I think) the perl installations (and specifically the pArl on the target machine and the pErl on your dev machine that runs pp) aren't entirely compatible, the XS.so in the .par will not be loaded. Instead, it will try to find a XS.so on the target system that's compatible to the target system's parl. - Such an XS.so is found, but after loading it, it turns out to be an incompatible VERSION! Let's suppose this is the case. I still wouldn't exactly call this a mistake on your part. It's simply a darned complicated situation. It's also one of the reasons why I don't like the "parl FOO.par" use case or PAR::Packer. But still, I don't think it's a downright PAR bug either. It's the behaviour you're implicitly asking for by not bundling the whole perl environment (i.e. "please use what's already there"). I hope this clears up confusion and doesn't add to it :) Best regards, Steffen P.S.: Food for thought and discussion: I just found out that adding -B to the "pp -p -o foo.par foo.pl" doesn't make "main.pl" clear @INC. I expected it would. We'll have to think about what the right behaviour is in that case. After all, it's bundling the core modules, so it should probably clear @INC before running, no?
Download extract-embedded.pl
text/x-perl 1.2k
#!/usr/bin/perl # code stolen from PAR script/parl.pl use File::Spec; use File::Basename; use File::Path; use strict; use warnings; @ARGV == 2 || die "usage: $0 executable directory_to_extract_into\n"; extract_embedded(@ARGV); sub extract_embedded { my ($exe, $xdir) = @_; open my $fh, '<', $exe or die qq[failed to open "$exe": $!]; binmode $fh; my $buf; seek $fh, -8, 2; read $fh, $buf, 8; die qq[no PAR signature found in "$exe"] unless $buf eq "\nPAR.pm\n"; seek $fh, -12, 2; read $fh, $buf, 4; seek $fh, -12 - unpack("N", $buf), 2; read $fh, $buf, 4; while ($buf eq "FILE") { read $fh, $buf, 4; read $fh, $buf, unpack("N", $buf); my $fullname = $buf; print STDERR qq[FILE "$fullname"...]; my $crc = ( $fullname =~ s|^([a-f\d]{8})/|| ) ? $1 : undef; my @path = split(/\//, $fullname); read $fh, $buf, 4; read $fh, $buf, unpack("N", $buf); my $file = File::Spec->catdir($xdir, @path); my $dir = dirname($file); mkpath($dir) unless -d $dir; open my $out, '>', $file or die qq[failed to open "$file": $!]; binmode $out; print $out $buf; close $out; print STDERR qq[ extracted to $file\n]; read $fh, $buf, 4; } close $fh; }
Download (untitled) / with headers
text/plain 913b
Sorry, jumping back into this bug again after a long break. The workaround I was using was simply to ensure the target system didn't install much at all into the system libraries, thus ensuring no conflicts, and then forgot about the issue. I've noticed the problem still seems to exist, even with the PAR being built on an identical system, so I'm sure the shared objects should be compatible. In fact, the PAR and parl executables were built on one system, and installed on the other, seeming to indicate binary compatibility between the two. The system-installed XS module's shared-library is loaded in preference to the one included in the PAR, thus causing DynaLoader to die. (in this case, it's Digest::SHA1 and Digest/SHA1.so) Is there anything else I can check in the meantime, that would be preventing the par-included SHA1.so to be loaded? I have verified it is included in the PAR. thanks.
Download (untitled) / with headers
text/plain 283b
On Thu Feb 04 03:03:13 2010, TJC wrote: Show quoted text
> Sorry, jumping back into this bug again after a long break.
Have you re-checked with the latest and greatest version of PAR::Packer? If so, does the minimal "hello world" $ pp -o hello -e 'print "hello world\n"' demonstrate the problem?
Download (untitled) / with headers
text/plain 670b
On Thu Feb 04 09:32:52 2010, RSCHUPP wrote: Show quoted text
> On Thu Feb 04 03:03:13 2010, TJC wrote:
> > Sorry, jumping back into this bug again after a long break.
> > Have you re-checked with the latest and greatest version of > PAR::Packer? > > If so, does the minimal "hello world" > > $ pp -o hello -e 'print "hello world\n"' > > demonstrate the problem?
Sorry, it does not. Small test case. Note that on the building system, File::Temp is at 0.22 (current) whereas the target system, File::Temp is, erm, slightly older (standard debian lenny) and doesn't have the newdir method. #!/usr/bin/perl use strict; use warnings; use File::Temp; my $dir = File::Temp->newdir;
Download (untitled) / with headers
text/plain 604b
On Thu Feb 04 18:12:00 2010, TJC wrote: Show quoted text
> Small test case. Note that on the building system, File::Temp is at 0.22 > (current) whereas the target system, File::Temp is, erm, slightly older > (standard debian lenny) and doesn't have the newdir method. > > #!/usr/bin/perl > use strict; > use warnings; > use File::Temp; > my $dir = File::Temp->newdir;
Can't reproduce this here - strace'ing the packed version of above shows that the system File::Temp is NOT used. Did you build PAR::Packer on the same system you're doing the packing? Or did you build it somewhere else (or got it from a distro)?
Download (untitled) / with headers
text/plain 971b
On Sat Feb 06 08:35:17 2010, RSCHUPP wrote: Show quoted text
> Did you build PAR::Packer on the same system you're doing > the packing? Or did you build it somewhere else (or got it > from a distro)?
While we're at it: if your.exe is the packed executable from the example above, please run $ strings -a your.exe and search for the string "File/Temp.pm" in the output. You should be looking at something like: 6b3c9548/File/Temp.pm 9#line 1 "/usr/share/perl/5.10/File/Temp.pm" package File::Temp; #line 138 # 5.6.0 gives us S_IWOTH, S_IWGRP, our and auto-vivifying filehandls # People would like a version on 5.004 so give them what they want :-) use 5.004; use strict; use Carp; use File::Spec 0.8; use File::Path qw/ rmtree /; ... That's actually a copy of File/Temp.pm - scroll down a bit and look for $VERSION = ... If that's the same version as on your target system (or at least older than that on your build system) I have an idea what's going on :) Cheers, Roderich
Download (untitled) / with headers
text/plain 1.5k
On Sat Feb 06 08:35:17 2010, RSCHUPP wrote: Show quoted text
> On Thu Feb 04 18:12:00 2010, TJC wrote:
> > Small test case. Note that on the building system, File::Temp is at
0.22 Show quoted text
> > (current) whereas the target system, File::Temp is, erm, slightly
older Show quoted text
> > (standard debian lenny) and doesn't have the newdir method. > > > > #!/usr/bin/perl > > use strict; > > use warnings; > > use File::Temp; > > my $dir = File::Temp->newdir;
> > Can't reproduce this here - strace'ing the packed version of above > shows that the system File::Temp is NOT used.
I build the par file with: pp -p -o testfile.par testfile.pl (where testfile.pl contains the above text only) If I extract the par with `unzip testfile.par` and then view the extracted file, I see it does contain File/Temp.pm, and it is the version from the source system, ie. 0.22. If I copy testfile.par to the remote system, and then execute either `parl testfile.par` or `par -MPAR testfile.par` I receive the error: tobyc@arya-tmp:~$ perl -MPAR filetest.par Can't locate object method "newdir" via package "File::Temp" at script/filetest.pl line 5. BEGIN failed--compilation aborted. (Note that the target system already has File::Temp 0.18 available in the system, but that version doesn't support the newdir() method.) Show quoted text
> Did you build PAR::Packer on the same system you're doing > the packing? Or did you build it somewhere else (or got it > from a distro)?
PAR::Packer was built on the source machine, and turned into a debian package, which was then installed on the target machine. Thanks for your help so far, Toby
Download (untitled) / with headers
text/plain 712b
To demonstrate the other/related problem, involving XS modules, create sha.pl as: #!/usr/bin/perl use strict; use warnings; use Digest::SHA1; my $digest = Digest::SHA1->new; Create a PAR with: pp -p -o sha.par sha.pl (Do this on a machine with version 2.12 of Digest::SHA1, eg. from CPAN) Then copy to new machine (which contains 2.11 of Digest::SHA1, eg. from Debian repository), and run it: tobyc@arya-tmp$ perl -MPAR sha.par Digest::SHA1 object version 2.11 does not match bootstrap parameter 2.12 at /usr/lib/perl/5.10/DynaLoader.pm line 219. Compilation failed in require at script/shatest.pl line 4. BEGIN failed--compilation aborted at script/shatest.pl line 4. BEGIN failed--compilation aborted.
Download (untitled) / with headers
text/plain 1010b
On Sun Feb 07 21:37:08 2010, TJC wrote: Show quoted text
> tobyc@arya-tmp$ perl -MPAR sha.par
Sorry, I misread your report until now - you're talking about a .par file, not a packed executable. I think what happens is a kind of chicken and egg problem: - your .par contains different versions of File::Temp etc than installed on the target machine - conceptually we would like "perl -MPAR foo.par" to behave like - extract file.par into temporary directory /tmp/foo123 - run "perl -I/tmp/foo123 /tmp/foo123/foo.pl" (i.e. "/tmp/foo123" is at the start of @INC) - in order to get to the modules in the .par file, PAR.pm needs some modules, e.g. File::Temp - hence it already has loaded the target system's File::Temp before it even gets to consider the (different) copy included in the .par All modules cited in your report are in this category "required by PAR.pm to extract stuff from a .par file". (Digest::SHA1 is a special case that's even weirder.) Sorry, I see no sane way to make this work.
Subject: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules
Date: Mon, 08 Feb 2010 09:45:10 +0100
To: bug-PAR [...] rt.cpan.org
From: Steffen Mueller <smueller [...] cpan.org>
Download (untitled) / with headers
text/plain 1.6k
Hi Toby, hi Roderich, RSCHUPP via RT wrote: Show quoted text
Show quoted text
> I think what happens is a kind of chicken and egg problem: > > - your .par contains different versions of File::Temp etc > than installed on the target machine > > - conceptually we would like "perl -MPAR foo.par" > to behave like > - extract file.par into temporary directory /tmp/foo123 > - run "perl -I/tmp/foo123 /tmp/foo123/foo.pl" > (i.e. "/tmp/foo123" is at the start of @INC) > > - in order to get to the modules in the .par file, PAR.pm > needs some modules, e.g. File::Temp > > - hence it already has loaded the target system's File::Temp > before it even gets to consider the (different) copy included > in the .par > > All modules cited in your report are in this category > "required by PAR.pm to extract stuff from a .par file". > (Digest::SHA1 is a special case that's even weirder.) > > Sorry, I see no sane way to make this work.
I agree. No sane way. But you may try to do something like this. I would probably not do it in production. - Pack Class::Unload into your .par. - Load the .par file (using system File::Temp) - Load Class::Unload. - Extract File/Temp.pm from the .par file into the appropriate location of the temporary storage location. Just have a look, the layout is basically a normal lib/ dir. PAR.pm has hooks for loading the contents of a file from any loaded .par file. - Run Class::Unload->unload('File::Temp') - require File::Temp This may or may not work. It's certainly one of the uglier hacks I've suggested in the past. Best regards, Steffen
Subject: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules
Date: Mon, 8 Feb 2010 22:11:47 +0000
To: bug-PAR [...] rt.cpan.org
From: Ben Morrow <ben [...] morrow.me.uk>
Download (untitled) / with headers
text/plain 1.2k
Quoth bug-PAR@rt.cpan.org: Show quoted text
> On Sun Feb 07 21:37:08 2010, TJC wrote:
> > tobyc@arya-tmp$ perl -MPAR sha.par
> > Sorry, I misread your report until now - you're talking > about a .par file, not a packed executable. > > I think what happens is a kind of chicken and egg problem: > > - your .par contains different versions of File::Temp etc > than installed on the target machine > > - conceptually we would like "perl -MPAR foo.par" > to behave like > - extract file.par into temporary directory /tmp/foo123 > - run "perl -I/tmp/foo123 /tmp/foo123/foo.pl" > (i.e. "/tmp/foo123" is at the start of @INC) > > - in order to get to the modules in the .par file, PAR.pm > needs some modules, e.g. File::Temp > > - hence it already has loaded the target system's File::Temp > before it even gets to consider the (different) copy included > in the .par > > All modules cited in your report are in this category > "required by PAR.pm to extract stuff from a .par file". > (Digest::SHA1 is a special case that's even weirder.) > > Sorry, I see no sane way to make this work.
How about doing the unpacking, and then re-execing perl with exactly that command-line? A little slow, perhaps, but PAR's startup (at least when it has to do unpacking) isn't exactly fast anyway. Ben
Download (untitled) / with headers
text/plain 499b
Should this issue still occur with executable packed archives? I ask because I am having a shot at that method instead, but still seem to be having the same problem. If i look in the unpacked /tmp/par-username/cache-xxxx/ directory, I see that I have version 2.024 of the .pm and .so files, but when I attempt to run the executable, I am told that it can't as object version 2.008 does not match bootstrap parameter 2.024. (The source system has 2.024, the target system has 2.008 installed).
Download (untitled) / with headers
text/plain 1.5k
On Wed Feb 10 01:44:53 2010, TJC wrote: Show quoted text
> Should this issue still occur with executable packed archives? > I ask because I am having a shot at that method instead, but still seem > to be having the same problem.
Basically it's the same problem, but at least you can do something about it :) The packed executable has the same chicken-and-egg problem as the .par archive. As a first approximation, a packed executable is just a custom built perl interpreter with a .par bolted on. In order to get at the stuff in the .par you need a minimal set of Perl modules. A .par takes these from the Perl environment where it is run (and doesn't care if different versions of these modules are included in the .par). The packed executable takes these modules from its own executable file (NOT from the appended .par). You may find them extracted below /tmp/par-username/cache-xxxx/, but you probably won't recognize them, because their file names have been mangled (some-checksum.pm). If you find, say, a File::Temp in the cache area that actually is called File/Temp.pm then this is the one extracted from the .par, but as in the .par case, it's not the one that gets loaded. See one of my earlier comments how to find out about the "built-in" modules in a packed executable. Which version of a specific module is "built-in" depends on the environment that was current when PAR::Packer was built. (That's why I asked where you got your PAR::Packer from.) If you rebuild PAR::Packer on your building system it will pick up the current versions of these modules.
Download (untitled) / with headers
text/plain 385b
On Wed Feb 10 04:59:03 2010, RSCHUPP wrote: Show quoted text
> If you rebuild PAR::Packer on your building system > it will pick up the current versions of these modules.
I'm fairly sure that PAR::Packer was built after those modules, since it (and PAR) were the modules that required these things to be updated from the default-installed versions. However I will rebuild it again and see. ta, Toby
On Wed Feb 10 18:14:24 2010, TJC wrote: Show quoted text
> On Wed Feb 10 04:59:03 2010, RSCHUPP wrote:
> > If you rebuild PAR::Packer on your building system > > it will pick up the current versions of these modules.
> > I'm fairly sure that PAR::Packer was built after those modules, since
it Show quoted text
> (and PAR) were the modules that required these things to be updated
from Show quoted text
> the default-installed versions. However I will rebuild it again and
see. Hmm, nope, same problem after rebuilding PAR::Packer. I think there might be a problem with the @INC order.. The old, system version of Compress::Raw::Zlib still exists, in /usr/lib/perl/5.10.0, and the new version is in /usr/local/lib/perl/5.10.0. The latter version is higher in @INC and is the version picked up by modules.. However it seems like pp is picking up the older version for some reason? .. Actually, I have deleted those copies, then rebuild PAR and PAR::Packer, then rebuild my test script (pp -o blah blah.pl) and I am *still* getting the error regarding versions on the target system.
Download (untitled) / with headers
text/plain 224b
Running "strings -a my_executeable" and looking for Compress::Raw::Zlib in the output shows that 2.024 (the newer version) appears to be packed in. (At least for the pm version.. How do I check for the .so that's packed?)
On Wed Feb 10 20:21:10 2010, TJC wrote: Show quoted text
> Running "strings -a my_executeable" and looking for Compress::Raw::Zlib > in the output shows that 2.024 (the newer version) appears to be packed > in. (At least for the pm version.. How do I check for the .so that's > packed?)
Run the attached script as $ perl extract-embedded.pl EXECUTABLE DIR it will extract all stuff (modules and shared libs) embedded in EXECUTABLE into DIR (using their original names). To check the version of .so I suggest to compare md5sums against possible candidates. Show quoted text
> I think there might be a problem with the @INC order.. > The old, system version of Compress::Raw::Zlib still exists, in > /usr/lib/perl/5.10.0, and the new version is in > /usr/local/lib/perl/5.10.0. > The latter version is higher in @INC and is the version picked > up by modules.. However it seems like pp is picking > up the older version for some reason?
Unlikely. pp will use the order as shown by $ perl -e 'print "@INC\n"' or does /usr/local/lib/perl/5.10.0 get prepended because you have PERL5LIB set in your environment? Chers, Roderich
Download (untitled) / with headers
text/plain 1.3k
On Thu Feb 11 05:25:53 2010, RSCHUPP wrote: Show quoted text
> Run the attached script as > > $ perl extract-embedded.pl EXECUTABLE DIR > > it will extract all stuff (modules and shared libs) embedded > in EXECUTABLE into DIR (using their original names). > To check the version of .so I suggest to compare md5sums against > possible candidates.
Checking the original executable: $ ./blah Compress::Raw::Zlib object version 2.008 does not match bootstrap parameter 2.024 at /usr/local/lib/perl/5.10.0/Compress/Raw/Zlib.pm line 97. Compilation failed in require at /home/tobyc/perl/lib/Archive/Zip.pm line 12. BEGIN failed--compilation aborted at /home/tobyc/perl/lib/Archive/Zip.pm line 12. Compilation failed in require at -e line 358. $ ./extract-embedded.pl blah extract/ ... $ cd extract $ find .|grep Zlib ./IO/Compress/Zlib ./IO/Compress/Zlib/Extra.pm ./Compress/Zlib.pm ./Compress/Raw/Zlib.pm ./auto/Compress/Raw/Zlib ./auto/Compress/Raw/Zlib/autosplit.ix ./auto/Compress/Raw/Zlib/Zlib.so ./auto/Compress/Zlib ./auto/Compress/Zlib/autosplit.ix $ grep VERSION ./Compress/Raw/Zlib.pm $VERSION = '2.024'; $ md5sum ./auto/Compress/Raw/Zlib/Zlib.so 04c2dc8c8fe0ac9606ef29aad5c959d4 # This matches the md5sum of version 2.024 on the build machine. So it seems like the executable PAR file contains both .pm and .so of version 2.024, yet when executed is still mysteriously trying to load the .so from the target machine.
I note that b3cf5856 = Compress/Raw/Zlib.pm and ce08cac2 = auto/Compress/Raw/Zlib/Zlib.so I tried 'strace'ing the executable, and noticed: 644: stat64("/tmp/par-tobyc/cache- 038134f668a1529182c945464ecd15936dab8d95/ce08cac2.so", {st_mode=S_IFREG|0755, st_size=331208, ...}) = 0 (no corresponding open() that I see) 1018: stat64("/tmp/par-tobyc/cache- 038134f668a1529182c945464ecd15936dab8d95/b3cf5856.pm", {st_mode=S_IFREG|0644, st_size=15076, ...}) = 0 1019: open("/tmp/par-tobyc/cache- 038134f668a1529182c945464ecd15936dab8d95/b3cf5856.pm", O_RDONLY|O_LARGEFILE) = 7 1062: stat64("CODE(0x9795f00)/auto/Compress/Raw/Zlib", 0x968f0c0) = -1 ENOENT (No such file or directory) 1069: stat64("/usr/lib/perl/5.10/auto/Compress/Raw/Zlib/Zlib.so", {st_mode=S_IFREG|0644, st_size=104100, ...}) = 0 1070: stat64("/usr/lib/perl/5.10/auto/Compress/Raw/Zlib/Zlib.bs", 0x968f0c0) = -1 ENOENT (No such file or directory) 1071: open("/usr/lib/perl/5.10/auto/Compress/Raw/Zlib/Zlib.so", O_RDONLY) = 7 I attach the full strace.
Subject: strace.txt
Download strace.txt
text/plain 79.4k

Message body is not shown because it is too large.

Download (untitled) / with headers
text/plain 395b
On Thu Feb 11 21:28:56 2010, TJC wrote: Show quoted text
> I tried 'strace'ing the executable, and noticed:
Yeah, the strace shows that .so's (but not corresponding .pm's) are still searched using the built-in @INC. This rings a bell, but I can't pin it yet. This is Perl 5.10.0, right? After using extract-embedded.pl on your executable please post the extracted XSLoader.pm and PAR/Heavy.pm Cheers, Roderich
Download (untitled) / with headers
text/plain 463b
On Fri Feb 12 05:41:41 2010, RSCHUPP wrote: Show quoted text
> On Thu Feb 11 21:28:56 2010, TJC wrote:
> > I tried 'strace'ing the executable, and noticed:
> > Yeah, the strace shows that .so's (but not corresponding .pm's) > are still searched using the built-in @INC. > This rings a bell, but I can't pin it yet. > > This is Perl 5.10.0, right?
Correct. Show quoted text
> After using extract-embedded.pl on your executable > please post the extracted XSLoader.pm and PAR/Heavy.pm
Attached.
Subject: XSLoader.pm
Download XSLoader.pm
text/x-perl 3.4k
#line 1 "/home/tobyc/perl/lib/i486-linux-gnu-thread-multi/XSLoader.pm" # Generated from XSLoader.pm.PL (resolved %Config::Config value) package XSLoader; $VERSION = "0.10"; #use strict; # enable debug/trace messages from DynaLoader perl code # $dl_debug = $ENV{PERL_DL_DEBUG} || 0 unless defined $dl_debug; my $dl_dlext = 'so'; package DynaLoader; # No prizes for guessing why we don't say 'bootstrap DynaLoader;' here. # NOTE: All dl_*.xs (including dl_none.xs) define a dl_error() XSUB boot_DynaLoader('DynaLoader') if defined(&boot_DynaLoader) && !defined(&dl_error); package XSLoader; sub load { package DynaLoader; die q{XSLoader::load('Your::Module', $Your::Module::VERSION)} unless @_; my($module) = $_[0]; # work with static linking too my $boots = "$module\::bootstrap"; goto &$boots if defined &$boots; goto retry; my @modparts = split(/::/,$module); my $modfname = $modparts[-1]; my $modpname = join('/',@modparts); my $modlibname = (caller())[1]; my $c = @modparts; $modlibname =~ s,[\\/][^\\/]+$,, while $c--; # Q&D basename my $file = "$modlibname/auto/$modpname/$modfname.$dl_dlext"; # print STDERR "XSLoader::load for $module ($file)\n" if $dl_debug; my $bs = $file; $bs =~ s/(\.\w+)?(;\d*)?$/\.bs/; # look for .bs 'beside' the library if (-s $bs) { # only read file if it's not empty # print STDERR "BS: $bs ($^O, $dlsrc)\n" if $dl_debug; eval { do $bs; }; warn "$bs: $@\n" if $@; } goto retry if not -f $file or -s $bs; my $bootname = "boot_$module"; $bootname =~ s/\W/_/g; @DynaLoader::dl_require_symbols = ($bootname); my $boot_symbol_ref; # Many dynamic extension loading problems will appear to come from # this section of code: XYZ failed at line 123 of DynaLoader.pm. # Often these errors are actually occurring in the initialisation # C code of the extension XS file. Perl reports the error as being # in this perl code simply because this was the last perl code # it executed. my $libref = dl_load_file($file, 0) or do { require Carp; Carp::croak("Can't load '$file' for module $module: " . dl_error()); }; push(@DynaLoader::dl_librefs,$libref); # record loaded object my @unresolved = dl_undef_symbols(); if (@unresolved) { require Carp; Carp::carp("Undefined symbols present after loading $file: @unresolved\n"); } $boot_symbol_ref = dl_find_symbol($libref, $bootname) or do { require Carp; Carp::croak("Can't find '$bootname' symbol in $file\n"); }; push(@DynaLoader::dl_modules, $module); # record loaded module boot: my $xs = dl_install_xsub($boots, $boot_symbol_ref, $file); # See comment block above push(@DynaLoader::dl_shared_objects, $file); # record files loaded return &$xs(@_); retry: my $bootstrap_inherit = DynaLoader->can('bootstrap_inherit') || XSLoader->can('bootstrap_inherit'); goto &$bootstrap_inherit; } # Versions of DynaLoader prior to 5.6.0 don't have this function. sub bootstrap_inherit { package DynaLoader; my $module = $_[0]; local *DynaLoader::isa = *{"$module\::ISA"}; local @DynaLoader::isa = (@DynaLoader::isa, 'DynaLoader'); # Cannot goto due to delocalization. Will report errors on a wrong line? require DynaLoader; DynaLoader::bootstrap(@_); } 1; __END__ #line 359
Subject: Heavy.pm
Download Heavy.pm
text/x-perl 4.9k
#line 1 "/home/tobyc/perl/lib/PAR/Heavy.pm" package PAR::Heavy; $PAR::Heavy::VERSION = '0.11'; #line 17 ######################################################################## # Dynamic inclusion of XS modules my ($bootstrap, $dl_findfile); # Caches for code references my ($dlext); # Cache for $Config{dlext} my ($cache_key); # The current file to find my $is_insensitive_fs = ( -s $0 and (-s lc($0) || -1) == (-s uc($0) || -1) and (-s lc($0) || -1) == -s $0 ); # Adds pre-hooks to Dynaloader's key methods sub _init_dynaloader { return if $bootstrap; return unless eval { require DynaLoader; DynaLoader::dl_findfile(); 1 }; $bootstrap = \&DynaLoader::bootstrap; $dl_findfile = \&DynaLoader::dl_findfile; local $^W; *{'DynaLoader::dl_expandspec'} = sub { return }; *{'DynaLoader::bootstrap'} = \&_bootstrap; *{'DynaLoader::dl_findfile'} = \&_dl_findfile; } # Return the cached location of .dll inside PAR first, if possible. sub _dl_findfile { return $FullCache{$cache_key} if exists $FullCache{$cache_key}; if ($is_insensitive_fs) { # We have a case-insensitive filesystem... my ($key) = grep { lc($_) eq lc($cache_key) } keys %FullCache; return $FullCache{$key} if defined $key; } return $dl_findfile->(@_); } # Find and extract .dll from PAR files for a given dynamic module. sub _bootstrap { my (@args) = @_; my ($module) = $args[0] or return; my @modparts = split(/::/, $module); my $modfname = $modparts[-1]; $modfname = &DynaLoader::mod2fname(\@modparts) if defined &DynaLoader::mod2fname; if (($^O eq 'NetWare') && (length($modfname) > 8)) { $modfname = substr($modfname, 0, 8); } # XXX: Multi-platform .dll support in PARs needs better than $Config. # FIXME: Config is always loaded by PAR.pm! $dlext ||= do { require Config; (defined %Config::Config) ? $Config::Config{dlext} : ''; }; my $modpname = join((($^O eq 'MacOS') ? ':' : '/'), @modparts); my $file = $cache_key = "auto/$modpname/$modfname.$dlext"; if ($FullCache{$file}) { # TODO: understand local $DynaLoader::do_expand = 1; return $bootstrap->(@args); } my $member; # First, try to find things in the peferentially loaded PARs: $member = PAR::_find_par_internals([@PAR::PAR_INC], undef, $file, 1) if defined &PAR::_find_par_internals; # If that failed to find the dll, let DynaLoader (try or) throw an error unless ($member) { my $filename = eval { $bootstrap->(@args) }; return $filename if not $@ and defined $filename; # Now try the fallback pars $member = PAR::_find_par_internals([@PAR::PAR_INC_LAST], undef, $file, 1) if defined &PAR::_find_par_internals; # If that fails, let dynaloader have another go JUST to throw an error # While this may seem wasteful, nothing really matters once we fail to # load shared libraries! unless ($member) { return $bootstrap->(@args); } } $FullCache{$file} = _dl_extract($member, $file); # Now extract all associated shared objs in the same auto/ dir # XXX: shouldn't this also set $FullCache{...} for those files? my $first = $member->fileName; my $path_pattern = $first; $path_pattern =~ s{[^/]*$}{}; if ($PAR::LastAccessedPAR) { foreach my $member ( $PAR::LastAccessedPAR->members ) { next if $member->isDirectory; my $name = $member->fileName; next if $name eq $first; next unless $name =~ m{^/?\Q$path_pattern\E\/[^/]*\.\Q$dlext\E[^/]*$}; $name =~ s{.*/}{}; _dl_extract($member, $file, $name); } } local $DynaLoader::do_expand = 1; return $bootstrap->(@args); } sub _dl_extract { my ($member, $file, $name) = @_; require File::Spec; require File::Temp; my ($fh, $filename); # fix borked tempdir from earlier versions if ($ENV{PAR_TEMP} and -e $ENV{PAR_TEMP} and !-d $ENV{PAR_TEMP}) { unlink($ENV{PAR_TEMP}); mkdir($ENV{PAR_TEMP}, 0755); } if ($ENV{PAR_CLEAN} and !$name) { ($fh, $filename) = File::Temp::tempfile( DIR => ($ENV{PAR_TEMP} || File::Spec->tmpdir), SUFFIX => ".$dlext", UNLINK => ($^O ne 'MSWin32' and $^O !~ /hpux/), ); ($filename) = $filename =~ /^([\x20-\xff]+)$/; } else { $filename = File::Spec->catfile( ($ENV{PAR_TEMP} || File::Spec->tmpdir), ($name || ($member->crc32String . ".$dlext")) ); ($filename) = $filename =~ /^([\x20-\xff]+)$/; open $fh, '>', $filename or die $! unless -r $filename and -e _ and -s _ == $member->uncompressedSize; } if ($fh) { binmode($fh); $member->extractToFileHandle($fh); close $fh; chmod 0755, $filename; } return $filename; } 1; #line 205
Download (untitled) / with headers
text/plain 1.1k
On Sat Feb 13 03:37:38 2010, TJC wrote: Show quoted text
> > This is Perl 5.10.0, right?
> > Correct.
I finally remembered what's going on :( PAR::Heavy interposes the loading of shared libraries by tricking DynaLoader into always calling DynaLoader::dl_expandspec (which PAR::Heavy has sneakily replaced by its own version). DynaLoader usually first tries to locate the shared library by explicity checking along @INC; only if this fails it will call dl_expandspec.) Unfortunately this trick (setting $DynaLoader::do_expand = 1) doesn't work in Perl 5.10.0 (look into DynaLoader.pm, $do_expand is defined, but nowhere used). It works in all versions before and after. So this explains what you see: the .pm part of a XS based module is correctly resolved from the packed executable, but the corresponding shared library is resolved from the target system. If the two happen to be of different versions, Perl aborts. Looking at the versions of DynaLoader.pm in Perl 5.10.0 and 5.10.1 (on Linux) I think that you may savely replace the former with the latter. If you do that remember to rebuild PAR::Packer afterwards so that packed executables will have the "fixed" DynaLoader.pm embedded.
Subject: PAR-based modules use system XS modules over included modules (on Perl 5.10.0)
Download (untitled) / with headers
text/plain 1.6k
On Sun Feb 14 12:44:51 2010, RSCHUPP wrote: Show quoted text
> On Sat Feb 13 03:37:38 2010, TJC wrote:
> > > This is Perl 5.10.0, right?
> > > > Correct.
> > I finally remembered what's going on :( > > PAR::Heavy interposes the loading of shared libraries > by tricking DynaLoader into always calling DynaLoader::dl_expandspec > (which PAR::Heavy has sneakily replaced by its own version). > DynaLoader usually first tries to locate the shared library > by explicity checking along @INC; only if this fails it > will call dl_expandspec.) > Unfortunately this trick (setting $DynaLoader::do_expand = 1) > doesn't work in Perl 5.10.0 (look into DynaLoader.pm, $do_expand > is defined, but nowhere used). It works in all versions before > and after. > > So this explains what you see: the .pm part of a XS based module > is correctly resolved from the packed executable, but the > corresponding shared library is resolved from the target system. > If the two happen to be of different versions, Perl aborts. > > Looking at the versions of DynaLoader.pm in Perl 5.10.0 and > 5.10.1 (on Linux) I think that you may savely replace the > former with the latter. If you do that remember to rebuild PAR::Packer > afterwards so that packed executables will have the "fixed" > DynaLoader.pm embedded.
Thanks for your help on this. Rather than backport Dynaloader to 5.10.0, I ended up making my own installation of 5.10.1 + modules; I was considering using it to build PARs but then decided to ship the entire tree with the application. It blows the size out quite a bit, but it's a lot simpler than trying to get Module::ScanDeps to locate everything. Cheers, Toby
Subject: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 9 Mar 2010 13:49:17 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 10 Mar 2010 09:46:09 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 11 Mar 2010 12:14:10 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 12 Mar 2010 13:46:30 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 13 Mar 2010 16:54:20 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 14 Mar 2010 12:27:36 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 16 Mar 2010 14:12:43 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 17 Mar 2010 12:07:53 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: Re: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 18 Mar 2010 08:26:17 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Subject: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: [rt.cpan.org #42986] PAR-based modules use system XS modules over included modules (on Perl 5.10.0) --> Ref #[1ML0X9862w03y4x]
Date: 18 Mar 2010 20:27:10 -0000
To: bug-PAR [...] rt.cpan.org
From: announcement [...] support.netzero.com
Download (untitled) / with headers
text/plain 641b
Hello, Thank you for contacting NetZero. This is an automated reply and no further response will be sent. Please do not reply to this email. If you are looking for help here are some useful links: * Online NetZero Help Center: http://www.netzero.com/support * Online Billing and Account Center: https://account.netzero.net * Reset/Retrieve your Password: http://my.netzero.com/s/resetpassword * Lookup Access Numbers in your area: http://my.netzero.com/s/numbers * Order/Download NetZero Software: http://www.netzero.com/support/start/download-index.html Thank you for choosing NetZero. Sincerely, NetZero Member Services
Wrong queue; already solved


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.