This queue is for tickets about the ExtUtils-Install CPAN distribution.

Report information
The Basics
Id:
130120
Status:
resolved
Priority:
Low/Low

People
Owner:
Nobody in particular
Requestors:
jkeenan [...] cpan.org
Cc:
AdminCc:

BugTracker
Severity:
Normal
Broken in:
2.14
Fixed in:
2.16



Subject: ExtUtils-Install/t/Installed.t: intermittent crashes in Perl 5 test suite
Today, while testing a tarball candidate for perl-5.31.2 on FreeBSD-11, I got this unexpected test failure: ##### ../cpan/ExtUtils-Install/t/Install.t ............................... ok ===( 15013;100 46/58 1/302 1/70 18/73 )========================Can't cd to (./t/xBxrnJZxku/Big-Dummy/) lib: No such file or directory at /usr/home/jkeenan/Downloads/perl-5.31.2/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. Can't cd to .. from ./t/xBxrnJZxku/Big-Dummy/install-test/lib: No such file or directory at /usr/home/jkeenan/Downloads/perl-5.31.2/cpan/ExtUtils-Install/../../lib/File/Find.pm line 470. cannot fetch initial working directory: No such file or directory at t/Installed.t line 83. # Looks like your test exited with 2 just after 18. ../cpan/ExtUtils-Install/t/Installed.t ............................. Dubious, test returned 2 (wstat 512, 0x200) Failed 55/73 subtests ... ../cpan/ExtUtils-Install/t/Installed.t (Wstat: 512 Tests: 18 Failed: 0) Non-zero exit status: 2 Parse errors: Bad plan. You planned 73 tests but ran 18. ##### Perhaps that's a race condition? This test failure has also been intermittently observed on smoke-tests of the core distribution, failing with a report like this: ##### ~~ ../cpan/ExtUtils-Install/t/Installed.t ...................... FAILED Non-zero exit status: 2 Bad plan. ##### In my experience, the "Non-zero exit status: 2 Bad plan" usually indicates a segfault. Can you investigate? Thank you very much. Jim Keenan
On Sat Jul 20 09:33:39 2019, JKEENAN wrote:
Show quoted text
> Today, while testing a tarball candidate for perl-5.31.2 on FreeBSD- > 11, I got this unexpected test failure: > > ##### > ../cpan/ExtUtils-Install/t/Install.t ............................... > ok > ===( 15013;100 46/58 1/302 1/70 18/73 > )========================Can't cd to (./t/xBxrnJZxku/Big-Dummy/) lib: > No such file or directory > at /usr/home/jkeenan/Downloads/perl-5.31.2/cpan/ExtUtils- > Install/../../lib/ExtUtils/Installed.pm line 201. > Can't cd to .. from ./t/xBxrnJZxku/Big-Dummy/install-test/lib: No such > file or directory at /usr/home/jkeenan/Downloads/perl- > 5.31.2/cpan/ExtUtils-Install/../../lib/File/Find.pm line 470. > cannot fetch initial working directory: No such file or directory at > t/Installed.t line 83. > # Looks like your test exited with 2 just after 18. > ../cpan/ExtUtils-Install/t/Installed.t ............................. > Dubious, test returned 2 (wstat 512, 0x200) > Failed 55/73 subtests > ... > ../cpan/ExtUtils-Install/t/Installed.t > (Wstat: 512 Tests: 18 Failed: 0) > Non-zero exit status: 2 > Parse errors: Bad plan. You planned 73 tests but ran 18. > ##### > > Perhaps that's a race condition? > > This test failure has also been intermittently observed on smoke-tests > of the core distribution, failing with a report like this: > > ##### > ~~ ../cpan/ExtUtils-Install/t/Installed.t ...................... > FAILED Non-zero exit status: 2 Bad plan. > ##### > > In my experience, the "Non-zero exit status: 2 Bad plan" usually > indicates a segfault. > > Can you investigate? > > Thank you very much. > Jim Keenan
Here is additional evidence of intermittent failures in cpan/ExtUtils-Install/t/Installed.t in the Perl 5 core distribution test suite. Several times each month I smoke-test blead (or, occasionally, other branches) on FreeBSD-11 with a configuration that exercises 3 different C-compilers -- gcc-7.3.0, g++-7.3.0, clang-6.0.1 -- with 6 different combinations of Configure switches for each. See, for example, http://perl5.test-smoke.org/report/109163. On 5 occasions within the past 90 days, my logs/smokecurrent/log*.log file has recorded warnings like this: ##### Can't cd to (/usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/auto/) ExtTest: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. ##### Usually the unit test runs to completion and Installed.t is graded PASS -- as is the smoke-test run as a whole. And usually this warning appears only once in the 18 different configurations in the run. However, in the smoke-test I ran yesterday (the URL above), I got the warning 3 times, one of which resulted in a FAIL for Installed.t and hence an overall grade of FAIL for the smoke-test run. Here are the 3 instances, each preceded by the configuration in effect at the time. ##### 178: [2020-03-13 17:13:38+0000] Configuration: -Dusedevel -Duseithreads -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" -Dcc="clang -Qunused-arguments" -Duselongdouble -DDEBUGGING [2020-03-13 17:13:38+0000] ------------------------------------------------------------------------------ 206: Can't cd to (/usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/auto/) ExtTest: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. 1811: [2020-03-13 21:04:06+0000] Configuration: -Dusedevel -Duseithreads -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" -Dcc="gcc" -Duselongdouble -DDEBUGGING [2020-03-13 21:04:06+0000] ------------------------------------------------------------------------------ 1980: Can't cd to ../../../.. from ./t/n4k6w86Mau/Big-Dummy/install-test/other_lib/perl/Big: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/File/Find.pm line 470. cannot fetch initial working directory: No such file or directory at t/Installed.t line 83. # Looks like your test exited with 2 just after 18. # Will use Digest::MD5 error while running harness target 'test_harness': 1 at /usr/home/jkeenan/p5smoke/lib/Test/Smoke/App/RunSmoke.pm line 118. 2197: [2020-03-13 21:41:46+0000] Configuration: -Dusedevel -Duseithreads -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" -Dcc="gcc" -Dusemorebits -DDEBUGGING [2020-03-13 21:41:46+0000] ------------------------------------------------------------------------------ 2366: Can't cd to (./t/) y69u52MVuL: No such file or directory at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils-Install/../../lib/ExtUtils/Installed.pm line 201. ##### The failures occur with different C-compilers and different Configure switches. The 3 instances yesterday were each DEBUGGING builds, but the number of instances is too small to attribute statistical significance to that. As previously reported, the test failure (and presumed segfault) happened after test 18 in Installed.t. ##### $ cd t;./perl harness -v ../cpan/ExtUtils-Install/t/Installed.t; cd - ok 1 - use ExtUtils::Installed; ok 2 - _is_prefix() should match valid path prefix ok 3 - ... should not match wrong prefix ok 4 - _is_type() should be true for type of "all" ok 5 - ... should find doc file in man1dir ok 6 - ... but not prog file in man1dir ok 7 - ... should find doc file in man3dir ok 8 - ... but not prog file in man3dir ok 9 - ... should find prog file under /usr/local ok 10 - ... should not find doc file outside path ok 11 - ... nor prog file outside path ok 12 - ... nor other type anywhere ok 13 - _is_under() should return true with no dirs ok 14 - ... should find no file not under dirs ok 15 - ... should find file under dir ok 16 ok 17 ok 18 ok 19 - An object of class 'ExtUtils::Installed' isa 'ExtUtils::Installed' ##### Here is the relevant section on Installed.t. Note "line 83". ##### rmtree 'auto/FakeMod'; ok( mkpath('auto/FakeMod') ); END { rmtree 'auto' } ### <-- line 83 - reported failure ok(open(PACKLIST, '>auto/FakeMod/.packlist')); print PACKLIST 'list'; close PACKLIST; ok(open(FAKEMOD, '>auto/FakeMod/FakeMod.pm')); print FAKEMOD <<'FAKE'; package FakeMod; use vars qw( $VERSION ); $VERSION = '1.1.1'; 1; FAKE close FAKEMOD; my $fake_mod_dir = File::Spec->catdir(cwd(), 'auto', 'FakeMod'); { # avoid warning and death by localizing glob local *ExtUtils::Installed::Config; %ExtUtils::Installed::Config = ( %Config, archlibexp => cwd(), sitearchexp => $fake_mod_dir, ); # should find $fake_mod_dir via '.' in @INC local @INC = @INC; push @INC, '.' if not $INC[-1] eq '.'; my $realei = ExtUtils::Installed->new(); isa_ok( $realei, 'ExtUtils::Installed' ); isa_ok( $realei->{Perl}{packlist}, 'ExtUtils::Packlist' ); is( $realei->{Perl}{version}, $Config{version}, 'new() should set Perl version from %Config' ); ok( exists $realei->{FakeMod}, 'new() should find modules with .packlists'); isa_ok( $realei->{FakeMod}{packlist}, 'ExtUtils::Packlist' ); is( $realei->{FakeMod}{version}, '1.1.1', '... should find version in modules' ); } ##### This gives us a clue to the source of the problem. Each of the test runs that either warned or errored was a threaded build. File::Path::rmtree() is documented to have problems with threaded builds. From the documentation for version 2.16: ##### File::Path "rmtree" and "remove_tree" will not work with multithreaded applications due to its use of "chdir". At this time, no warning or error is generated in this situation. You will certainly encounter unexpected results. The implementation that surfaces this limitation will not be changed. ##### Can Installed.t be adapted to work consistently on threaded builds? Thank you very much. Jim Keenan
On Sat Mar 14 09:16:55 2020, JKEENAN wrote:
Show quoted text
> On Sat Jul 20 09:33:39 2019, JKEENAN wrote:
> > Today, while testing a tarball candidate for perl-5.31.2 on FreeBSD-
[snip]
Show quoted text
> > ##### > $ cd t;./perl harness -v ../cpan/ExtUtils-Install/t/Installed.t; cd - > > ok 1 - use ExtUtils::Installed; > ok 2 - _is_prefix() should match valid path prefix > ok 3 - ... should not match wrong prefix > ok 4 - _is_type() should be true for type of "all" > ok 5 - ... should find doc file in man1dir > ok 6 - ... but not prog file in man1dir > ok 7 - ... should find doc file in man3dir > ok 8 - ... but not prog file in man3dir > ok 9 - ... should find prog file under /usr/local > ok 10 - ... should not find doc file outside path > ok 11 - ... nor prog file outside path > ok 12 - ... nor other type anywhere > ok 13 - _is_under() should return true with no dirs > ok 14 - ... should find no file not under dirs > ok 15 - ... should find file under dir > ok 16 > ok 17 > ok 18 > ok 19 - An object of class 'ExtUtils::Installed' isa > 'ExtUtils::Installed' > #####
To assist in diagnosing this problem I have submitted a pull request providing descriptions for the 3 tests above which lack them. https://github.com/Perl-Toolchain-Gang/ExtUtils-Install/pull/17 Thank you very much. Jim Keenan
On Sat Mar 14 09:16:55 2020, JKEENAN wrote: [snip]
Show quoted text
> > Here is additional evidence of intermittent failures in cpan/ExtUtils- > Install/t/Installed.t in the Perl 5 core distribution test suite. > > Several times each month I smoke-test blead (or, occasionally, other > branches) on FreeBSD-11 with a configuration that exercises 3 > different C-compilers -- gcc-7.3.0, g++-7.3.0, clang-6.0.1 -- with 6 > different combinations of Configure switches for each. See, for > example, http://perl5.test-smoke.org/report/109163. > > On 5 occasions within the past 90 days, my logs/smokecurrent/log*.log > file has recorded warnings like this: > > ##### > Can't cd to (/usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils- > Install/../../lib/auto/) ExtTest: No such file or directory > at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils- > Install/../../lib/ExtUtils/Installed.pm line 201. > ##### > > Usually the unit test runs to completion and Installed.t is graded > PASS -- as is the smoke-test run as a whole. And usually this warning > appears only once in the 18 different configurations in the run. > > However, in the smoke-test I ran yesterday (the URL above), I got the > warning 3 times, one of which resulted in a FAIL for Installed.t and > hence an overall grade of FAIL for the smoke-test run. Here are the 3 > instances, each preceded by the configuration in effect at the time. > > ##### > 178: > [2020-03-13 17:13:38+0000] Configuration: -Dusedevel -Duseithreads > -Doptimize="-O2 -pipe -fstack-protector -fno-strict-aliasing" > -Dcc="clang -Qunused-arguments" -Duselongdouble -DDEBUGGING > [2020-03-13 17:13:38+0000] > ------------------------------------------------------------------------------ > > 206: > Can't cd to (/usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils- > Install/../../lib/auto/) ExtTest: No such file or directory > at /usr/home/jkeenan/p5smoke/perl-current/cpan/ExtUtils- > Install/../../lib/ExtUtils/Installed.pm line 201. >
Further weirdness: The string 'ExtTest' is not found in the ExtUtils-Install distribution. The only place where it occurs in the Perl 5 source code is in this test file in a *different* cpan/ distro: ##### [perl] 553 $ ack ExtTest . cpan/ExtUtils-Constant/t/Constant.t 50:my $package = "ExtTest"; 157: # Writing Makefile for ExtTest 285: # Need to make distclean to remove ../../lib/ExtTest.pm 660:} elsif ($@ !~ /Your vendor has not defined ExtTest macro NOTDEF/) { 668:my $notthere = eval { &ExtTest::NOTTHERE; }; 671:} elsif ($@ !~ /NOTTHERE is not a valid ExtTest macro/) { 707:$notdef = eval { &ExtTest::So }; 710:} elsif ($@ !~ /^So is not a valid ExtTest macro/) { 718:$notdef = eval { &ExtTest::EW }; 721:} elsif ($@ !~ /^EW is not a valid ExtTest macro/) { 835:ExtTest.xs: In function `constant_1': 836:ExtTest.xs:80: warning: multi-character character constant 837:ExtTest.xs:80: warning: case value out of range ##### jimk
On Sat Jul 20 09:33:39 2019, JKEENAN wrote:
Show quoted text
> Today, while testing a tarball candidate for perl-5.31.2 on FreeBSD- > 11, I got this unexpected test failure: > > ##### > ../cpan/ExtUtils-Install/t/Install.t ............................... > ok > ===( 15013;100 46/58 1/302 1/70 18/73 > )========================Can't cd to (./t/xBxrnJZxku/Big-Dummy/) lib: > No such file or directory > at /usr/home/jkeenan/Downloads/perl-5.31.2/cpan/ExtUtils- > Install/../../lib/ExtUtils/Installed.pm line 201. > Can't cd to .. from ./t/xBxrnJZxku/Big-Dummy/install-test/lib: No such > file or directory at /usr/home/jkeenan/Downloads/perl- > 5.31.2/cpan/ExtUtils-Install/../../lib/File/Find.pm line 470. > cannot fetch initial working directory: No such file or directory at > t/Installed.t line 83. > # Looks like your test exited with 2 just after 18. > ../cpan/ExtUtils-Install/t/Installed.t ............................. > Dubious, test returned 2 (wstat 512, 0x200) > Failed 55/73 subtests > ... > ../cpan/ExtUtils-Install/t/Installed.t > (Wstat: 512 Tests: 18 Failed: 0) > Non-zero exit status: 2 > Parse errors: Bad plan. You planned 73 tests but ran 18. > ##### > > Perhaps that's a race condition? > > This test failure has also been intermittently observed on smoke-tests > of the core distribution, failing with a report like this: > > ##### > ~~ ../cpan/ExtUtils-Install/t/Installed.t ...................... > FAILED Non-zero exit status: 2 Bad plan. > ##### > > In my experience, the "Non-zero exit status: 2 Bad plan" usually > indicates a segfault. >
As I studied this problem further I realized that t/Installed.t was structured differently from t/Install.t, t/InstallWithMM.t and t/Installapi2.t. Those test files use File::Temp::tempdir() to create a temporary directory underneath which directories and files needed to simulate module installation are created. t/Installed.t was creating that simulated installation tree directly underneath t/. This is a recipe for a race condition. Please review this pull request: https://github.com/Perl-Toolchain-Gang/ExtUtils-Install/pull/22 After merging to the master branch, we should get a new CPAN release at your earliest convenience. Thank you very much. Jim Keenan
On Thu Mar 26 21:28:11 2020, JKEENAN wrote:
Show quoted text
> On Sat Jul 20 09:33:39 2019, JKEENAN wrote:
> > Today, while testing a tarball candidate for perl-5.31.2 on FreeBSD- > > 11, I got this unexpected test failure: > > > > ##### > > ../cpan/ExtUtils-Install/t/Install.t ............................... > > ok > > ===( 15013;100 46/58 1/302 1/70 18/73 > > )========================Can't cd to (./t/xBxrnJZxku/Big-Dummy/) lib: > > No such file or directory > > at /usr/home/jkeenan/Downloads/perl-5.31.2/cpan/ExtUtils- > > Install/../../lib/ExtUtils/Installed.pm line 201. > > Can't cd to .. from ./t/xBxrnJZxku/Big-Dummy/install-test/lib: No > > such > > file or directory at /usr/home/jkeenan/Downloads/perl- > > 5.31.2/cpan/ExtUtils-Install/../../lib/File/Find.pm line 470. > > cannot fetch initial working directory: No such file or directory at > > t/Installed.t line 83. > > # Looks like your test exited with 2 just after 18. > > ../cpan/ExtUtils-Install/t/Installed.t ............................. > > Dubious, test returned 2 (wstat 512, 0x200) > > Failed 55/73 subtests > > ... > > ../cpan/ExtUtils-Install/t/Installed.t > > (Wstat: 512 Tests: 18 Failed: 0) > > Non-zero exit status: 2 > > Parse errors: Bad plan. You planned 73 tests but ran 18. > > ##### > > > > Perhaps that's a race condition? > > > > This test failure has also been intermittently observed on smoke- > > tests > > of the core distribution, failing with a report like this: > > > > ##### > > ~~ ../cpan/ExtUtils-Install/t/Installed.t ...................... > > FAILED Non-zero exit status: 2 Bad plan. > > ##### > > > > In my experience, the "Non-zero exit status: 2 Bad plan" usually > > indicates a segfault. > >
> > As I studied this problem further I realized that t/Installed.t was > structured differently from t/Install.t, t/InstallWithMM.t and > t/Installapi2.t. Those test files use File::Temp::tempdir() to create > a temporary directory underneath which directories and files needed to > simulate module installation are created. t/Installed.t was creating > that simulated installation tree directly underneath t/. This is a > recipe for a race condition. > > Please review this pull request: > https://github.com/Perl-Toolchain-Gang/ExtUtils-Install/pull/22 > > After merging to the master branch, we should get a new CPAN release > at your earliest convenience. > > Thank you very much. > Jim Keenan
With the June 20 2020 release of ExtUtils-Install-2.16, this problem should be fixed. So I'm closing this ticket. Thank you for your investigation.


This service runs on Request Tracker, is sponsored by The Perl Foundation, and maintained by Best Practical Solutions.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.