Skip Menu |
 

This queue is for tickets about the Test-Harness CPAN distribution.

Report information
The Basics
Id: 59186
Status: resolved
Priority: 0/
Queue: Test-Harness

People
Owner: andy [...] hexten.net
Requestors: avar [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 3.21
Fixed in: (no value)



Subject: Passing arguments to tests with :: depends on the shebang
Download (untitled) / with headers
text/plain 948b
Git uses *.sh tests that generate TAP. I can't pass arguments to them because prove will seemingly only pass through arguments perl files. This isn't consistent with its documentation, or what it should be doing. Arguments that you provide should be passed indiscriminately to all tests that prove runs. Here's a simple test case that demonstrates the issue: $ (echo '#!/bin/sh' && echo 'echo "Options are: $@"') > test.sh $ (echo '#!/usr/bin/perl' && echo 'print "Options are: @ARGV\n"') > test.pl $ prove -v ./test.* :: --some --options ./test.pl .. Options are: --some --options No subtests run ./test.sh .. Options are: No subtests run Test Summary Report ------------------- ./test.pl (Wstat: 0 Tests: 0 Failed: 0) Parse errors: No plan found in TAP output ./test.sh (Wstat: 0 Tests: 0 Failed: 0) Parse errors: No plan found in TAP output Files=2, Tests=0, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) Result: FAIL
Subject: Re: [rt.cpan.org #59186] AutoReply: Passing arguments to tests with :: depends on the shebang
Date: Thu, 8 Jul 2010 16:03:00 +0000
To: bug-Test-Harness [...] rt.cpan.org
From: Ævar Arnfjörð Bjarmason <avar [...] cpan.org>
Download (untitled) / with headers
text/plain 1.7k
On Thu, Jul 8, 2010 at 01:34, Bugs in Test-Harness via RT <bug-Test-Harness@rt.cpan.org> wrote: Show quoted text
> Git uses *.sh tests that generate TAP. I can't pass arguments to > them because prove will seemingly only pass through arguments > perl files. > > This isn't consistent with its documentation, or what it should > be doing. Arguments that you provide should be passed > indiscriminately to all tests that prove runs. > > Here's a simple test case that demonstrates the issue: > > $ (echo '#!/bin/sh' && echo 'echo  "Options are: $@"') > test.sh > $ (echo '#!/usr/bin/perl' && echo 'print "Options are: > @ARGV\n"') > test.pl > $ prove -v ./test.* :: --some --options > ./test.pl .. > Options are: --some --options > No subtests run > ./test.sh .. > Options are: > No subtests run
I've read a bit more about this and discovered that this is due to the TAP SourceHandlers. The .sh ends up being handled by the Executable class, which it seems never passes any arguments to the things it's testing: $ TAP_HARNESS_SOURCE_FACTORY_VOTES=1 prove -v ./test.* :: --some --options votes: TAP::Parser::SourceHandler::Perl: 0.9 ./test.pl .. Options are: --some --options No subtests run votes: TAP::Parser::SourceHandler::Perl: 0.25, TAP::Parser::SourceHandler::Executable: 0.8 ./test.sh .. Options are: No subtests run This is due to detect_source: http://search.cpan.org/~andya/Test-Harness-3.21/lib/TAP/Parser/IteratorFactory.pm#detect_source Passing e.g. --source=Perl to override it doesn't work. And in any case silently ignoring options that I've explicitly specified should be passed to the tests seems to be the wrong behavior. Is there some workaround that I've missed that I can use for passing through the options unscathed?
CC: Steve Purkis <steve [...] purkis.ca>
Subject: Re: [rt.cpan.org #59186] AutoReply: Passing arguments to tests with :: depends on the shebang
Date: Thu, 8 Jul 2010 18:31:36 +0000
To: bug-Test-Harness [...] rt.cpan.org
From: Ævar Arnfjörð Bjarmason <avar [...] cpan.org>
Download (untitled) / with headers
text/plain 2.6k
On Thu, Jul 8, 2010 at 16:03, Ævar Arnfjörð Bjarmason <avar@cpan.org> wrote: Show quoted text
> On Thu, Jul 8, 2010 at 01:34, Bugs in Test-Harness via RT > <bug-Test-Harness@rt.cpan.org> wrote:
>> Git uses *.sh tests that generate TAP. I can't pass arguments to >> them because prove will seemingly only pass through arguments >> perl files. >> >> This isn't consistent with its documentation, or what it should >> be doing. Arguments that you provide should be passed >> indiscriminately to all tests that prove runs. >> >> Here's a simple test case that demonstrates the issue: >> >> $ (echo '#!/bin/sh' && echo 'echo  "Options are: $@"') > test.sh >> $ (echo '#!/usr/bin/perl' && echo 'print "Options are: >> @ARGV\n"') > test.pl >> $ prove -v ./test.* :: --some --options >> ./test.pl .. >> Options are: --some --options >> No subtests run >> ./test.sh .. >> Options are: >> No subtests run
> > I've read a bit more about this and discovered that this is due to the > TAP SourceHandlers. The .sh ends up being handled by the Executable > class, which it seems never passes any arguments to the things it's > testing: > >    $ TAP_HARNESS_SOURCE_FACTORY_VOTES=1 prove -v ./test.* :: --some --options >    votes: TAP::Parser::SourceHandler::Perl: 0.9 >    ./test.pl .. >    Options are: --some --options >    No subtests run >    votes: TAP::Parser::SourceHandler::Perl: 0.25, > TAP::Parser::SourceHandler::Executable: 0.8 >    ./test.sh .. >    Options are: >    No subtests run > > This is due to detect_source: > http://search.cpan.org/~andya/Test-Harness-3.21/lib/TAP/Parser/IteratorFactory.pm#detect_source > > Passing e.g. --source=Perl to override it doesn't work. And in any > case silently ignoring options that I've explicitly specified should > be passed to the tests seems to be the wrong behavior. > > Is there some workaround that I've missed that I can use for passing > through the options unscathed?
I've discovered that this is a regression. The first bad commit is 0be9972d756f37c6dd6527cfd51cb5c6104718ef by Steve Purkis. And the first bad release is 3.17_01. Before that commit (3.17): $ TAP_HARNESS_SOURCE_FACTORY_VOTES=1 perl -I ~/g/Test-Harness/lib ~/g/Test-Harness/bin/prove -v ./test.* :: --some --options ./test.pl .. Options are: --some --options No subtests run ./test.sh .. Options are: --some --options And after (3.17_01): $ TAP_HARNESS_SOURCE_FACTORY_VOTES=1 perl -I ~/g/Test-Harness/lib ~/g/Test-Harness/bin/prove -v ./test.* :: --some --options votes: TAP::Parser::SourceHandler::Perl: 0.9 ./test.pl .. Options are: --some --options No subtests run votes: TAP::Parser::SourceHandler::Perl: 0.25, TAP::Parser::SourceHandler::Executable: 0.8 ./test.sh ..
RT-Send-CC: steve [...] purkis.ca
Thanks! This is fixed in 80cf03.
Subject: Re: [rt.cpan.org #59186] Passing arguments to tests with :: depends on the shebang
Date: Mon, 12 Jul 2010 14:15:20 +0100
To: bug-Test-Harness [...] rt.cpan.org
From: Steve Purkis <steve [...] purkis.ca>
Download (untitled) / with headers
text/plain 213b
On 11 Jul 2010, at 12:39, Andy Armstrong via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=59186 > > > Thanks! This is fixed in 80cf03.
And now tested in 3a934f8 ;-) Thanks again, Ævar. -Steve
Subject: Re: [rt.cpan.org #59186] Passing arguments to tests with :: depends on the shebang
Date: Mon, 12 Jul 2010 14:40:58 +0100
To: bug-Test-Harness [...] rt.cpan.org
From: Andy Armstrong <andy [...] hexten.net>
Download (untitled) / with headers
text/plain 261b
On 12 Jul 2010, at 14:21, Steve Purkis via RT wrote: Show quoted text
>> <URL: https://rt.cpan.org/Ticket/Display.html?id=59186 > >> >> Thanks! This is fixed in 80cf03.
> > And now tested in 3a934f8 ;-)
I wrote the failing test first Steve :P -- Andy Armstrong, Hexten


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.