Skip Menu |
 

This queue is for tickets about the Test-Harness CPAN distribution.

Report information
The Basics
Id: 115522
Status: open
Priority: 0/
Queue: Test-Harness

People
Owner: Nobody in particular
Requestors: PETDANCE [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 3.36
Fixed in: (no value)



Subject: Failing subtest within a subtest within a TODO puts errors to the screen
Download (untitled) / with headers
text/plain 1.3k
A failing subtest within a subtest within a TODO will emit the failures on prove. I would expect all the diagnostics for the failures to be hidden, but the nested ones make their way out to the user. The test passes overall, but has messy diagnostics that we don't want to see. The horse/sheep mismatch is suppressed correctly, but the cow/pig mismatch goes to the screen. $ cat foo.t use warnings; use strict; use Test::More 'no_plan'; diag "Perl $], Test::More $Test::More::VERSION\n"; TODO: { local $TODO = 'Have not changed laws of biology'; subtest 'Farm animals' => sub { is( 'horse', 'sheep', 'Horses and sheep are the same' ); subtest 'Barn animals' => sub { is( 'cow', 'pig', 'Cows and pigs are the same' ); }; is( 'duck', 'goose', 'Duck and goose are the same' ); }; } $ prove foo.t [16:04:05] foo.t .. # Perl 5.020003, Test::More 1.302015 # Failed test 'Cows and pigs are the same' # at foo.t line 13. # got: 'cow' # expected: 'pig' # Looks like you failed 1 test of 1. [16:04:05] foo.t .. ok 63 ms ( 0.00 usr 0.00 sys + 0.06 cusr 0.00 csys = 0.06 CPU) [16:04:05] All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.06 cusr 0.00 csys = 0.09 CPU) Result: PASS $ prove --version TAP::Harness v3.36 and Perl v5.20.3
Subject: Re: [rt.cpan.org #115522] Failing subtest within a subtest within a TODO puts errors to the screen
Date: Wed, 22 Jun 2016 00:11:53 +0200
To: bug-Test-Harness [...] rt.cpan.org
From: Leon Timmermans <leont [...] cpan.org>
Download (untitled) / with headers
text/plain 1.6k
On Tue, Jun 21, 2016 at 11:16 PM, Andy Lester via RT < bug-Test-Harness@rt.cpan.org> wrote: Show quoted text
> A failing subtest within a subtest within a TODO will emit the failures on > prove. I would expect all the diagnostics for the failures to be hidden, > but the nested ones make their way out to the user. The test passes > overall, but has messy diagnostics that we don't want to see. > > The horse/sheep mismatch is suppressed correctly, but the cow/pig mismatch > goes to the screen. > > > > $ cat foo.t > use warnings; > use strict; > use Test::More 'no_plan'; > > diag "Perl $], Test::More $Test::More::VERSION\n"; > TODO: { > local $TODO = 'Have not changed laws of biology'; > > subtest 'Farm animals' => sub { > is( 'horse', 'sheep', 'Horses and sheep are the same' ); > > subtest 'Barn animals' => sub { > is( 'cow', 'pig', 'Cows and pigs are the same' ); > }; > > is( 'duck', 'goose', 'Duck and goose are the same' ); > }; > } > > > $ prove foo.t > [16:04:05] foo.t .. # Perl 5.020003, Test::More 1.302015 > > > # Failed test 'Cows and pigs are the same' > # at foo.t line 13. > # got: 'cow' > # expected: 'pig' > # Looks like you failed 1 test of 1. > > > [16:04:05] foo.t .. ok 63 ms ( 0.00 usr 0.00 sys + 0.06 cusr 0.00 > csys = 0.06 CPU) > [16:04:05] > All tests successful. > Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.06 cusr > 0.00 csys = 0.09 CPU) > Result: PASS >
I can't reproduce this. Why do you think this is a bug in Test::Harness and not Test::More? What version of Test::More are you using? What happens when you run the file directly (and redirect stdout/stderr) ? Leon
Subject: Re: [rt.cpan.org #115522] Failing subtest within a subtest within a TODO puts errors to the screen
Date: Wed, 22 Jun 2016 00:27:21 +0200
To: bug-Test-Harness [...] rt.cpan.org
From: Leon Timmermans <fawaka [...] gmail.com>
Download (untitled) / with headers
text/plain 442b
On Wed, Jun 22, 2016 at 12:11 AM, Leon Timmermans <leont@cpan.org> wrote: Show quoted text
> I can't reproduce this. Why do you think this is a bug in Test::Harness > and not Test::More? What version of Test::More are you using? What happens > when you run the file directly (and redirect stdout/stderr) ? >
I now see you had included the Test::More version, and as I suspected it's a Test2 based version. That would be my main suspect right now TBH. Leon
Download (untitled) / with headers
text/plain 501b
My apologies for not trying any different versions before submitting this. I just tried it on an older Test::More and it passes. $ prove --version TAP::Harness v3.29 and Perl v5.10.1 $ prove foo.t [21:11:26] foo.t .. # Perl 5.010001, Test::More 1.001002 [21:11:26] foo.t .. ok 33 ms [21:11:26] All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.00 csys = 0.03 CPU) Result: PASS I submitted it as https://github.com/Test-More/test-more/issues/684.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.