Skip Menu |
 

This queue is for tickets about the Test-Class CPAN distribution.

Report information
The Basics
Id: 103224
Status: open
Priority: 0/
Queue: Test-Class

People
Owner: Nobody in particular
Requestors: WolfSage [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: (no value)
Fixed in: (no value)



Subject: Test::Class doesn't detect early exit from tests
Download (untitled) / with headers
text/plain 1.2k
Sometimes the code you're testing might have embedded somewhere deep within it a call to exit (or, a call to die, and a $SIG{__DIE__} handler that causes it to exit gracefully.) With Test::More, this will be caught as long as you define a plan or use done_testing: $ cat test2.pl #!/usr/bin/perl use strict; use warnings; use Test::More; ok(1); exit; ok(2); done_testing; $ perl test2.pl ok 1 # Tests were run but no plan was declared and done_testing() was not seen. However, with Test::Class, even if you define a plan in your subtests, an early exit can cause the rest of them to run and it seems like everything is okay: $ cat test.pl #!/usr/bin/perl use strict; use warnings; package My::Tests; use base qw(Test::Class); use Test::More; sub t01_the_test : Tests(3) { my ($self) = @_; ok(1); exit; } sub t02_more_tests : Tests { ok(1); } My::Tests->runtests; $ perl test.pl ok 1 - t01 the test 1..1 I think early exits should be caught - at least in the case where a plan is specified, but realistically in both if possible. (I'd like to pretend there's an explicit done_testing at the end of every subtest that doesn't have a plan defined.) -- Matthew Horsfall (alh)
Download (untitled) / with headers
text/plain 773b
On 2015-03-31 06:33:45, WOLFSAGE wrote: Show quoted text
> However, with Test::Class, even if you define a plan in your subtests, > an early exit can cause the rest of them to run and it seems like > everything is okay:
This is almost "by design": https://metacpan.org/pod/Test::Class#RETURNING-EARLY However, dies are correctly detected: https://metacpan.org/pod/Test::Class#HANDLING-EXCEPTIONS Show quoted text
> I'd like to pretend there's an explicit done_testing at the end of every subtest that doesn't have a plan defined.
Test::Class predates done_testing. In fact it doesn't even use subtests to encapsulate each test method as its own test, instead doing some convoluted backflips to swap out the Test::Builder object. It most definitely needs a rewrite to use more modern TB internals. :( :(
Subject: Re: [rt.cpan.org #103224] Test::Class doesn't detect early exit from tests
Date: Tue, 31 Mar 2015 17:33:03 -0400
To: "bug-Test-Class [...] rt.cpan.org" <bug-Test-Class [...] rt.cpan.org>
From: "Matthew Horsfall (alh)" <wolfsage [...] gmail.com>
Download (untitled) / with headers
text/plain 1.1k
Show quoted text
> On Mar 31, 2015, at 5:17 PM, "Karen Etheridge via RT" <bug-Test-Class@rt.cpan.org> wrote: > > <URL: https://rt.cpan.org/Ticket/Display.html?id=103224 > >
>> On 2015-03-31 06:33:45, WOLFSAGE wrote: >> >> However, with Test::Class, even if you define a plan in your subtests, >> an early exit can cause the rest of them to run and it seems like >> everything is okay:
> > This is almost "by design": https://metacpan.org/pod/Test::Class#RETURNING-EARLY >
That's bad enough, but at least early returns are limited to the enclosing test method and the return statement must be in that method. An exit causes all subsequent test methods to never run and can be hidden anywhere in the program :/ Show quoted text
> However, dies are correctly detected: https://metacpan.org/pod/Test::Class#HANDLING-EXCEPTIONS >
>> I'd like to pretend there's an explicit done_testing at the end of every subtest that doesn't have a plan defined.
> > Test::Class predates done_testing. In fact it doesn't even use subtests to encapsulate each test method as its own test, instead doing some convoluted backflips to swap out the Test::Builder object. It most definitely needs a rewrite to use more modern TB internals. :( :(


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.