Skip Menu |
 

This queue is for tickets about the Test-Class CPAN distribution.

Report information
The Basics
Id: 56636
Status: resolved
Priority: 0/
Queue: Test-Class

People
Owner: Nobody in particular
Requestors: kfox [...] vulpes.com
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: (no value)

Attachments


Subject: feature request: option to turn off auto-skipping uncompleted tests
Download (untitled) / with headers
text/plain 638b
There's a recurring problem in my company's test code where test methods accidentally return from a test and Test::Class assumes they were intentionally skipped. It is very easy to skip tests explicitly, so it would be beneficial to us to turn off this feature. With auto-skipping on, sloppy_test passes. With auto-skipping off, sloppy_test should fail. sub sloppy_test :Test(2) { my ($self) = @_; ok(1); } sub precise_test :Test(2) { my ($self) = @_; ok(1); $self->builder->skip("not tested"); } sub another_precise_test :Test(2) { my ($self) = @_; ok(1); SKIP: { skip "not tested", 1; } }
Download (untitled) / with headers
text/plain 1012b
On Thu Apr 15 09:19:18 2010, KENFOX wrote: Show quoted text
> There's a recurring problem in my company's test code where test
methods Show quoted text
> accidentally return from a test and Test::Class assumes they were > intentionally skipped. It is very easy to skip tests explicitly, so it > would be beneficial to us to turn off this feature. > > With auto-skipping on, sloppy_test passes. With auto-skipping off, > sloppy_test should fail. > > > sub sloppy_test :Test(2) { > my ($self) = @_; > ok(1); > } > > sub precise_test :Test(2) { > my ($self) = @_; > ok(1); > $self->builder->skip("not tested"); > } > > sub another_precise_test :Test(2) { > my ($self) = @_; > ok(1); > SKIP: { > skip "not tested", 1; > } > }
This sounds reasonable to me. I assume you would want tests which would have been "auto-skipped" to be failures? Can you suggest an unobtrusive interface for how this should be implemented? I've commit rights now and have been addressing some issues here. Cheers, Ovid
Subject: Re: [rt.cpan.org #56636] feature request: option to turn off auto-skipping uncompleted tests
Date: Thu, 22 Apr 2010 09:47:05 -0400
To: bug-Test-Class [...] rt.cpan.org
From: Ken Fox <ken [...] vulpes.com>
Download (untitled) / with headers
text/plain 1.6k
I'd like to have the test attributes support something like :Test(exactly 2). Tests with "exactly" would pass that info back to _run_method which would turn off implicit autoskip. Thanks for taking a look at this. I've messed around a bit with Test::Class, so if you'd rather have me give you a patch, that's fine with me. It would be more fun to hack on the code if it were designed for subclassing. On Thu, Apr 22, 2010 at 5:35 AM, Curtis 'Ovid' Poe via RT < bug-Test-Class@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=56636 > > > On Thu Apr 15 09:19:18 2010, KENFOX wrote:
> > There's a recurring problem in my company's test code where test
> methods
> > accidentally return from a test and Test::Class assumes they were > > intentionally skipped. It is very easy to skip tests explicitly, so it > > would be beneficial to us to turn off this feature. > > > > With auto-skipping on, sloppy_test passes. With auto-skipping off, > > sloppy_test should fail. > > > > > > sub sloppy_test :Test(2) { > > my ($self) = @_; > > ok(1); > > } > > > > sub precise_test :Test(2) { > > my ($self) = @_; > > ok(1); > > $self->builder->skip("not tested"); > > } > > > > sub another_precise_test :Test(2) { > > my ($self) = @_; > > ok(1); > > SKIP: { > > skip "not tested", 1; > > } > > }
> > This sounds reasonable to me. I assume you would want tests which would > have been "auto-skipped" to be failures? Can you suggest an unobtrusive > interface for how this should be implemented? I've commit rights now > and have been addressing some issues here. > > Cheers, > Ovid >
From: bitcard-20100921 [...] djce.org.uk
Download (untitled) / with headers
text/plain 518b
On Thu Apr 15 09:19:18 2010, KENFOX wrote: Show quoted text
> There's a recurring problem in my company's test code where test methods > accidentally return from a test and Test::Class assumes they were > intentionally skipped. It is very easy to skip tests explicitly, so it > would be beneficial to us to turn off this feature.
See attached for a patch which I believe addresses this issue. It introduces an option called "fail_if_returned_early" at the class level; if true, then an early return causes failures instead of skips.
Subject: Test-Class-0.36-fail-if-returned-early.patch
diff -ru Test-Class-0.36/lib/Test/Class.pm Test-Class-0.36-fail-if-returned-early/lib/Test/Class.pm --- Test-Class-0.36/lib/Test/Class.pm 2010-08-19 14:25:59.000000000 +0100 +++ Test-Class-0.36-fail-if-returned-early/lib/Test/Class.pm 2010-09-20 11:22:36.000000000 +0100 @@ -286,13 +286,19 @@ $skip_reason = "$method died"; $exception = ''; } else { - $Builder->skip( $skip_reason ); + if ($self->fail_if_returned_early) { + $Builder->ok(0, "(returned before plan complete)"); + } else { + $Builder->skip( $skip_reason ); + } }; }; }; return(_all_ok_from($self, $num_start)); }; +sub fail_if_returned_early { 0 } + sub _show_header { my ($self, @tests) = @_; return if $Builder->has_plan; @@ -871,9 +877,25 @@ Startup methods are a special case. Since startup methods will usually be creating state needed by all the other test methods an exception within a startup method will prevent all other test methods running. +=head1 RETURNING EARLY + +If a test method returns before it has run all of its tests, by default the missing tests are deemed to have been skipped; see L<"Skipped Tests"> for more information. + +However, if the class's C<fail_if_returned_early> method returns true, then the missing tests will be deemed to have failed. For example, + + package MyClass; + use base 'Test::Class'; + sub fail_if_returned_early { 1 } + + sub oops : Tests(8) { + for (my $n=1; $n*$n<50; ++$n) { + ok 1, "$n squared is less than fifty"; + } + } + =head1 SKIPPED TESTS -You can skip the rest of the tests in a method by returning from the method before all the test have finished running. The value returned is used as the reason for the tests being skipped. +You can skip the rest of the tests in a method by returning from the method before all the test have finished running (but see L<"Returning Early"> for how to change this). The value returned is used as the reason for the tests being skipped. This makes managing tests that can be skipped for multiple reasons very simple. For example: @@ -1495,6 +1517,10 @@ See the section on the L</"GENERAL FILTERING OF TESTS"> for more information. +=item B<fail_if_returned_early> + +Controls what happens if a method returns before it has run all of its tests. It is called with no arguments in boolean context; if it returns true, then the missing tests fail, otherwise, they skip. See L<"Returning Early"> and L<"Skipped Tests">. + =back diff -ru Test-Class-0.36/t/runtests_return.t Test-Class-0.36-fail-if-returned-early/t/runtests_return.t --- Test-Class-0.36/t/runtests_return.t 2010-08-19 14:25:59.000000000 +0100 +++ Test-Class-0.36-fail-if-returned-early/t/runtests_return.t 2010-09-20 10:01:40.000000000 +0100 @@ -13,9 +13,25 @@ ok(-w "/Library", "/Library writable") }; +package Bar; +use Test::More; +use base qw(Test::Class); + +sub fail_if_returned_early { 1 } + +sub darwin_only : Test { + return("darwin only test");# unless $^O eq "darwin"; + ok(-w "/Library", "/Library writable") +}; + package main; -use Test::Builder::Tester tests => 1; +use Test::Builder::Tester tests => 2; test_out("ok 1 # skip darwin only test"); Foo->runtests; -test_test("early return handled"); +test_test("early return handled (skip)"); + +test_out("not ok 1 - (returned before plan complete)"); +test_err(qr/.*in Bar->darwin_only.*/s); +Bar->runtests; +test_test("early return handled (fail)");
Download (untitled) / with headers
text/plain 119b
fail_if_returned_early() is now in the 0.37 release on github - will be released onto CPAN in the next couple of days.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.