Skip Menu |
 

This queue is for tickets about the Test-Harness CPAN distribution.

Report information
The Basics
Id: 11723
Status: resolved
Priority: 0/
Queue: Test-Harness

People
Owner: andy [...] hexten.net
Requestors: kevin.benton [...] amd.com
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: 2.46
Fixed in: (no value)



Subject: Wishlist: Add ability to stop testing on failure
Download (untitled) / with headers
text/plain 668b
For developers who use Test::Harness, it would be helpful if we had the ability to tell the system to abort as soon as it saw a discrepancy. It would also be nice to be able to specify verbosity for error messages. I'm doing Bugzilla development currently and the testing suite doesn't tell me the filename having issues making it more difficult for me to locate the problem. The testing suite is Test::Harness. I've found ways around the limitation, however, it would be to everyone's advantage if we could tell the test suite to stop and give us more information about the problem found. I'm using Test-Harness-2.46 on Perl 5.8.0 under Linux 2.4.20-31.9 (RH9).
Date: Thu, 3 Mar 2005 16:04:41 -0500
From: Michael G Schwern <schwern [...] pobox.com>
To: Guest via RT <bug-Test-Harness [...] rt.cpan.org>
CC: undisclosed-recipients: ;
Subject: Re: [cpan #11723] Wishlist: Add ability to stop testing on failure
RT-Send-Cc:
Download (untitled) / with headers
text/plain 2.6k
On Wed, Mar 02, 2005 at 04:38:30PM -0500, Guest via RT wrote: Show quoted text
> For developers who use Test::Harness, it would be helpful if we had the ability to tell the system to abort as soon as it saw a discrepancy. It would also be nice to be able to specify verbosity for error messages. I'm doing Bugzilla development currently and the testing suite doesn't tell me the filename having issues making it more difficult for me to locate the problem. The testing suite is Test::Harness. I've found ways around the limitation, however, it would be to everyone's advantage if we could tell the test suite to stop and give us more information about the problem found. > > I'm using Test-Harness-2.46 on Perl 5.8.0 under Linux 2.4.20-31.9 (RH9).
First, a clarification. Test::Harness is not a testing suite. It runs the individual test files, parses their output and provides a summary of the failure. The test files themselves are written using a variety of different techniques and testing libraries, the most popular of them being Test::More. Regardless, Test::Harness should be reporting which testing file contained the error like so: 0 ~/tmp$ perl -MTest::Harness -e 'runtests @ARGV' test.t test....FAILED test 1 Failed 1/1 tests, 0.00% okay Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- test.t 1 1 100.00% 1 Failed 1/1 test scripts, 0.00% okay. 1/1 subtests failed, 0.00% okay. Additional diagnostic information may be provided by your test programs assuming they are using a decent testing library such as Test::More. This has nothing to do with Test::Harness. 0 ~/tmp$ perl -MTest::Harness -e 'runtests @ARGV' new.t new....NOK 13 # Failed test (new.t at line 41) # 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' # doesn't match '(?-xism:^Mozilla.+compatible.+VMS)' # Looks like you failed 1 test of 14. new....dubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 13 Failed 1/14 tests, 92.86% okay Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- new.t 1 256 14 1 7.14% 13 Failed 1/1 test scripts, 0.00% okay. 1/14 subtests failed, 92.86% okay. Given that Test::Harness reports which test file failed we'd need more information as to what your problem is. Perhaps its a Bugzilla limitation?
From: nikc [...] cpan.org
Download (untitled) / with headers
text/plain 880b
[guest - Wed Mar 2 16:38:28 2005]: Show quoted text
> For developers who use Test::Harness, it would be helpful if we had > the ability to tell the system to abort as soon as it saw a > discrepancy. It would also be nice to be able to specify verbosity > for error messages. I'm doing Bugzilla development currently and > the testing suite doesn't tell me the filename having issues making > it more difficult for me to locate the problem. The testing suite > is Test::Harness. I've found ways around the limitation, however, > it would be to everyone's advantage if we could tell the test suite > to stop and give us more information about the problem found. > > I'm using Test-Harness-2.46 on Perl 5.8.0 under Linux 2.4.20-31.9 > (RH9).
You can do this. See the documentation for "Bail out!" in the Test::Harness::TAP documentation, and Test::More::BAIL_OUT.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.