Skip Menu |
 

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the CPAN-Reporter CPAN distribution.

Report information
The Basics
Id: 31373
Status: resolved
Priority: 0/
Queue: CPAN-Reporter

People
Owner: Nobody in particular
Requestors: ANDK [...] cpan.org
Cc: ANDYA [...] cpan.org
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



CC: ANDYA [...] cpan.org
Subject: Test::Harness 3.05 breaks CPAN::Reporter
MIME-Version: 1.0
X-Mailer: MIME-tools 5.418 (Entity 5.418)
Charset: utf8
Content-Disposition: inline
Content-Type: text/plain
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 3145
Subject line says all. Andy, can you please run a CPAN::Reporter test before a release? This is not the only test failing, there are more, but please test yourself: t/52_test_report_unknown...... # Failed test 't-NoOutput: 'make test' returned 1' # at t/52_test_report_unknown.t line 69. # Failed test 't-NoOutput: 'make test' grade reported as 'unknown'' # at t/52_test_report_unknown.t line 69. # got: 'fail' # expected: 'unknown' # STDOUT: # make[4]: Entering directory `/tmp/QZ_WiyY7O7' # cp lib/Bogus/Module.pm blib/lib/Bogus/Module.pm # PERL_DL_NONLAZY=1 /home/src/perl/repoperls/installed-perls/perl/pKhaQOm/perl-5.8.0@32603/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t # t/01_Bogus...... No subtests run # # Test Summary Report # ------------------- # t/01_Bogus.t (Wstat: 0 Tests: 0 Failed: 0) # Parse errors: No plan found in TAP output # Files=1, Tests=0, 1 wallclock secs ( 0.02 usr 0.00 sys + 0.04 cusr 0.00 csys = 0.06 CPU) # Result: FAIL # Failed 1/1 test programs. 0/0 subtests failed. # make[4]: *** [test_dynamic] Error 255 # make[4]: Leaving directory `/tmp/QZ_WiyY7O7' # (make test exited with 512) # CPAN::Reporter: Test result is 'fail', One or more tests failed. # CPAN::Reporter: preparing a CPAN Testers report for Bogus-Module-1.23 # Do you want to edit the test report? (yes/no) [no] no # CPAN::Reporter: sending test report with 'fail' to no_one@example.com, JOHNQP@cpan.org # # # STDERR: # # Failed test 't-NoOutput: 'make test' grade explanation correct' # at t/52_test_report_unknown.t line 69. # got: 'One or more tests failed.' # expected: 'No tests were run.' # Failed test 't-NoOutput: 'perl Build test' returned 1' # at t/52_test_report_unknown.t line 69. # Failed test 't-NoOutput: 'perl Build test' grade reported as 'unknown'' # at t/52_test_report_unknown.t line 69. # got: 'fail' # expected: 'unknown' # STDOUT: # Copying lib/Bogus/Module.pm -> blib/lib/Bogus/Module.pm # t/01_Bogus...... No subtests run # # Test Summary Report # ------------------- # t/01_Bogus.t (Wstat: 0 Tests: 0 Failed: 0) # Parse errors: No plan found in TAP output # Files=1, Tests=0, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.04 cusr 0.01 csys = 0.07 CPU) # Result: FAIL # Failed 1/1 test programs. 0/0 subtests failed. # (/home/src/perl/repoperls/installed-perls/perl/pKhaQOm/perl-5.8.0@32603/bin/perl Build test exited with 65280) # CPAN::Reporter: Test result is 'fail', One or more tests failed. # CPAN::Reporter: preparing a CPAN Testers report for Bogus-Module-1.23 # Do you want to edit the test report? (yes/no) [no] no # CPAN::Reporter: sending test report with 'fail' to no_one@example.com, JOHNQP@cpan.org # # # STDERR: # # Failed test 't-NoOutput: 'perl Build test' grade explanation correct' # at t/52_test_report_unknown.t line 69. # got: 'One or more tests failed.' # expected: 'No tests were run.' # Looks like you failed 6 tests of 46. Dubious, test returned 6 (wstat 1536, 0x600) Failed 6/46 subtests Hope this helps, I have no time left to investigate.
MIME-Version: 1.0
X-Mailer: MIME-tools 5.418 (Entity 5.418)
Content-Disposition: inline
Charset: utf8
Message-Id: <rt-3.6.HEAD-27056-1197380223-927.31373-0-0 [...] rt.cpan.org>
Content-Type: text/plain
Content-Transfer-Encoding: binary
From: DAGOLDEN [...] cpan.org
X-RT-Original-Encoding: utf-8
X-RT-Original-Encoding: utf-8
Content-Length: 503
Download (untitled) / with headers
text/plain 503b
On Tue Dec 11 03:20:13 2007, ANDK wrote: Show quoted text
> # Failed test 't-NoOutput: 'make test' grade reported as 'unknown'' > # at t/52_test_report_unknown.t line 69. > # got: 'fail' > # expected: 'unknown'
Looks like 3.05 treats no output as a test failure instead of unknown. When I run t/dist/No-Output manually, I get "Parse errors: No plan found in TAP output". If that's the new standard, I can update CPAN::Reporter, but that's not consistent with the old behavior of Test::Harness. David
MIME-Version: 1.0
In-Reply-To: <rt-3.6.HEAD-27056-1197380223-927.31373-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.418 (Entity 5.418)
Content-Disposition: inline
Charset: utf8
References: <rt-3.6.HEAD-27056-1197380223-927.31373-0-0 [...] rt.cpan.org>
Message-Id: <rt-3.6.HEAD-27040-1197382121-1651.31373-0-0 [...] rt.cpan.org>
Content-Type: text/plain
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Original-Encoding: utf-8
Content-Length: 1837
Download (untitled) / with headers
text/plain 1.7k
On Tue Dec 11 08:37:04 2007, DAGOLDEN wrote: Show quoted text
> On Tue Dec 11 03:20:13 2007, ANDK wrote:
> > # Failed test 't-NoOutput: 'make test' grade reported as 'unknown'' > > # at t/52_test_report_unknown.t line 69. > > # got: 'fail' > > # expected: 'unknown'
> > Looks like 3.05 treats no output as a test failure instead of unknown. > When I run t/dist/No-Output manually, I get "Parse errors: No plan found > in TAP output". > > If that's the new standard, I can update CPAN::Reporter, but that's not > consistent with the old behavior of Test::Harness.
Sorry David, Andreas. Once we resolve this problem I'll add a run of CPAN::Reporter's tests to one of our smoke boxes which should ensure it can't happen again. The current behaviour is return ( $has_errors || $total != $passed ) ? 'FAIL' : $total ? 'PASS' : 'NOTESTS'; $has_errors covers things like 'no plan in TAP'. David: this feature exists primarily for CPAN::Reporter - so it should do what you want it to. Can we bottom out what the semantics should be? Currently: PASS: A non-zero number of tests passed, no parse errors in TAP NOTESTS: No tests run (perhaps all skipped) and no errors in TAP FAIL: Test failures, TAP errors A test that outputs nothing - or produces output that has TAP parse errors is a fail. prove t/dist/t-NoOutput/t/01_Bogus.t t/dist/t-NoOutput/t/01_Bogus...... No subtests run Test Summary Report ------------------- t/dist/t-NoOutput/t/01_Bogus.t (Wstat: 0 Tests: 0 Failed: 0) Parse errors: No plan found in TAP output Files=1, Tests=0, 0 wallclock secs ( 0.01 usr 0.01 sys + 0.02 cusr 0.00 csys = 0.04 CPU) Result: FAIL We need to decide: should a test that outputs nothing be FAIL or NOTESTS. I think it's a FAIL because NOTESTS means 'no tests expected' rather than 'no tests run'.
MIME-Version: 1.0
In-Reply-To: <rt-3.6.HEAD-27040-1197382121-1651.31373-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.418 (Entity 5.418)
Content-Disposition: inline
Charset: utf8
References: <rt-3.6.HEAD-27056-1197380223-927.31373-0-0 [...] rt.cpan.org> <rt-3.6.HEAD-27040-1197382121-1651.31373-0-0 [...] rt.cpan.org>
Message-Id: <rt-3.6.HEAD-27040-1197382422-284.31373-0-0 [...] rt.cpan.org>
Content-Type: text/plain
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Original-Encoding: utf-8
Content-Length: 120
Download (untitled) / with headers
text/plain 120b
As an aside I didn't initially see this because it's in the CPAN::Reporter queue. It should probably be in T::H's queue.
MIME-Version: 1.0
In-Reply-To: <rt-3.6.HEAD-27040-1197382121-1651.31373-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.418 (Entity 5.418)
Content-Disposition: inline
Charset: utf8
References: <rt-3.6.HEAD-27056-1197380223-927.31373-0-0 [...] rt.cpan.org> <rt-3.6.HEAD-27040-1197382121-1651.31373-0-0 [...] rt.cpan.org>
Message-Id: <rt-3.6.HEAD-27055-1197383428-682.31373-0-0 [...] rt.cpan.org>
Content-Type: text/plain
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Original-Encoding: utf-8
Content-Length: 821
Download (untitled) / with headers
text/plain 821b
On Tue Dec 11 09:08:41 2007, ANDYA wrote: Show quoted text
> David: this feature exists primarily for CPAN::Reporter - so it should > do what you want it to. > Can we bottom out what the semantics should be?
I put the question to perl-qa and tapx-dev. You should see those shortly if you haven't already. We can do this either way once everyone agrees on what the right semantics should be. Either way, it looks like I'll have to update CPAN::Reporter. I just added a new t-NoOutput-die test distribution (call Test::More then use a non-existent module) and T::H 2.64 doesn't see that as a failure, but as unknown. So in that sense, T::H 3.05 is right to flag non-zero exit as failure, regardless of output. So that means more fun custom testing logic depending on whether we have 2.XX, 3.0[0-4], 3.05, 3.0[>5] and so on. David
MIME-Version: 1.0
X-Mailer: MIME-tools 5.418 (Entity 5.418)
Content-Disposition: inline
Charset: utf8
Message-Id: <rt-3.6.HEAD-27040-1197468148-1158.31373-0-0 [...] rt.cpan.org>
Content-Type: text/plain
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Original-Encoding: utf-8
Content-Length: 15
Fixed in 1.0601


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.