Skip Menu |
 

This queue is for tickets about the Test-Harness CPAN distribution.

Report information
The Basics
Id: 14275
Status: resolved
Priority: 0/
Queue: Test-Harness

People
Owner: Nobody in particular
Requestors: adamk [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: (no value)
Fixed in: (no value)



Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
Subject: prove runs tests in unusual order
X-RT-Original-Encoding: iso-8859-1
Content-Length: 844
Download (untitled) / with headers
text/plain 844b
The prove tool is nice, but runs the tests in an unusual order. Specifically, in whatever order is returned when reading the directory. Although this means it runs them in a mostly deterministic order, it's an order that doesn't match the order that 'make test' will latter run it in. While it could be argued that this is a good thing, the --shuffle option already exists specifically to handle this case. The trivial fix (which should be enough) would be to change 115: return @hits; to 115: return sort @hits; This is a small thing, but when you intentionally rely on the test order (for example you have ordered it explicitly so that the first test that fails is where the ACTUAL bug is) then it becomes a problem. "Random and deterministic" and "Random and non-deterministic" is not the best choice to offer people :)
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
RT-Send-CC: cwest [...] cpan.org
X-RT-Original-Encoding: iso-8859-1
Content-Length: 17
Fixed in 2.53_02.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.