|Subject:||Strawberry needs a post-build QA test process|
|X-Mailer:||MIME-tools 5.427 (Entity 5.427)|
Over the last 6 months or so, most of the serious bugs that have been reported relate to mistakes that we really should have picked up in beta testing, but didn't. Examples: Bad PPM repository, badly broken C stuff, distributions missing support DLLs etc. I think we've seen enough of these that we should consider this to be a systematic flaw in the Strawberry production process rather than a collection of small oversights. I'd like to propose we develop a post-installation QA mechanism, so that beta testers and other can drop the installer onto a machine, fire up CPAN and just "install Perl::Dist::Strawberry::QA" or something similar. These post-production tests would potentially require that the test machines have MySQL, Postgres etc databases installed on them, and have Test::Database configured. It might try some basic tests for MySQL and Postgres (not necesarily the entire test suite), SQLite, installation via PPM, etc (installing the QA module itself tests CPAN installation). We could add a series of different regression tests to this module as we uncover and fix various problems which aren't caught by the module's own test scripts.