Re: [Qemu-devel] Suggestion for testing framework

[Avi Kivity <avi@qumranet.com>]

Ian Jackson wrote:
> Paul Brook writes ("Re: [Qemu-devel] Suggestion for testing framework"):
>   
>> We were talking about a tester that does periodic long running tests off svn 
>> trunk, and reports the results. Individual developers are not directly 
>> involved.
>>     
>
> Just reporting the results is all very well, but as others have said
> testing is only useful with a very firm commitment to deal with the
> results of reports.  Few Free Software projects can manage to do this
> without some kind of formal and automatic linkage of QA passes into
> code distribution or propagation.  Without that, continuous discipline
> is needed - and if it ever slips, the test failures become an
> ever-deepening swamp.
>
> One common approach to this problem used with success in very
> different ways by both Debian and Xen and probably many others, is to
> maintain parallel branches: one (call it `unstable') is changed
> freely, and the other (call it `testing') is updated from unstable
> only when the QA criteria (whatever those are) are met.
>   

IMO parallel branches are overkill, especially given the current lack of 
reviewing and committing.

I'd suggest having just two rules:
- patch authors make a reasonable effort to ensure no regression in 
compilation or execution; this does not include running the entire test 
suite
- if a regression is discovered by the test suite, the offending patch 
is either fixed in a short time or reverted

This way progress can still be made, but quality is maintained

-- 
error compiling committee.c: too many arguments to function