Re: [Qemu-devel] Status and RFC of patchew testings on QEMU

[Cornelia Huck <cohuck@redhat.com>]

On Fri, 28 Jul 2017 14:33:37 +0800
Fam Zheng <famz@redhat.com> wrote:

> On Fri, 07/28 02:46, Philippe Mathieu-Daudé wrote:
> > Hi Fam,
> > 
> > On 07/17/2017 03:35 AM, Fam Zheng wrote:  
> > > So far we have these tests running by patchew on each patch series:
> > > 
> > >    * Docker tests
> > >      Basically it is
> > >          make docker-test-quick@centos6 \
> > >               docker-test-build@min-glib \
> > >               docker-test-mingw@fedora"
> > > 
> > >    * checkpatch.pl
> > >      Each patch is fed to ./scripts/checkpatch.pl and all errors are reported.
> > > 
> > >    * s390x
> > >      It runs on a machine shared by Fedora team, basically only "./configure and
> > >      make", because "make check" hanging is tricky to deal with from an
> > >      automation perspective. (Ideas?)
> > > 
> > >    * FreeBSD
> > >      Like s390x.
> > >   
> > [...]  
> > > Q4: Any other improvements/features you want? (E.g. some documentation? :)  
> > 
> > I'm unsure if someone already asked about it, but is it possible to sort the
> > tests, like:
> > - fastest/cheapest
> > - more likely to fail
> > - ...
> > - scarce resources at last
> > 
> > And abort/send report on first failure.  
> 
> Different hosts run tests in parallel, currently they are three: FreeBSD, Fedora
> s390 and Fedora x86_64, and there isn't a strict order between them.
> 
> Also I'm not sure it's a good idea to _not_ run build tests if checkpatch test
> fails.

Especially as sometimes checkpatch.pl reports things that can be
overridden by the maintainer with a good conscience.

Generally, I find it useful to get *all* problems instead of playing
whack-a-mole.

> 
> One thing I agree is that checkpatch reports should be generated as
> soon as possible, which is not the case at the moment. Thanks.

+1