Re: [Qemu-devel] Slow tests on "make check" (was Re: [PULL 6/9] Convert error_report() to warn_report())

["Daniel P. Berrange" <berrange@redhat.com>]

On Tue, Jul 25, 2017 at 04:08:18PM -0300, Eduardo Habkost wrote:
> On Mon, Jul 24, 2017 at 09:30:06AM +0200, Paolo Bonzini wrote:
> > On 24/07/2017 08:30, Markus Armbruster wrote:
> > > Markus Armbruster <armbru@redhat.com> writes:
> > > 
> > >> Kevin Wolf <kwolf@redhat.com> writes:
> > >>
> > >>> Am 13.07.2017 um 15:27 hat Markus Armbruster geschrieben:
> > >>>> From: Alistair Francis <alistair.francis@xilinx.com>
> > >>>>
> > >>>> Convert all uses of error_report("warning:"... to use warn_report()
> > >>>> instead. This helps standardise on a single method of printing warnings
> > >>>> to the user.
> > >>>>
> > >>>> All of the warnings were changed using these two commands:
> > >>>>     find ./* -type f -exec sed -i \
> > >>>>       's|error_report(".*warning[,:] |warn_report("|Ig' {} +
> > >>>>
> > >>>> Indentation fixed up manually afterwards.
> > >>>>
> > >>>> The test-qdev-global-props test case was manually updated to ensure that
> > >>>> this patch passes make check (as the test cases are case sensitive).
> > >>>
> > >>> This patch broke qemu-iotests 051 because it neglected to update the
> > >>> reference output. Not sure if a change of the message was even intended,
> > >>> but with a error location prefix, the order changes:
> > >>>
> > >>> -(qemu) QEMU_PROG: -drive if=scsi,media=cdrom: warning: bus=0,unit=0 is deprecated with this machine type
> > >>> +(qemu) warning: qemu-system-x86_64: -drive if=scsi,media=cdrom: bus=0,unit=0 is deprecated with this machine type
> > >>>
> > >>> Personally, I would expect the error location or at least the program
> > >>> name to come first even for warnings.
> > >>
> > >> I'll fix it.
> > >>
> > >> While focusing on something other than block, I forget qemu-iotests
> > >> exist.  My fault, but it's a pretty common fault.  I reiterate my plea
> > >> to include (a sensible subset of) it in "make check".
> > > 
> > > Oh, 051 isn't run by "make check-block".  It could just as well not
> > > exist then.
> > > 
> > > What's the recommended way to run all iotests a build of QEMU can run?
> > 
> > I use "path-to-srcdir/tests/check-block.sh".  It takes about 20 minutes.
> > 
> > There are two issues with check-block:
> > 
> > * tests that require QEMU are skipped (rationale: you might be building
> > without the x86_64-softmmu target).
> > 
> > * it only runs the "quick" group, but I think we should reverse the
> > direction and mark slow tests specifically (that would cut about 10
> > minutes out of the 20).
> 
> About slow tests: I never included device-crash-test on "make
> check" because it would take too long to run.  Also, "make
> check-help" lies when it says "make check" runs all tests,
> because it doesn't run check-block.

Even if you also run "check-block", it would still be a lie because you'd
not be running block tests for anything other than qcow2, which is far
from "all tests".


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|