Re: [Qemu-devel] [PULL 42/43] piix4: add acpi pci hotplug support

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Oct 16, 2013 at 11:26:11PM +0200, Paolo Bonzini wrote:
> Il 16/10/2013 20:37, Michael S. Tsirkin ha scritto:
> > Gleb, Paolo, what do you think? OK to merge kvm unit test
> > into qemu? It depends on qemu anyway, in-tree will make it easier.
> > Maybe someone's looking at this already?
> 
> I think merging KVM unit tests doesn't make much sense because, with
> some small exceptions, it is mostly a test or a benchmark for KVM.

But why keep them separate? They need qemu to work, don't they?

What I wanted to use from kvm unit test is the infrastructure
to generating the kernel binary for qemu.

> What
> may make sense is to have a quick way to run autotest on a QEMU tree,
> with a subset of testcases that doesn't take too much time (let's say <4
> hours)

That's not really reasonable for make check though.

> and is more or less guaranteed to pass.

That's still the main challenge.

> KVM unit tests are run
> by autotest, that should be enough.
> 
> I agree with Anthony that device model code should be tested by qtest.
> I'm not sure this extends to firmware interfaces, though, for two reasons:
> 
> (1) any testcase you could have written would have likely not shown the
> kind of problem that Igor and Gerd found in your previous versions.
> Black box unit testing can only do so much for something as complex as a
> DSDT, while black box integration testing works well.
> 
> (2) IMO qtest's main advantage is that, at least in principle, the same
> testcases could run on all the rarely-used almost-unmaintained targets
> (the endianness-test already does that for example).  This does not
> apply to most firmware interfaces, though.
> 
> 
> By the way, this advantage of qtest is also being mostly negated by the
> immaturity (or sheer absence) of infrastructure.
> Looking at bugs that were reported, at least these two from Igor are
> probably best handled with integration tests (like autotest or Anthony's
> qemu-test):
> 
> * WS2008R2x64 BSODs with ACPI error on boot when 64bit PCI hole is
> present, but it boots fine with upstream QEMU
> 
> * hotadd CPU to guest, reboot guest, only initial CPUs are visible to guest
> 
> 
> qtest could at best host some sanity checks on the ACPI tables, which
> would catch the MCFG problems that Gerd reported on v5.

Depends on how deep the test understands ACPI - the signature
was wrong I think.
Note I was testing this too - comparing tables between
revisions. I just didn't notice that list of tables
to test included was generated by me on piix, so
MCFG wasn't tested.

> Gerd also reported some segfaults, not sure how they escaped mst's
> testing so I cannot judge what kind of testing could have exposed them
> preemptively.
> 
> Paolo

Mostly forgot to commit mistakes. I since added a script
that checks my tree is clean before build.