From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60585) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VWZ92-0004ZD-BE for qemu-devel@nongnu.org; Wed, 16 Oct 2013 18:00:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VWZ8w-0001lh-5y for qemu-devel@nongnu.org; Wed, 16 Oct 2013 18:00:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:8971) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VWZ8v-0001jB-D0 for qemu-devel@nongnu.org; Wed, 16 Oct 2013 18:00:37 -0400 Date: Thu, 17 Oct 2013 01:03:01 +0300 From: "Michael S. Tsirkin" Message-ID: <20131016220301.GA10908@redhat.com> References: <525D51C3.2050201@redhat.com> <20131015143530.GA7763@redhat.com> <525D5630.1030801@redhat.com> <87txgimrd6.fsf@codemonkey.ws> <20131015201706.GB9579@redhat.com> <20131016181800.GB7100@redhat.com> <20131016183732.GA8157@redhat.com> <525F0473.6060408@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <525F0473.6060408@redhat.com> Subject: Re: [Qemu-devel] [PULL 42/43] piix4: add acpi pci hotplug support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: Peter Maydell , gleb@redhat.com, marcel.a@redhat.com, qemu-devel , Gerd Hoffmann , Anthony Liguori , Igor Mammedov On Wed, Oct 16, 2013 at 11:26:11PM +0200, Paolo Bonzini wrote: > Il 16/10/2013 20:37, Michael S. Tsirkin ha scritto: > > Gleb, Paolo, what do you think? OK to merge kvm unit test > > into qemu? It depends on qemu anyway, in-tree will make it easier. > > Maybe someone's looking at this already? > > I think merging KVM unit tests doesn't make much sense because, with > some small exceptions, it is mostly a test or a benchmark for KVM. But why keep them separate? They need qemu to work, don't they? What I wanted to use from kvm unit test is the infrastructure to generating the kernel binary for qemu. > What > may make sense is to have a quick way to run autotest on a QEMU tree, > with a subset of testcases that doesn't take too much time (let's say <4 > hours) That's not really reasonable for make check though. > and is more or less guaranteed to pass. That's still the main challenge. > KVM unit tests are run > by autotest, that should be enough. > > I agree with Anthony that device model code should be tested by qtest. > I'm not sure this extends to firmware interfaces, though, for two reasons: > > (1) any testcase you could have written would have likely not shown the > kind of problem that Igor and Gerd found in your previous versions. > Black box unit testing can only do so much for something as complex as a > DSDT, while black box integration testing works well. > > (2) IMO qtest's main advantage is that, at least in principle, the same > testcases could run on all the rarely-used almost-unmaintained targets > (the endianness-test already does that for example). This does not > apply to most firmware interfaces, though. > > > By the way, this advantage of qtest is also being mostly negated by the > immaturity (or sheer absence) of infrastructure. > Looking at bugs that were reported, at least these two from Igor are > probably best handled with integration tests (like autotest or Anthony's > qemu-test): > > * WS2008R2x64 BSODs with ACPI error on boot when 64bit PCI hole is > present, but it boots fine with upstream QEMU > > * hotadd CPU to guest, reboot guest, only initial CPUs are visible to guest > > > qtest could at best host some sanity checks on the ACPI tables, which > would catch the MCFG problems that Gerd reported on v5. Depends on how deep the test understands ACPI - the signature was wrong I think. Note I was testing this too - comparing tables between revisions. I just didn't notice that list of tables to test included was generated by me on piix, so MCFG wasn't tested. > Gerd also reported some segfaults, not sure how they escaped mst's > testing so I cannot judge what kind of testing could have exposed them > preemptively. > > Paolo Mostly forgot to commit mistakes. I since added a script that checks my tree is clean before build.