Re: [Qemu-devel] [PATCH RFC 0/1] qtest: Generic PCI device test

[John Snow <jsnow@redhat.com>]

On 01/29/2015 09:58 AM, Markus Armbruster wrote:
> The test uses QMP introspection to find PCI devices, then tries to
> cold-plug each of them.  Could be extended to hot-plug and unplug.
>
> The tests' QMP introspection part is patterned after Andreas's
> qom-test, which uses QMP to find machine types.
>
> The following devices pass the test:
>
>      AC97                ich9-ahci           pcnet
>      ES1370              ich9-intel-hda      piix3-ide
>      VGA                 ich9-usb-ehci1      piix3-ide-xen
>      am53c974            ich9-usb-ehci2      piix3-usb-uhci
>      cirrus-vga          ich9-usb-uhci1      piix4-ide
>      dc390               ich9-usb-uhci2      piix4-usb-uhci
>      e1000               ich9-usb-uhci3      pvscsi
>      e1000-82540em       ich9-usb-uhci4      qxl-vga
>      e1000-82544gc       ich9-usb-uhci5      rtl8139
>      e1000-82545em       ich9-usb-uhci6      sdhci-pci
>      edu                 intel-hda           secondary-vga
>      i6300esb            ioh3420             tpci200
>      i82550              lsi53c810           usb-ehci
>      i82551              lsi53c895a          virtio-balloon-pci
>      i82557a             megasas             virtio-blk-pci
>      i82557b             megasas-gen2        virtio-net-pci
>      i82557c             ne2k_pci            virtio-rng-pci
>      i82558a             nec-usb-xhci        virtio-scsi-pci
>      i82558b             nvme                virtio-serial-pci
>      i82559a             pci-bridge          vmware-svga
>      i82559b             pci-ohci            vmxnet3
>      i82559c             pci-serial          vt82c686b-usb-uhci
>      i82559er            pci-serial-2x       x3130-upstream
>      i82562              pci-serial-4x       xen-pvdevice
>      i82801              pci-testdev         xio3130-downstream
>      i82801b11-bridge
>
> Many of them are not currently tested at all.
>
> This test does everything a number of existing tests currently do:
>
>      ac97-test.c e1000-test.c es1370-test.c eepro100-test.c
>      ne2000-test.c nvme-test.c pcnet-test.c rtl8139-test.c
>      tpci200-test.c virtio-balloon-test.c virtio-rng-test.c
>      vmxnet3-test.c
>
> They are all marked "TODO: Replace with functional tests".  Options
>
> * Delete them now, undelete when we add functional tests
>
> * Keep them, blacklist the devices in pci-devs-test.c
>
> * Live with the duplicated testing
>

I'd be happy with stubbing out some of those tests, so we at least have 
some record and starting place for beginning tests again in the future.

So... Delete the test interface, keep the tests?

Or: Remove these devices from the generic test, but factor the generic 
test code such that these device tests can just invoke the generic 
routines from their own tests.

That way there's no duplication, everybody gets to use the shared code, 
and we keep the record that we'd like to enhance testing of these devices.

> Andreas, I guess you got an opinion here.
>
> There's overlap with a few others:
>
>      i82801b11-test.c usb-hcd-ehci-test.c usb-hcd-ohci-test.c
>      usb-hcd-xhci-test.c virtio-blk-test.c virtio-net-test.c
>      virtio-scsi-test.c virtio-serial-test.c
>
> Options:
>
> * Blacklist the devices in pci-devs-test.c
>
> * Live with the duplicated testing
>

If you factor out the core generic PCI testing as above, you can just 
cull the common parts out of these tests and let them do their own 
functional tests on top, then blacklist them from the generic dispatcher.

> Andreas?
>
> I manually blacklisted devices unavailable with -device, because QMP
> introspection can't tell (pity).
>
> I further blacklisted devices that require a suitable host device to
> pass through:
>
>      kvm-pci-assign     vhost-scsi-pci       xen-pci-passthrough
>      vfio-pci
>
> I blacklisted virtio-9p-pci and ivshmem for now, because they require
> funky backends.
>
> I blacklisted qxl and xen-platform, because they fail the test.
>
> The test runs only for the x86 targets, just like all of
> $(check-qtest-pci-y).  It could run on other targets, as long as we
> can find a machine type with a PCI bus.
>
> Markus Armbruster (1):
>    qtest: Add generic PCI device test
>
>   tests/Makefile        |   2 +
>   tests/pci-devs-test.c | 299 ++++++++++++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 301 insertions(+)
>   create mode 100644 tests/pci-devs-test.c
>