Re: [Qemu-devel] [PATCH 0/6] virtio: refactor host notifiers

[Paolo Bonzini <pbonzini@redhat.com>]

On 24/03/2016 17:15, Cornelia Huck wrote:
> Here's the next version of my refactoring of the virtio host notifiers.
> This one actually survives a bit of testing for me (reboot loop).
> 
> As this patchset fixes a latent bug exposed by the recent dataplane
> changes (we have a deassigned ioeventfd for a short period of time
> during dataplane start, which leads to the virtqueue handler being
> called in both the vcpu thread and the iothread simultaneously), I'd
> like to see this in 2.6.

Tested-by: Paolo Bonzini <pbonzini@redhat.com>

Resisted 6 minutes versus 10 seconds.  At about 2.5 seconds per reboot, 
that means the failure happened at the fourth reboot before, and 
resisted about 150 reboots with your patches.

My testcase was to add "systemd.unit=reboot.target" to a Fedora 21's 
kernel command line and run the following

./+build/x86_64-softmmu/qemu-system-x86_64 --enable-kvm -m 512 \
  -smp 4 -serial mon:stdio -display none -object iothread,id=io \
  -drive if=none,id=hd,file=/vm/virt_test/images/jeos-21-64.qcow2 \
  -device virtio-blk-pci,drive=hd,iothread=io \
  -drive if=none,file=null-co://,id=n1 \
  -drive if=none,file=null-co://,id=n2 \
  -drive if=none,file=null-co://,id=n3 \
  -drive if=none,file=null-co://,id=n4 \
  -drive if=none,file=null-co://,id=n5 \
  -drive if=none,file=null-co://,id=n6 \
  -drive if=none,file=null-co://,id=n7 \
  -drive if=none,file=null-co://,id=n8 \
  -device virtio-blk-pci,iothread=io,drive=n1 \
  -device virtio-blk-pci,iothread=io,drive=n2 \
  -device virtio-blk-pci,iothread=io,drive=n3 \
  -device virtio-blk-pci,iothread=io,drive=n4 \
  -device virtio-blk-pci,iothread=io,drive=n5 \
  -device virtio-blk-pci,iothread=io,drive=n6 \
  -device virtio-blk-pci,iothread=io,drive=n7 \
  -device virtio-blk-pci,iothread=io,drive=n8

with the assertion patch applied:

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index cb710f1..d0b8248 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -591,6 +591,7 @@
         return;
     }
 
+    assert(atomic_fetch_inc(&s->reentrancy_test) == 0);
     blk_io_plug(s->blk);
 
     while ((req = virtio_blk_get_request(s))) {
@@ -602,6 +603,7 @@
     }
 
     blk_io_unplug(s->blk);
+    atomic_dec(&s->reentrancy_test);
 }
 
 static void virtio_blk_dma_restart_bh(void *opaque)
diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h
index ae84d92..5cb66cd 100644
--- a/include/hw/virtio/virtio-blk.h
+++ b/include/hw/virtio/virtio-blk.h
@@ -54,6 +54,7 @@ typedef struct VirtIOBlock {
     bool original_wce;
     VMChangeStateEntry *change;
     bool dataplane_started;
+    int reentrancy_test;
     struct VirtIOBlockDataPlane *dataplane;
 } VirtIOBlock;
 

Thanks for your help!

Paolo

> Changes from RFC:
> - Fixed some silly errors (checking for !disabled instead of disabled,
>   virtio_ccw_stop_ioeventfd() calling virtio_bus_start_ioeventfd()).
> - Completely reworked set_host_notifier(): We only want to set/unset
>   the actual handler function and don't want to do anything to the
>   ioeventfd backing, so reduce the function to actually doing only
>   that.
> - With the change above, we can lose the 'assign' parameter in
>   virtio_bus_stop_ioeventfd() again.
> - Added more comments that hopefully make it clearer what is going on.
> 
> I'd appreciate it if people could give it some testing; I'll be back
> to look at the fallout after Easter.
> 
> Cornelia Huck (6):
>   virtio-bus: common ioeventfd infrastructure
>   virtio-bus: have callers tolerate new host notifier api
>   virtio-ccw: convert to ioeventfd callbacks
>   virtio-pci: convert to ioeventfd callbacks
>   virtio-mmio: convert to ioeventfd callbacks
>   virtio-bus: remove old set_host_notifier callback
> 
>  hw/block/dataplane/virtio-blk.c |   6 +-
>  hw/s390x/virtio-ccw.c           | 133 ++++++++++++++--------------------------
>  hw/scsi/virtio-scsi-dataplane.c |   9 ++-
>  hw/virtio/vhost.c               |  13 ++--
>  hw/virtio/virtio-bus.c          | 132 +++++++++++++++++++++++++++++++++++++++
>  hw/virtio/virtio-mmio.c         | 128 +++++++++++++-------------------------
>  hw/virtio/virtio-pci.c          | 124 +++++++++++++------------------------
>  include/hw/virtio/virtio-bus.h  |  31 +++++++++-
>  8 files changed, 303 insertions(+), 273 deletions(-)
>