From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40215) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj8iK-000710-91 for qemu-devel@nongnu.org; Thu, 24 Mar 2016 13:06:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aj8iG-0000GT-4b for qemu-devel@nongnu.org; Thu, 24 Mar 2016 13:06:28 -0400 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]:33005) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj8iF-0000GO-OK for qemu-devel@nongnu.org; Thu, 24 Mar 2016 13:06:24 -0400 Received: by mail-wm0-x244.google.com with SMTP id u125so14288382wmg.0 for ; Thu, 24 Mar 2016 10:06:23 -0700 (PDT) Sender: Paolo Bonzini References: <1458836125-73613-1-git-send-email-cornelia.huck@de.ibm.com> From: Paolo Bonzini Message-ID: <56F41E8D.5000609@redhat.com> Date: Thu, 24 Mar 2016 18:06:21 +0100 MIME-Version: 1.0 In-Reply-To: <1458836125-73613-1-git-send-email-cornelia.huck@de.ibm.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH 0/6] virtio: refactor host notifiers List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Cornelia Huck , qemu-devel@nongnu.org Cc: tubo@linux.vnet.ibm.com, borntraeger@de.ibm.com, famz@redhat.com, stefanha@redhat.com, mst@redhat.com On 24/03/2016 17:15, Cornelia Huck wrote: > Here's the next version of my refactoring of the virtio host notifiers. > This one actually survives a bit of testing for me (reboot loop). > > As this patchset fixes a latent bug exposed by the recent dataplane > changes (we have a deassigned ioeventfd for a short period of time > during dataplane start, which leads to the virtqueue handler being > called in both the vcpu thread and the iothread simultaneously), I'd > like to see this in 2.6. Tested-by: Paolo Bonzini Resisted 6 minutes versus 10 seconds. At about 2.5 seconds per reboot, that means the failure happened at the fourth reboot before, and resisted about 150 reboots with your patches. My testcase was to add "systemd.unit=reboot.target" to a Fedora 21's kernel command line and run the following ./+build/x86_64-softmmu/qemu-system-x86_64 --enable-kvm -m 512 \ -smp 4 -serial mon:stdio -display none -object iothread,id=io \ -drive if=none,id=hd,file=/vm/virt_test/images/jeos-21-64.qcow2 \ -device virtio-blk-pci,drive=hd,iothread=io \ -drive if=none,file=null-co://,id=n1 \ -drive if=none,file=null-co://,id=n2 \ -drive if=none,file=null-co://,id=n3 \ -drive if=none,file=null-co://,id=n4 \ -drive if=none,file=null-co://,id=n5 \ -drive if=none,file=null-co://,id=n6 \ -drive if=none,file=null-co://,id=n7 \ -drive if=none,file=null-co://,id=n8 \ -device virtio-blk-pci,iothread=io,drive=n1 \ -device virtio-blk-pci,iothread=io,drive=n2 \ -device virtio-blk-pci,iothread=io,drive=n3 \ -device virtio-blk-pci,iothread=io,drive=n4 \ -device virtio-blk-pci,iothread=io,drive=n5 \ -device virtio-blk-pci,iothread=io,drive=n6 \ -device virtio-blk-pci,iothread=io,drive=n7 \ -device virtio-blk-pci,iothread=io,drive=n8 with the assertion patch applied: diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index cb710f1..d0b8248 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -591,6 +591,7 @@ return; } + assert(atomic_fetch_inc(&s->reentrancy_test) == 0); blk_io_plug(s->blk); while ((req = virtio_blk_get_request(s))) { @@ -602,6 +603,7 @@ } blk_io_unplug(s->blk); + atomic_dec(&s->reentrancy_test); } static void virtio_blk_dma_restart_bh(void *opaque) diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index ae84d92..5cb66cd 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -54,6 +54,7 @@ typedef struct VirtIOBlock { bool original_wce; VMChangeStateEntry *change; bool dataplane_started; + int reentrancy_test; struct VirtIOBlockDataPlane *dataplane; } VirtIOBlock; Thanks for your help! Paolo > Changes from RFC: > - Fixed some silly errors (checking for !disabled instead of disabled, > virtio_ccw_stop_ioeventfd() calling virtio_bus_start_ioeventfd()). > - Completely reworked set_host_notifier(): We only want to set/unset > the actual handler function and don't want to do anything to the > ioeventfd backing, so reduce the function to actually doing only > that. > - With the change above, we can lose the 'assign' parameter in > virtio_bus_stop_ioeventfd() again. > - Added more comments that hopefully make it clearer what is going on. > > I'd appreciate it if people could give it some testing; I'll be back > to look at the fallout after Easter. > > Cornelia Huck (6): > virtio-bus: common ioeventfd infrastructure > virtio-bus: have callers tolerate new host notifier api > virtio-ccw: convert to ioeventfd callbacks > virtio-pci: convert to ioeventfd callbacks > virtio-mmio: convert to ioeventfd callbacks > virtio-bus: remove old set_host_notifier callback > > hw/block/dataplane/virtio-blk.c | 6 +- > hw/s390x/virtio-ccw.c | 133 ++++++++++++++-------------------------- > hw/scsi/virtio-scsi-dataplane.c | 9 ++- > hw/virtio/vhost.c | 13 ++-- > hw/virtio/virtio-bus.c | 132 +++++++++++++++++++++++++++++++++++++++ > hw/virtio/virtio-mmio.c | 128 +++++++++++++------------------------- > hw/virtio/virtio-pci.c | 124 +++++++++++++------------------------ > include/hw/virtio/virtio-bus.h | 31 +++++++++- > 8 files changed, 303 insertions(+), 273 deletions(-) >