Re: [Qemu-devel] [PATCH 1/1] virtio: fallback from irqfd to non-irqfd notify

[Paolo Bonzini <pbonzini@redhat.com>]

On 01/03/2017 14:22, Halil Pasic wrote:
> Here a trace:
> 
> 135871@1488304024.512533:virtio_blk_req_complete req 0x2aa6b117e10 status 0
> 135871@1488304024.512541:virtio_notify_irqfd vdev 0x2aa6b0e19d8 vq 0x2aa6b4c0870
> 135871@1488304024.522607:virtio_blk_req_complete req 0x2aa6b118980 status 0
> 135871@1488304024.522616:virtio_blk_req_complete req 0x2aa6b119260 status 0
> 135871@1488304024.522627:virtio_notify_irqfd vdev 0x2aa6b0e19d8 vq 0x2aa6b4c0870
> 135871@1488304024.527386:virtio_blk_req_complete req 0x2aa6b118980 status 0
> 135871@1488304024.527431:virtio_notify_irqfd vdev 0x2aa6b0e19d8 vq 0x2aa6b4c0870
> 135871@1488304024.528611:virtio_guest_notifier_read vdev 0x2aa6b0e61c8 vq 0x2aa6b4de880
> 135871@1488304024.528628:virtio_guest_notifier_read vdev 0x2aa6b0e61c8 vq 0x2aa6b4de8f8
> 135871@1488304024.528753:virtio_blk_data_plane_stop dataplane 0x2aa6b0e5540
>                          ^== DATAPLANE STOP  
> 135871@1488304024.530709:virtio_blk_req_complete req 0x2aa6b117e10 status 0
> 135871@1488304024.530752:virtio_guest_notifier_read vdev 0x2aa6b0e19d8 vq 0x2aa6b4c0870
>                          ^== comes from k->set_guest_notifiers(qbus->parent, nvqs, false);
>                              in virtio_blk_data_plane_stop and done immediately after
>                              irqfd is cleaned up by the transport
> 135871@1488304024.530836:virtio_notify_irqfd vdev 0x2aa6b0e19d8 vq 0x2aa6b4c0870
> halil: error in event_notifier_set: Bad file descriptor
>                          ^== here we have the problem
> 
> If you want a stacktrace that can be arranged to.
> 
>> like a reset should cause it (the only call in virtio-blk is from
>> virtio_blk_data_plane_stop), and then the guest doesn't care anymore
>> about interrupts.
> I do not understand this with 'doesn't care anymore about interrupts'.
> I was debugging a virtio-blk device being stuck waiting for a host
> notification (interrupt) after migration.

Ok, this explains it better then.  The issue is that
virtio_blk_data_plane_stop doesn't flush the bottom half, which you want
to do when the caller is, for example, virtio_ccw_vmstate_change.

Does it work if you call to qemu_bh_cancel(s->bh) and notify_guest_bh(s)
after

    blk_set_aio_context(s->conf->conf.blk, qemu_get_aio_context());

?

Paolo

>> That path also does a qemu_bh_delete, so the notify_guest_bh should not
>> be invoked at all.
>>
> That's only for destroy. I'm migrating.
> 
> Seems I tried to fix this is the wrong way. Was not too confident about it
> in the first place. Suggestions welcome!