Re: [Qemu-devel] [patch v6 11/12] vfio: register aer resume notification handler for aer resume

[Alex Williamson <alex.williamson@redhat.com>]

On Tue, 24 May 2016 13:49:12 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Tue, Apr 26, 2016 at 08:48:15AM -0600, Alex Williamson wrote:
> > I think that means that if we want to switch from a
> > simple halt-on-error to a mechanism for the guest to handle recovery,
> > we need to disable access to the device between being notified that the
> > error occurred and being notified to resume.  
> 
> But this isn't what happens on bare metal.
> Errors are reported asynchronously and host might access the device
> meanwhile.  These accesses might or might not trigger more errors, but
> fundamentally this should not matter too much as device is going to be
> reset.

Bare metal also doesn't have a hypervisor underneath performing a PCI
bus reset, there's only one OS trying to control the device at a time,
so we have some clear differences from bare metal that I don't know we
can avoid.  The thought here was that we need to notify the guest at the
earliest point we can, but let the host recovery run to completion
before allowing the user to interact with the device.  Perhaps there is
no need to block region access to the device (ie. config space & BAR
resources), but I think we do need to somehow synchronize the bus resets
or else we get situations like that observed previously where the bus is
still in reset while userspace trys to proceed with using it.

The next question then would be whether that's QEMU's job or something
that should be done in the host kernel.  It's been proposed to add yet
another eventfd for the kernel vfio-pci to signal QEMU when a resume
notification has occured, but perhaps the better approach would be for
the hot reset ioctl (and base reset ioctl) to handle this situation more
transparently.  We could immediately return -EAGAIN and allow QEMU to
delay itself for any reset ioctl received after the AER error detected
event, but before the resume event.  We could also allow some sort of
timeout, that the ioctl might enter an interruptible sleep, woken on
the resume notification or timeout.  That sounds a bit better to me as
the specification of what's allowed between the error detected
notification and the resume notification is otherwise pretty poorly
defined.  Do you think we can run completely asynchronous, letting the
host and guest bus resets race?  Thanks,

Alex