Re: [PATCH] vfio/pci: Restore standard PCI config space in .slot_reset()

[Keith Busch <kbusch@kernel.org>]

On Mon, Aug 17, 2026 at 03:50:06PM -0600, Alex Williamson wrote:
> > In scenarios where userspace simply needs the underlying hardware
> > restored to its known-good baseline state post-reset, having vfio-pci
> > implement .slot_reset() and leverage the existing PCI core snapshot via
> > pci_restore_state() bridges this gap cleanly.
> 
> Does it though?  Even for a simple reset to initial state we need to
> prevent the host and guest stepping on each other across the reset as
> well as tear down user modified state, like interrupts.

Specifically considering host-guest interactions, I don't think this
scenario is handled at all. After 14 years, this is the state of QEMU:

  static void vfio_err_notifier_handler(void *opaque)
  {
      VFIOPCIDevice *vdev = opaque;
  
      if (!event_notifier_test_and_clear(&vdev->err_notifier)) {
          return;
      }
  
      /*
       * TBD. Retrieve the error details and decide what action
       * needs to be taken. One of the actions could be to pass
       * the error to the guest and have the guest driver recover
       * from the error. This requires that PCIe capabilities be
       * exposed to the guest. For now, we just terminate the
       * guest to contain the error.
       */
  
      error_report("%s(%s) Unrecoverable error detected. Please collect any data possible and then kill the guest", __func__, vdev->vbasedev.name);
  
      vm_stop(RUN_STATE_INTERNAL_ERROR);
  }

Is there another common VMM that actually does something useful to
continue from this event?

Outside virtualization, I'm more interested in enabling user space
DPDK-like drivers. I don't want to break anyone, so starting small here:
restoring the config space to the baseline before the device was handed
to a user space driver feels right.

> > A few questions I'd like to put to the list:
> > 1. Since the PCI core already takes responsibility for holding the early 
> >    config space checkpoint, is calling pci_restore_state() during
> >    .slot_reset() the appropriate place for vfio-pci to apply it, or
> >    should this be explicitly driven/triggered via a VFIO ioctl?
> > 2. Does returning PCI_ERS_RESULT_RECOVERED here create subtle state
> >    issues if userspace directly modified config space registers that
> >    were not captured in pdev->saved_config_space?
> > 3. Should we pair this with an explicit "link restored / reset complete"
> >    eventfd notification so userspace knows exactly when it is safe to
> >    resume access?
> > 
> > Appreciate any feedback or historical context on how VFIO and PCI error
> > recovery should interact here.
> 
> Certainly the host saved state doesn't take into account user
> manipulation of the device since the last snapshot. 

Yeah, the kernel emits an eventfd that an error occured. The user side
should have some baseline from which to proceed. It feels outside the
scope of user space to save and restore such low level and early
initialization things like the PCI BAR config space.

Also consider that the user space side may not have even been
initialized at the time a PCIe error occured. What happens then?

> But also, vfio-pci error handling is currently limited to generating
> an event when a non-recoverable error has occurred.  I don't think we
> can nudge it forward in any meaningful way by implementing a
> .slot_reset to restore a prior host snapshot.  That misses the user
> modified state, coordination across error handling, and may not even
> match the hand-off state of the device to the user.

I totally agree. Lacking a notification, user space can at best poll
something specific to their device, but it'd be better to generically
coordinate this sequence with the kernel's error handling.

Would it be acceptable to introduce additional eventfd's for each part
in the pcie error handling? Privately, I've proposed and tested the
user-space component to quiesce on .error_detected, then start from
scratch after the .slot_reset. But I still need something to restore the
config space, and I feel kernel is the right place to do it.