Re: [RFC PATCH 1/1] vfio/pci: Disable sriov on PF device close

[Alex Williamson <alex@shazbot.org>]

On Tue, 11 Aug 2026 09:52:37 +0000
"Tian, Kevin" <kevin.tian@intel.com> wrote:

> > From: Alex Williamson <alex@shazbot.org>
> > Sent: Monday, August 10, 2026 11:32 PM
> > 
> > On Thu, 6 Aug 2026 16:47:39 -0300
> > Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >   
> > > Adding the sriov disable here at least makes it defensibly safe, that
> > > we do still clear the PF on close, and we don't take risks that the
> > > active VFs will crash the system during the FLR blip.
> > >
> > > Though I understand it was not the original intention, I feel we have
> > > ended up in a strange place with the SRIOV PF feature ..  
> > 
> > I think we have 3 potential options for PF reset:  
> 
> Is there any coordination between in-kernel PF driver and in-kernel VF
> drivers regarding to reset? At a glance looks that requesting FLR on PF
> via sysfs would silently affect active VFs anyway...

Yes, and this is a key aspect that the RFC I sent last night tackles.
pci_reset_function(), and it's exposure through pci-sysfs, is intended
to have function level scope.  We could walk the VFs, save and restore
their PCI state, and trigger their reset callbacks, but this is a
destructive operation relative to the VFs and their drivers.  A better
fit of our scope model would be if we simply declare that while a PF has
active VFs, the PF does not have a function scoped reset.

This immediately, and cleanly, makes the open/close, config space, and
ioctl resets in vfio-pci inaccessible for PFs in this state.  We can
then further push the vfio-pci hot-reset interface to require SR-IOV is
first disabled on any affected PF devices, as the VFs are not currently
accounted for in the set of affected devices.


> >  a) All resets are blocked or deferred while SR-IOV is active.
> > 
> >  b) VF mappings are zapped/invalidated at PF reset and require
> >     coordination to be reinstated.
> > 
> >  c) VFs are torn down on all PF resets.
> > 
> > Option (a) is complex and doesn't look like stable material.  A deferred
> > reset on .close needs to materialize on .remove or .sriov_configure
> > (and of course .open), but to allow in-kernel VF drivers we can't rely
> > on vfio usage counts, or even our vf-token trust boundaries, which is
> > what leads to the .sriov_configure hook since we need to take advantage
> > of every opportunity to issue a deferred reset when SR-IOV is not
> > active when we can only rely on the PF state.
> > 
> > Option (b) is potentially more simple, it implements the blocking of
> > access on reset in the kernel for enforcement, but defers the
> > coordination of reinstantiating mappings to userspace, which is really
> > how the vf_token model is intended to work.  This is incompatible with
> > in-kernel VF drivers, which I think means tainting on VF unbind from
> > vfio-pci and those working outside the model would tread carefully.
> > 
> > Option (c) is the more heavy handed approach, it means that a PF driver
> > cannot fail or exit and re-attach.  We have existing logic that handles
> > a PF driver re-opening the PF device with active VFs and authenticating
> > the vf_token (logic also broken by in-kernel VF drivers that bypass the
> > vf_token mechanics).  There also appears to be significant locking
> > challenges in this approach, ex. the inversion of getting the device
> > lock for SR-IOV teardown from the ioctl, config space, and .close
> > contexts.
> > 
> > I'm open to suggestions, PF resets with live VFs is very much a gap
> > that I'd like to close in the vfio-pci SR-IOV model.  Thanks,
> >   
> 
> Before closing the open on reset, does it make sense to first fit it into
> the coming trust infrastructure [1]? e.g. initially set to TRUST_NONE
> for any VF with a PF owned by vfio-pci, preventing any bind to
> in-kernel VF drivers. Then opt-in is allowed to promote the trust of
> such VFs to TRUST_ADVERSARY, allowing driver binding but also put
> it in precaution with IOMMU protection. So a malicious userspace
> PF driver cannot indirectly affect VFs to do dma-based attack.
> 
> somehow VFs in this scenario feel akin to Thunderbolt devices...
> 
> [1] https://lore.kernel.org/linux-coco/20260705220819.2472765-10-djbw@kernel.org/

That certainly puts a more cohesive driver-core story around binding VFs
from a userspace owned PF to in-kernel drivers.

We currently have our own hand-rolled version for the PF/VF use case.
Each bound PF registers a bus notifier that monitors actions for VFs
whose physfn is the vfio-pci bound PF.  On BUS_NOTIFY_ADD_DEVICE we
write the VF driver_override to the PF driver name, preventing any other
in-kernel driver from binding the VF without a userspace override.  We
also then monitor BUS_NOTIFY_BOUND_DRIVER and generate a pci_warn() if
a VF is bound to any other driver.

So semantically, it still requires an administrative opt-in, but once
opt'd in, the receiving in-kernel driver has no idea the device is
driven by userspace.

Maybe the trust level of the device becomes more evident with an
adversarial tag, but coordination between drivers relative to things
like reset still seems like an orthogonal topic.  Thanks,

Alex