Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, May 05, 2021 at 08:30:54AM -0400, Tony Krowiak wrote:
> On 5/4/21 11:30 AM, Cornelia Huck wrote:
> > > So, it looks like nothing is done with the mdev until the fd for the
> > > mdev is closed when the guest is shut down, at which time the
> > > mdev is removed.
> > You probably want to wire up the request callback and notify userspace.
>
> Not sure what you mean by this, but I also don't think it matters.

request is triggered by the vfio core to expedite closing of the FD in
userspace.

> After coding up the fix for this and testing it, I've learned that
> if a user attempts to remove an mdev via the sysfs 'remove'
> attribute while the mdev fd is still open (i.e., in use by the
> guest), the mdev remove callback is not invoked until the fd is
> closed

Right the vfio_del_group_dev() called along this path and it blocks
until all FD related accesses are prevented. After it returns no ops
callback can be invoked.

> (i.e., the guest is shut down). During that time, the mdev is
> physically removed from sysfs so no further actions can be taken
> on it;

sysfs files are removed before we get to the remove callback - this is
part of why remove cannot fail, the damage is already done.

Jason