Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

From: Jason Gunthorpe via iommu <iommu@lists.linux-foundation.org>
To: "Liu, Yi L" <yi.l.liu@intel.com>
Cc: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"jasowang@redhat.com" <jasowang@redhat.com>,
	"kwankhede@nvidia.com" <kwankhede@nvidia.com>,
	"hch@lst.de" <hch@lst.de>,
	"jean-philippe@linaro.org" <jean-philippe@linaro.org>,
	"Jiang, Dave" <dave.jiang@intel.com>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"parav@mellanox.com" <parav@mellanox.com>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"lkml@metux.net" <lkml@metux.net>,
	"david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>,
	"dwmw2@infradead.org" <dwmw2@infradead.org>,
	"Tian, Jun J" <jun.j.tian@intel.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"lushenming@huawei.com" <lushenming@huawei.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>
Subject: Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices
Date: Mon, 1 Nov 2021 09:50:13 -0300	[thread overview]
Message-ID: <20211101125013.GL2744544@nvidia.com> (raw)
In-Reply-To: <PH0PR11MB56586D2EC89F282C915AF18DC3879@PH0PR11MB5658.namprd11.prod.outlook.com>

On Fri, Oct 29, 2021 at 09:47:27AM +0000, Liu, Yi L wrote:
> Hi Jason,
> 
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Monday, October 25, 2021 8:53 PM
> > 
> > On Mon, Oct 25, 2021 at 06:28:09AM +0000, Liu, Yi L wrote:
> > >    thanks for the guiding. will also refer to your vfio_group_cdev series.
> > >
> > >    Need to double confirm here. Not quite following on the kfree. Is
> > >    this kfree to free the vfio_device structure? But now the
> > >    vfio_device pointer is provided by callers (e.g. vfio-pci). Do
> > >    you want to let vfio core allocate the vfio_device struct and
> > >    return the pointer to callers?
> > 
> > There are several common patterns for this problem, two that would be
> > suitable:
> > 
> > - Require each driver to provide a release op inside vfio_device_ops
> >   that does the kfree. Have the core provide a struct device release
> >   op that calls this one. Keep the kalloc/kfree in the drivers
> 
> this way sees to suit the existing vfio registration manner listed
> below. right? 

Not really, most drivers are just doing kfree. The need for release
comes if the drivers are doing more stuff.

> But device drivers needs to do the kfree in the
> newly added release op instead of doing it on their own (e.g.
> doing kfree in remove).

Yes

> > struct ib_device *_ib_alloc_device(size_t size);
> > #define ib_alloc_device(drv_struct, member)                                    \
> >         container_of(_ib_alloc_device(sizeof(struct drv_struct) +              \
> >                                       BUILD_BUG_ON_ZERO(offsetof(              \
> >                                               struct drv_struct, member))),    \
> >                      struct drv_struct, member)
> > 
> 
> thanks for the example. If this way, still requires driver to provide
> a release op inside vfio_device_ops. right?

No, it would optional. It would contain the stuff the driver is doing
before kfree()

For instance mdev looks like the only driver that cares:

	vfio_uninit_group_dev(&mdev_state->vdev);
	kfree(mdev_state->pages);
	kfree(mdev_state->vconfig);
	kfree(mdev_state);

pages/vconfig would logically be in a release function

On the other hand ccw needs to rcu free the vfio_device, so that would
have to be global overhead with this api design.

Jason
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu