Re: iommufd dirty page logging overview

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Mar 18, 2022 at 09:23:49AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Thursday, March 17, 2022 7:51 AM
> > 
> > > there a rough idea of how the new dirty page logging will look like?
> > > Is this already explained in the email threads an I missed it?
> > 
> > I'm hoping to get something to show in the next few weeks, but what
> > I've talked about previously is to have two things:
> > 
> > 1) Control and reporting of dirty tracking via the system IOMMU
> >    through the iommu_domain interface exposed by iommufd
> > 
> > 2) Control and reporting of dirty tracking via a VFIO migration
> >    capable device's internal tracking through a VFIO_DEVICE_FEATURE
> >    interface similar to the v2 migration interface
> > 
> > The two APIs would be semantically very similar but target different
> > HW blocks. Userspace would be in charge to decide which dirty tracker
> > to use and how to configure it.
> > 
> 
> for the 2nd option I suppose userspace is expected to retrieve
> dirty bits via VFIO_DEVICE_FEATURE before every iommufd 
> unmap operation in precopy phase, just like why we need return
> the dirty bitmap to userspace in iommufd unmap interface in
> the 1st option. Correct?

It would have to be after unmap, not before

> Is there any value of having iommufd pull dirty bitmap from
> vfio driver then the userspace can just stick to a unified
> iommufd interface for dirty pages no matter they are tracked
> by system IOMMU or device IP? Sorry if this has been discussed
> in previous threads which I haven't fully checked.

It is something to discuss, this is sort of what the current vfio
interface imagines

But to do it we need to build a whole bunch of infrastructure to
register and control these things and add new ioctls to vfio to
support this. I'm not sure we get a sufficient benifit to be
worthwhile, infact it is probably a net loss as we loose the ability
for userspace to pull the dirty bits from multiple device trackers in
parallel with threading.

Jason