Re: [PATCH v14 vfio 6/8] vfio/pds: Add support for dirty page tracking

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 10 Aug 2023 14:19:40 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:

> On Thu, Aug 10, 2023 at 10:47:34AM -0600, Alex Williamson wrote:
> > On Thu, 10 Aug 2023 02:47:15 +0000
> > "Tian, Kevin" <kevin.tian@intel.com> wrote:
> >   
> > > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > > Sent: Thursday, August 10, 2023 2:06 AM
> > > > 
> > > > On Wed, Aug 09, 2023 at 11:33:00AM -0600, Alex Williamson wrote:
> > > >     
> > > > > Shameer, Kevin, Jason, Yishai, I'm hoping one or more of you can
> > > > > approve this series as well.  Thanks,    
> > > > 
> > > > I've looked at it a few times now, I think it is OK, aside from the
> > > > nvme issue.
> > > >     
> > > 
> > > My only concern is the duplication of backing storage management
> > > of the migration file which I didn't take time to review.
> > > 
> > > If all others are fine to leave it as is then I will not insist.  
> > 
> > There's leverage now if you feel strongly about it, but code
> > consolidation could certainly come later.
> > 
> > Are either of you willing to provide a R-b?  
> 
> The code structure is good enough (though I agree with Kevin), so sure:
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> 
> > What are we looking for relative to NVMe?  AIUI, the first couple
> > revisions of this series specified an NVMe device ID, then switched to
> > a wildcard, then settled on an Ethernet device ID, all with no obvious
> > changes that would suggest support is limited to a specific device
> > type.  I think we're therefore concerned that migration of an NVMe VF
> > could be enabled by overriding/adding device IDs, whereas we'd like to
> > standardize NVMe migration to avoid avoid incompatible implementations.  
> 
> Yeah
> 
> > It's somewhat a strange requirement since we have no expectation of
> > compatibility between vendors for any other device type, but how far
> > are we going to take it?  Is it enough that the device table here only
> > includes the Ethernet VF ID or do we want to actively prevent what
> > might be a trivial enabling of migration for another device type
> > because we envision it happening through an industry standard that
> > currently doesn't exist?  Sorry if I'm not familiar with the dynamics
> > of the NVMe working group or previous agreements.  Thanks,  
> 
> I don't really have a solid answer. Christoph and others in the NVMe
> space are very firm that NVMe related things must go through
> standards, I think that is their right.
> 
> It does not seem good to allow undermining that approach.

If we wanted to enforce something like this the probe function could
reject NVMe class devices, but...
 
> On the flip side, if we are going to allow this driver, why are we not
> letting them enable their full device functionality with all their
> non-compliant VF/PF combinations? They shouldn't have to hide what
> they are actually doing just to get merged.

This.  Is it enough that this appears to implement device type agnostic
migration support for devices hosted by this distributed services card
and NVMe happens to be one of those device types?  Is that a high
enough bar that this is not simply a vendor specific NVMe migration
implementation?
 
> If we want to block anything it should be to block the PCI spec
> non-compliance of having PF/VF IDs that are different.

PCI Express® Base Specification Revision 6.0.1, pg 1461:

  9.3.3.11 VF Device ID (Offset 1Ah)

  This field contains the Device ID that should be presented for every VF to the SI.

  VF Device ID may be different from the PF Device ID...

That?  Thanks,

Alex