Re: [PATCH V9 mlx5-next 10/15] vfio: Extend the device migration protocol with RUNNING_P2P

[Cornelia Huck <cohuck@redhat.com>]

On Thu, Feb 24 2022, Alex Williamson <alex.williamson@redhat.com> wrote:

> On Thu, 24 Feb 2022 12:13:30 -0400
> Jason Gunthorpe <jgg@nvidia.com> wrote:
>
>> On Thu, Feb 24, 2022 at 08:30:42AM -0700, Alex Williamson wrote:
>> > On Thu, 24 Feb 2022 16:21:11 +0100
>> > Cornelia Huck <cohuck@redhat.com> wrote:

>> > > conflicts with it. We should not create the impression that STOP_COPY
>> > > will neccessarily be mandatory for all time.  
>> 
>> We really *should* create that impression because a userspace that
>> does not test STOP_COPY in the cases required above is *broken* and
>> must be strongly discouraged from existing.

Well yes, you need STOP_COPY with the current implementation. I'm not
arguing against that.

>> 
>> The purpose of this comment is to inform the userspace implementator,
>> not to muse about possible future expansion options for kernel
>> developers. We all agree this expansion path exists and is valid, we
>> need to keep that option open by helping userspace implement
>> correctly.
>
> Chatting with Connie offline, I think the clarification that might help
> is something alone the lines that the combination of bits must support
> migration, which currently requires the STOP_COPY and RESUMING states.
> The VFIO_MIGRATION_P2P flag alone does not provide these states.  The
> only flag in the current specification to provide these states is
> VFIO_MIGRATION_STOP_COPY.  I don't think we want to preclude that some
> future flag might provide variants of STOP_COPY and RESUMING, so it's
> not so much that VFIO_MIGRATION_STOP_COPY is mandatory, but it is
> currently the only flag which provides the base degree of migration
> support.

Indeed.

>
> How or if that translates to an actual documentation update, I'm not
> sure.  As it stands, we're not speculating about future support, we're
> only stating these two combinations are valid.  Future combinations may
> or may not include VFIO_MIGRATION_STOP_COPY.  As the existing proposed
> comment indicates, other combinations are TBD.  Connie?  Thanks,
>
> Alex

Hm... "a flag indicating support for a migration state machine such as
VFIO_MIGRATION_STOP_COPY is mandatory"?