Re: [PATCH V7 mlx5-next 08/15] vfio: Define device migration protocol v2

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Feb 15, 2022 at 10:58:58AM +0000, Tian, Kevin wrote:

> > Another interesting thing (not an immediate concern on this series)
> > is how to handle devices which may have long time (e.g. due to
> > draining outstanding requests, even w/o vPRI) to enter the STOP
> > state. that time is not as deterministic as pending bytes thus cannot
> > be reported back to the user before the operation is actually done.
> > 
> > Similarly to what we discussed for vPRI an eventfd will be beneficial
> > so the user can timeout-wait on it, but it also needs an arc to create
> > the eventfd between RUNNING->STOP...
> > 
> 
> type too fast. it doesn’t need a new arc. Just a new capability to say
> that STOP returns an event fd for the user to wait for completion,
> when supporting such devices is required. 😊

I think it is better to add a new arc rather than radically redefine
the behavior of existing ones:

  RUNNING -> RUNNING_PRI_DRAIN

Should return the event fd and allow the async sleep. Then you
alter the FSM so that RUNNING -> STOP is not allowed anymore and
userspace has to accomodate this new behavior.

Overall it will be some extension like the PRE_COPY and P2P, though
probably not transparently backwards compatabile..

Jason