Re: [PATCH vfio 0/7] Enhances the vfio-virtio driver to support live migration

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Oct 28, 2024 at 10:13:48AM -0600, Alex Williamson wrote:
> On Sun, 27 Oct 2024 12:07:44 +0200
> Yishai Hadas <yishaih@nvidia.com> wrote:
> > 
> > - According to the Virtio specification, a device has only two states:
> >   RUNNING and STOPPED. Consequently, certain VFIO transitions (e.g.,
> >   RUNNING_P2P->STOP, STOP->RUNNING_P2P) are treated as no-ops. When
> >   transitioning to RUNNING_P2P, the device state is set to STOP and
> >   remains STOPPED until it transitions back from RUNNING_P2P->RUNNING, at
> >   which point it resumes its RUNNING state.
> 
> Does this assume the virtio device is not a DMA target for another
> device?  If so, how can we make such an assumption?  Otherwise, what
> happens on a DMA write to the stopped virtio device?

I was told the virtio spec says that during VFIO STOP it only stops
doing outgoing DMA, it still must accept incoming operations.

It was a point of debate if the additional step (stop everything vs
stop outgoing only) was necessary and the virtio folks felt that stop
outgoing was good enough.

> If the virtio spec doesn't support partial contexts, what makes it
> beneficial here?  

It stil lets the receiver 'warm up', like allocating memory and
approximately sizing things.

> If it is beneficial, why is it beneficial to send initial data more than
> once?  

I guess because it is allowed to change and the benefit is highest
when the pre copy data closely matches the final data..

Rate limiting does seem better to me

Jason