Re: [PATCH v4 0/7] Move memory listener register to vhost_vdpa_init

[Si-Wei Liu <si-wei.liu@oracle.com>]

For the series:

Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com>

On 5/7/2025 11:46 AM, Jonah Palmer wrote:
> Current memory operations like pinning may take a lot of time at the
> destination.  Currently they are done after the source of the migration is
> stopped, and before the workload is resumed at the destination.  This is a
> period where neigher traffic can flow, nor the VM workload can continue
> (downtime).
>
> We can do better as we know the memory layout of the guest RAM at the
> destination from the moment that all devices are initializaed.  So
> moving that operation allows QEMU to communicate the kernel the maps
> while the workload is still running in the source, so Linux can start
> mapping them.
>
> As a small drawback, there is a time in the initialization where QEMU
> cannot respond to QMP etc.  By some testing, this time is about
> 0.2seconds.  This may be further reduced (or increased) depending on the
> vdpa driver and the platform hardware, and it is dominated by the cost
> of memory pinning.
>
> This matches the time that we move out of the called downtime window.
> The downtime is measured as checking the trace timestamp from the moment
> the source suspend the device to the moment the destination starts the
> eight and last virtqueue pair.  For a 39G guest, it goes from ~2.2526
> secs to 2.0949.
>
> Future directions on top of this series may include to move more things ahead
> of the migration time, like set DRIVER_OK or perform actual iterative migration
> of virtio-net devices.
>
> Comments are welcome.
>
> This series is a different approach of series [1]. As the title does not
> reflect the changes anymore, please refer to the previous one to know the
> series history.
>
> This series is based on [2], it must be applied after it.
>
> [Jonah Palmer]
> This series was rebased after [3] was pulled in, as [3] was a prerequisite
> fix for this series.
>
> v4:
> ---
> * Add memory listener unregistration to vhost_vdpa_reset_device.
> * Remove memory listener unregistration from vhost_vdpa_reset_status.
>
> v3:
> ---
> * Rebase
>
> v2:
> ---
> * Move the memory listener registration to vhost_vdpa_set_owner function.
> * Move the iova_tree allocation to net_vhost_vdpa_init.
>
> v1 at https://lists.gnu.org/archive/html/qemu-devel/2024-01/msg02136.html.
>
> [1] https://patchwork.kernel.org/project/qemu-devel/cover/20231215172830.2540987-1-eperezma@redhat.com/
> [2] https://lists.gnu.org/archive/html/qemu-devel/2024-01/msg05910.html
> [3] https://lore.kernel.org/qemu-devel/20250217144936.3589907-1-jonah.palmer@oracle.com/
>
> Jonah - note: I'll be on vacation from May 10-19. Will respond to
>                comments when I return.
>
> Eugenio Pérez (7):
>    vdpa: check for iova tree initialized at net_client_start
>    vdpa: reorder vhost_vdpa_set_backend_cap
>    vdpa: set backend capabilities at vhost_vdpa_init
>    vdpa: add listener_registered
>    vdpa: reorder listener assignment
>    vdpa: move iova_tree allocation to net_vhost_vdpa_init
>    vdpa: move memory listener register to vhost_vdpa_init
>
>   hw/virtio/vhost-vdpa.c         | 107 +++++++++++++++++++++------------
>   include/hw/virtio/vhost-vdpa.h |  22 ++++++-
>   net/vhost-vdpa.c               |  34 +----------
>   3 files changed, 93 insertions(+), 70 deletions(-)
>