Re: [PATCH v4 0/7] Move memory listener register to vhost_vdpa_init

[Jonah Palmer <jonah.palmer@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 4266 bytes --]


On 5/14/25 11:49 AM, Eugenio Perez Martin wrote:
> On Wed, May 7, 2025 at 8:47 PM Jonah Palmer<jonah.palmer@oracle.com> wrote:
>> Current memory operations like pinning may take a lot of time at the
>> destination.  Currently they are done after the source of the migration is
>> stopped, and before the workload is resumed at the destination.  This is a
>> period where neigher traffic can flow, nor the VM workload can continue
>> (downtime).
>>
>> We can do better as we know the memory layout of the guest RAM at the
>> destination from the moment that all devices are initializaed.  So
>> moving that operation allows QEMU to communicate the kernel the maps
>> while the workload is still running in the source, so Linux can start
>> mapping them.
>>
>> As a small drawback, there is a time in the initialization where QEMU
>> cannot respond to QMP etc.  By some testing, this time is about
>> 0.2seconds.  This may be further reduced (or increased) depending on the
>> vdpa driver and the platform hardware, and it is dominated by the cost
>> of memory pinning.
>>
>> This matches the time that we move out of the called downtime window.
>> The downtime is measured as checking the trace timestamp from the moment
>> the source suspend the device to the moment the destination starts the
>> eight and last virtqueue pair.  For a 39G guest, it goes from ~2.2526
>> secs to 2.0949.
>>
> Hi Jonah,
>
> Could you update this benchmark? I don't think it changed a lot but
> just to be as updated as possible.

Yes, will update this for 39G guest and for 128G guests :)

>
> I think I cannot ack the series as I sent the first revision. Jason or
> Si-Wei, could you ack it?
>
> Thanks!
>
>> Future directions on top of this series may include to move more things ahead
>> of the migration time, like set DRIVER_OK or perform actual iterative migration
>> of virtio-net devices.
>>
>> Comments are welcome.
>>
>> This series is a different approach of series [1]. As the title does not
>> reflect the changes anymore, please refer to the previous one to know the
>> series history.
>>
>> This series is based on [2], it must be applied after it.
>>
>> [Jonah Palmer]
>> This series was rebased after [3] was pulled in, as [3] was a prerequisite
>> fix for this series.
>>
>> v4:
>> ---
>> * Add memory listener unregistration to vhost_vdpa_reset_device.
>> * Remove memory listener unregistration from vhost_vdpa_reset_status.
>>
>> v3:
>> ---
>> * Rebase
>>
>> v2:
>> ---
>> * Move the memory listener registration to vhost_vdpa_set_owner function.
>> * Move the iova_tree allocation to net_vhost_vdpa_init.
>>
>> v1 athttps://urldefense.com/v3/__https://lists.gnu.org/archive/html/qemu-devel/2024-01/msg02136.html__;!!ACWV5N9M2RV99hQ!IEW1otcaS4OGOE7TX094yfNmZ7WbibjJQv_DaSJxTjMB4HYFNEjgaFdHMUKQMiGgWKeRhMBCS86V7C4DccE$ .
>>
>> [1]https://urldefense.com/v3/__https://patchwork.kernel.org/project/qemu-devel/cover/20231215172830.2540987-1-eperezma@redhat.com/__;!!ACWV5N9M2RV99hQ!IEW1otcaS4OGOE7TX094yfNmZ7WbibjJQv_DaSJxTjMB4HYFNEjgaFdHMUKQMiGgWKeRhMBCS86VTze8nNQ$ 
>> [2]https://urldefense.com/v3/__https://lists.gnu.org/archive/html/qemu-devel/2024-01/msg05910.html__;!!ACWV5N9M2RV99hQ!IEW1otcaS4OGOE7TX094yfNmZ7WbibjJQv_DaSJxTjMB4HYFNEjgaFdHMUKQMiGgWKeRhMBCS86VNYsAGaI$ 
>> [3]https://urldefense.com/v3/__https://lore.kernel.org/qemu-devel/20250217144936.3589907-1-jonah.palmer@oracle.com/__;!!ACWV5N9M2RV99hQ!IEW1otcaS4OGOE7TX094yfNmZ7WbibjJQv_DaSJxTjMB4HYFNEjgaFdHMUKQMiGgWKeRhMBCS86VXyDekTU$ 
>>
>> Jonah - note: I'll be on vacation from May 10-19. Will respond to
>>                comments when I return.
>>
>> Eugenio Pérez (7):
>>    vdpa: check for iova tree initialized at net_client_start
>>    vdpa: reorder vhost_vdpa_set_backend_cap
>>    vdpa: set backend capabilities at vhost_vdpa_init
>>    vdpa: add listener_registered
>>    vdpa: reorder listener assignment
>>    vdpa: move iova_tree allocation to net_vhost_vdpa_init
>>    vdpa: move memory listener register to vhost_vdpa_init
>>
>>   hw/virtio/vhost-vdpa.c         | 107 +++++++++++++++++++++------------
>>   include/hw/virtio/vhost-vdpa.h |  22 ++++++-
>>   net/vhost-vdpa.c               |  34 +----------
>>   3 files changed, 93 insertions(+), 70 deletions(-)
>>
>> --
>> 2.43.5
>>

[-- Attachment #2: Type: text/html, Size: 5998 bytes --]