Re: [RFC v2 04/14] virtio-net: detect VirtIODevice status mid-migration change

[Jonah Palmer <jonah.palmer@oracle.com>]

On 3/24/26 6:45 AM, Eugenio Perez Martin wrote:
> On Fri, Mar 20, 2026 at 3:20 PM Jonah Palmer <jonah.palmer@oracle.com> wrote:
>>
>> This patch introduces the mechanism in which this series will use to
>> detect VirtIODevice & VirtIONet state deltas, starting with the
>> VirtIODevice's 'status' member of a virtio-net device.
>>
>> Before we send the device's state early, we save each piece of its state
>> in a temporary structure via virtio_net_early_pre_save. Later, once the
>> source VM has been paused,
> 
> While I ack this approach should cover 99% of cases—as I think it is
> unlikely the guest changes device properties during a live migration—I
> still believe these changes should be sent when the guest modifies the
> property, before it is paused. That way we shrink the downtime even
> more. This comparison fits naturally: Instead of sending one state and
> then the deltas at the switchover, we send the state and then the
> delta for every change.
> 
> However, I'm happy enough with this approach because it is more
> self-contained within the migration code. It also reduces the state
> sent over the migration channel if the guest changes its device status
> many times.
> 
> CCing Juraj as we discussed this at the virtio-net upstream meeting.
> 

Right. I agree it would be optimal that, for every change made during 
live migration, the destination is updated with this change and it's not 
left to be handled once the source has been paused, which would impact 
guest-visible downtime (especially if it's an expensive change).

However, the only way to handle mid-migration changes like this is via 
the SaveVMHandlers framework. I did give a stab at this approach in my 
v1 RFC series: 
https://lore.kernel.org/qemu-devel/20250722124127.2497406-1-jonah.palmer@oracle.com/

However, Peter believed that this early-migration work would be better 
suited using the VMStateDescription framework instead. And the VMSD 
framework does not support handling these mid-migration changes before 
the stop-and-copy phase.

>> we compare the current values of those pieces
>> to what we saved earlier. If any mismatch is found, virtio-net's VMSD
>> (vmstate_virtio_net) is enabled and resends all state and device prep
>> work (as it's normally done today when live migrating a virtio-net
>> device).
>>
>> Once all relevant delta checks are in place, a no-delta case will skip
>> vmstate_virtio_net and only resend updated VQ indices.
>>
>> For this patch, keep a temporary always-true fallback in
>> virtio_net_has_delta/virtio_net_needed until follow-up patches cover the
>> remaining VirtIONet & VirtIODevice deltas.
>>
>> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
>> ---
>>   hw/net/virtio-net.c        | 40 ++++++++++++++++++++++++++++++++++++++
>>   include/hw/virtio/virtio.h |  2 ++
>>   2 files changed, 42 insertions(+)
>>
>> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
>> index 5d71ad235e..2733e0130c 100644
>> --- a/hw/net/virtio-net.c
>> +++ b/hw/net/virtio-net.c
>> @@ -3864,6 +3864,16 @@ static bool failover_hide_primary_device(DeviceListener *listener,
>>       return qatomic_read(&n->failover_primary_hidden);
>>   }
>>
>> +static int virtio_net_early_pre_save(void *opaque)
>> +{
>> +    VirtIONet *n = opaque;
>> +    VirtIODevice *vdev = VIRTIO_DEVICE(n);
>> +    VirtIODevMigration *vdev_mig = vdev->migration;
>> +
>> +    vdev_mig->status_early = vdev->status;
>> +    return 0;
>> +}
>> +
>>   static int virtio_net_early_pre_load(void *opaque)
>>   {
>>       VirtIONet *n = opaque;
>> @@ -3887,6 +3897,7 @@ static const VMStateDescription vmstate_virtio_net_early = {
>>       .minimum_version_id = VIRTIO_NET_VM_VERSION,
>>       .version_id = VIRTIO_NET_VM_VERSION,
>>       .early_setup = true,
>> +    .pre_save = virtio_net_early_pre_save,
>>       .pre_load = virtio_net_early_pre_load,
>>       .post_load = virtio_net_early_post_load,
>>       .fields = (const VMStateField[]) {
>> @@ -4231,10 +4242,39 @@ static bool dev_unplug_pending(void *opaque)
>>       return vdc->primary_unplug_pending(dev);
>>   }
>>
>> +static bool virtio_net_has_delta(VirtIONet *n, VirtIODevice *vdev)
>> +{
>> +    VirtIODevMigration *vdev_mig = vdev->migration;
>> +
>> +    /* Has the VirtIODevice's status changed? */
>> +    if (vdev->status != vdev_mig->status_early) {
>> +        return true;
>> +    }
> 
> To clarify my point, I think we should add something like this:
> 
> if (vdev->status != status) {
>    // process it.
>    virtio_send_status_update_to_destination()
> }
> 
> At virtio_net_set_status, or similar.
> 

See comment above.

>> +
>> +    /*
>> +     * Always return true for now until we're able to detect all possible
>> +     * changes to a VirtIONet device.
>> +     */
>> +    return true;
>> +}
>> +
>> +static bool virtio_net_needed(void *opaque)
>> +{
>> +    VirtIONet *n = opaque;
>> +    VirtIODevice *vdev = VIRTIO_DEVICE(n);
>> +
>> +    if (!n->early_mig) {
>> +        return true;
>> +    }
>> +
>> +    return virtio_net_has_delta(n, vdev);
>> +}
>> +
>>   static const VMStateDescription vmstate_virtio_net = {
>>       .name = "virtio-net",
>>       .minimum_version_id = VIRTIO_NET_VM_VERSION,
>>       .version_id = VIRTIO_NET_VM_VERSION,
>> +    .needed = virtio_net_needed,
>>       .fields = (const VMStateField[]) {
>>           VMSTATE_VIRTIO_DEVICE,
>>           VMSTATE_END_OF_LIST()
>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>> index 74fed8c324..752c46ce53 100644
>> --- a/include/hw/virtio/virtio.h
>> +++ b/include/hw/virtio/virtio.h
>> @@ -102,9 +102,11 @@ enum virtio_device_endian {
>>   /**
>>    * struct VirtIODevMigration - Common VirtIODevice migration structure
>>    * @early_load: Flag to indicate an early virtio_load for the device.
>> + * @status_early: Device status at the time it was sent early.
>>    */
>>   typedef struct VirtIODevMigration {
>>       bool early_load;
>> +    uint8_t status_early;
>>   } VirtIODevMigration;
>>
>>   /**
>> --
>> 2.51.0
>>
>