[PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Fiona Ebner]

Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
version 8.1 can fail with:

> kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
> kvm: Failed to load virtio-net:virtio
> kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
> kvm: load of migration failed: Operation not permitted

The series

53da8b5a99 virtio-net: Add support for USO features
9da1684954 virtio-net: Add USO flags to vhost support.
f03e0cf63b tap: Add check for USO features
2ab0ec3121 tap: Add USO support to tap device.

only landed in QEMU 8.2, so the compatibility flags should be part of
machine version 8.1.

Moving the flags unfortunately breaks forward migration with machine
version 8.1 from a binary without this patch to a binary with this
patch.

Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 hw/core/machine.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index c7ceb11501..95051b80db 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -50,15 +50,15 @@ GlobalProperty hw_compat_8_1[] = {
     { "ramfb", "x-migrate", "off" },
     { "vfio-pci-nohotplug", "x-ramfb-migrate", "off" },
     { "igb", "x-pcie-flr-init", "off" },
+    { TYPE_VIRTIO_NET, "host_uso", "off"},
+    { TYPE_VIRTIO_NET, "guest_uso4", "off"},
+    { TYPE_VIRTIO_NET, "guest_uso6", "off"},
 };
 const size_t hw_compat_8_1_len = G_N_ELEMENTS(hw_compat_8_1);
 
 GlobalProperty hw_compat_8_0[] = {
     { "migration", "multifd-flush-after-each-section", "on"},
     { TYPE_PCI_DEVICE, "x-pcie-ari-nextfn-1", "on" },
-    { TYPE_VIRTIO_NET, "host_uso", "off"},
-    { TYPE_VIRTIO_NET, "guest_uso4", "off"},
-    { TYPE_VIRTIO_NET, "guest_uso6", "off"},
 };
 const size_t hw_compat_8_0_len = G_N_ELEMENTS(hw_compat_8_0);
 
-- 
2.39.2

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Fabiano Rosas]

Fiona Ebner <f.ebner@proxmox.com> writes:

> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
> version 8.1 can fail with:
>
>> kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
>> kvm: Failed to load virtio-net:virtio
>> kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
>> kvm: load of migration failed: Operation not permitted
>
> The series
>
> 53da8b5a99 virtio-net: Add support for USO features
> 9da1684954 virtio-net: Add USO flags to vhost support.
> f03e0cf63b tap: Add check for USO features
> 2ab0ec3121 tap: Add USO support to tap device.
>
> only landed in QEMU 8.2, so the compatibility flags should be part of
> machine version 8.1.
>
> Moving the flags unfortunately breaks forward migration with machine
> version 8.1 from a binary without this patch to a binary with this
> patch.
>
> Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>

Reviewed-by: Fabiano Rosas <farosas@suse.de>

I'll get to it eventually, but is this another one where just having
-device virtio-net in the command line when testing cross-version
migration would already have caught the issue?

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Jason Wang]

On Tue, May 21, 2024 at 6:23 AM Fabiano Rosas <farosas@suse.de> wrote:
>
> Fiona Ebner <f.ebner@proxmox.com> writes:
>
> > Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
> > version 8.1 can fail with:
> >
> >> kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
> >> kvm: Failed to load virtio-net:virtio
> >> kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
> >> kvm: load of migration failed: Operation not permitted
> >
> > The series
> >
> > 53da8b5a99 virtio-net: Add support for USO features
> > 9da1684954 virtio-net: Add USO flags to vhost support.
> > f03e0cf63b tap: Add check for USO features
> > 2ab0ec3121 tap: Add USO support to tap device.
> >
> > only landed in QEMU 8.2, so the compatibility flags should be part of
> > machine version 8.1.
> >
> > Moving the flags unfortunately breaks forward migration with machine
> > version 8.1 from a binary without this patch to a binary with this
> > patch.
> >
> > Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
> > Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
>
> Reviewed-by: Fabiano Rosas <farosas@suse.de>
>
> I'll get to it eventually, but is this another one where just having
> -device virtio-net in the command line when testing cross-version
> migration would already have caught the issue?

Yes if you are using Qemu >= 8.2. Qemu has a default machine type for
each version.

Thanks

>

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Fiona Ebner]

Am 21.05.24 um 00:22 schrieb Fabiano Rosas:
> Fiona Ebner <f.ebner@proxmox.com> writes:
> 
>> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
>> version 8.1 can fail with:
>>
>>> kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
>>> kvm: Failed to load virtio-net:virtio
>>> kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
>>> kvm: load of migration failed: Operation not permitted
>>
>> The series
>>
>> 53da8b5a99 virtio-net: Add support for USO features
>> 9da1684954 virtio-net: Add USO flags to vhost support.
>> f03e0cf63b tap: Add check for USO features
>> 2ab0ec3121 tap: Add USO support to tap device.
>>
>> only landed in QEMU 8.2, so the compatibility flags should be part of
>> machine version 8.1.
>>
>> Moving the flags unfortunately breaks forward migration with machine
>> version 8.1 from a binary without this patch to a binary with this
>> patch.
>>
>> Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
>> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> 
> Reviewed-by: Fabiano Rosas <farosas@suse.de>
> 
> I'll get to it eventually, but is this another one where just having
> -device virtio-net in the command line when testing cross-version
> migration would already have caught the issue?
> 
AFAIU, the guest kernel needs to be recent enough to support the feature
too. I don't seem to run into the issue with a Debian 11 (kernel 5.10)
guest, but I do run into the issue with an Ubuntu 23.10 (kernel 6.5)
guest. Seems like it got added in kernel 6.2 with 418044e1de30
("drivers/net/virtio_net.c: Added USO support.")

Best Regards,
Fiona

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Fabiano Rosas]

Fiona Ebner <f.ebner@proxmox.com> writes:

> Am 21.05.24 um 00:22 schrieb Fabiano Rosas:
>> Fiona Ebner <f.ebner@proxmox.com> writes:
>> 
>>> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
>>> version 8.1 can fail with:
>>>
>>>> kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
>>>> kvm: Failed to load virtio-net:virtio
>>>> kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
>>>> kvm: load of migration failed: Operation not permitted
>>>
>>> The series
>>>
>>> 53da8b5a99 virtio-net: Add support for USO features
>>> 9da1684954 virtio-net: Add USO flags to vhost support.
>>> f03e0cf63b tap: Add check for USO features
>>> 2ab0ec3121 tap: Add USO support to tap device.
>>>
>>> only landed in QEMU 8.2, so the compatibility flags should be part of
>>> machine version 8.1.
>>>
>>> Moving the flags unfortunately breaks forward migration with machine
>>> version 8.1 from a binary without this patch to a binary with this
>>> patch.
>>>
>>> Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
>>> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
>> 
>> Reviewed-by: Fabiano Rosas <farosas@suse.de>
>> 
>> I'll get to it eventually, but is this another one where just having
>> -device virtio-net in the command line when testing cross-version
>> migration would already have caught the issue?
>> 
> AFAIU, the guest kernel needs to be recent enough to support the feature
> too. I don't seem to run into the issue with a Debian 11 (kernel 5.10)
> guest, but I do run into the issue with an Ubuntu 23.10 (kernel 6.5)
> guest. Seems like it got added in kernel 6.2 with 418044e1de30
> ("drivers/net/virtio_net.c: Added USO support.")

Ah ok, so this is more complex, the tests wouldn't have caught it even
with the device options addition. A test for this will have to come at a
second moment once we figure out how to deal with guest-code dependent
issues.

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Jason Wang]

On Fri, May 17, 2024 at 3:54 PM Fiona Ebner <f.ebner@proxmox.com> wrote:
>
> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
> version 8.1 can fail with:
>
> > kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
> > kvm: Failed to load virtio-net:virtio
> > kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
> > kvm: load of migration failed: Operation not permitted
>
> The series
>
> 53da8b5a99 virtio-net: Add support for USO features
> 9da1684954 virtio-net: Add USO flags to vhost support.
> f03e0cf63b tap: Add check for USO features
> 2ab0ec3121 tap: Add USO support to tap device.
>
> only landed in QEMU 8.2, so the compatibility flags should be part of
> machine version 8.1.
>
> Moving the flags unfortunately breaks forward migration with machine
> version 8.1 from a binary without this patch to a binary with this
> patch.
>
> Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>

Acked-by: Jason Wang <jasowang@redhat.com>

Thanks

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Peter Xu]

On Fri, May 17, 2024 at 09:53:36AM +0200, Fiona Ebner wrote:
> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
> version 8.1 can fail with:
> 
> > kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
> > kvm: Failed to load virtio-net:virtio
> > kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
> > kvm: load of migration failed: Operation not permitted
> 
> The series
> 
> 53da8b5a99 virtio-net: Add support for USO features
> 9da1684954 virtio-net: Add USO flags to vhost support.
> f03e0cf63b tap: Add check for USO features
> 2ab0ec3121 tap: Add USO support to tap device.
> 
> only landed in QEMU 8.2, so the compatibility flags should be part of
> machine version 8.1.
> 
> Moving the flags unfortunately breaks forward migration with machine
> version 8.1 from a binary without this patch to a binary with this
> patch.
> 

I think it'll be nicer we always copy stable in the commit log, so that
"whether to copy stable" is stick with the patch:

Cc: qemu-stable <qemu-stable@nongnu.org>

Even though it's not required for qemu stable process (I'd say that's
because we have a good stable tree maintainer..).

So we need this in qemu 8.1, 8.2 and 9.0?

> Fixes: 53da8b5a99 ("virtio-net: Add support for USO features")
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>

Reviewed-by: Peter Xu <peterx@redhat.com>

> ---
>  hw/core/machine.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index c7ceb11501..95051b80db 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -50,15 +50,15 @@ GlobalProperty hw_compat_8_1[] = {
>      { "ramfb", "x-migrate", "off" },
>      { "vfio-pci-nohotplug", "x-ramfb-migrate", "off" },
>      { "igb", "x-pcie-flr-init", "off" },
> +    { TYPE_VIRTIO_NET, "host_uso", "off"},
> +    { TYPE_VIRTIO_NET, "guest_uso4", "off"},
> +    { TYPE_VIRTIO_NET, "guest_uso6", "off"},
>  };
>  const size_t hw_compat_8_1_len = G_N_ELEMENTS(hw_compat_8_1);
>  
>  GlobalProperty hw_compat_8_0[] = {
>      { "migration", "multifd-flush-after-each-section", "on"},
>      { TYPE_PCI_DEVICE, "x-pcie-ari-nextfn-1", "on" },
> -    { TYPE_VIRTIO_NET, "host_uso", "off"},
> -    { TYPE_VIRTIO_NET, "guest_uso4", "off"},
> -    { TYPE_VIRTIO_NET, "guest_uso6", "off"},
>  };
>  const size_t hw_compat_8_0_len = G_N_ELEMENTS(hw_compat_8_0);
>  
> -- 
> 2.39.2
> 
> 
> 

-- 
Peter Xu

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

[Fabiano Rosas]

On Fri, 17 May 2024 09:53:36 +0200, Fiona Ebner wrote:
> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine
> version 8.1 can fail with:
> 
> > kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7
> > kvm: Failed to load virtio-net:virtio
> > kvm: error while loading state for instance 0x0 of device '0000:00:12.0/virtio-net'
> > kvm: load of migration failed: Operation not permitted
> 
> [...]

Queued, thanks!