[Qemu-devel] Re: persistent tun & different virtual NICs & dead guest network

[Qemu-devel] Re: persistent tun & different virtual NICs & dead guest network

[Avi Kivity]

(cc qemu-devel)

Michael Tokarev wrote:
> Hello.
>
> 2 days debugging an.. issue here, and finally got it.
> To make the long and painful (it was for me anyway)
> story short...
>
> kvm provides a way to control various offload settings
> on the "host side" of the tun network device (I mean
> the `-net tap' setup) from within guest.  I.e., guest
> can set/clear various offload bits according to its
> capabilities/wishes.
>
> The problem is that different virtual NICs as used by
> kvm/qemu expects and sets different offload bits for
> the virtual NIC.  And sets only those bits which -
> as they "think" - differs from the default (all-off).
>
> This means that when changing virtual NIC model AND
> using persistent tun device, it's very likely to get
> inconsistent flags.
>
> For example, here's how the offload settings on the
> host looks like after using e1000 driver in guest
> (freshly created persistent tun device):
>
>  rx-checksumming: on
>  tx-checksumming: on
>  scatter-gather: on
>  tcp segmentation offload: on
>  udp fragmentation offload: off
>  generic segmentation offload: off
>  large receive offload: off
>
> Here's the same setting when using virtio_net
> instead:
>
>  rx-checksumming: on
>  tx-checksumming: off
>  scatter-gather: off
>  tcp segmentation offload: off
>  udp fragmentation offload: off
>  generic segmentation offload: off
>  large receive offload: off
>
> I.e., only rx-checksumming.  When using virtio_net
> from 2.6.29, which supports LRO, it also turns on
> large receive offload.
>
> Now, say, I tried a host with e1000 driver, and it
> turned on tx, sg and tso bits.  And now I'm trying
> to run a guest with new virtio-net NIC instead.  It
> turns on lro bit, but the network does not work anyway:
> almost any packet that's being sent from host to the
> guest has incorrect checksum - because the NIC is marked
> as able to do tx-checksumming but it does not do it.
> The network is dead.
>
> Now, after trying that and this, not understanding
> what's going on etc, let's reboot back with e1000
> NIC which worked a few minutes ago... just to discover
> that it does not work anymore too!  Because previous
> attempt with virtio_net resulted in lro being on, but
> the driver does not support it!  So now, we've non-
> working network again, and now, it does not matter
> which driver we'll try: neither of them will work
> because the offload settings are broken.
>
> It's more: one can't control this stuff from the
> host side using standard ethtool: it says that
> the operation is not supported (I wonder how kvm
> performs the settings changes).
>
> The solution here is to re-create the tun device
> before changing the virtual NIC model.  But it
> isn't always possible, esp. when guests are
> being run from non-root user (where persistent
> tun devices are most useful).
>
> Can this be fixed somehow please?
>
> I think all the settings should be reset to 0
> when opening the tun device.

This should definitely be fixed.  I'll look at writing a patch.

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: persistent tun & different virtual NICs & dead guest network

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

Avi Kivity wrote:
>> I think all the settings should be reset to 0
>> when opening the tun device.
>
> This should definitely be fixed.  I'll look at writing a patch.
>

Okay, that's not in upstream qemu, so I committed a fix to 
kvm-userspace.git.

Attached if you want to test it.

-- 
error compiling committee.c: too many arguments to function


[-- Attachment #2: 0001-kvm-qemu-clear-tap-features-on-initialization.patch --]
[-- Type: text/x-patch, Size: 926 bytes --]

>From 25971710409c374e9486c960c297f324a9164a65 Mon Sep 17 00:00:00 2001
From: Avi Kivity <avi@redhat.com>
Date: Sun, 5 Apr 2009 15:08:55 +0300
Subject: [PATCH] kvm: qemu: clear tap features on initialization

tap features change how tap interprets data, so they must be cleared on
initialization to prevent old settings from interfering with new guest
instances.

Signed-off-by: Avi Kivity <avi@redhat.com>
---
 qemu/net.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/qemu/net.c b/qemu/net.c
index d753fa0..703d01c 100644
--- a/qemu/net.c
+++ b/qemu/net.c
@@ -930,6 +930,7 @@ static TAPState *net_tap_fd_init(VLANState *vlan,
 #endif
 #ifdef TUNSETOFFLOAD
     s->vc->set_offload = tap_set_offload;
+    tap_set_offload(s->vc, 0, 0, 0, 0);
 #endif
     qemu_set_fd_handler2(s->fd, tap_can_send, tap_send, NULL, s);
     snprintf(s->vc->info_str, sizeof(s->vc->info_str), "fd=%d", fd);
-- 
1.6.0.6

[Qemu-devel] Re: persistent tun & different virtual NICs & dead guest network

[Michael S. Tsirkin]

On Sun, Apr 05, 2009 at 03:12:04PM +0300, Avi Kivity wrote:
> Avi Kivity wrote:
>>> I think all the settings should be reset to 0
>>> when opening the tun device.
>>
>> This should definitely be fixed.  I'll look at writing a patch.
>>
>
> Okay, that's not in upstream qemu, so I committed a fix to  
> kvm-userspace.git.
>
> Attached if you want to test it.
>
> -- 
> error compiling committee.c: too many arguments to function
>

> >From 25971710409c374e9486c960c297f324a9164a65 Mon Sep 17 00:00:00 2001
> From: Avi Kivity <avi@redhat.com>
> Date: Sun, 5 Apr 2009 15:08:55 +0300
> Subject: [PATCH] kvm: qemu: clear tap features on initialization
> 
> tap features change how tap interprets data, so they must be cleared on
> initialization to prevent old settings from interfering with new guest
> instances.
> 
> Signed-off-by: Avi Kivity <avi@redhat.com>
> ---
>  qemu/net.c |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/qemu/net.c b/qemu/net.c
> index d753fa0..703d01c 100644
> --- a/qemu/net.c
> +++ b/qemu/net.c
> @@ -930,6 +930,7 @@ static TAPState *net_tap_fd_init(VLANState *vlan,
>  #endif
>  #ifdef TUNSETOFFLOAD
>      s->vc->set_offload = tap_set_offload;
> +    tap_set_offload(s->vc, 0, 0, 0, 0);

BTW, should not these bits be restored on load?
I couldn't find code that does this.

>  #endif
>      qemu_set_fd_handler2(s->fd, tap_can_send, tap_send, NULL, s);
>      snprintf(s->vc->info_str, sizeof(s->vc->info_str), "fd=%d", fd);

Looks good to me.  Just a thought: do we want kernel to give us an
option to tie the offload bits to the character device, so that it gets
cleaned automatically when qemu dies?

-- 
MST