networking checksum errors again

networking checksum errors again

[Jason]

Hey gang,   I have run into the dredded tcp and udp checksum error in xen, but with a twist.
Regardless of whether or not I use a dom0 kernel or a domU kernel I get failed checksum packets
whenever I try to route across a virtual interface.  I have a pure virtual setup (ala I just have
xend bring up a new interface with an ip on it that I route traffic to the domU on) and just like
in the FAQ, ping works but no other traffic does because of checksums.  I have tried the ethtool
thing on all interfaces, tried changing the code in netback.c to force a checksum and have had no
luck.   Any else having trouble?

-- 
Jason
The place where you made your stand never mattered,
only that you were there... and still on your feet

Re: networking checksum errors again

[Florian Kirstein]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

Hallo,

> Regardless of whether or not I use a dom0 kernel or a domU kernel I
> get failed checksum packets whenever I try to route across a virtual
> interface.  I have a pure virtual setup
Ah, I just dived into that recently... Is there a bugzilla id
for that already? I had the same problem as you, Dom0 can't communicate
with DomU due to the checksum errors in a routed setup, everything
works when bridged. But I don't want to bridge :)

> thing on all interfaces, tried changing the code in netback.c to
> force a checksum and have had no luck.   Any else having trouble?
Which netback.c patch did you use? I currently have a working setup
using the "interface.c" patch which removes the NETIF_F_NO_CSUM from
the dev->features in Dom0 (attached). Using this "sum ok" packets go
from Dom0 to DomU (also leaving Dom0 intact), DomU to Dom0 packets
are broken leaving DomU but Dom0 ignores that. Using
ethtool -K eth0 tx off
in DomU I can bring DomU to generate valid sums also, but that's cosmetic
and not needed for a working setup. I'd rather also patch this in the
kernel instead of running ethtool in every DomU but I haven't searched
for that yet. Should be easy, though :)

As far as I understood the discussion, this is not an easy issue and
possibly will be dealt with in in clean way after the 3.0.2 release?
For now I have other problems, the above setup seems to work fine
for me (also for DomU-DomU communication, which of course is routed
over Dom0. Will have to test tunnels/vpn in DomU).

(:ul8er, r@y

[-- Attachment #2: netback-interface.patch --]
[-- Type: text/plain, Size: 656 bytes --]

diff -U 3 -r xen-unstable-orig/linux-2.6-xen-sparse/drivers/xen/netback/interface.c xen-unstable/linux-2.6-xen-sparse/drivers/xen/netback/interface.c
--- xen-unstable-orig/linux-2.6-xen-sparse/drivers/xen/netback/interface.c	2006-03-28 06:11:37.000000000 +0200
+++ xen-unstable/linux-2.6-xen-sparse/drivers/xen/netback/interface.c	2006-03-30 08:17:01.000000000 +0200
@@ -101,7 +101,7 @@
 	dev->get_stats       = netif_be_get_stats;
 	dev->open            = net_open;
 	dev->stop            = net_close;
-	dev->features        = NETIF_F_NO_CSUM;
+	dev->features        = 0; /* replaced NETIF_F_NO_CSUM */
 
 	/* Disable queuing. */
 	dev->tx_queue_len = 0;

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: networking checksum errors again

[Jason]

My bad, I meant to say interface.c  not netback.   I didn't have any luck the first time I ran a
kernel with that patch, but Im going to try again because Im not 100% sure I ran the correct
kernel.

-- 
Jason
The place where you made your stand never mattered,
only that you were there... and still on your feet

On Fri, 31 Mar 2006, Florian Kirstein wrote:

> Hallo,
>
>> Regardless of whether or not I use a dom0 kernel or a domU kernel I
>> get failed checksum packets whenever I try to route across a virtual
>> interface.  I have a pure virtual setup
> Ah, I just dived into that recently... Is there a bugzilla id
> for that already? I had the same problem as you, Dom0 can't communicate
> with DomU due to the checksum errors in a routed setup, everything
> works when bridged. But I don't want to bridge :)
>
>> thing on all interfaces, tried changing the code in netback.c to
>> force a checksum and have had no luck.   Any else having trouble?
> Which netback.c patch did you use? I currently have a working setup
> using the "interface.c" patch which removes the NETIF_F_NO_CSUM from
> the dev->features in Dom0 (attached). Using this "sum ok" packets go
> from Dom0 to DomU (also leaving Dom0 intact), DomU to Dom0 packets
> are broken leaving DomU but Dom0 ignores that. Using
> ethtool -K eth0 tx off
> in DomU I can bring DomU to generate valid sums also, but that's cosmetic
> and not needed for a working setup. I'd rather also patch this in the
> kernel instead of running ethtool in every DomU but I haven't searched
> for that yet. Should be easy, though :)
>
> As far as I understood the discussion, this is not an easy issue and
> possibly will be dealt with in in clean way after the 3.0.2 release?
> For now I have other problems, the above setup seems to work fine
> for me (also for DomU-DomU communication, which of course is routed
> over Dom0. Will have to test tunnels/vpn in DomU).
>
> (:ul8er, r@y
>

Re: networking checksum errors again

[Jason]

w00t!  That fixed it.  I read your email on the logic for not doing the tcp checksums and, while I
am by no means in possesion of the brains the developers have, I am left wondering why that
decision was made.  Would anyone care to comment?

-- 
Jason
The place where you made your stand never mattered,
only that you were there... and still on your feet

On Fri, 31 Mar 2006, Florian Kirstein wrote:

> Hi,
>
>> My bad, I meant to say interface.c  not netback.   I didn't have any luck the first time I ran a
>
> It also took me two tries, had a second problem with iptables preventing
> me from detecting the working setup correctly %-) But as said, for me
> patching interface.c in Dom0 solved the problem. You should see
> the intact packets with tcpdump -vvv -n -i eth0   in DomU then.
>
> (:ul8er, r@y
>

Re: networking checksum errors again

[Nivedita Singhvi]

Jason wrote:
> w00t!  That fixed it.  I read your email on the logic for not doing the 
> tcp checksums and, while I
> am by no means in possesion of the brains the developers have, I am left 
> wondering why that
> decision was made.  Would anyone care to comment?
> 
I think I missed a post somewhere along this thread...

Which decision are you referring to? Avoiding the
TCP checksum between domains? The rationale for not
doing the checksum is that it is a significant savings
in performance. Even for traffic that goes out to
a public net and must contain a checksum, deferring
it to dom0 when the OS can offload it to the physical
NIC (most NICs these days are capable of computing the
checksums in hardware) is a significant saving.

The native Linux stack offloads the calculation of
the checksum to the NIC, and not doing so in the virtual
environment increases the gap when comparing Xen to
bare metal Linux.

There are a lot of other issues here to resolve, of
course, and fixing some of the current issues is being
worked. It's likely going to be an issue for discussion
when we're up for mainline inclusion.

thanks,
Nivedita

Re: networking checksum errors again

[Jason]

If I understand correctly, the offloading to a physical NIC only works when a physical NIC is
present and being used for that traffic. In a purely virtual route setup, that traffic should be
subject to this checksum issue.  Is this just a recent development or is routing a fairly uncommon 
configuration in comparison to bridging with xen?

-- 
Jason
The place where you made your stand never mattered,
only that you were there... and still on your feet

On Sat, 1 Apr 2006, Nivedita Singhvi wrote:

> Jason wrote:
>> w00t!  That fixed it.  I read your email on the logic for not doing the tcp 
>> checksums and, while I
>> am by no means in possesion of the brains the developers have, I am left 
>> wondering why that
>> decision was made.  Would anyone care to comment?
>> 
> I think I missed a post somewhere along this thread...
>
> Which decision are you referring to? Avoiding the
> TCP checksum between domains? The rationale for not
> doing the checksum is that it is a significant savings
> in performance. Even for traffic that goes out to
> a public net and must contain a checksum, deferring
> it to dom0 when the OS can offload it to the physical
> NIC (most NICs these days are capable of computing the
> checksums in hardware) is a significant saving.
>
> The native Linux stack offloads the calculation of
> the checksum to the NIC, and not doing so in the virtual
> environment increases the gap when comparing Xen to
> bare metal Linux.
>
> There are a lot of other issues here to resolve, of
> course, and fixing some of the current issues is being
> worked. It's likely going to be an issue for discussion
> when we're up for mainline inclusion.
>
> thanks,
> Nivedita
>
>