[mdlabriola@yahoo.com: Re: Re: [Xen-users] DomU routed traffic disappearing in vif.]

[mdlabriola@yahoo.com: Re: Re: [Xen-users] DomU routed traffic disappearing in vif.]

[Konrad Rzeszutek Wilk]

> On Sat, 2010-05-08 at 02:53 +0200, Matej Zary wrote:
> > Hello,
> > 
> > 
> > XEN 4.0 with pvops 2.6.32.11 Dom0 kernel. Configured PV DomU with the same pvops Xen kernel on top of it.
> > 
> > Physical computers has 3 NICs - every NIC in own bridge (first one created by Xen).
> > 
> > eth0		8000.00138fe78f1b	no		peth0
> > mst1		8000.0040f4b5286e	no		eth1
> > 							vif5.0
> > mst2		8000.000e2e68db10	no		eth2
> > 							vif5.1
> > 
> > Attached 2 virtual NICs to the PV DomU and bridges - I want to use that DomU as router between that two bridges (enabled ip_forwarding in DomU) - mst1 and mst2.
> > 
> > I can ping from netwok on eth1 the vif5.0 interface.
> > I can ping from network on eth2 the vif5.1 interface.
> > I can see with tcpdump, that the DomU routes the traffic between its eth0 and eth1 interfaces (tcpdump inside the domu).
> > I can see with tcpdump the incoming traffic in mst1 and vif5.0.
> > But there's nothing on the vif5.1and mst2 bridge - even when tcpdump on the eth1 in that PV DomU shows the routed traffic ("works"  the same in opposite direction too).
> > 
> > Looks like all traffic routed by that PV DomU cant't get from DomU eth interface to the vif interface. Ping from the DomU works (non routed traffic), but no routed traffic can get thu this Dom0.
> > 
> > IPltables flushed with -F 
> > 
> > Thanks for every idea, i'm clueless right now. :(
> > 
> > Regars 
> > 
> > Matej
> 
> Well, it's the dreaded cksum incorrect error (and Real(BAD)tek PCI
> NICs). Tried to set off the tx check-summing on all interfaces (eth
> interfaces in DomU, eth and bridge interfaces in Dom0) with ethtool -K
> iface tx off, and it changed the situation a little bit - now are these
> packets with incorrect checksum visible on the outgoing eth2 physical
> NIC in the mst2 bridge and also on physical host in the physical network
> connected to mst2 bridge via the eth2 NIC. Final result ist still the
> same, physical hosts routed via the DomU can't communicate (now they are
> getting packets with wrong checksum at least :D). 
> 
> Any chances that newer pv_ops kernel that the 2.6.32.11 (git stable)
> version will solve this issue? Or I have to switch to the 2.6.18 and the
> forwardported oldstyle kernels? 

Ian had a patch that he backported from XCP that might fix this, not
sure thought. Jeremy is trying it out but he ran in some other issues
(pvgrub messing up UDP checksums). Stay tuned.

Re: [mdlabriola@yahoo.com: Re: Re: [Xen-users] DomU routed traffic disappearing in vif.]

[Ian Campbell]

On Mon, 2010-05-10 at 16:18 +0100, Konrad Rzeszutek Wilk wrote:
> Ian had a patch that he backported from XCP that might fix this, not
> sure thought. 

If possible can someone who is seeing this issue try this patch:

--- 

xen/netback: correctly setup skb->ip_summed on receive

In 2.6.18 CHECKSUM_PARTIAL and CHECKSUM_UNNECESSARY were both synonyms for
CHECKSUM_HW. This is no longer the case and we need to select the correct one.

  data_validated csum_blank -> ip_summed
  0              0             CHECKSUM_NONE
  0              1             CHECKSUM_PARTIAL
  1              0             CHECKSUM_UNNECESSARY
  1              1             CHECKSUM_PARTIAL

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff -r 3a1424756f50 drivers/xen/netback/netback.c
--- a/drivers/xen/netback/netback.c	Fri May 07 14:38:37 2010 +0100
+++ b/drivers/xen/netback/netback.c	Fri May 07 14:40:31 2010 +0100
@@ -1384,14 +1384,10 @@
 			netif_idx_release(pending_idx);
 		}
 
-		/*
-		 * Old frontends do not assert data_validated but we
-		 * can infer it from csum_blank so test both flags.
-		 */
-		if (txp->flags & (NETTXF_data_validated|NETTXF_csum_blank))
+		if (txp->flags & NETTXF_csum_blank)
 			skb->ip_summed = CHECKSUM_PARTIAL;
-		else
-			skb->ip_summed = CHECKSUM_NONE;
+		else if (txp->flags & NETTXF_data_validated)
+			skb->ip_summed = CHECKSUM_UNNECESSARY;
 
 		netbk_fill_frags(skb);

Re: [mdlabriola@yahoo.com: Re: Re: [Xen-users] DomU routed traffic disappearing in vif.]

[Michael D Labriola]

xen-users-bounces@lists.xensource.com wrote on 05/10/2010 11:32:33 AM:

> Ian Campbell <Ian.Campbell@citrix.com> 
> Sent by: xen-users-bounces@lists.xensource.com
> 
> 05/10/2010 11:32 AM
> 
> To
> 
> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> 
> cc
> 
> "zary@cvtisr.sk" <zary@cvtisr.sk>, "xen-devel@lists.xensource.com" 
> <xen-devel@lists.xensource.com>, "xen-users@lists.xensource.com" 
> <xen-users@lists.xensource.com>, "mdlabriola@yahoo.com" 
<mdlabriola@yahoo.com>
> 
> Subject
> 
> Re: [mdlabriola@yahoo.com: [Xen-devel] Re: Re: [Xen-users] DomU 
> routed traffic disappearing in vif.]
> 
> On Mon, 2010-05-10 at 16:18 +0100, Konrad Rzeszutek Wilk wrote:
> > Ian had a patch that he backported from XCP that might fix this, not
> > sure thought. 
> 
> If possible can someone who is seeing this issue try this patch:
> 

I'll go patch my tree right now.

---
Michael D Labriola
Electric Boat
mlabriol@gdeb.com
401-848-8871 (desk)
401-848-8513 (lab)
401-316-9844 (cell)

Re: [mdlabriola@yahoo.com: Re: Re: [Xen-users] DomU routed traffic disappearing in vif.]

[Michael D Labriola]

xen-users-bounces@lists.xensource.com wrote on 05/10/2010 01:17:26 PM:

> Michael D Labriola <mlabriol@gdeb.com> 
> Sent by: xen-users-bounces@lists.xensource.com
> 
> 05/10/2010 01:17 PM
> 
> To
> 
> Ian Campbell <Ian.Campbell@citrix.com>
> 
> cc
> 
> "zary@cvtisr.sk" <zary@cvtisr.sk>, "xen-devel@lists.xensource.com" 
> <xen-devel@lists.xensource.com>, xen-users-
> bounces@lists.xensource.com, Konrad Rzeszutek Wilk 
> <konrad.wilk@oracle.com>, "mdlabriola@yahoo.com" 
> <mdlabriola@yahoo.com>, "xen-users@lists.xensource.com" <xen-
> users@lists.xensource.com>
> 
> Subject
> 
> Re: [mdlabriola@yahoo.com: [Xen-devel] Re: Re: [Xen-users] DomU 
> routed traffic disappearing in vif.]
> 
> xen-users-bounces@lists.xensource.com wrote on 05/10/2010 11:32:33 AM:
> 
> > Ian Campbell <Ian.Campbell@citrix.com> 
> > Sent by: xen-users-bounces@lists.xensource.com
> > 
> > 05/10/2010 11:32 AM
> > 
> > To
> > 
> > Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > 
> > cc
> > 
> > "zary@cvtisr.sk" <zary@cvtisr.sk>, "xen-devel@lists.xensource.com" 
> > <xen-devel@lists.xensource.com>, "xen-users@lists.xensource.com" 
> > <xen-users@lists.xensource.com>, "mdlabriola@yahoo.com" 
> <mdlabriola@yahoo.com>
> > 
> > Subject
> > 
> > Re: [mdlabriola@yahoo.com: [Xen-devel] Re: Re: [Xen-users] DomU 
> > routed traffic disappearing in vif.]
> > 
> > On Mon, 2010-05-10 at 16:18 +0100, Konrad Rzeszutek Wilk wrote:
> > > Ian had a patch that he backported from XCP that might fix this, not
> > > sure thought. 
> > 
> > If possible can someone who is seeing this issue try this patch:
> > 
> 
> I'll go patch my tree right now.

Ok, that seems to fix my vanishing routed domU VLAN traffic issue.  :-)

I've got some other problems that I was hoping would also get fixed...  On 
just one of my boxes, only in Xen, my kernel Oopses when I receive 
multicast traffic.  Also, on this same box, I have a couple different X 
crashes that only happen when XDMCP'd into a domU hosted by that dom0 (so 
logged in over the eth0 bridge).  Guess I'll start new threads for those 
with more details, unless anyone happens to have had similar problems and 
fixed it.

---
Michael D Labriola
Electric Boat
mlabriol@gdeb.com
401-848-8871 (desk)
401-848-8513 (lab)
401-316-9844 (cell)

Re: Re: [mdlabriola@yahoo.com: [Xen-devel] Re: Re: DomU routed traffic disappearing in vif.]

[Matej Zary]

[-- Attachment #1: Type: text/plain, Size: 1881 bytes --]

On Mon, 2010-05-10 at 16:32 +0100, Ian Campbell wrote:
> On Mon, 2010-05-10 at 16:18 +0100, Konrad Rzeszutek Wilk wrote:
> > Ian had a patch that he backported from XCP that might fix this, not
> > sure thought. 
> 
> If possible can someone who is seeing this issue try this patch:
> 
> --- 
> 
> xen/netback: correctly setup skb->ip_summed on receive
> 
> In 2.6.18 CHECKSUM_PARTIAL and CHECKSUM_UNNECESSARY were both synonyms for
> CHECKSUM_HW. This is no longer the case and we need to select the correct one.
> 
>   data_validated csum_blank -> ip_summed
>   0              0             CHECKSUM_NONE
>   0              1             CHECKSUM_PARTIAL
>   1              0             CHECKSUM_UNNECESSARY
>   1              1             CHECKSUM_PARTIAL
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> 
> diff -r 3a1424756f50 drivers/xen/netback/netback.c
> --- a/drivers/xen/netback/netback.c	Fri May 07 14:38:37 2010 +0100
> +++ b/drivers/xen/netback/netback.c	Fri May 07 14:40:31 2010 +0100
> @@ -1384,14 +1384,10 @@
>  			netif_idx_release(pending_idx);
>  		}
>  
> -		/*
> -		 * Old frontends do not assert data_validated but we
> -		 * can infer it from csum_blank so test both flags.
> -		 */
> -		if (txp->flags & (NETTXF_data_validated|NETTXF_csum_blank))
> +		if (txp->flags & NETTXF_csum_blank)
>  			skb->ip_summed = CHECKSUM_PARTIAL;
> -		else
> -			skb->ip_summed = CHECKSUM_NONE;
> +		else if (txp->flags & NETTXF_data_validated)
> +			skb->ip_summed = CHECKSUM_UNNECESSARY;
>  
>  		netbk_fill_frags(skb);
>  
> 
> 
Works for me!

The DomU router now works like charm. Thanks! :) Also the problem with
ICMP traffic disappeared (non UDP/TCP packet dropped in syslog).

Patched the 2.6.32.11 kernel, performance wise, the 2.6.18.8 had bit
better some numbers (~ 60 Mbit difference with Iperf), will have some
more numbers later. 

Regards

Matej

[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 4185 bytes --]

[-- Attachment #3: Type: text/plain, Size: 137 bytes --]

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Re: Re: [mdlabriola@yahoo.com: Re: Re: [Xen-users] DomU routed traffic disappearing in vif.]

[Ian Campbell]

On Tue, 2010-05-11 at 01:43 +0100, Matej Zary wrote:
> On Mon, 2010-05-10 at 16:32 +0100, Ian Campbell wrote:
> > On Mon, 2010-05-10 at 16:18 +0100, Konrad Rzeszutek Wilk wrote:
> > > Ian had a patch that he backported from XCP that might fix this, not
> > > sure thought.
> >
> > If possible can someone who is seeing this issue try this patch:
> >
> > ---
> >
> > xen/netback: correctly setup skb->ip_summed on receive
> >
> > In 2.6.18 CHECKSUM_PARTIAL and CHECKSUM_UNNECESSARY were both synonyms for
> > CHECKSUM_HW. This is no longer the case and we need to select the correct one.
> >
> >   data_validated csum_blank -> ip_summed
> >   0              0             CHECKSUM_NONE
> >   0              1             CHECKSUM_PARTIAL
> >   1              0             CHECKSUM_UNNECESSARY
> >   1              1             CHECKSUM_PARTIAL
> >
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> >
> > diff -r 3a1424756f50 drivers/xen/netback/netback.c
> > --- a/drivers/xen/netback/netback.c   Fri May 07 14:38:37 2010 +0100
> > +++ b/drivers/xen/netback/netback.c   Fri May 07 14:40:31 2010 +0100
> > @@ -1384,14 +1384,10 @@
> >                       netif_idx_release(pending_idx);
> >               }
> >
> > -             /*
> > -              * Old frontends do not assert data_validated but we
> > -              * can infer it from csum_blank so test both flags.
> > -              */
> > -             if (txp->flags & (NETTXF_data_validated|NETTXF_csum_blank))
> > +             if (txp->flags & NETTXF_csum_blank)
> >                       skb->ip_summed = CHECKSUM_PARTIAL;
> > -             else
> > -                     skb->ip_summed = CHECKSUM_NONE;
> > +             else if (txp->flags & NETTXF_data_validated)
> > +                     skb->ip_summed = CHECKSUM_UNNECESSARY;
> >
> >               netbk_fill_frags(skb);
> >
> >
> >
> Works for me!

Thanks everyone who tested.

Jeremy/Konrad,

The following changes since commit 7ec9d1e805338fd18bbce73e8c72e4d09da8f2d5:
  Ian Campbell (1):
        xen/netback: correctly setup skb->ip_summed on receive

are available in the git repository at:

  git://xenbits.xensource.com/people/ianc/linux-2.6.git for-jeremy/netback

Re: Re: [mdlabriola@yahoo.com: [Xen-devel] Re: Re: DomU routed traffic disappearing in vif.]

[Konrad Rzeszutek Wilk]

> Works for me!

Awesome.
> 
> The DomU router now works like charm. Thanks! :) Also the problem with
> ICMP traffic disappeared (non UDP/TCP packet dropped in syslog).
> 
> Patched the 2.6.32.11 kernel, performance wise, the 2.6.18.8 had bit
> better some numbers (~ 60 Mbit difference with Iperf), will have some
> more numbers later. 

Somebody pointed out that when they disabled most of the debug options
(don't know which ones), the performance was on par with 2.6.18.