Re: TSO/GRO/LRO/somethingO breaks LVS on 2.6.36

[Simon Horman <horms@verge.net.au>]

On Wed, Jan 26, 2011 at 04:48:05PM -0800, Simon Kirby wrote:
> On Thu, Jan 13, 2011 at 03:34:22PM +0900, Simon Horman wrote:
> 
> > Hi Simon,
> > 
> > thanks for prodding me to respond to this post offline and sorry for not
> > responding earlier.
> > 
> > Firstly, I think that this is a receive-side problem so I don't believe
> > that GSO (generic segmentation offload) or other transmit-side options are
> > likely to have any affect.
> > 
> > My understanding is that on the receive-side there are two options which
> > when enabled can result in the behaviour that you describe.
> > 
> > * LRO (large receive offload)
> > 
> >   You have this disabled, and assuming it really is disabled it
> >   shouldn't be causing a problem.
> > 
> > * GRO (generic receive offload)
> > 
> >   This does not seem to be in the output of your ethtool commands at all.
> >   So I wonder if your ethtool is too old to support this option?
> 
> So, this was the case.  Our ethtool (lenny) was too old to see the GRO
> option, only GSO.  Disabling GRO on eth1.39 has no effect, but disabling
> it on eth1 caused it to stop receiving the merged frames, fixing the LVS
> packet loss (due to no sending GSO support from LVS/IPVS).
> 
> Speaking of this, did your patch for LVS/IPVS GSO support go anywhere? 

The patch for IPVS GRO support has been merged and should appear in 2.6.39.
This is somewhat later than I previously anticipated due to a merge mix-up
on my part.

> >   In any case, I was able to reproduce the problem that you describe (or at
> >   least something very similar) using 2.6.36 with GRO enabled on eth1.1 and
> >   the problem did not manifest when I disabled GRO on eth1.1.
> 
> It worked for you to do ethtool -K eth1.1 gro off, then?  For me on
> 2.6.37, it seemed to be that "ethtool -K eth1 gro off" was needed, even
> though packets arrive on eth1.39.

I will recheck my results, but in general I think it is a bit
of an open question as to how ethtool settings should be propagated
between related devices.

> Also, strangely, 2.6.35.4's default state (with no received merged frames)
> has GRO on for eth1 but off for eth1.39:
> 
> # ethtool -k eth1
> Offload parameters for eth1:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp-segmentation-offload: on
> udp-fragmentation-offload: off
> generic-segmentation-offload: on
> generic-receive-offload: on
> large-receive-offload: off
> ntuple-filters: off
> receive-hashing: off
> 
> # ethtool -k eth1.39
> Offload parameters for eth1.39:
> rx-checksumming: on
> tx-checksumming: off
> scatter-gather: off
> tcp-segmentation-offload: off
> udp-fragmentation-offload: off
> generic-segmentation-offload: off
> generic-receive-offload: off
> large-receive-offload: off
> ntuple-filters: off
> receive-hashing: off
> 
> If I set 2.6.37 to have all of the same options, I still see GRO frames
> on 2.6.37 (tg3), which is weird.

Yes that is a weird.

There has been quite a lot of work on VLANs recently and
I suspect that the behaviour that you are observing with 2.6.37
is a regression that occurred during that work. It would
be good to fix things to restore the 2.6.35 behaviour.