From: Simon Kirby <sim@hostway.ca>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Simon Horman <horms@verge.net.au>, netdev@vger.kernel.org
Subject: Re: TSO/GRO/LRO/somethingO breaks LVS on 2.6.36
Date: Wed, 22 Dec 2010 15:39:48 -0800 [thread overview]
Message-ID: <20101222233948.GC17581@hostway.ca> (raw)
In-Reply-To: <1291863119.2795.29.camel@edumazet-laptop>
On Thu, Dec 09, 2010 at 03:51:59AM +0100, Eric Dumazet wrote:
> Le mercredi 08 d??cembre 2010 ?? 18:35 -0800, Simon Kirby a ??crit :
>
> > Right. So, on 2.6.35, ethtool shows:
> >
> > # ethtool -k eth1
> > Offload parameters for eth1:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: on
> > tcp segmentation offload: on
> > udp fragmentation offload: off
> > generic segmentation offload: on
> > large receive offload: off
> >
> > # ethtool -k eth1.39
> > Offload parameters for eth1.39:
> > rx-checksumming: on
> > tx-checksumming: off
> > scatter-gather: off
> > tcp segmentation offload: off
> > udp fragmentation offload: off
> > generic segmentation offload: off
> > large receive offload: off
> >
> > On 2.6.36, ethtool shows:
> >
> > # ethtool -k eth1
> > Offload parameters for eth1:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: on
> > tcp segmentation offload: on
> > udp fragmentation offload: off
> > generic segmentation offload: on
> > large receive offload: off
> >
> > # ethtool -k eth1.39
> > Offload parameters for eth1.39:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: on
> > tcp segmentation offload: on
> > udp fragmentation offload: off
> > generic segmentation offload: on
> > large receive offload: off
> >
> > And if I set it with ethtool -K eth1.39 gso off; ethtool -K eth1 gso off,
> > it says "off", but I still see merged frames, as verified with an nc <
> > /dev/zero and tcpdump -i any -n 'length > 1500'.
>
> Yep, but you also have "tso on" by default on 2.6.36 and eth1.39 (vlan)
and turning it off makes no difference, either. Even if I down/up the
interface after. I still see merged packets on receive, even if I try to
use ethtool to turn off everything that is "on". It's as if the new code
to enable it on vlans works, but the disable from ethtool does not.
In other words, I cannot find a way to disable it (and thus make LVS
work) on 2.6.36 or 2.6.37-rc6.
One interesting note is that if I destroy and recreate the VLAN interface,
it comes back with gso enabled:
# ethtool -K eth1 gso off
# ethtool -K eth1.39 gso off
# ethtool -K eth1 tso off
# ethtool -K eth1.39 tso off
# ifdown eth1.39
Removed VLAN -:eth1.39:-
# ifconfig eth1 down
# ifup eth1.39
Added VLAN with VID == 39 to IF -:eth1:-
# ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
udp fragmentation offload: off
generic segmentation offload: off
large receive offload: off
# ethtool -k eth1.39
Offload parameters for eth1.39:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
udp fragmentation offload: off
generic segmentation offload: on
large receive offload: off
# ethtool -K eth1.39 gso off
# ethtool -k eth1.39
Offload parameters for eth1.39:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
udp fragmentation offload: off
generic segmentation offload: off
large receive offload: off
# ifdown eth1.39; ifup eth1.39
Removed VLAN -:eth1.39:-
Added VLAN with VID == 39 to IF -:eth1:-
# ethtool -k eth1.39
Offload parameters for eth1.39:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
udp fragmentation offload: off
generic segmentation offload: on
large receive offload: off
So, the bug seems to related using it on VLANs.
Simon-
> > > I will prepare a backport of the "ipvs: allow transmit of GRO aggregated
> > > skbs" patch to v2.6.36 and post it shortly. Testing to see if that
> > > resolves the problem that you are seeing would probably be a good start.
> >
> > I can test this in our case, and it should specifically work on the
> > servers we were seeing the problem on, but it still needs some
> > complicated logic to be safe for all cases (like when LVS decides to
> > route out an interface without GRO). Anyway, thanks!
> >
> > Simon-
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2010-12-22 23:39 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-03 10:34 TSO/GRO/LRO/somethingO breaks LVS on 2.6.36 Simon Kirby
2010-12-03 11:29 ` Eric Dumazet
2010-12-03 12:36 ` Simon Horman
2010-12-03 12:42 ` Simon Horman
2010-12-03 13:27 ` Ben Hutchings
2010-12-03 13:39 ` Simon Horman
2010-12-09 2:35 ` Simon Kirby
2010-12-09 2:51 ` Eric Dumazet
2010-12-22 23:39 ` Simon Kirby [this message]
2011-01-13 6:34 ` Simon Horman
2011-01-27 0:48 ` Simon Kirby
2011-01-27 1:36 ` Simon Horman
2011-01-27 7:42 ` Eric Dumazet
2011-01-27 14:42 ` Simon Horman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101222233948.GC17581@hostway.ca \
--to=sim@hostway.ca \
--cc=eric.dumazet@gmail.com \
--cc=horms@verge.net.au \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).