From: Eric Dumazet <eric.dumazet@gmail.com>
To: Ben Hutchings <bhutchings@solarflare.com>
Cc: Simon Horman <horms@verge.net.au>, netdev@vger.kernel.org
Subject: Re: Bonding, GRO and tcp_reordering
Date: Tue, 30 Nov 2010 17:04:33 +0100 [thread overview]
Message-ID: <1291133073.2904.128.camel@edumazet-laptop> (raw)
In-Reply-To: <1291131776.21077.27.camel@bwh-desktop>
Le mardi 30 novembre 2010 à 15:42 +0000, Ben Hutchings a écrit :
> On Tue, 2010-11-30 at 22:55 +0900, Simon Horman wrote:
> > The only other parameter that seemed to have significant effect was to
> > increase the mtu. In the case of MTU=9000, GRO seemed to have a negative
> > impact on throughput, though a significant positive effect on CPU
> > utilisation.
> [...]
>
> Increasing MTU also increases the interval between packets on a TCP flow
> using maximum segment size so that it is more likely to exceed the
> difference in delay.
>
GRO really is operational _if_ we receive in same NAPI run several
packets for the same flow.
As soon as we exit NAPI mode, GRO packets are flushed.
Big MTU --> bigger delays between packets, so big chance that GRO cannot
trigger at all, since NAPI runs for one packet only.
One possibility with big MTU is to tweak "ethtool -c eth0" params
rx-usecs: 20
rx-frames: 5
rx-usecs-irq: 0
rx-frames-irq: 5
so that "rx-usecs" is bigger than the delay between two MTU full sized
packets.
Gigabit speed means 1 nano second per bit, and MTU=9000 means 72 us
delay between packets.
So try :
ethtool -C eth0 rx-usecs 100
to get chance that several packets are delivered at once by NIC.
Unfortunately, this also add some latency, so it helps bulk transferts,
and slowdown interactive traffic
next prev parent reply other threads:[~2010-11-30 16:04 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-30 13:55 Bonding, GRO and tcp_reordering Simon Horman
2010-11-30 15:42 ` Ben Hutchings
2010-11-30 16:04 ` Eric Dumazet [this message]
2010-12-01 4:34 ` Simon Horman
2010-12-01 4:47 ` Eric Dumazet
2010-12-02 6:39 ` Simon Horman
2010-12-03 13:38 ` Simon Horman
2010-12-01 4:31 ` Simon Horman
2010-11-30 17:56 ` Rick Jones
2010-11-30 18:14 ` Eric Dumazet
2010-12-01 4:30 ` Simon Horman
2010-12-01 19:42 ` Rick Jones
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1291133073.2904.128.camel@edumazet-laptop \
--to=eric.dumazet@gmail.com \
--cc=bhutchings@solarflare.com \
--cc=horms@verge.net.au \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).