From: James Yonan <james@openvpn.net>
To: Rick Jones <rick.jones2@hp.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>, netdev <netdev@vger.kernel.org>
Subject: Re: GSO/GRO and UDP performance
Date: Fri, 06 Sep 2013 13:26:46 -0600 [thread overview]
Message-ID: <522A2C76.10203@openvpn.net> (raw)
In-Reply-To: <522A05E9.3090206@hp.com>
On 06/09/2013 10:42, Rick Jones wrote:
> On 09/06/2013 06:07 AM, Eric Dumazet wrote:
>> On Fri, 2013-09-06 at 03:22 -0600, James Yonan wrote:
>>
>>> So I think that playing well with GSO/GRO is essential to get speedup in
>>> UDP apps because of this 43x multiplier.
>>>
>>
>> Thats not true. GRO cannot aggregate more than 16+1 packets.
Where does the 16+1 come from? I'm getting my 43x from the ratio of max
legal IP packet size (64KB) / internet MTU (1500). Are you saying that
GRO cannot aggregate up to 64 KB?
>> I think we cannot aggregate UDP packets, because UDP lacks sequence
>> numbers, so reorders would be a problem.
>> You really need something that is not UDP generic.
Right -- that's why I'm proposing a hook for UDP GSO/GRO providers that
know about specific app-layer protocols and can provide segmentation and
aggregation methods for them. Such a provider would be implemented in a
kernel module and would know about the specific app-layer protocol, so
it would be able to losslessly segment and aggregate it (i.e. it could
use a sequence number from the app-layer protocol).
> It may not be as sexy, and it cannot get the 43x multiplier (just what
> *is* the service demand change on a netperf TCP_STREAM test these days
> between GSO/GRO on and off anyway?)
That's something I haven't really looked too closely at yet. With
MAX_GRO_SKBS set to only 8, how well would this really scale?
> but looking for basic path-length reductions would be goodness.
Path is fairly optimized as-is.
Direction 1: udp_encap_recv -> tunnel decapsulation -> netif_rx
Direction 2: ndo_start_xmit -> tunnel encapsulation -> ip_local_out
I've also looked into getting closer to driver TX by using
dev_queue_xmit instead of ip_local_out.
Even though this is a virtual driver without interrupts, I'm also
looking at NAPI as a way of getting packet flows into GRO on the RX side.
Bottom line is that I want to saturate 10 GigE with UDP packets without
breaking a sweat. ixgbe or other drivers in that class can handle it if
the per-packet overhead in the network stack can be reduced enough.
James
next prev parent reply other threads:[~2013-09-06 19:26 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-04 10:07 GSO/GRO and UDP performance James Yonan
2013-09-04 11:53 ` Eric Dumazet
2013-09-06 9:22 ` James Yonan
2013-09-06 13:07 ` Eric Dumazet
2013-09-06 16:42 ` Rick Jones
2013-09-06 19:26 ` James Yonan [this message]
2013-09-06 19:32 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=522A2C76.10203@openvpn.net \
--to=james@openvpn.net \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=rick.jones2@hp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).