From: Eric Dumazet <eric.dumazet@gmail.com>
To: Willy Tarreau <w@1wt.eu>, Marek Majkowski <marek@cloudflare.com>
Cc: netdev@vger.kernel.org
Subject: Re: splice() performance for TCP socket forwarding
Date: Thu, 13 Dec 2018 05:37:11 -0800 [thread overview]
Message-ID: <de250149-ac07-6ac3-e770-eda64dd0a84d@gmail.com> (raw)
In-Reply-To: <20181213125553.GA16149@1wt.eu>
On 12/13/2018 04:55 AM, Willy Tarreau wrote:
>
> It's quite strange, it doesn't match at all what I'm used to. In haproxy
> we're using splicing as well between sockets, and for medium to large
> objects we always get much better performance with splicing than without.
> 3 years ago during a test, we reached 60 Gbps on a 4-core machine using
> 2 40G NICs, which is not an exceptional sizing. And between processes on
> the loopback, numbers around 100G are totally possible. By the way this
> is one test you should start with, to verify if the issue is more on the
> splice side or on the NIC's side. It might be that your network driver is
> totally inefficient when used with GRO/GSO. In my case, multi-10G using
> ixgbe and 40G using mlx5 have always shown excellent results.
Maybe mlx5 driver is in LRO mode, packing TCP payload in 4K pages ?
bnx2x GRO/LRO has this mode, meaning that around 8 pages are used for a GRO packets of ~32 KB,
while mlx4 for instance would use one page frag for every ~1428 bytes of payload.
next prev parent reply other threads:[~2018-12-13 13:37 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-13 11:25 splice() performance for TCP socket forwarding Marek Majkowski
2018-12-13 12:49 ` Eric Dumazet
2018-12-13 13:17 ` Marek Majkowski
2018-12-13 13:18 ` Marek Majkowski
2018-12-13 13:33 ` Marek Majkowski
2018-12-13 14:03 ` Eric Dumazet
2018-12-13 14:05 ` Eric Dumazet
2018-12-13 14:04 ` Willy Tarreau
2018-12-13 12:55 ` Willy Tarreau
2018-12-13 13:37 ` Eric Dumazet [this message]
2018-12-13 13:57 ` Willy Tarreau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=de250149-ac07-6ac3-e770-eda64dd0a84d@gmail.com \
--to=eric.dumazet@gmail.com \
--cc=marek@cloudflare.com \
--cc=netdev@vger.kernel.org \
--cc=w@1wt.eu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).