From: Jason Wang <jasowang@redhat.com>
To: Michael Dalton <mwdalton@google.com>,
"David S. Miller" <davem@davemloft.net>
Cc: netdev@vger.kernel.org, Eric Dumazet <edumazet@google.com>,
virtualization@lists.linux-foundation.org,
"Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [PATCH net-next 3/3] net: auto-tune mergeable rx buffer size for improved performance
Date: Thu, 26 Dec 2013 15:33:01 +0800 [thread overview]
Message-ID: <52BBDBAD.8030609@redhat.com> (raw)
In-Reply-To: <1387239389-13216-3-git-send-email-mwdalton@google.com>
On 12/17/2013 08:16 AM, Michael Dalton wrote:
> Commit 2613af0ed18a ("virtio_net: migrate mergeable rx buffers to page frag
> allocators") changed the mergeable receive buffer size from PAGE_SIZE to
> MTU-size, introducing a single-stream regression for benchmarks with large
> average packet size. There is no single optimal buffer size for all
> workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net
> header-sized buffers are preferred as larger buffers reduce the TCP window
> due to SKB truesize. However, single-stream workloads with large average
> packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers
> are used.
>
> This commit auto-tunes the mergeable receiver buffer packet size by
> choosing the packet buffer size based on an EWMA of the recent packet
> sizes for the receive queue. Packet buffer sizes range from MTU_SIZE +
> virtio-net header len to PAGE_SIZE. This improves throughput for
> large packet workloads, as any workload with average packet size >=
> PAGE_SIZE will use PAGE_SIZE buffers.
>
> These optimizations interact positively with recent commit
> ba275241030c ("virtio-net: coalesce rx frags when possible during rx"),
> which coalesces adjacent RX SKB fragments in virtio_net. The coalescing
> optimizations benefit buffers of any size.
>
> Benchmarks taken from an average of 5 netperf 30-second TCP_STREAM runs
> between two QEMU VMs on a single physical machine. Each VM has two VCPUs
> with all offloads & vhost enabled. All VMs and vhost threads run in a
> single 4 CPU cgroup cpuset, using cgroups to ensure that other processes
> in the system will not be scheduled on the benchmark CPUs. Trunk includes
> SKB rx frag coalescing.
>
> net-next w/ virtio_net before 2613af0ed18a (PAGE_SIZE bufs): 14642.85Gb/s
> net-next (MTU-size bufs): 13170.01Gb/s
> net-next + auto-tune: 14555.94Gb/s
>
> Signed-off-by: Michael Dalton <mwdalton@google.com>
The patch looks good to me and test this patch with mlx4, it help to
increase the rx performance from about 22Gb/s to about 26 Gb/s.
next prev parent reply other threads:[~2013-12-26 7:33 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-17 0:16 [PATCH net-next 1/3] net: allow > 0 order atomic page alloc in skb_page_frag_refill Michael Dalton
2013-12-17 0:16 ` [PATCH net-next 2/3] virtio-net: use per-receive queue page frag alloc for mergeable bufs Michael Dalton
2013-12-23 8:12 ` Jason Wang
2013-12-23 17:27 ` Eric Dumazet
2013-12-23 19:37 ` Michael S. Tsirkin
2013-12-26 21:28 ` Michael Dalton
2013-12-26 21:37 ` Michael S. Tsirkin
2013-12-26 22:00 ` Eric Dumazet
2014-01-08 17:21 ` Michael S. Tsirkin
2014-01-08 18:09 ` Eric Dumazet
2014-01-08 18:57 ` Michael S. Tsirkin
2014-01-08 19:54 ` David Miller
2014-01-08 21:16 ` Rick Jones
2013-12-26 21:56 ` Eric Dumazet
2013-12-27 4:55 ` Jason Wang
2013-12-27 5:46 ` Eric Dumazet
2013-12-27 6:12 ` Jason Wang
2013-12-23 13:31 ` Michael S. Tsirkin
2013-12-17 0:16 ` [PATCH net-next 3/3] net: auto-tune mergeable rx buffer size for improved performance Michael Dalton
2013-12-23 12:51 ` Michael S. Tsirkin
2013-12-23 13:33 ` Michael S. Tsirkin
2013-12-30 10:14 ` Amos Kong
2014-01-08 17:41 ` Michael S. Tsirkin
2013-12-26 7:33 ` Jason Wang [this message]
2013-12-26 20:06 ` Michael Dalton
2013-12-26 20:24 ` Michael S. Tsirkin
2013-12-27 3:04 ` Jason Wang
2013-12-27 21:41 ` Michael Dalton
2013-12-30 4:50 ` Jason Wang
2013-12-30 5:38 ` Jason Wang
2014-01-08 17:37 ` Michael S. Tsirkin
2013-12-19 19:58 ` [PATCH net-next 1/3] net: allow > 0 order atomic page alloc in skb_page_frag_refill David Miller
2013-12-23 13:35 ` Michael S. Tsirkin
2013-12-23 7:52 ` Jason Wang
2013-12-23 17:24 ` Eric Dumazet
2013-12-23 12:53 ` Michael S. Tsirkin
2013-12-23 17:30 ` Eric Dumazet
2013-12-23 19:19 ` Michael S. Tsirkin
2013-12-24 22:46 ` David Miller
2014-01-03 0:42 ` Debabrata Banerjee
2014-01-03 0:56 ` Eric Dumazet
2014-01-03 1:26 ` Eric Dumazet
2014-01-03 1:59 ` Debabrata Banerjee
2014-01-03 22:47 ` Debabrata Banerjee
2014-01-03 22:54 ` Eric Dumazet
2014-01-03 23:27 ` Debabrata Banerjee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52BBDBAD.8030609@redhat.com \
--to=jasowang@redhat.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=mst@redhat.com \
--cc=mwdalton@google.com \
--cc=netdev@vger.kernel.org \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).