netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jason Wang <jasowang@redhat.com>
To: Michael Dalton <mwdalton@google.com>,
	"David S. Miller" <davem@davemloft.net>
Cc: netdev@vger.kernel.org, Eric Dumazet <edumazet@google.com>,
	virtualization@lists.linux-foundation.org,
	"Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [PATCH net-next 3/3] net: auto-tune mergeable rx buffer size for improved performance
Date: Thu, 26 Dec 2013 15:33:01 +0800	[thread overview]
Message-ID: <52BBDBAD.8030609@redhat.com> (raw)
In-Reply-To: <1387239389-13216-3-git-send-email-mwdalton@google.com>

On 12/17/2013 08:16 AM, Michael Dalton wrote:
> Commit 2613af0ed18a ("virtio_net: migrate mergeable rx buffers to page frag
> allocators") changed the mergeable receive buffer size from PAGE_SIZE to
> MTU-size, introducing a single-stream regression for benchmarks with large
> average packet size. There is no single optimal buffer size for all
> workloads.  For workloads with packet size <= MTU bytes, MTU + virtio-net
> header-sized buffers are preferred as larger buffers reduce the TCP window
> due to SKB truesize. However, single-stream workloads with large average
> packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers
> are used.
>
> This commit auto-tunes the mergeable receiver buffer packet size by
> choosing the packet buffer size based on an EWMA of the recent packet
> sizes for the receive queue. Packet buffer sizes range from MTU_SIZE +
> virtio-net header len to PAGE_SIZE. This improves throughput for
> large packet workloads, as any workload with average packet size >=
> PAGE_SIZE will use PAGE_SIZE buffers.
>
> These optimizations interact positively with recent commit
> ba275241030c ("virtio-net: coalesce rx frags when possible during rx"),
> which coalesces adjacent RX SKB fragments in virtio_net. The coalescing
> optimizations benefit buffers of any size.
>
> Benchmarks taken from an average of 5 netperf 30-second TCP_STREAM runs
> between two QEMU VMs on a single physical machine. Each VM has two VCPUs
> with all offloads & vhost enabled. All VMs and vhost threads run in a
> single 4 CPU cgroup cpuset, using cgroups to ensure that other processes
> in the system will not be scheduled on the benchmark CPUs. Trunk includes
> SKB rx frag coalescing.
>
> net-next w/ virtio_net before 2613af0ed18a (PAGE_SIZE bufs): 14642.85Gb/s
> net-next (MTU-size bufs):  13170.01Gb/s
> net-next + auto-tune: 14555.94Gb/s
>
> Signed-off-by: Michael Dalton <mwdalton@google.com>

The patch looks good to me and test this patch with mlx4, it help to
increase the rx performance from about 22Gb/s to about 26 Gb/s.

  parent reply	other threads:[~2013-12-26  7:33 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-12-17  0:16 [PATCH net-next 1/3] net: allow > 0 order atomic page alloc in skb_page_frag_refill Michael Dalton
2013-12-17  0:16 ` [PATCH net-next 2/3] virtio-net: use per-receive queue page frag alloc for mergeable bufs Michael Dalton
2013-12-23  8:12   ` Jason Wang
2013-12-23 17:27     ` Eric Dumazet
2013-12-23 19:37       ` Michael S. Tsirkin
2013-12-26 21:28         ` Michael Dalton
2013-12-26 21:37           ` Michael S. Tsirkin
2013-12-26 22:00             ` Eric Dumazet
2014-01-08 17:21               ` Michael S. Tsirkin
2014-01-08 18:09                 ` Eric Dumazet
2014-01-08 18:57                   ` Michael S. Tsirkin
2014-01-08 19:54                   ` David Miller
2014-01-08 21:16                   ` Rick Jones
2013-12-26 21:56           ` Eric Dumazet
2013-12-27  4:55             ` Jason Wang
2013-12-27  5:46               ` Eric Dumazet
2013-12-27  6:12                 ` Jason Wang
2013-12-23 13:31   ` Michael S. Tsirkin
2013-12-17  0:16 ` [PATCH net-next 3/3] net: auto-tune mergeable rx buffer size for improved performance Michael Dalton
2013-12-23 12:51   ` Michael S. Tsirkin
2013-12-23 13:33   ` Michael S. Tsirkin
2013-12-30 10:14     ` Amos Kong
2014-01-08 17:41       ` Michael S. Tsirkin
2013-12-26  7:33   ` Jason Wang [this message]
2013-12-26 20:06     ` Michael Dalton
2013-12-26 20:24       ` Michael S. Tsirkin
2013-12-27  3:04       ` Jason Wang
2013-12-27 21:41         ` Michael Dalton
2013-12-30  4:50           ` Jason Wang
2013-12-30  5:38           ` Jason Wang
2014-01-08 17:37           ` Michael S. Tsirkin
2013-12-19 19:58 ` [PATCH net-next 1/3] net: allow > 0 order atomic page alloc in skb_page_frag_refill David Miller
2013-12-23 13:35   ` Michael S. Tsirkin
2013-12-23  7:52 ` Jason Wang
2013-12-23 17:24   ` Eric Dumazet
2013-12-23 12:53 ` Michael S. Tsirkin
2013-12-23 17:30   ` Eric Dumazet
2013-12-23 19:19     ` Michael S. Tsirkin
2013-12-24 22:46 ` David Miller
2014-01-03  0:42   ` Debabrata Banerjee
2014-01-03  0:56     ` Eric Dumazet
2014-01-03  1:26       ` Eric Dumazet
2014-01-03  1:59         ` Debabrata Banerjee
2014-01-03 22:47           ` Debabrata Banerjee
2014-01-03 22:54             ` Eric Dumazet
2014-01-03 23:27               ` Debabrata Banerjee

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=52BBDBAD.8030609@redhat.com \
    --to=jasowang@redhat.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=mst@redhat.com \
    --cc=mwdalton@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).