From: Jesper Dangaard Brouer <brouer@redhat.com>
To: Jesper Dangaard Brouer <brouer@redhat.com>, netdev@vger.kernel.org
Cc: Alexander Duyck <alexander.h.duyck@intel.com>,
Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
Daniel Borkmann <dborkman@redhat.com>,
Florian Westphal <fw@strlen.de>,
"David S. Miller" <davem@davemloft.net>,
Stephen Hemminger <shemminger@vyatta.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Robert Olsson <robert@herjulf.se>,
Ben Greear <greearb@candelatech.com>,
John Fastabend <john.r.fastabend@intel.com>,
danieltt@kth.se, zhouzhouyi@gmail.com
Subject: [net-next PATCH 0/5] Optimizing "pktgen" for single CPU performance
Date: Wed, 14 May 2014 16:17:38 +0200 [thread overview]
Message-ID: <20140514141545.20309.28343.stgit@dragon> (raw)
I'm on a quest to push the packet per sec (pps) limits of our network
stack, with a special focus on single CPU performance.
My first action is to measure and identify bottlenecks in the transmit
path. For achieving this goal, I need a fast in-kernel packet
generator, like "pktgen". It turned out that "pktgen" were too slow.
Thus, this series focus on optimizing "pktgen" for single CPU performance.
Overview 1xCPU performance Packet Per Sec (pps) stats:
* baseline: 3,930,068 pps
* patch2: 5,362,722 pps -- TXSZ=1024
* patch3: 5,608,781 pps --> 178.29ns per pkt
* patch4: 5,857,065 pps --> 170.73ns ( -7.56ns)
* patch5: 6,346,500 pps --> 157.56ns (-13.17ns)
* No-lock: 6,642,948 pps --> 150.53ns ( -7.03ns)
The last result "No-lock" removes the HARD_TX_{UN}LOCK, and is not
applicable to upstream. It removes two "LOCK" instructions (cost 8ns
each), thus I were expecting to see an improvement of 16ns, but we
only see 7ns. This leads me to believe, that we have reached the
ixgbe driver limit, single queue.
Setup according to blogpost:
http://netoptimizer.blogspot.dk/2014/04/basic-tuning-for-network-overload.html
Hardware:
System: CPU E5-2630
NIC: Intel ixgbe/82599 chip
Testing done with net-next git tree on top of
commit 79e0f1c9f (ipv6: Need to sock_put on csum error).
Pktgen script exercising race condition:
https://github.com/netoptimizer/network-testing/blob/master/pktgen/unit_test01_race_add_rem_device_loop.sh
---
Jesper Dangaard Brouer (5):
pktgen: RCU'ify "if_list" to remove lock in next_to_run()
pktgen: avoid expensive set_current_state() call in loop
pktgen: avoid atomic_inc per packet in xmit loop
ixgbe: increase default TX ring buffer to 1024
ixgbe: trivial fixes while reading code
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2
net/core/pktgen.c | 115 +++++++++++++------------
3 files changed, 61 insertions(+), 58 deletions(-)
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
next reply other threads:[~2014-05-14 14:18 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-14 14:17 Jesper Dangaard Brouer [this message]
2014-05-14 14:17 ` [net-next PATCH 1/5] ixgbe: trivial fixes while reading code Jesper Dangaard Brouer
2014-05-14 14:17 ` [net-next PATCH 2/5] ixgbe: increase default TX ring buffer to 1024 Jesper Dangaard Brouer
2014-05-14 14:28 ` David Laight
2014-05-14 19:25 ` Jesper Dangaard Brouer
2014-05-14 16:28 ` Alexander Duyck
2014-05-14 17:49 ` David Miller
2014-05-14 19:09 ` Jesper Dangaard Brouer
2014-05-14 19:54 ` David Miller
2014-05-15 9:16 ` David Laight
2014-05-29 15:29 ` Jesper Dangaard Brouer
2014-05-14 14:17 ` [net-next PATCH 3/5] pktgen: avoid atomic_inc per packet in xmit loop Jesper Dangaard Brouer
2014-05-14 14:35 ` Eric Dumazet
2014-05-14 15:13 ` Jesper Dangaard Brouer
2014-05-14 15:35 ` Eric Dumazet
2014-05-14 14:17 ` [net-next PATCH 4/5] pktgen: avoid expensive set_current_state() call in loop Jesper Dangaard Brouer
2014-05-14 14:18 ` [net-next PATCH 5/5] pktgen: RCU'ify "if_list" to remove lock in next_to_run() Jesper Dangaard Brouer
2014-06-26 11:16 ` [net-next PATCH V2 0/3] Optimizing pktgen for single CPU performance Jesper Dangaard Brouer
2014-06-26 11:16 ` [net-next PATCH V2 1/3] pktgen: document tuning for max NIC performance Jesper Dangaard Brouer
2014-06-26 11:16 ` [net-next PATCH V2 2/3] pktgen: avoid expensive set_current_state() call in loop Jesper Dangaard Brouer
2014-06-26 11:16 ` [net-next PATCH V2 3/3] pktgen: RCU-ify "if_list" to remove lock in next_to_run() Jesper Dangaard Brouer
2014-07-01 22:51 ` [net-next PATCH V2 0/3] Optimizing pktgen for single CPU performance David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140514141545.20309.28343.stgit@dragon \
--to=brouer@redhat.com \
--cc=alexander.h.duyck@intel.com \
--cc=danieltt@kth.se \
--cc=davem@davemloft.net \
--cc=dborkman@redhat.com \
--cc=fw@strlen.de \
--cc=greearb@candelatech.com \
--cc=jeffrey.t.kirsher@intel.com \
--cc=john.r.fastabend@intel.com \
--cc=netdev@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=robert@herjulf.se \
--cc=shemminger@vyatta.com \
--cc=zhouzhouyi@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).