From: Rick Jones <rick.jones2@hp.com>
To: Christoph Lameter <cl@linux.com>
Cc: netdev@vger.kernel.org
Subject: Re: Network latency regressions from 2.6.22 to 2.6.29
Date: Thu, 16 Apr 2009 10:21:10 -0700 [thread overview]
Message-ID: <49E76906.2060205@hp.com> (raw)
In-Reply-To: <alpine.DEB.1.10.0904161151330.13531@qirst.com>
Christoph Lameter wrote:
> The following are results of lantency measurements using udpping
> (available from http://gentwo.org/ll). It shows that significant latencies
> were added since 2.6.27. I surely wish we could get back to times below 90
> microseconds.
>
> The tests were done over 1G ethernet using
> 09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754
> Gigabit Ethernet PCI Express (rev 02)
>
> Results:
>
> 2.6.22 2.6.23 2.6.24 2.6.25 2.6.26 2.6.27 2.6.28 2.6.29
> 40 Bytes 89.50 90.75 89.61 91.51 91.89 99.17 99.80 99.34
> 400 Bytes 98.58 101.44 97.85 99.61 100.36 117.96 118.10 126.79
> 1400 Bytes 152.76 153.75 153.85 156.22 156.66 163.92 165.54 166.04
>
> Compared to 2.6.22 2.6.23 2.6.24 2.6.25 2.6.26 2.6.27 2.6.28 2.6.29
> 40 Bytes -1.4% -0.1% -2.2% -2.6% -9.8% -10.3% -9.9%
> 400 Bytes -2.8% 0.7% -1.0% -1.8% -16.4% -16.5%-22.2%
> 1400 Bytes -0.6% -0.7% -2.2% -2.5% -6.8% -7.7% -8.0%
>
> I presented these numbers with some nice graphs at the Linux Collab Summit
> last week.
>
> See
> http://www.kernel.org/pub/linux/kernel/people/christoph/collab-spring-2009/Collab-summit-2009-sf.pdf
Does udpping have a concept of service demand a la netperf? That could help show
how much was code bloat vs say some tweak to interrupt coalescing parameters in
the NIC/driver.
[root@bl870c1 netperf2_trunk]# netperf -T 0 -c -C -t UDP_RR -H bl870c2.west -v 2
UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to bl870c2.west
(10.208.0.210) port 0 AF_INET : histogram : first burst 0 : cpu bind
Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
Send Recv Size Size Time Rate local remote local remote
bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
126976 126976 1 1 10.00 7550.46 2.33 2.41 24.721 25.551
126976 126976
Histogram of request/reponse times.
UNIT_USEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
TEN_USEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 1
HUNDRED_USEC : 0: 75508: 0: 0: 0: 0: 0: 0: 0: 0
UNIT_MSEC : 0: 2: 0: 0: 0: 0: 0: 0: 0: 0
TEN_MSEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
HUNDRED_MSEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
UNIT_SEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
TEN_SEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
>100_SECS: 0
HIST_TOTAL: 75511
I guess one of these days I have to tweak netperf to be able to get latencies
when doing bursts on the connection...
[root@bl870c1 netperf2_trunk]# ethtool -C eth0 rx-frames 1
[do the same on the other end]
[root@bl870c1 netperf2_trunk]# netperf -T 0 -c -C -t UDP_RR -H bl870c2.west -v 2
UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to bl870c2.west
(10.208.0.210) port 0 AF_INET : histogram : first burst 0 : cpu bind
Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
Send Recv Size Size Time Rate local remote local remote
bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
126976 126976 1 1 10.00 11126.15 2.60 3.43 18.711 24.633
126976 126976
Histogram of request/reponse times.
UNIT_USEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
TEN_USEC : 0: 0: 0: 0: 0: 0: 0: 16: 71799: 38583
HUNDRED_USEC : 0: 856: 8: 0: 0: 0: 0: 0: 0: 0
UNIT_MSEC : 0: 1: 0: 0: 0: 0: 1: 0: 0: 0
TEN_MSEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
HUNDRED_MSEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
UNIT_SEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
TEN_SEC : 0: 0: 0: 0: 0: 0: 0: 0: 0: 0
>100_SECS: 0
HIST_TOTAL: 111264
next prev parent reply other threads:[~2009-04-16 17:21 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-16 16:10 Network latency regressions from 2.6.22 to 2.6.29 Christoph Lameter
2009-04-16 17:21 ` Rick Jones [this message]
2009-04-16 19:06 ` Christoph Lameter
2009-04-16 19:29 ` Eric Dumazet
2009-04-16 19:33 ` Christoph Lameter
2009-04-16 22:47 ` David Miller
2009-04-17 13:46 ` Christoph Lameter
2009-04-17 21:43 ` Ilpo Järvinen
2009-04-16 20:05 ` Rick Jones
2009-04-16 18:07 ` Ben Hutchings
2009-04-16 19:02 ` Christoph Lameter
2009-04-16 21:19 ` Ben Hutchings
2009-04-16 22:47 ` David Miller
2009-04-16 19:43 ` Eric Dumazet
2009-04-16 19:50 ` Eric Dumazet
2009-04-16 20:01 ` Christoph Lameter
2009-04-16 23:00 ` David Miller
2009-04-17 16:42 ` Network latency regressions from 2.6.22 to 2.6.29 (results with IRQ affinity) Christoph Lameter
2009-04-18 8:18 ` Eric Dumazet
2009-04-18 8:20 ` Eric Dumazet
2009-04-18 19:43 ` Eric Dumazet
2009-04-20 17:29 ` Christoph Lameter
2009-04-20 17:57 ` Eric Dumazet
2009-04-20 18:13 ` Christoph Lameter
2009-04-20 18:46 ` Eric Dumazet
2009-04-20 19:16 ` Christoph Lameter
2009-04-20 20:07 ` Eric Dumazet
2009-04-20 21:14 ` Christoph Lameter
2009-04-20 21:52 ` Eric Dumazet
2009-04-21 14:00 ` Christoph Lameter
2009-04-21 19:36 ` Network latency regressions from 2.6.22 to 2.6.29 (MSI off) Christoph Lameter
2009-04-20 19:44 ` Network latency regressions from 2.6.22 to 2.6.29 (results with IRQ affinity) Evgeniy Polyakov
2009-04-16 19:55 ` Network latency regressions from 2.6.22 to 2.6.29 Christoph Lameter
2009-04-16 21:57 ` Michael Chan
2009-04-17 13:47 ` Christoph Lameter
2009-04-16 22:59 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=49E76906.2060205@hp.com \
--to=rick.jones2@hp.com \
--cc=cl@linux.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.