netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* tx-nocache-copy performance
@ 2014-01-06 20:27 Benjamin Poirier
  2014-01-06 20:57 ` Eric Dumazet
  2014-01-06 20:59 ` tx-nocache-copy performance Tom Herbert
  0 siblings, 2 replies; 9+ messages in thread
From: Benjamin Poirier @ 2014-01-06 20:27 UTC (permalink / raw)
  To: Tom Herbert; +Cc: netdev

Hi Tom,

In commit "c6e1a0d net: Allow no-cache copy from user on transmit
(v3.0-rc1)" you introduced the tx-nocache-copy performance optimization
and set it to on by default. I've tried to reproduce your testcase, as
well as a few more, but I did not find any performance improvement from
turning on tx-nocache-copy. Do you think tx-nocache-copy is still a
worthwhile optimization and it should remain on by default? In which
situations does it help?

I've ran latency tests similar to the ones you described in the commit
log. I've also tested how the option affects single stream throughput
tests. According to the results I obtained, it seems that
tx-nocache-copy has either no impact (in the latency test) or a negative
impact (in the throughput test).

My test results follow. I tested using 3.12.6 on one Intel Xeon W3565
and one i7 920 connected by ixgbe adapters. The results are from the
Xeon, but they're similar on the i7. All numbers report the mean±stddev
over 10 runs of 10s.

1) latency tests similar to what you described
There is no statistically significant difference between tx-nocache-copy
on/off.
nic irqs spread out (one queue per cpu)

200x netperf -r 1400,1
tx-nocache-copy off
        692000±1000 tps
        50/90/95/99% latency (us): 275±2/643.8±0.4/799±1/2474.4±0.3
tx-nocache-copy on
        693000±1000 tps
        50/90/95/99% latency (us): 274±1/644.1±0.7/800±2/2474.5±0.7

200x netperf -r 14000,14000
tx-nocache-copy off
        86450±80 tps
        50/90/95/99% latency (us): 334.37±0.02/838±1/2100±20/3990±40
tx-nocache-copy on
        86110±60 tps
        50/90/95/99% latency (us): 334.28±0.01/837±2/2110±20/3990±20

2) single stream throughput tests
tx-nocache-copy leads to higher service demand

                        throughput  cpu0        cpu1        demand
                        (Gb/s)      (Gcycle)    (Gcycle)    (cycle/B)

nic irqs and netperf on cpu0 (1x netperf -T0,0 -t omni -- -d send)

tx-nocache-copy off     9402±5      9.4±0.2                 0.80±0.01
tx-nocache-copy on      9403±3      9.85±0.04               0.838±0.004

nic irqs on cpu0, netperf on cpu1 (1x netperf -T1,1 -t omni -- -d send)

tx-nocache-copy off     9401±5      5.83±0.03   5.0±0.1     0.923±0.007
tx-nocache-copy on      9404±2      5.74±0.03   5.523±0.009 0.958±0.002

-Benjamin

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2014-01-07 21:20 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-06 20:27 tx-nocache-copy performance Benjamin Poirier
2014-01-06 20:57 ` Eric Dumazet
2014-01-06 21:13   ` Eric Dumazet
2014-01-06 23:20   ` [PATCH] net: Do not enable tx-nocache-copy by default Benjamin Poirier
2014-01-07  1:00     ` David Miller
2014-01-07 15:11       ` [PATCH v2] " Benjamin Poirier
2014-01-07 21:20         ` David Miller
2014-01-06 20:59 ` tx-nocache-copy performance Tom Herbert
2014-01-06 21:20   ` David Miller

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).