From: Eric Dumazet <dada1@cosmosbay.com>
To: "Paweł Staszewski" <pstaszewski@itcare.pl>
Cc: Linux Network Development list <netdev@vger.kernel.org>
Subject: Re: results with diff affinity settings using ixgbe
Date: Tue, 27 Jan 2009 16:53:24 +0100 [thread overview]
Message-ID: <497F2DF4.4000507@cosmosbay.com> (raw)
In-Reply-To: <497F24CE.6060201@itcare.pl>
Paweł Staszewski a écrit :
> I make some tests with affinity on my machine with some traffic load.
> If someone want look:
>
>
>
> Traffic in all test was 350Mbit/s TX and the same on RX
>
> RX queues on cpu6 / TX queues on cpu7 / intel e1000e on cpu4
> Average: CPU %user %nice %sys %iowait %irq %soft
> %steal %idle intr/s
> Average: all 0.05 0.00 0.00 0.00 1.89 10.03
> 0.00 88.03 221511.70
> Average: 0 0.40 0.00 0.00 0.00 0.00 0.00
> 0.00 99.60 0.70
> Average: 1 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 0.80
> Average: 2 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 0.70
> Average: 3 0.00 0.00 0.10 0.00 0.00 0.40
> 0.00 99.50 0.70
> Average: 4 0.00 0.00 0.00 0.00 2.06 18.74
> 0.00 79.20 12870.50
> Average: 5 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 1.30
> Average: 6 0.00 0.00 0.00 0.00 2.65 57.49
> 0.00 39.86 59410.60
> Average: 7 0.00 0.00 0.00 0.00 10.50 5.00
> 0.00 84.50 53977.90
>
>
> RX queues on cpu6 / TX queues on cpu7 / intel e1000e on cpu1
> Average: CPU %user %nice %sys %iowait %irq %soft
> %steal %idle intr/s
> Average: all 0.08 0.00 0.09 0.00 0.75 14.24
> 0.00 84.84 215755.20
> Average: 0 0.50 0.00 0.00 0.00 0.00 0.00
> 0.00 99.50 1.30
> Average: 1 0.00 0.00 0.53 0.00 2.34 36.91
> 0.00 60.21 9771.90
> Average: 2 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 1.40
> Average: 3 0.00 0.00 0.00 0.00 0.00 0.20
> 0.00 99.80 1.30
> Average: 4 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 1.30
> Average: 5 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 1.80
> Average: 6 0.00 0.00 0.20 0.00 2.65 78.51
> 0.00 18.64 54691.70
> Average: 7 0.00 0.00 0.00 0.00 1.03 0.51
> 0.00 98.46 54060.60
>
>
> All RX/TX queues from ixgbe on one cpu7 / intel e1000e on cpu1
> Average: CPU %user %nice %sys %iowait %irq %soft
> %steal %idle intr/s
> Average: all 0.07 0.00 0.10 0.00 1.19 9.44
> 0.00 89.19 210155.50
> Average: 0 0.60 0.00 0.00 0.00 0.00 0.00
> 0.00 99.40 0.60
> Average: 1 0.00 0.00 0.41 0.00 2.87 16.27
> 0.00 80.45 13045.30
> Average: 2 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 0.70
> Average: 3 0.10 0.00 0.30 0.00 0.00 0.20
> 0.00 99.40 0.60
> Average: 4 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 0.70
> Average: 5 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 1.10
> Average: 6 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 0.60
> Average: 7 0.00 0.00 0.10 0.00 6.77 60.00
> 0.00 33.13 107242.50
>
>
> All RX/TX queues from ixgbe on one cpu7 / intel e1000e on cpu4
> Average: CPU %user %nice %sys %iowait %irq %soft
> %steal %idle intr/s
> Average: all 0.05 0.00 0.04 0.00 1.24 9.55
> 0.00 89.12 212186.90
> Average: 0 0.40 0.00 0.00 0.00 0.00 0.00
> 0.00 99.60 2.20
> Average: 1 0.10 0.00 0.00 0.00 0.00 0.00
> 0.00 99.90 2.30
> Average: 2 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 2.20
> Average: 3 0.00 0.00 0.00 0.00 0.00 0.30
> 0.00 99.70 2.20
> Average: 4 0.00 0.00 0.00 0.00 1.03 14.68
> 0.00 84.28 11965.50
> Average: 5 0.00 0.00 0.10 0.00 0.00 0.00
> 0.00 99.90 2.80
> Average: 6 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 100.00 2.30
> Average: 7 0.00 0.00 0.20 0.00 8.97 62.69
> 0.00 28.13 110943.50
>
>
> Affinity set to "ffffffff" / and intel e1000e is on cpu4
> Average: CPU %user %nice %sys %iowait %irq %soft
> %steal %idle intr/s
> Average: all 0.10 0.00 0.14 0.01 1.16 15.07
> 0.00 83.52 185043.10
> Average: 0 0.00 0.00 0.00 0.10 2.90 36.90
> 0.00 60.10 9008.60
> Average: 1 0.00 0.00 0.10 0.00 2.30 28.36
> 0.00 69.24 9009.10
> Average: 2 0.80 0.00 0.40 0.00 0.50 13.76
> 0.00 84.54 9008.90
> Average: 3 0.00 0.00 0.20 0.00 0.20 4.61
> 0.00 94.98 9008.90
> Average: 4 0.00 0.00 0.10 0.00 2.35 19.55
> 0.00 77.99 14142.10
> Average: 5 0.00 0.00 0.10 0.00 0.20 3.85
> 0.00 95.84 9009.10
> Average: 6 0.00 0.00 0.10 0.00 0.40 4.22
> 0.00 95.28 9008.90
> Average: 7 0.00 0.00 0.10 0.00 0.51 8.69
> 0.00 90.70 9009.00
>
> And if i split more RX/TX queues to different CPU-s i have more load
> done by:
> CPU: Core 2, speed 2999.98 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a
> unit mask of 0x00 (Unhalted core cycles) count 100000
> samples % image name app name
> symbol name
> 458001 29.5565 vmlinux vmlinux
> native_read_tsc
>
> If all interrupts are on one CPU i have the best results
> So for me the best is to put ixgbe work on one CPU or make affinity to
> balance over all cpus
I am confused by all these numbers. What is your goal exactly ?
If one CPU is able to do the whole workload, it will be faster too.
All other cpus are idle and wont fight with cache lines.
>
> But something is wrong with my intel e1000e
> because if i put intel e1000e with ixgbe on the same CPU system have
> 100% cpu load and all hangs.
Try to reproduce the bug and submit it ?
prev parent reply other threads:[~2009-01-27 15:53 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-01-27 15:14 results with diff affinity settings using ixgbe Paweł Staszewski
2009-01-27 15:53 ` Eric Dumazet [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=497F2DF4.4000507@cosmosbay.com \
--to=dada1@cosmosbay.com \
--cc=netdev@vger.kernel.org \
--cc=pstaszewski@itcare.pl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox