From: Stefan Priebe - Profihost AG <s.priebe@profihost.ag>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Linux Netdev List <netdev@vger.kernel.org>
Subject: Re: 10GBE performance drop with net.ipv4.tcp_timestamps=0
Date: Wed, 20 Jun 2012 11:50:52 +0200 [thread overview]
Message-ID: <4FE19CFC.8030408@profihost.ag> (raw)
In-Reply-To: <1340185645.4604.853.camel@edumazet-glaptop>
Am 20.06.2012 11:47, schrieb Eric Dumazet:
> On Wed, 2012-06-20 at 11:33 +0200, Stefan Priebe - Profihost AG wrote:
>
>> Sure. In that case i get 4Gbit/s in both variants. I also tried two
>> other different machines same result.
>>
>
> So 3.5 on receiver is the problem, it seems ?
Yes.
> And you checked all the stuff about irq affinities, i presume, since a
> lot of things might have changed between 2.6.32 and 3.5 ?
It is a single core E5 Xeon - i've set the affinity like this:
eth2 mask=1 for /proc/irq/83/smp_affinity
eth2 mask=2 for /proc/irq/84/smp_affinity
eth2 mask=4 for /proc/irq/85/smp_affinity
eth2 mask=8 for /proc/irq/86/smp_affinity
eth2 mask=10 for /proc/irq/87/smp_affinity
eth2 mask=20 for /proc/irq/88/smp_affinity
eth2 mask=40 for /proc/irq/89/smp_affinity
eth2 mask=80 for /proc/irq/90/smp_affinity
> cat /proc/interrupts
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
CPU6 CPU7
0: 141 0 0 0 0 0
0 0 IO-APIC-edge timer
1: 1 8 0 0 0 0
0 0 IO-APIC-edge i8042
9: 0 0 0 0 0 0
0 0 IO-APIC-fasteoi acpi
12: 0 3 0 0 0 0
0 0 IO-APIC-edge i8042
14: 0 0 0 0 0 0
0 0 IO-APIC-edge ide0
15: 0 0 0 0 0 0
0 0 IO-APIC-edge ide1
16: 0 0 26 0 0 0
0 0 IO-APIC-fasteoi ehci_hcd:usb1
23: 0 0 30 0 0 0
0 0 IO-APIC-fasteoi ehci_hcd:usb2
64: 0 0 0 81979 0 0
0 0 PCI-MSI-edge ahci
65: 0 0 0 1 0 0
0 0 PCI-MSI-edge eth0
66: 0 0 0 0 1090 0
0 0 PCI-MSI-edge eth0-TxRx-0
67: 0 0 0 0 411 0
0 0 PCI-MSI-edge eth0-TxRx-1
68: 0 0 0 0 592 0
0 0 PCI-MSI-edge eth0-TxRx-2
69: 0 0 0 0 472 0
0 0 PCI-MSI-edge eth0-TxRx-3
70: 0 0 0 0 0 1196
0 0 PCI-MSI-edge eth0-TxRx-4
71: 0 0 0 0 0 374
0 0 PCI-MSI-edge eth0-TxRx-5
72: 0 0 0 0 0 405
0 0 PCI-MSI-edge eth0-TxRx-6
73: 0 0 0 0 0 468
0 0 PCI-MSI-edge eth0-TxRx-7
83: 31278 0 0 65 0 0
0 0 PCI-MSI-edge eth2-TxRx-0
84: 0 36311 0 0 61 0
0 0 PCI-MSI-edge eth2-TxRx-1
85: 0 0 46189 0 61 0
0 0 PCI-MSI-edge eth2-TxRx-2
86: 0 0 0 28712 67 0
0 0 PCI-MSI-edge eth2-TxRx-3
87: 0 0 0 0 28089 0
0 0 PCI-MSI-edge eth2-TxRx-4
88: 0 0 0 0 0 34982
0 0 PCI-MSI-edge eth2-TxRx-5
89: 0 0 0 0 0 61
32420 0 PCI-MSI-edge eth2-TxRx-6
90: 0 0 0 0 0 61
0 25922 PCI-MSI-edge eth2-TxRx-7
91: 0 0 0 0 0 3
0 0 PCI-MSI-edge eth2
NMI: 13 12 15 22 5 5
5 5 Non-maskable interrupts
LOC: 58919 61420 65519 82647 35519 40489
27141 30228 Local timer interrupts
SPU: 0 0 0 0 0 0
0 0 Spurious interrupts
PMI: 13 12 15 22 5 5
5 5 Performance monitoring interrupts
IWI: 0 0 0 0 0 0
0 0 IRQ work interrupts
RTR: 6 0 0 0 0 0
0 0 APIC ICR read retries
RES: 15116 4521 2418 1814 2375 1615
1488 1367 Rescheduling interrupts
CAL: 134 148 100 162 170 172
172 172 Function call interrupts
TLB: 422 486 415 483 460 460
476 398 TLB shootdowns
TRM: 0 0 0 0 0 0
0 0 Thermal event interrupts
THR: 0 0 0 0 0 0
0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0
0 0 Machine check exceptions
MCP: 4 4 4 4 4 4
4 4 Machine check polls
> what kind of NIC it is ?
# lspci | grep 10-Giga
06:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
06:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
Stefan
next prev parent reply other threads:[~2012-06-20 9:50 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-19 21:08 10GBE performance drop with net.ipv4.tcp_timestamps=0 Stefan Priebe
2012-06-19 21:31 ` Eric Dumazet
2012-06-20 7:00 ` Stefan Priebe - Profihost AG
2012-06-20 7:22 ` Eric Dumazet
2012-06-20 8:21 ` Stefan Priebe - Profihost AG
2012-06-20 8:41 ` Eric Dumazet
2012-06-20 9:06 ` Stefan Priebe - Profihost AG
2012-06-20 9:12 ` Stefan Priebe - Profihost AG
2012-06-20 9:17 ` Eric Dumazet
2012-06-20 9:25 ` Stefan Priebe - Profihost AG
2012-06-20 9:28 ` Eric Dumazet
2012-06-20 9:33 ` Stefan Priebe - Profihost AG
2012-06-20 9:47 ` Eric Dumazet
2012-06-20 9:50 ` Stefan Priebe - Profihost AG [this message]
2012-06-20 10:06 ` Eric Dumazet
2012-06-20 11:08 ` Eric Dumazet
2012-06-20 9:16 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4FE19CFC.8030408@profihost.ag \
--to=s.priebe@profihost.ag \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).