From: Stefan Priebe - Profihost AG <s.priebe@profihost.ag>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Linux Netdev List <netdev@vger.kernel.org>
Subject: Re: 10GBE performance drop with net.ipv4.tcp_timestamps=0
Date: Wed, 20 Jun 2012 11:50:52 +0200 [thread overview]
Message-ID: <4FE19CFC.8030408@profihost.ag> (raw)
In-Reply-To: <1340185645.4604.853.camel@edumazet-glaptop>
Am 20.06.2012 11:47, schrieb Eric Dumazet:
> On Wed, 2012-06-20 at 11:33 +0200, Stefan Priebe - Profihost AG wrote:
>
>> Sure. In that case i get 4Gbit/s in both variants. I also tried two
>> other different machines same result.
>>
>
> So 3.5 on receiver is the problem, it seems ?
Yes.
> And you checked all the stuff about irq affinities, i presume, since a
> lot of things might have changed between 2.6.32 and 3.5 ?
It is a single core E5 Xeon - i've set the affinity like this:
eth2 mask=1 for /proc/irq/83/smp_affinity
eth2 mask=2 for /proc/irq/84/smp_affinity
eth2 mask=4 for /proc/irq/85/smp_affinity
eth2 mask=8 for /proc/irq/86/smp_affinity
eth2 mask=10 for /proc/irq/87/smp_affinity
eth2 mask=20 for /proc/irq/88/smp_affinity
eth2 mask=40 for /proc/irq/89/smp_affinity
eth2 mask=80 for /proc/irq/90/smp_affinity
> cat /proc/interrupts
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
CPU6 CPU7
0: 141 0 0 0 0 0
0 0 IO-APIC-edge timer
1: 1 8 0 0 0 0
0 0 IO-APIC-edge i8042
9: 0 0 0 0 0 0
0 0 IO-APIC-fasteoi acpi
12: 0 3 0 0 0 0
0 0 IO-APIC-edge i8042
14: 0 0 0 0 0 0
0 0 IO-APIC-edge ide0
15: 0 0 0 0 0 0
0 0 IO-APIC-edge ide1
16: 0 0 26 0 0 0
0 0 IO-APIC-fasteoi ehci_hcd:usb1
23: 0 0 30 0 0 0
0 0 IO-APIC-fasteoi ehci_hcd:usb2
64: 0 0 0 81979 0 0
0 0 PCI-MSI-edge ahci
65: 0 0 0 1 0 0
0 0 PCI-MSI-edge eth0
66: 0 0 0 0 1090 0
0 0 PCI-MSI-edge eth0-TxRx-0
67: 0 0 0 0 411 0
0 0 PCI-MSI-edge eth0-TxRx-1
68: 0 0 0 0 592 0
0 0 PCI-MSI-edge eth0-TxRx-2
69: 0 0 0 0 472 0
0 0 PCI-MSI-edge eth0-TxRx-3
70: 0 0 0 0 0 1196
0 0 PCI-MSI-edge eth0-TxRx-4
71: 0 0 0 0 0 374
0 0 PCI-MSI-edge eth0-TxRx-5
72: 0 0 0 0 0 405
0 0 PCI-MSI-edge eth0-TxRx-6
73: 0 0 0 0 0 468
0 0 PCI-MSI-edge eth0-TxRx-7
83: 31278 0 0 65 0 0
0 0 PCI-MSI-edge eth2-TxRx-0
84: 0 36311 0 0 61 0
0 0 PCI-MSI-edge eth2-TxRx-1
85: 0 0 46189 0 61 0
0 0 PCI-MSI-edge eth2-TxRx-2
86: 0 0 0 28712 67 0
0 0 PCI-MSI-edge eth2-TxRx-3
87: 0 0 0 0 28089 0
0 0 PCI-MSI-edge eth2-TxRx-4
88: 0 0 0 0 0 34982
0 0 PCI-MSI-edge eth2-TxRx-5
89: 0 0 0 0 0 61
32420 0 PCI-MSI-edge eth2-TxRx-6
90: 0 0 0 0 0 61
0 25922 PCI-MSI-edge eth2-TxRx-7
91: 0 0 0 0 0 3
0 0 PCI-MSI-edge eth2
NMI: 13 12 15 22 5 5
5 5 Non-maskable interrupts
LOC: 58919 61420 65519 82647 35519 40489
27141 30228 Local timer interrupts
SPU: 0 0 0 0 0 0
0 0 Spurious interrupts
PMI: 13 12 15 22 5 5
5 5 Performance monitoring interrupts
IWI: 0 0 0 0 0 0
0 0 IRQ work interrupts
RTR: 6 0 0 0 0 0
0 0 APIC ICR read retries
RES: 15116 4521 2418 1814 2375 1615
1488 1367 Rescheduling interrupts
CAL: 134 148 100 162 170 172
172 172 Function call interrupts
TLB: 422 486 415 483 460 460
476 398 TLB shootdowns
TRM: 0 0 0 0 0 0
0 0 Thermal event interrupts
THR: 0 0 0 0 0 0
0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0
0 0 Machine check exceptions
MCP: 4 4 4 4 4 4
4 4 Machine check polls
> what kind of NIC it is ?
# lspci | grep 10-Giga
06:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
06:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
Stefan
next prev parent reply other threads:[~2012-06-20 9:50 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-19 21:08 10GBE performance drop with net.ipv4.tcp_timestamps=0 Stefan Priebe
2012-06-19 21:31 ` Eric Dumazet
2012-06-20 7:00 ` Stefan Priebe - Profihost AG
2012-06-20 7:22 ` Eric Dumazet
2012-06-20 8:21 ` Stefan Priebe - Profihost AG
2012-06-20 8:41 ` Eric Dumazet
2012-06-20 9:06 ` Stefan Priebe - Profihost AG
2012-06-20 9:12 ` Stefan Priebe - Profihost AG
2012-06-20 9:17 ` Eric Dumazet
2012-06-20 9:25 ` Stefan Priebe - Profihost AG
2012-06-20 9:28 ` Eric Dumazet
2012-06-20 9:33 ` Stefan Priebe - Profihost AG
2012-06-20 9:47 ` Eric Dumazet
2012-06-20 9:50 ` Stefan Priebe - Profihost AG [this message]
2012-06-20 10:06 ` Eric Dumazet
2012-06-20 11:08 ` Eric Dumazet
2012-06-20 9:16 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4FE19CFC.8030408@profihost.ag \
--to=s.priebe@profihost.ag \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.