RE: >10% performance degradation since 2.6.18

Ma, Chinang
2009-07-06T17:57:20Z

>-----Original Message-----
>From: Matthew Wilcox [mailto:matthew@wil.cx]
>Sent: Monday, July 06, 2009 10:42 AM
>To: Ma, Chinang
>Cc: Rick Jones; Herbert Xu; Jeff Garzik; andi@firstfloor.org;
>arjan@infradead.org; jens.axboe@oracle.com; linux-kernel@vger.kernel.org;
>Styner, Douglas W; Prickett, Terry O; Wilcox, Matthew R;
>netdev@vger.kernel.org; Brandeburg, Jesse
>Subject: Re: >10% performance degradation since 2.6.18
>
>On Mon, Jul 06, 2009 at 10:36:11AM -0700, Ma, Chinang wrote:
>> For OLTP workload we are not pushing much network throughput. Lower
>network latency is more important for OLTP performance. For the original
>Nehalem 2 sockets OLTP result in this mail thread, we bound the two NIC
>interrupts to cpu1 and cpu9 (one NIC per sockets). Database processes are
>divided into two groups and pinned to socket and each processe only
>received request from the NIC it bound to. This binding scheme gave us >1%
>performance boost pre-Nehalem date. We also see positive impact on this NHM
>system.
>
>So you've tried spreading the four RX and TX interrupts for each card
>out over, say, CPUs 1, 3, 5, 7 for eth1 and then 9, 11, 13, 15 for eth0,
>and it produces worse performance than having all four tied to CPUs 1
>and 9? Interesting.

I was comparing 2 NIC on 2 sockets to 2 NIC on the same socket. I have not tried spreading out the interrupt for a NIC to cpus in the same sockets. Is there good reason for trying this?

>
>Can you try changing IGB_MAX_RX_QUEUES (in drivers/net/igb/igb.h, about
>line 60) to 1, and seeing if performance improves that way?
>

I suppose this should wait until we find out whether spread out NIC interrupt in socket helps or not.

>--
>Matthew Wilcox Intel Open Source Technology Centre
>"Bill, look, we understand that you're interested in selling us this
>operating system, but compare it to ours. We can't possibly take such
>a retrograde step."