Re: bnx2/BCM5709: why 5 interrupts on a 4 core system (2.6.33.3)

["Krzysztof Olędzki" <ole@ans.pl>]

On 2010-05-16 22:47, Eric Dumazet wrote:
> Le dimanche 16 mai 2010 à 22:34 +0200, Krzysztof Olędzki a écrit :
>> On 2010-05-16 22:15, Eric Dumazet wrote:
>
>>> All tx packets through bonding will use txqueue 0, since bnx2 doesnt
>>> provide a ndo_select_queue() function.
>>
>> OK, that explains everything. Thank you Eric. I assume it may take some
>> time for bonding to become multiqueue aware and/or bnx2x to provide
>> ndo_select_queue?
>>
>
> bonding might become multiqueue aware, there are several patches
> floating around.
>
> But with your ping tests, it wont change the selected txqueue anyway (it
> will be the same for any targets, because skb_tx_hash() wont hash the
> destination address, only the skb->protocol.

What do you mean by "wont hash the destination address, only the 
skb->protocol"? It won't hash the destination address for ICMP or for 
all IP protocols?

My normal workload is TCP and UDP based so if it is only ICMP then there 
is no problem. Actually I have noticeably more UDP traffic than an 
average network, mainly because of LWAPP/CAPWAP, so I'm interested in 
good performance for both TCP and UDP.

During my initial tests ICMP ping showed the same behavior like UDP/TCP 
with iperf, so I sticked with it. I'll redo everyting with UDP and TCP 
of course. :)

>> BTW: With a normal router workload, should I expect big performance drop
>> when receiving and forwarding the same packet using different CPUs?
>> Bonding provides very important functionality, I'm not able to drop it. :(
>>
>
> Not sure what you mean by forwarding same packet using different CPUs.
> You probably meant different queues, because in normal case, only one
> cpu is involved (the one receiving the packet is also the one
> transmitting it, unless you have congestion or trafic shaping)

I mean to receive it on a one CPU and to send it on a different one. I 
would like to assing different vectors (eth1-0 .. eth1-4) to different 
CPUs, but with bnx2x+bonding packets are received on queues 1-4 (eth1-1 
.. eth1-4) and sent from queue 0 (eth1-0). So, for a one packet, two 
different CPUs will be involved (RX on q1-q4, TX on q0).

> If you have 4 cpus, you can use following patch and have a transparent
> bonding against multiqueue.

Thanks! If I get it right: with the patch, packets should be sent using 
the same CPU (queue?) that was used when receiving?

> Still bonding xmit path hits a global
> rwlock, so performance is not what you can get without bonding.

It may not be perfect, but it should be much better than nothing, right?

Best regards,

			Krzysztof Olędzki