does the broadcom bnx2x support RSS/multi queue

does the broadcom bnx2x support RSS/multi queue

[Jon Zhou]

hi there

I am not sure if my Broadcom 10G nic driver(bnx2x) support RSS/multi queue

ibm-bc-53:/home/ruizhou/nprobe # uname -a
Linux ibm-bc-53 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64 x86_64 x86_64 GNU/Linux

ibm-bc-53:/home/ruizhou/nprobe # ethtool -S eth5
NIC statistics:
     rx_bytes: 68100170
     rx_error_bytes: 0
     tx_bytes: 0
     tx_error_bytes: 0
     rx_ucast_packets: 201654
     rx_mcast_packets: 0
     rx_bcast_packets: 0
     tx_packets: 0
     tx_mac_errors: 0
     tx_carrier_errors: 0
     rx_crc_errors: 0
     rx_align_errors: 0
     tx_single_collisions: 0
     tx_multi_collisions: 0
     tx_deferred: 0
     tx_excess_collisions: 0
     tx_late_collisions: 0
     tx_total_collisions: 0
     rx_fragments: 0
     rx_jabbers: 0
     rx_undersize_packets: 0
     rx_oversize_packets: 0
     tx_64_byte_packets: 0
     tx_65_to_127_byte_packets: 0
     tx_128_to_255_byte_packets: 0
     tx_256_to_511_byte_packets: 0
     tx_512_to_1023_byte_packets: 0
     tx_1024_to_1522_byte_packets: 0
     tx_1523_to_9022_byte_packets: 0
     rx_xon_frames: 0
     rx_xoff_frames: 0
     tx_xon_frames: 0
     tx_xoff_frames: 0
     rx_mac_ctrl_frames: 0
     rx_filtered_packets: 0
     rx_discards: 0
     rx_fw_discards: 0
     brb_discard: 0
     brb_truncate: 0
     rx_phy_ip_err_discards: 0
     rx_skb_alloc_discard: 0
     rx_csum_offload_errors: 6

the driver ver is:
bnx2x_main.c
#define DRV_MODULE_VERSION      "1.45.26"

looks not support?

thanks
jon

Re: does the broadcom bnx2x support RSS/multi queue

[Eric Dumazet]

Le mercredi 12 mai 2010 à 00:31 -0700, Jon Zhou a écrit :
> hi there
> 
> I am not sure if my Broadcom 10G nic driver(bnx2x) support RSS/multi queue
> 
> ibm-bc-53:/home/ruizhou/nprobe # uname -a
> Linux ibm-bc-53 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64 x86_64 x86_64 GNU/Linux
> 
> ibm-bc-53:/home/ruizhou/nprobe # ethtool -S eth5
> NIC statistics:
>      rx_bytes: 68100170
>      rx_error_bytes: 0
>      tx_bytes: 0
>      tx_error_bytes: 0
>      rx_ucast_packets: 201654
>      rx_mcast_packets: 0
>      rx_bcast_packets: 0
>      tx_packets: 0
>      tx_mac_errors: 0
>      tx_carrier_errors: 0
>      rx_crc_errors: 0
>      rx_align_errors: 0
>      tx_single_collisions: 0
>      tx_multi_collisions: 0
>      tx_deferred: 0
>      tx_excess_collisions: 0
>      tx_late_collisions: 0
>      tx_total_collisions: 0
>      rx_fragments: 0
>      rx_jabbers: 0
>      rx_undersize_packets: 0
>      rx_oversize_packets: 0
>      tx_64_byte_packets: 0
>      tx_65_to_127_byte_packets: 0
>      tx_128_to_255_byte_packets: 0
>      tx_256_to_511_byte_packets: 0
>      tx_512_to_1023_byte_packets: 0
>      tx_1024_to_1522_byte_packets: 0
>      tx_1523_to_9022_byte_packets: 0
>      rx_xon_frames: 0
>      rx_xoff_frames: 0
>      tx_xon_frames: 0
>      tx_xoff_frames: 0
>      rx_mac_ctrl_frames: 0
>      rx_filtered_packets: 0
>      rx_discards: 0
>      rx_fw_discards: 0
>      brb_discard: 0
>      brb_truncate: 0
>      rx_phy_ip_err_discards: 0
>      rx_skb_alloc_discard: 0
>      rx_csum_offload_errors: 6
> 
> the driver ver is:
> bnx2x_main.c
> #define DRV_MODULE_VERSION      "1.45.26"
> 
> looks not support?
> 
> thanks
> jon

Per queue stats were added last year only (Thu Feb 12 08:36:33 2009)

You might check "grep eth5 /proc/interrupts"

Or upgrade to 2.6.33.x kernel :)

Re: does the broadcom bnx2x support RSS/multi queue

[Eilon Greenstein]

On Wed, 2010-05-12 at 00:41 -0700, Eric Dumazet wrote:
> Le mercredi 12 mai 2010 à 00:31 -0700, Jon Zhou a écrit :
> > hi there
> > 
> > I am not sure if my Broadcom 10G nic driver(bnx2x) support RSS/multi queue
> > 
> > ibm-bc-53:/home/ruizhou/nprobe # uname -a
> > Linux ibm-bc-53 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64 x86_64 x86_64 GNU/Linux
> > 
> > ibm-bc-53:/home/ruizhou/nprobe # ethtool -S eth5
> > NIC statistics:
> >      rx_bytes: 68100170
> >      rx_error_bytes: 0
> >      tx_bytes: 0
> >      tx_error_bytes: 0
> >      rx_ucast_packets: 201654
> >      rx_mcast_packets: 0
> >      rx_bcast_packets: 0
> >      tx_packets: 0
> >      tx_mac_errors: 0
> >      tx_carrier_errors: 0
> >      rx_crc_errors: 0
> >      rx_align_errors: 0
> >      tx_single_collisions: 0
> >      tx_multi_collisions: 0
> >      tx_deferred: 0
> >      tx_excess_collisions: 0
> >      tx_late_collisions: 0
> >      tx_total_collisions: 0
> >      rx_fragments: 0
> >      rx_jabbers: 0
> >      rx_undersize_packets: 0
> >      rx_oversize_packets: 0
> >      tx_64_byte_packets: 0
> >      tx_65_to_127_byte_packets: 0
> >      tx_128_to_255_byte_packets: 0
> >      tx_256_to_511_byte_packets: 0
> >      tx_512_to_1023_byte_packets: 0
> >      tx_1024_to_1522_byte_packets: 0
> >      tx_1523_to_9022_byte_packets: 0
> >      rx_xon_frames: 0
> >      rx_xoff_frames: 0
> >      tx_xon_frames: 0
> >      tx_xoff_frames: 0
> >      rx_mac_ctrl_frames: 0
> >      rx_filtered_packets: 0
> >      rx_discards: 0
> >      rx_fw_discards: 0
> >      brb_discard: 0
> >      brb_truncate: 0
> >      rx_phy_ip_err_discards: 0
> >      rx_skb_alloc_discard: 0
> >      rx_csum_offload_errors: 6
> > 
> > the driver ver is:
> > bnx2x_main.c
> > #define DRV_MODULE_VERSION      "1.45.26"
> > 
> > looks not support?
> > 
> > thanks
> > jon
> 
> Per queue stats were added last year only (Thu Feb 12 08:36:33 2009)
> 
> You might check "grep eth5 /proc/interrupts"
> 
> Or upgrade to 2.6.33.x kernel :)
> 
The HW and current driver support multi-queue. However, you are using a version which is too old.

RE: does the broadcom bnx2x support RSS/multi queue

[Jon Zhou]

hi eilon:

do you think I need to update the kernel also?

thanks!
jon

-----Original Message-----
From: Eilon Greenstein [mailto:eilong@broadcom.com] 
Sent: Wednesday, May 12, 2010 5:19 PM
To: Eric Dumazet
Cc: Jon Zhou; netdev@vger.kernel.org
Subject: Re: does the broadcom bnx2x support RSS/multi queue

On Wed, 2010-05-12 at 00:41 -0700, Eric Dumazet wrote:
> Le mercredi 12 mai 2010 à 00:31 -0700, Jon Zhou a écrit :
> > hi there
> > 
> > I am not sure if my Broadcom 10G nic driver(bnx2x) support RSS/multi queue
> > 
> > ibm-bc-53:/home/ruizhou/nprobe # uname -a
> > Linux ibm-bc-53 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64 x86_64 x86_64 GNU/Linux
> > 
> > ibm-bc-53:/home/ruizhou/nprobe # ethtool -S eth5
> > NIC statistics:
> >      rx_bytes: 68100170
> >      rx_error_bytes: 0
> >      tx_bytes: 0
> >      tx_error_bytes: 0
> >      rx_ucast_packets: 201654
> >      rx_mcast_packets: 0
> >      rx_bcast_packets: 0
> >      tx_packets: 0
> >      tx_mac_errors: 0
> >      tx_carrier_errors: 0
> >      rx_crc_errors: 0
> >      rx_align_errors: 0
> >      tx_single_collisions: 0
> >      tx_multi_collisions: 0
> >      tx_deferred: 0
> >      tx_excess_collisions: 0
> >      tx_late_collisions: 0
> >      tx_total_collisions: 0
> >      rx_fragments: 0
> >      rx_jabbers: 0
> >      rx_undersize_packets: 0
> >      rx_oversize_packets: 0
> >      tx_64_byte_packets: 0
> >      tx_65_to_127_byte_packets: 0
> >      tx_128_to_255_byte_packets: 0
> >      tx_256_to_511_byte_packets: 0
> >      tx_512_to_1023_byte_packets: 0
> >      tx_1024_to_1522_byte_packets: 0
> >      tx_1523_to_9022_byte_packets: 0
> >      rx_xon_frames: 0
> >      rx_xoff_frames: 0
> >      tx_xon_frames: 0
> >      tx_xoff_frames: 0
> >      rx_mac_ctrl_frames: 0
> >      rx_filtered_packets: 0
> >      rx_discards: 0
> >      rx_fw_discards: 0
> >      brb_discard: 0
> >      brb_truncate: 0
> >      rx_phy_ip_err_discards: 0
> >      rx_skb_alloc_discard: 0
> >      rx_csum_offload_errors: 6
> > 
> > the driver ver is:
> > bnx2x_main.c
> > #define DRV_MODULE_VERSION      "1.45.26"
> > 
> > looks not support?
> > 
> > thanks
> > jon
> 
> Per queue stats were added last year only (Thu Feb 12 08:36:33 2009)
> 
> You might check "grep eth5 /proc/interrupts"
> 
> Or upgrade to 2.6.33.x kernel :)
> 
The HW and current driver support multi-queue. However, you are using a version which is too old.

RE: does the broadcom bnx2x support RSS/multi queue

[Eilon Greenstein]

On Wed, 2010-05-12 at 02:34 -0700, Jon Zhou wrote:
> hi eilon:
> 
> do you think I need to update the kernel also?

Kernel 2.6.27 supports Tx multi queue (Rx multi-queue was added even
before that), so theoretically you can update only the bnx2x. However,
you cannot use the bnx2x from the current kernel due to other changes
between 2.6.27 and the current kernel. So you need to download the bnx2x
from the Broadcom site.

Enjoy,
Eilon

RE: does the broadcom bnx2x support RSS/multi queue

[Eric Dumazet]

Le mercredi 12 mai 2010 à 02:34 -0700, Jon Zhou a écrit :
> hi eilon:
> 
> do you think I need to update the kernel also?
> 
> thanks!
> jon

I believe both of us (Eilong and me) stated your kernel version was too
old.

In order to play with multiqueue, you should use a very recent kernel,
or hit various bottlenecks and bugs.

RE: does the broadcom bnx2x support RSS/multi queue

[Jon Zhou]

hi eric:

"very recent kernel" -- do you you mean "receive packet steering"?

thanks
jon




-----Original Message-----
From: Eric Dumazet [mailto:eric.dumazet@gmail.com] 
Sent: Wednesday, May 12, 2010 6:00 PM
To: Jon Zhou
Cc: eilong@broadcom.com; netdev@vger.kernel.org
Subject: RE: does the broadcom bnx2x support RSS/multi queue

Le mercredi 12 mai 2010 à 02:34 -0700, Jon Zhou a écrit :
> hi eilon:
> 
> do you think I need to update the kernel also?
> 
> thanks!
> jon

I believe both of us (Eilong and me) stated your kernel version was too
old.

In order to play with multiqueue, you should use a very recent kernel,
or hit various bottlenecks and bugs.

RE: does the broadcom bnx2x support RSS/multi queue

[Eric Dumazet]

Le lundi 17 mai 2010 à 01:05 -0700, Jon Zhou a écrit :
> hi eric:
> 
> "very recent kernel" -- do you you mean "receive packet steering"?
> 

Not necessarly. RPS is the "poor man solution", when multique NICS are
not available, or a refinement over it on special cases.

I would say 2.6.33.4 or 2.6.34 would be ok.

RE: does the broadcom bnx2x support RSS/multi queue

[Jon Zhou]

hi:

I just tried to insall 1.52.12 driver:

insmod ./PF_RING/drivers/broadcom/netxtreme2-5.2.50/bnx2x-1.52.12/src/bnx2x.ko multi_mode=1 num_queues=4 int_mode=3 debug=1
ifconfig eth5 up

but seems MSI not enabled:
34:         26          0          0          0          0          0          0          0          0          0          0          0   49470611   63427140          0          0   IO-APIC-fasteoi   eth5
4314:          2          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0      none-<NULL>
4316:          2          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0      none-<NULL>
4318:          2          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0      none-<NULL>
4321:       6510        728       1786          0         14          0         57        380          0          0          0          0        100        139          0          0   PCI-MSI-edge      eth1
4322:       1102          0       2556        788          0          5          0       1338          0          0          0          0        174         45        411          0   PCI-MSI-edge      eth1
4323:         21          0          0          0          0          0          0          0          0          0          0          0        784       1103         56         87   PCI-MSI-edge      eth1
43

could give some tips?
thanks!

May 13 03:37:59 ibm-bc-53 kernel: Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.12 ($DateTime: 2009/12/17 12:14:50 $)
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:15:00.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:15:00.0: setting latency timer to 64
May 13 03:37:59 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-473033
May 13 03:37:59 ibm-bc-53 kernel: eth2: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem a0000000, IRQ 24, node addr ffff8801db810210
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:15:00.1: PCI INT B -> GSI 34 (level, low) -> IRQ 34
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:15:00.1: setting latency timer to 64
May 13 03:37:59 ibm-bc-53 kernel: eth2 renamed to eth3 by udevd [24569]
May 13 03:37:59 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-473033
May 13 03:37:59 ibm-bc-53 kernel: udev: renamed network interface eth2 to eth3
May 13 03:37:59 ibm-bc-53 ifup:     eth3      device: Broadcom Corporation NetXtreme II BCM57711 10Gigabit PCIe
May 13 03:37:59 ibm-bc-53 SuSEfirewall2: SuSEfirewall2 not active
May 13 03:37:59 ibm-bc-53 kernel: eth2: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem a0800000, IRQ 34, node addr ffff880200828210
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:1a:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:1a:00.0: setting latency timer to 64
May 13 03:37:59 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-473033
May 13 03:37:59 ibm-bc-53 kernel: eth2 renamed to eth5 by udevd [24642]
May 13 03:37:59 ibm-bc-53 kernel: udev: renamed network interface eth2 to eth5
May 13 03:37:59 ibm-bc-53 ifup:     eth5      device: Broadcom Corporation NetXtreme II BCM57711 10Gigabit PCIe
May 13 03:37:59 ibm-bc-53 ifup:     eth5      Startmode is 'off'
May 13 03:37:59 ibm-bc-53 kernel: eth2: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem 9a000000, IRQ 26, node addr ffff8802030f8210
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:1a:00.1: PCI INT B -> GSI 25 (level, low) -> IRQ 25
May 13 03:37:59 ibm-bc-53 kernel: bnx2x 0000:1a:00.1: setting latency timer to 64
May 13 03:37:59 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-473033
May 13 03:37:59 ibm-bc-53 ifup:     eth2      device: Broadcom Corporation NetXtreme II BCM57711 10Gigabit PCIe
May 13 03:37:59 ibm-bc-53 SuSEfirewall2: SuSEfirewall2 not active
May 13 03:38:00 ibm-bc-53 kernel: eth4: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem 9a800000, IRQ 25, node addr ffff88028d5a8210
May 13 03:38:00 ibm-bc-53 ifup:     eth4      device: Broadcom Corporation NetXtreme II BCM57711 10Gigabit PCIe
May 13 03:38:00 ibm-bc-53 SuSEfirewall2: SuSEfirewall2 not active
May 13 03:38:12 ibm-bc-53 kernel: bnx2x: eth5 NIC Link is Up, 10000 Mbps full duplex, receive & transmit flow control ON

I think there should be "using MSI-X" or something like that


-----Original Message-----
From: Eilon Greenstein [mailto:eilong@broadcom.com] 
Sent: Wednesday, May 12, 2010 5:19 PM
To: Eric Dumazet
Cc: Jon Zhou; netdev@vger.kernel.org
Subject: Re: does the broadcom bnx2x support RSS/multi queue

On Wed, 2010-05-12 at 00:41 -0700, Eric Dumazet wrote:
> Le mercredi 12 mai 2010 à 00:31 -0700, Jon Zhou a écrit :
> > hi there
> > 
> > I am not sure if my Broadcom 10G nic driver(bnx2x) support RSS/multi queue
> > 
> > ibm-bc-53:/home/ruizhou/nprobe # uname -a
> > Linux ibm-bc-53 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64 x86_64 x86_64 GNU/Linux
> > 
> > ibm-bc-53:/home/ruizhou/nprobe # ethtool -S eth5
> > NIC statistics:
> >      rx_bytes: 68100170
> >      rx_error_bytes: 0
> >      tx_bytes: 0
> >      tx_error_bytes: 0
> >      rx_ucast_packets: 201654
> >      rx_mcast_packets: 0
> >      rx_bcast_packets: 0
> >      tx_packets: 0
> >      tx_mac_errors: 0
> >      tx_carrier_errors: 0
> >      rx_crc_errors: 0
> >      rx_align_errors: 0
> >      tx_single_collisions: 0
> >      tx_multi_collisions: 0
> >      tx_deferred: 0
> >      tx_excess_collisions: 0
> >      tx_late_collisions: 0
> >      tx_total_collisions: 0
> >      rx_fragments: 0
> >      rx_jabbers: 0
> >      rx_undersize_packets: 0
> >      rx_oversize_packets: 0
> >      tx_64_byte_packets: 0
> >      tx_65_to_127_byte_packets: 0
> >      tx_128_to_255_byte_packets: 0
> >      tx_256_to_511_byte_packets: 0
> >      tx_512_to_1023_byte_packets: 0
> >      tx_1024_to_1522_byte_packets: 0
> >      tx_1523_to_9022_byte_packets: 0
> >      rx_xon_frames: 0
> >      rx_xoff_frames: 0
> >      tx_xon_frames: 0
> >      tx_xoff_frames: 0
> >      rx_mac_ctrl_frames: 0
> >      rx_filtered_packets: 0
> >      rx_discards: 0
> >      rx_fw_discards: 0
> >      brb_discard: 0
> >      brb_truncate: 0
> >      rx_phy_ip_err_discards: 0
> >      rx_skb_alloc_discard: 0
> >      rx_csum_offload_errors: 6
> > 
> > the driver ver is:
> > bnx2x_main.c
> > #define DRV_MODULE_VERSION      "1.45.26"
> > 
> > looks not support?
> > 
> > thanks
> > jon
> 
> Per queue stats were added last year only (Thu Feb 12 08:36:33 2009)
> 
> You might check "grep eth5 /proc/interrupts"
> 
> Or upgrade to 2.6.33.x kernel :)
> 
The HW and current driver support multi-queue. However, you are using a version which is too old.

RE: does the broadcom bnx2x support RSS/multi queue

[Eilon Greenstein]

On Thu, 2010-05-13 at 02:44 -0700, Jon Zhou wrote:
> insmod ./PF_RING/drivers/broadcom/netxtreme2-5.2.50/bnx2x-1.52.12/src/bnx2x.ko multi_mode=1 num_queues=4 int_mode=3 debug=1
There is no need to set multi_mode or int_mode - you are using the
default values. However, I need more information on why you are using
INTA and not MSI-X, so please set the debug to 0x20

> but seems MSI not enabled:
> 34:         26          0          0          0          0          0          0          0          0          0          0          0   49470611   63427140          0          0   IO-APIC-fasteoi   eth5
Indeed, you are using INT# - this is why you do not have multi-queue.

Eilon

RE: does the broadcom bnx2x support RSS/multi queue

[Jon Zhou]

hi eilon:

I got these log when applied debug=0x20,why " MSI is not attainable"?
thanks


May 13 18:53:26 ibm-bc-53 kernel: Broadcom NetXtreme II 5771x 10Gigabit Ethernet
 Driver bnx2x 1.52.12 ($DateTime: 2009/12/17 12:14:50 $)
May 13 18:53:26 ibm-bc-53 kernel: bnx2x 0000:15:00.0: PCI INT A -> GSI 24 (level
, low) -> IRQ 24
May 13 18:53:26 ibm-bc-53 kernel: bnx2x 0000:15:00.0: setting latency timer to 6
4
May 13 18:53:26 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-
473033
May 13 18:53:26 ibm-bc-53 kernel: eth2: Broadcom NetXtreme II BCM57711 XGb (A0)
PCI-E x8 5GHz (Gen2) found at mem a0000000, IRQ 24, node addr ffff8801db9b0210
May 13 18:53:26 ibm-bc-53 kernel: bnx2x 0000:15:00.1: PCI INT B -> GSI 34 (level
, low) -> IRQ 34
May 13 18:53:26 ibm-bc-53 kernel: bnx2x 0000:15:00.1: setting latency timer to 6
4
May 13 18:53:26 ibm-bc-53 kernel: eth2 renamed to eth3 by udevd [30749]
May 13 18:53:26 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-
473033
May 13 18:53:26 ibm-bc-53 kernel: udev: renamed network interface eth2 to eth3
May 13 18:53:26 ibm-bc-53 ifup:     eth3      device: Broadcom Corporation NetXt
reme II BCM57711 10Gigabit PCIe
May 13 18:53:26 ibm-bc-53 SuSEfirewall2: SuSEfirewall2 not active
May 13 18:53:26 ibm-bc-53 kernel: eth2: Broadcom NetXtreme II BCM57711 XGb (A0)
PCI-E x8 5GHz (Gen2) found at mem a0800000, IRQ 34, node addr ffff8801db9a8210
May 13 18:53:26 ibm-bc-53 kernel: bnx2x 0000:1a:00.0: PCI INT A -> GSI 26 (level
, low) -> IRQ 26
May 13 18:53:26 ibm-bc-53 kernel: bnx2x 0000:1a:00.0: setting latency timer to 6
4
May 13 18:53:26 ibm-bc-53 kernel: eth2 renamed to eth5 by udevd [30822]
May 13 18:53:26 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-
473033
May 13 18:53:26 ibm-bc-53 kernel: udev: renamed network interface eth2 to eth5
May 13 18:53:27 ibm-bc-53 ifup:     eth5      device: Broadcom Corporation NetXt
reme II BCM57711 10Gigabit PCIe
May 13 18:53:27 ibm-bc-53 ifup:     eth5      Startmode is 'off'
May 13 18:53:27 ibm-bc-53 kernel: eth2: Broadcom NetXtreme II BCM57711 XGb (A0)
PCI-E x8 5GHz (Gen2) found at mem 9a000000, IRQ 26, node addr ffff880203c18210
May 13 18:53:27 ibm-bc-53 kernel: bnx2x 0000:1a:00.1: PCI INT B -> GSI 25 (level
, low) -> IRQ 25
May 13 18:53:27 ibm-bc-53 kernel: bnx2x 0000:1a:00.1: setting latency timer to 6
4
May 13 18:53:27 ibm-bc-53 kernel: bnx2x: part number 394D4342-31373735-31314131-
473033
May 13 18:53:27 ibm-bc-53 ifup:     eth2      device: Broadcom Corporation NetXt
reme II BCM57711 10Gigabit PCIe
May 13 18:53:27 ibm-bc-53 SuSEfirewall2: SuSEfirewall2 not active


May 13 18:53:27 ibm-bc-53 kernel: eth4: Broadcom NetXtreme II BCM57711 XGb (A0)
PCI-E x8 5GHz (Gen2) found at mem 9a800000, IRQ 25, node addr ffff8801f7940210
May 13 18:53:27 ibm-bc-53 ifup:     eth4      device: Broadcom Corporation NetXt
reme II BCM57711 10Gigabit PCIe
May 13 18:53:27 ibm-bc-53 SuSEfirewall2: SuSEfirewall2 not active
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_set_num_queues:8134(eth5)]set number of queues to 4
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7625(eth5)]msix_table[0].entry = 0 (slowpath)
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7630(eth5)]msix_table[1].entry = 1 (CNIC)
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7637(eth5)]msix_table[2].entry = 2 (fastpath #0)
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7637(eth5)]msix_table[3].entry = 3 (fastpath #1)
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7637(eth5)]msix_table[4].entry = 4 (fastpath #2)
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7637(eth5)]msix_table[5].entry = 5 (fastpath #3)
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msix:7667(eth5)]MSI-X is not attainable  rc -28
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_enable_msi:7726(eth5)]MSI is not attainable
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_nic_init:6148(eth5)]queue[0]:  bnx2x_init_sb(ffff8801db9a8780,ffff8801d70fc000)  cl_id 0  sb 1  cos 0
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_init_rx_rings:5386(eth5)]mtu 1500  rx_buf_size 1650
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_set_storm_rx_mode:5753(eth5)]rx mode 0  mask 0x1
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_init_internal_func:6061(eth5)]All MIN values are zeroes  fairness will be disabled
May 13 18:54:05 ibm-bc-53 kernel: [bnx2x_init_ind_table:5688(eth5)]Initializing indirection table  multi_mode 1
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_wait_ramrod:7941(eth5)]waiting for state to become 3000 on IDX [0]
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_sp_event:1191(eth5)]got setup ramrod
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_mac_addr_e1h_gen:7927(eth5)]setting MAC (001a:6476:0367)  E1HOV 0  CLID mask 1
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_wait_ramrod:7941(eth5)]waiting for state to become 0 on IDX [0]
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_sp_event:1215(eth5)]got set mac ramrod
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_rx_mode:12833(eth5)]dev->flags = 1002
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_storm_rx_mode:5753(eth5)]rx mode 1  mask 0x1
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_rx_mode:12833(eth5)]dev->flags = 1003
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_storm_rx_mode:5753(eth5)]rx mode 1  mask 0x1
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_rx_mode:12833(eth5)]dev->flags = 1003
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_rx_mode:12927(eth5)]Adding mcast MAC: ffff8803f58a4c08
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_storm_rx_mode:5753(eth5)]rx mode 1  mask 0x1
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_rx_mode:12833(eth5)]dev->flags = 1003
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_rx_mode:12927(eth5)]Adding mcast MAC: ffff8803f58a4c08
May 13 18:54:06 ibm-bc-53 kernel: [bnx2x_set_storm_rx_mode:5753(eth5)]rx mode 1  mask 0x1
May 13 18:54:06 ibm-bc-53 kernel: bnx2x: eth5 NIC Link is Up, 10000 Mbps full duplex, receive & transmit flow control ON




-----Original Message-----
From: Eilon Greenstein [mailto:eilong@broadcom.com] 
Sent: Thursday, May 13, 2010 7:32 PM
To: Jon Zhou
Cc: Eric Dumazet; netdev@vger.kernel.org
Subject: RE: does the broadcom bnx2x support RSS/multi queue

On Thu, 2010-05-13 at 02:44 -0700, Jon Zhou wrote:
> insmod ./PF_RING/drivers/broadcom/netxtreme2-5.2.50/bnx2x-1.52.12/src/bnx2x.ko multi_mode=1 num_queues=4 int_mode=3 debug=1
There is no need to set multi_mode or int_mode - you are using the
default values. However, I need more information on why you are using
INTA and not MSI-X, so please set the debug to 0x20

> but seems MSI not enabled:
> 34:         26          0          0          0          0          0          0          0          0          0          0          0   49470611   63427140          0          0   IO-APIC-fasteoi   eth5
Indeed, you are using INT# - this is why you do not have multi-queue.

Eilon