bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Krzysztof Olędzki]

Hello,

I would like to connect a dual port SFP+ NetXtreme II BCM57711
10-Gigabit NIC to a HP J9309A ProCurve 4-Port 10GbE SFP+ zl Module
using a HP ProCurve 10-GbE SFP+ 7m Direct Attach Cable (J9285B).

Unfortuantely, it does not work. :( After connecting the switch and
the server together and loading the bnx2x module the switch logs:

I 05/20/10 18:32:23 ports: port E4 is Blocked by STP
I 05/20/10 18:32:23 ports: port E4 is now on-line

Here is the dmesg output from the server:

Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.1-5 (2009/11/09)
bnx2x 0000:04:00.0: PCI INT A -> GSI 38 (level, low) -> IRQ 38
bnx2x 0000:04:00.0: setting latency timer to 64
bnx2x: part number 394D4342-31373735-31314131-473331
bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
bnx2x 0000:04:00.0: firmware: requesting bnx2x-e1h-5.2.7.0.fw
eth4: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dc000000, IRQ 38, node addr 00:10:18:5f:e4:b4
bnx2x 0000:04:00.1: PCI INT B -> GSI 45 (level, low) -> IRQ 45
bnx2x 0000:04:00.1: setting latency timer to 64
bnx2x: part number 394D4342-31373735-31314131-473331
bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
bnx2x 0000:04:00.1: firmware: requesting bnx2x-e1h-5.2.7.0.fw
eth5: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dd000000, IRQ 45, node addr 00:10:18:5f:e4:b6
bnx2x 0000:04:00.0: irq 97 for MSI/MSI-X
bnx2x 0000:04:00.0: irq 98 for MSI/MSI-X
bnx2x 0000:04:00.0: irq 99 for MSI/MSI-X
bnx2x 0000:04:00.0: irq 100 for MSI/MSI-X
bnx2x 0000:04:00.0: irq 101 for MSI/MSI-X
bnx2x 0000:04:00.0: irq 102 for MSI/MSI-X
bnx2x: eth4: using MSI-X  IRQs: sp 97  fp[0] 99 ... fp[3] 102
ADDRCONF(NETDEV_UP): eth4: link is not ready
bnx2x 0000:04:00.1: irq 103 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 104 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 105 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 106 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 107 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 108 for MSI/MSI-X
bnx2x: eth5: using MSI-X  IRQs: sp 103  fp[0] 105 ... fp[3] 108
ADDRCONF(NETDEV_UP): eth5: link is not ready
bnx2x: eth5 NIC Link is Down
bnx2x: eth5 NIC Link is Down

Loading the driver with debug mode enabled (modprobe bnx2x debug=0x20004) I got:

bnx2x 0000:04:00.1: irq 103 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 104 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 105 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 106 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 107 for MSI/MSI-X
bnx2x 0000:04:00.1: irq 108 for MSI/MSI-X
bnx2x: eth5: using MSI-X  IRQs: sp 103  fp[0] 105 ... fp[3] 108
[bnx2x_nic_load:7524(eth5)]pmf 1
[bnx2x_link_reset:6171(eth5)]Resetting the link of port 1
[bnx2x_set_led:5804(eth5)]bnx2x_set_led: port 1, mode 0
[bnx2x_set_led:5806(eth5)]speed 0x0, hw_led_mode 0x1
[bnx2x_stats_init:4597(eth5)]port_stx 0xa72f4  func_stx 0x0
[bnx2x_phy_init:5966(eth5)]Phy Initialization started
[bnx2x_phy_init:5968(eth5)]req_speed 10000, req_flowctrl 0
[bnx2x_emac_init:255(eth5)]EMAC reset reg is 1024
[bnx2x_phy_deassert:534(eth5)]bnx2x_phy_deassert:XGXS
[bnx2x_phy_init:6140(eth5)]Phy address = 0x1
[bnx2x_ext_phy_reset:2092(eth5)]Port 1: bnx2x_ext_phy_reset
[bnx2x_calc_ieee_aneg_adv:1383(eth5)]ieee_fc = 0x1a0
[bnx2x_ext_phy_init:3672(eth5)]control reg 0x2040 (after 0 ms)
[bnx2x_ext_phy_init:4136(eth5)]Initializing BCM8727
[bnx2x_ext_phy_init:4285(eth5)]Setting TX_CTRL1 0xc000,TX_CTRL2 0x40
[bnx2x_link_int_enable:5338(eth5)]enabled XGXS interrupt
[bnx2x_link_int_enable:5345(eth5)]enabled external phy int
[bnx2x_link_int_enable:5366(eth5)]port 1, is_xgxs 1, int_status 0x0
[bnx2x_link_int_enable:5370(eth5)] int_mask 0x3c8001, MI_INT 1, SERDES_LINK 0
[bnx2x_link_int_enable:5373(eth5)] 10G 0, XGXS_LINK 0
[bnx2x_stats_handle:4469(eth5)]state 0 -> event 3 -> state 0
[bnx2x_link_update:6375(eth5)]port 1, XGXS?1, int_status 0x0
[bnx2x_link_update:6383(eth5)]int_mask 0x0 MI_INT 1, SERDES_LINK 0
[bnx2x_link_update:6387(eth5)] 10G 0, XGXS_LINK 0
[bnx2x_ext_phy_is_link_up:4809(eth5)]8727 RX_ALARM_STATUS 0x439
ADDRCONF(NETDEV_UP): eth5: link is not ready
[bnx2x_ext_phy_is_link_up:4819(eth5)]8727 LASI status 0x6
[bnx2x_8727_handle_mod_abs:4622(eth5)]MOD_ABS indication show module is present
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_handle_mod_abs:4660(eth5)]SFP+ module is not initialized
[bnx2x_8727_handle_mod_abs:4664(eth5)]8727 RX_ALARM_STATUS 0x438
[bnx2x_ext_phy_is_link_up:4916(eth5)]Tx is disabled
[bnx2x_link_settings_status:1958(eth5)]phy link down
[bnx2x_link_settings_status:1976(eth5)]gp_status 0x208  phy_link_up 0 line_speed 2710
[bnx2x_link_settings_status:1980(eth5)]duplex 1  flow_ctrl 0x400 autoneg 0x0
[bnx2x_link_settings_status:1981(eth5)]link_status 0x0
[bnx2x_update_link_down:6285(eth5)]Port 1: Link is down
[bnx2x_set_led:5804(eth5)]bnx2x_set_led: port 1, mode 0
[bnx2x_set_led:5806(eth5)]speed 0x0, hw_led_mode 0x1
bnx2x: eth5 NIC Link is Down
[bnx2x_stats_handle:4469(eth5)]state 0 -> event 3 -> state 0
[bnx2x_link_update:6375(eth5)]port 1, XGXS?1, int_status 0x0
[bnx2x_link_update:6383(eth5)]int_mask 0x0 MI_INT 1, SERDES_LINK 0
[bnx2x_link_update:6387(eth5)] 10G 0, XGXS_LINK 0
[bnx2x_ext_phy_is_link_up:4809(eth5)]8727 RX_ALARM_STATUS 0x41c
[bnx2x_ext_phy_is_link_up:4819(eth5)]8727 LASI status 0x2
[bnx2x_ext_phy_is_link_up:4916(eth5)]Tx is disabled
[bnx2x_link_settings_status:1958(eth5)]phy link down
[bnx2x_link_settings_status:1976(eth5)]gp_status 0x208  phy_link_up 0 line_speed 0
[bnx2x_link_settings_status:1980(eth5)]duplex 1  flow_ctrl 0x400 autoneg 0x0
[bnx2x_link_settings_status:1981(eth5)]link_status 0x0
[bnx2x_update_link_down:6285(eth5)]Port 1: Link is down
[bnx2x_set_led:5804(eth5)]bnx2x_set_led: port 1, mode 0
[bnx2x_set_led:5806(eth5)]speed 0x0, hw_led_mode 0x1
bnx2x: eth5 NIC Link is Down

# uname -r
2.6.33.3

Is there something I can do to fix this? Like for example kernel/driver or the firmware upgrade?

Best regards,

			Krzysztof Olędzki

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Eilon Greenstein]

On Thu, 2010-05-20 at 10:08 -0700, Krzysztof Olędzki wrote:
> Hello,
> 
> I would like to connect a dual port SFP+ NetXtreme II BCM57711
> 10-Gigabit NIC to a HP J9309A ProCurve 4-Port 10GbE SFP+ zl Module
> using a HP ProCurve 10-GbE SFP+ 7m Direct Attach Cable (J9285B).
> 
> Unfortuantely, it does not work. :( After connecting the switch and
> the server together and loading the bnx2x module the switch logs:
> 
> I 05/20/10 18:32:23 ports: port E4 is Blocked by STP
> I 05/20/10 18:32:23 ports: port E4 is now on-line
> 
> Here is the dmesg output from the server:
> 
> Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.1-5 (2009/11/09)
> bnx2x 0000:04:00.0: PCI INT A -> GSI 38 (level, low) -> IRQ 38
> bnx2x 0000:04:00.0: setting latency timer to 64
> bnx2x: part number 394D4342-31373735-31314131-473331
> bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
> bnx2x 0000:04:00.0: firmware: requesting bnx2x-e1h-5.2.7.0.fw
> eth4: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dc000000, IRQ 38, node addr 00:10:18:5f:e4:b4
> bnx2x 0000:04:00.1: PCI INT B -> GSI 45 (level, low) -> IRQ 45
> bnx2x 0000:04:00.1: setting latency timer to 64
> bnx2x: part number 394D4342-31373735-31314131-473331
> bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
> bnx2x 0000:04:00.1: firmware: requesting bnx2x-e1h-5.2.7.0.fw
> eth5: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dd000000, IRQ 45, node addr 00:10:18:5f:e4:b6
> bnx2x 0000:04:00.0: irq 97 for MSI/MSI-X
> bnx2x 0000:04:00.0: irq 98 for MSI/MSI-X
> bnx2x 0000:04:00.0: irq 99 for MSI/MSI-X
> bnx2x 0000:04:00.0: irq 100 for MSI/MSI-X
> bnx2x 0000:04:00.0: irq 101 for MSI/MSI-X
> bnx2x 0000:04:00.0: irq 102 for MSI/MSI-X
> bnx2x: eth4: using MSI-X  IRQs: sp 97  fp[0] 99 ... fp[3] 102
> ADDRCONF(NETDEV_UP): eth4: link is not ready
> bnx2x 0000:04:00.1: irq 103 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 104 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 105 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 106 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 107 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 108 for MSI/MSI-X
> bnx2x: eth5: using MSI-X  IRQs: sp 103  fp[0] 105 ... fp[3] 108
> ADDRCONF(NETDEV_UP): eth5: link is not ready
> bnx2x: eth5 NIC Link is Down
> bnx2x: eth5 NIC Link is Down
> 
> Loading the driver with debug mode enabled (modprobe bnx2x debug=0x20004) I got:
Thank you for this debug information! You saved one email round trip :)

However, I still need some more information about the FW version and
nvram settings. Can you please send me the output of ethtool -i and
ethtool -e? Since ethtool -e is quite long, it is best to send it as an
attached file.

> 
> bnx2x 0000:04:00.1: irq 103 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 104 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 105 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 106 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 107 for MSI/MSI-X
> bnx2x 0000:04:00.1: irq 108 for MSI/MSI-X
> bnx2x: eth5: using MSI-X  IRQs: sp 103  fp[0] 105 ... fp[3] 108
> [bnx2x_nic_load:7524(eth5)]pmf 1
> [bnx2x_link_reset:6171(eth5)]Resetting the link of port 1
> [bnx2x_set_led:5804(eth5)]bnx2x_set_led: port 1, mode 0
> [bnx2x_set_led:5806(eth5)]speed 0x0, hw_led_mode 0x1
> [bnx2x_stats_init:4597(eth5)]port_stx 0xa72f4  func_stx 0x0
> [bnx2x_phy_init:5966(eth5)]Phy Initialization started
> [bnx2x_phy_init:5968(eth5)]req_speed 10000, req_flowctrl 0
> [bnx2x_emac_init:255(eth5)]EMAC reset reg is 1024
> [bnx2x_phy_deassert:534(eth5)]bnx2x_phy_deassert:XGXS
> [bnx2x_phy_init:6140(eth5)]Phy address = 0x1
> [bnx2x_ext_phy_reset:2092(eth5)]Port 1: bnx2x_ext_phy_reset
> [bnx2x_calc_ieee_aneg_adv:1383(eth5)]ieee_fc = 0x1a0
> [bnx2x_ext_phy_init:3672(eth5)]control reg 0x2040 (after 0 ms)
> [bnx2x_ext_phy_init:4136(eth5)]Initializing BCM8727
> [bnx2x_ext_phy_init:4285(eth5)]Setting TX_CTRL1 0xc000,TX_CTRL2 0x40
> [bnx2x_link_int_enable:5338(eth5)]enabled XGXS interrupt
> [bnx2x_link_int_enable:5345(eth5)]enabled external phy int
> [bnx2x_link_int_enable:5366(eth5)]port 1, is_xgxs 1, int_status 0x0
> [bnx2x_link_int_enable:5370(eth5)] int_mask 0x3c8001, MI_INT 1, SERDES_LINK 0
> [bnx2x_link_int_enable:5373(eth5)] 10G 0, XGXS_LINK 0
> [bnx2x_stats_handle:4469(eth5)]state 0 -> event 3 -> state 0
> [bnx2x_link_update:6375(eth5)]port 1, XGXS?1, int_status 0x0
> [bnx2x_link_update:6383(eth5)]int_mask 0x0 MI_INT 1, SERDES_LINK 0
> [bnx2x_link_update:6387(eth5)] 10G 0, XGXS_LINK 0
> [bnx2x_ext_phy_is_link_up:4809(eth5)]8727 RX_ALARM_STATUS 0x439
> ADDRCONF(NETDEV_UP): eth5: link is not ready
> [bnx2x_ext_phy_is_link_up:4819(eth5)]8727 LASI status 0x6
> [bnx2x_8727_handle_mod_abs:4622(eth5)]MOD_ABS indication show module is present
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_read_sfp_module_eeprom:2818(eth5)]Got bad status 0x0 when reading from SFP+ EEPROM
> [bnx2x_8727_handle_mod_abs:4660(eth5)]SFP+ module is not initialized
> [bnx2x_8727_handle_mod_abs:4664(eth5)]8727 RX_ALARM_STATUS 0x438
> [bnx2x_ext_phy_is_link_up:4916(eth5)]Tx is disabled
> [bnx2x_link_settings_status:1958(eth5)]phy link down
> [bnx2x_link_settings_status:1976(eth5)]gp_status 0x208  phy_link_up 0 line_speed 2710
> [bnx2x_link_settings_status:1980(eth5)]duplex 1  flow_ctrl 0x400 autoneg 0x0
> [bnx2x_link_settings_status:1981(eth5)]link_status 0x0
> [bnx2x_update_link_down:6285(eth5)]Port 1: Link is down
> [bnx2x_set_led:5804(eth5)]bnx2x_set_led: port 1, mode 0
> [bnx2x_set_led:5806(eth5)]speed 0x0, hw_led_mode 0x1
> bnx2x: eth5 NIC Link is Down
> [bnx2x_stats_handle:4469(eth5)]state 0 -> event 3 -> state 0
> [bnx2x_link_update:6375(eth5)]port 1, XGXS?1, int_status 0x0
> [bnx2x_link_update:6383(eth5)]int_mask 0x0 MI_INT 1, SERDES_LINK 0
> [bnx2x_link_update:6387(eth5)] 10G 0, XGXS_LINK 0
> [bnx2x_ext_phy_is_link_up:4809(eth5)]8727 RX_ALARM_STATUS 0x41c
> [bnx2x_ext_phy_is_link_up:4819(eth5)]8727 LASI status 0x2
> [bnx2x_ext_phy_is_link_up:4916(eth5)]Tx is disabled
> [bnx2x_link_settings_status:1958(eth5)]phy link down
> [bnx2x_link_settings_status:1976(eth5)]gp_status 0x208  phy_link_up 0 line_speed 0
> [bnx2x_link_settings_status:1980(eth5)]duplex 1  flow_ctrl 0x400 autoneg 0x0
> [bnx2x_link_settings_status:1981(eth5)]link_status 0x0
> [bnx2x_update_link_down:6285(eth5)]Port 1: Link is down
> [bnx2x_set_led:5804(eth5)]bnx2x_set_led: port 1, mode 0
> [bnx2x_set_led:5806(eth5)]speed 0x0, hw_led_mode 0x1
> bnx2x: eth5 NIC Link is Down
> 
> # uname -r
> 2.6.33.3
> 
> Is there something I can do to fix this? Like for example kernel/driver or the firmware upgrade?
> 
> Best regards,
> 
> 			Krzysztof Olędzki
> 
I assume it is the same when you unplug the module and insert it again
when the driver is already up and running, right? I also assumed you
verified it is firmly plugged in (no offense, bug I had to ask...)

Eilon

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Eilon Greenstein]

On Thu, 2010-05-20 at 11:09 -0700, Krzysztof Olędzki wrote:
> On 2010-05-20 19:49, Eilon Greenstein wrote:
> > On Thu, 2010-05-20 at 10:08 -0700, Krzysztof Olędzki wrote:
> >> Hello,
> >>
> >> I would like to connect a dual port SFP+ NetXtreme II BCM57711
> >> 10-Gigabit NIC to a HP J9309A ProCurve 4-Port 10GbE SFP+ zl Module
> >> using a HP ProCurve 10-GbE SFP+ 7m Direct Attach Cable (J9285B).
> >>
> >> Unfortuantely, it does not work. :( After connecting the switch and
> >> the server together and loading the bnx2x module the switch logs:
> >>
> >> I 05/20/10 18:32:23 ports: port E4 is Blocked by STP
> >> I 05/20/10 18:32:23 ports: port E4 is now on-line
> >>
> >> Here is the dmesg output from the server:
> >>
> >> Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.1-5 (2009/11/09)
> >> bnx2x 0000:04:00.0: PCI INT A ->  GSI 38 (level, low) ->  IRQ 38
> >> bnx2x 0000:04:00.0: setting latency timer to 64
> >> bnx2x: part number 394D4342-31373735-31314131-473331
> >> bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
> >> bnx2x 0000:04:00.0: firmware: requesting bnx2x-e1h-5.2.7.0.fw
> >> eth4: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dc000000, IRQ 38, node addr 00:10:18:5f:e4:b4
> >> bnx2x 0000:04:00.1: PCI INT B ->  GSI 45 (level, low) ->  IRQ 45
> >> bnx2x 0000:04:00.1: setting latency timer to 64
> >> bnx2x: part number 394D4342-31373735-31314131-473331
> >> bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
> >> bnx2x 0000:04:00.1: firmware: requesting bnx2x-e1h-5.2.7.0.fw
> >> eth5: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dd000000, IRQ 45, node addr 00:10:18:5f:e4:b6
> >> bnx2x 0000:04:00.0: irq 97 for MSI/MSI-X
> >> bnx2x 0000:04:00.0: irq 98 for MSI/MSI-X
> >> bnx2x 0000:04:00.0: irq 99 for MSI/MSI-X
> >> bnx2x 0000:04:00.0: irq 100 for MSI/MSI-X
> >> bnx2x 0000:04:00.0: irq 101 for MSI/MSI-X
> >> bnx2x 0000:04:00.0: irq 102 for MSI/MSI-X
> >> bnx2x: eth4: using MSI-X  IRQs: sp 97  fp[0] 99 ... fp[3] 102
> >> ADDRCONF(NETDEV_UP): eth4: link is not ready
> >> bnx2x 0000:04:00.1: irq 103 for MSI/MSI-X
> >> bnx2x 0000:04:00.1: irq 104 for MSI/MSI-X
> >> bnx2x 0000:04:00.1: irq 105 for MSI/MSI-X
> >> bnx2x 0000:04:00.1: irq 106 for MSI/MSI-X
> >> bnx2x 0000:04:00.1: irq 107 for MSI/MSI-X
> >> bnx2x 0000:04:00.1: irq 108 for MSI/MSI-X
> >> bnx2x: eth5: using MSI-X  IRQs: sp 103  fp[0] 105 ... fp[3] 108
> >> ADDRCONF(NETDEV_UP): eth5: link is not ready
> >> bnx2x: eth5 NIC Link is Down
> >> bnx2x: eth5 NIC Link is Down
> >>
> >> Loading the driver with debug mode enabled (modprobe bnx2x debug=0x20004) I got:
> > Thank you for this debug information! You saved one email round trip :)
> 
> Hehe, thanks.
> 
> > However, I still need some more information about the FW version and
> > nvram settings. Can you please send me the output of ethtool -i
> 
> # ethtool -i eth5
> driver: bnx2x
> version: 1.52.1-5
> firmware-version: BC:5.0.13 PHY:0aa0:0406
> bus-info: 0000:04:00.1
> 
> > and ethtool -e? Since ethtool -e is quite long, it is best to send
> > it as an attached file.
> 
> Attached.

Almost everything seems to be in order. Almost - since you don't get
link... I don't think I have tried using this kind of Direct Attach
Cable - so maybe it just needs some more time. Let's see if the
following makes any difference (other than delay the failure for another
2.7 seconds):

diff --git a/drivers/net/bnx2x_link.c b/drivers/net/bnx2x_link.c
index ff70be8..bcee38c 100644
--- a/drivers/net/bnx2x_link.c
+++ b/drivers/net/bnx2x_link.c
@@ -3113,7 +3113,7 @@ static u8
bnx2x_wait_for_sfp_module_initialized(struct link_params *params)
        u16 timeout;
        /* Initialization time after hot-plug may take up to 300ms for
some
        phys type ( e.g. JDSU ) */
-       for (timeout = 0; timeout < 60; timeout++) {
+       for (timeout = 0; timeout < 600; timeout++) {
                if (bnx2x_read_sfp_module_eeprom(params, 1, 1, &val)
                    == 0) {
                        DP(NETIF_MSG_LINK, "SFP+ module initialization "

If it does help, be sure to let me know how much time it took (you
should have this debug print).

Regards,
Eilon

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Krzysztof Olędzki]

On 2010-05-20 20:45, Eilon Greenstein wrote:
> On Thu, 2010-05-20 at 11:09 -0700, Krzysztof Olędzki wrote:
>> On 2010-05-20 19:49, Eilon Greenstein wrote:
>>> On Thu, 2010-05-20 at 10:08 -0700, Krzysztof Olędzki wrote:
>>>> Hello,
>>>>
>>>> I would like to connect a dual port SFP+ NetXtreme II BCM57711
>>>> 10-Gigabit NIC to a HP J9309A ProCurve 4-Port 10GbE SFP+ zl Module
>>>> using a HP ProCurve 10-GbE SFP+ 7m Direct Attach Cable (J9285B).
>>>>
>>>> Unfortuantely, it does not work. :( After connecting the switch and
>>>> the server together and loading the bnx2x module the switch logs:
>>>>
>>>> I 05/20/10 18:32:23 ports: port E4 is Blocked by STP
>>>> I 05/20/10 18:32:23 ports: port E4 is now on-line
>>>>
>>>> Here is the dmesg output from the server:
>>>>
>>>> Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.1-5 (2009/11/09)
>>>> bnx2x 0000:04:00.0: PCI INT A ->   GSI 38 (level, low) ->   IRQ 38
>>>> bnx2x 0000:04:00.0: setting latency timer to 64
>>>> bnx2x: part number 394D4342-31373735-31314131-473331
>>>> bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
>>>> bnx2x 0000:04:00.0: firmware: requesting bnx2x-e1h-5.2.7.0.fw
>>>> eth4: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dc000000, IRQ 38, node addr 00:10:18:5f:e4:b4
>>>> bnx2x 0000:04:00.1: PCI INT B ->   GSI 45 (level, low) ->   IRQ 45
>>>> bnx2x 0000:04:00.1: setting latency timer to 64
>>>> bnx2x: part number 394D4342-31373735-31314131-473331
>>>> bnx2x: Loading bnx2x-e1h-5.2.7.0.fw
>>>> bnx2x 0000:04:00.1: firmware: requesting bnx2x-e1h-5.2.7.0.fw
>>>> eth5: Broadcom NetXtreme II BCM57711 XGb (A0) PCI-E x8 5GHz (Gen2) found at mem dd000000, IRQ 45, node addr 00:10:18:5f:e4:b6
>>>> bnx2x 0000:04:00.0: irq 97 for MSI/MSI-X
>>>> bnx2x 0000:04:00.0: irq 98 for MSI/MSI-X
>>>> bnx2x 0000:04:00.0: irq 99 for MSI/MSI-X
>>>> bnx2x 0000:04:00.0: irq 100 for MSI/MSI-X
>>>> bnx2x 0000:04:00.0: irq 101 for MSI/MSI-X
>>>> bnx2x 0000:04:00.0: irq 102 for MSI/MSI-X
>>>> bnx2x: eth4: using MSI-X  IRQs: sp 97  fp[0] 99 ... fp[3] 102
>>>> ADDRCONF(NETDEV_UP): eth4: link is not ready
>>>> bnx2x 0000:04:00.1: irq 103 for MSI/MSI-X
>>>> bnx2x 0000:04:00.1: irq 104 for MSI/MSI-X
>>>> bnx2x 0000:04:00.1: irq 105 for MSI/MSI-X
>>>> bnx2x 0000:04:00.1: irq 106 for MSI/MSI-X
>>>> bnx2x 0000:04:00.1: irq 107 for MSI/MSI-X
>>>> bnx2x 0000:04:00.1: irq 108 for MSI/MSI-X
>>>> bnx2x: eth5: using MSI-X  IRQs: sp 103  fp[0] 105 ... fp[3] 108
>>>> ADDRCONF(NETDEV_UP): eth5: link is not ready
>>>> bnx2x: eth5 NIC Link is Down
>>>> bnx2x: eth5 NIC Link is Down
>>>>
>>>> Loading the driver with debug mode enabled (modprobe bnx2x debug=0x20004) I got:
>>> Thank you for this debug information! You saved one email round trip :)
>>
>> Hehe, thanks.
>>
>>> However, I still need some more information about the FW version and
>>> nvram settings. Can you please send me the output of ethtool -i
>>
>> # ethtool -i eth5
>> driver: bnx2x
>> version: 1.52.1-5
>> firmware-version: BC:5.0.13 PHY:0aa0:0406
>> bus-info: 0000:04:00.1
>>
>>> and ethtool -e? Since ethtool -e is quite long, it is best to send
>>> it as an attached file.
>>
>> Attached.
>
> Almost everything seems to be in order. Almost - since you don't get
> link... I don't think I have tried using this kind of Direct Attach
> Cable - so maybe it just needs some more time. Let's see if the
> following makes any difference (other than delay the failure for another
> 2.7 seconds):
>
> diff --git a/drivers/net/bnx2x_link.c b/drivers/net/bnx2x_link.c
> index ff70be8..bcee38c 100644
> --- a/drivers/net/bnx2x_link.c
> +++ b/drivers/net/bnx2x_link.c
> @@ -3113,7 +3113,7 @@ static u8
> bnx2x_wait_for_sfp_module_initialized(struct link_params *params)
>          u16 timeout;
>          /* Initialization time after hot-plug may take up to 300ms for
> some
>          phys type ( e.g. JDSU ) */
> -       for (timeout = 0; timeout<  60; timeout++) {
> +       for (timeout = 0; timeout<  600; timeout++) {
>                  if (bnx2x_read_sfp_module_eeprom(params, 1, 1,&val)
>                      == 0) {
>                          DP(NETIF_MSG_LINK, "SFP+ module initialization "
>
> If it does help, be sure to let me know how much time it took (you
> should have this debug print).

Still no luck. :( The kernel printed many more "Got bad status 0x0 when 
reading from SFP+ EEPROM" messages. Finally I got:
  "SFP+ module is not initialized".

Best regards,

			Krzysztof Olędzki

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Rick Jones]

Some simple/simplistic thoughts/questions...

Has the DAC been used successfully prior to this?

Do you have another HP ProCurve 10-GbE SFP+ 7m Direct Attach Cable (J9285B) to try?

There's a transceiver and presumably an EEPROM at both ends of a DAC right?  If 
the EEPROM at one end were "bad" might the 57711 be happier with the other end 
of the DAC?  Getting some sort of error message at the switch side, which may 
(or may not) have more detailed diagnostics might help.

rick jones

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Krzysztof Olędzki]

On 2010-05-20 22:25, Rick Jones wrote:
> Some simple/simplistic thoughts/questions...
>
> Has the DAC been used successfully prior to this?

Yes. It was successfully used to connect two HP switches, before I 
received SFP+ SR modules, that allowed me to put the switches into 
distanced rooms.

> Do you have another HP ProCurve 10-GbE SFP+ 7m Direct Attach Cable (J9285B) to try?

Yes. The same situation.

> There's a transceiver and presumably an EEPROM at both ends of a DAC right?

Yes, I think there should be one. ;)

> If
> the EEPROM at one end were "bad" might the 57711 be happier with the other end
> of the DAC?

Tested both ends. The same situation. :|

> Getting some sort of error message at the switch side, which may
> (or may not) have more detailed diagnostics might help.

There is no error message at the switch side. The switch shows that 
everything is correct.

Best regards,

				Krzysztof Olędzki

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Eilon Greenstein]

On Thu, 2010-05-20 at 13:54 -0700, Krzysztof Olędzki wrote:
> On 2010-05-20 22:25, Rick Jones wrote:
> > Some simple/simplistic thoughts/questions...
> >
> > Has the DAC been used successfully prior to this?
> 
> Yes. It was successfully used to connect two HP switches, before I 
> received SFP+ SR modules, that allowed me to put the switches into 
> distanced rooms.

Krzysztof,

I would like to check in what speed the I2C is running at, and while we
are there, to make sure it is at low speed (100KHz and not 400KHz). Can
you please add this patch and let me know what you see:

diff --git a/drivers/net/bnx2x_link.c b/drivers/net/bnx2x_link.c
index ff70be8..6e05f99 100644
--- a/drivers/net/bnx2x_link.c
+++ b/drivers/net/bnx2x_link.c
@@ -2766,6 +2766,21 @@ static u8
bnx2x_8727_read_sfp_module_eeprom(struct link_params *params,
                      MDIO_PMA_REG_SFP_TWO_WIRE_CTRL,
                      &val);
 
+       /* Make sure we are working in 100KHz */
+       bnx2x_cl45_read(bp, port,
+                     PORT_HW_CFG_XGXS_EXT_PHY_TYPE_BCM8727,
+                     ext_phy_addr,
+                     MDIO_PMA_DEVAD,
+                     8005,
+                     &val);
+       BNX2X_ERR("I2C 2W speed 0x%x, bit 8 %d\n", val, val & 8);
+       bnx2x_cl45_write(bp, port,
+                      ext_phy_type,
+                      ext_phy_addr,
+                      MDIO_PMA_DEVAD,
+                      8005,
+                      val & ~8);
+
        /* Set the read command byte count */
        bnx2x_cl45_write(bp, port,
                       ext_phy_type,

Since we are already off for the weekend, I couldn't find a machine with
this kind of board to test it for myself. On top of that, I don't have
this type of DAC cable, so I need your help in this debug process.

Thanks,
Eilon

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Krzysztof Olędzki]

On 2010-05-21 14:58, Eilon Greenstein wrote:
> On Thu, 2010-05-20 at 13:54 -0700, Krzysztof Olędzki wrote:
>> On 2010-05-20 22:25, Rick Jones wrote:
>>> Some simple/simplistic thoughts/questions...
>>>
>>> Has the DAC been used successfully prior to this?
>>
>> Yes. It was successfully used to connect two HP switches, before I
>> received SFP+ SR modules, that allowed me to put the switches into
>> distanced rooms.
>
> Krzysztof,
>
> I would like to check in what speed the I2C is running at, and while we
> are there, to make sure it is at low speed (100KHz and not 400KHz). Can
> you please add this patch and let me know what you see:

Sure.

>
> diff --git a/drivers/net/bnx2x_link.c b/drivers/net/bnx2x_link.c

<CUT>

[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2774(eth5)]I2C 2W speed 0x0, bit 8 0

> Since we are already off for the weekend, I couldn't find a machine with
> this kind of board to test it for myself. On top of that, I don't have
> this type of DAC cable, so I need your help in this debug process.

No problem. There is no need to ask for a help a person who depends on 
your support. ;) I would like to solve it as soon as possible so I'm 
willing to test everything you send me. ;)

best regards,

			Krzysztof Olędzki

Re: bnx2x + SFP+ DA/2.6.33.3: Got bad status 0x0 when reading from SFP+ EEPROM -> SFP+ module is not initialized

[Krzysztof Olędzki]

On 2010-05-21 15:19, Krzysztof Olędzki wrote:
> On 2010-05-21 14:58, Eilon Greenstein wrote:
>> On Thu, 2010-05-20 at 13:54 -0700, Krzysztof Olędzki wrote:
>>> On 2010-05-20 22:25, Rick Jones wrote:
>>>> Some simple/simplistic thoughts/questions...
>>>>
>>>> Has the DAC been used successfully prior to this?
>>>
>>> Yes. It was successfully used to connect two HP switches, before I
>>> received SFP+ SR modules, that allowed me to put the switches into
>>> distanced rooms.
>>
>> Krzysztof,
>>
>> I would like to check in what speed the I2C is running at, and while we
>> are there, to make sure it is at low speed (100KHz and not 400KHz). Can
>> you please add this patch and let me know what you see:
> 
> Sure.
> 
>>
>> diff --git a/drivers/net/bnx2x_link.c b/drivers/net/bnx2x_link.c
> 
> <CUT>
> 
> [bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
> [bnx2x_8727_read_sfp_module_eeprom:2774(eth5)]I2C 2W speed 0x0, bit 8 0
> 
>> Since we are already off for the weekend, I couldn't find a machine with
>> this kind of board to test it for myself. On top of that, I don't have
>> this type of DAC cable, so I need your help in this debug process.
> 
> No problem. There is no need to ask for a help a person who depends on
> your support. ;) I would like to solve it as soon as possible so I'm
> willing to test everything you send me. ;)

Hello again,

To make sure the NICs are OK I made a test and plugged a SFP+ SR module (HP J9150A):

[bnx2x_8727_handle_mod_abs:4639(eth4)]MOD_ABS indication show module is present
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2835(eth4)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2835(eth4)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2835(eth4)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2835(eth4)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2835(eth4)]Got bad status 0x0 when reading from SFP+ EEPROM
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_wait_for_sfp_module_initialized:3138(eth4)]SFP+ module initialization took 25 ms
[bnx2x_sfp_module_detection:3197(eth4)]SFP+ module plugged in/out detected on port 0
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_get_edc_mode:2930(eth4)]Optic module detected
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_get_edc_mode:2954(eth4)]EDC mode is set to 0x44
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
[bnx2x_8727_read_sfp_module_eeprom:2774(eth4)]I2C 2W speed 0x0, bit 8 0
bnx2x: Warning: Unqualified SFP+ module detected on eth4, Port 0 from MergeOptics GmbH part number TRX10GVP2010CA01
[bnx2x_sfp_module_detection:3205(eth4)]Module verification failed!!

So, both NICs and SFP+ DA cables are OK. The only problem we have is to make them talk together. :|

Best regards,

			Krzysztof Olędzki