* Broadcom 5709 take long time to bring up @ 2010-08-18 14:34 Joe Jin 2010-08-18 16:08 ` Benjamin Li 0 siblings, 1 reply; 8+ messages in thread From: Joe Jin @ 2010-08-18 14:34 UTC (permalink / raw) To: Michael Chan; +Cc: joe.jin, terry.liu, netdev Hi Michael, We try boning two broadcom 5709 nics with active-backup mode, we try to down/up one of the nics, I found some ping packets loss as below: # ifdown eth0 # ifup eth0 <-- no pkg loss # ifdown eth1 # ifup eth1 <-- 2 pkg loss # ifdown eth0 # ifup eth0 <-- 10 pkg loss # ifdown eth1 # ifup eth1 <-- no pkg loss # ifdown eth0 # ifup eth0 <-- 10 pkg loss module bond parameter are: "mode=1 miimon=100 primary=eth0 downdelay=100 updelay=100" we have tried the latest driver(2.0.8c) of broadcom and Redhat el5.5 official driver(2.0.2), result are same. I checked and debugged the driver found it caused by take long time(~3s) to bring up the device. here are link state when call if up: # ifdown eth0 # ifup eth0 #ip -o link | grep eth0 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff # ifdown eth1 # ifup eth1 #ip -o link | grep eth1 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff # ifdown eth0 # ifup eth0 #ip -o link | grep eth0 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff # ifdown eth1 # ifup eth1 #ip -o link | grep eth1 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff # ifdown eth0 # ifup eth0 #ip -o link | grep eth0 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff So, the question is how long alway will take when bring up the interface? 3 seconds as expection or no? if not, is that a firmware issure or driver issue? Thanks, Joe ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-18 14:34 Broadcom 5709 take long time to bring up Joe Jin @ 2010-08-18 16:08 ` Benjamin Li 2010-08-18 20:44 ` Mitchell Erblich 2010-08-19 1:13 ` Joe Jin 0 siblings, 2 replies; 8+ messages in thread From: Benjamin Li @ 2010-08-18 16:08 UTC (permalink / raw) To: Joe Jin; +Cc: Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org Hi Joe, Just a couple of quick questions: 1. It is possible that autonegotiation could take in the orders of seconds depending on the remote partner. Could you describe the remote partner? 2. Are there iSCSI offload connections? With iSCSI there is additionial overhead to cleanup/setup iSCSI connections on the chip/firmware when ifup'ing/ifdown'ing a network interface. 3. Could you provide the kernel logs during this up/down test? I wanted to see if there was anything interesting in the kernel logs. Thanks again. -Ben On Wed, 2010-08-18 at 07:34 -0700, Joe Jin wrote: > Hi Michael, > > We try boning two broadcom 5709 nics with active-backup mode, we try to > down/up one of the nics, I found some ping packets loss as below: > # ifdown eth0 > # ifup eth0 <-- no pkg loss > # ifdown eth1 > # ifup eth1 <-- 2 pkg loss > # ifdown eth0 > # ifup eth0 <-- 10 pkg loss > # ifdown eth1 > # ifup eth1 <-- no pkg loss > # ifdown eth0 > # ifup eth0 <-- 10 pkg loss > > module bond parameter are: > "mode=1 miimon=100 primary=eth0 downdelay=100 updelay=100" > > we have tried the latest driver(2.0.8c) of broadcom and Redhat el5.5 > official driver(2.0.2), result are same. > > I checked and debugged the driver found it caused by take long time(~3s) > to bring up the device. here are link state when call if up: > # ifdown eth0 > # ifup eth0 > #ip -o link | grep eth0 > 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff > # ifdown eth1 > # ifup eth1 > #ip -o link | grep eth1 > 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff > # ifdown eth0 > # ifup eth0 > #ip -o link | grep eth0 > 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff > # ifdown eth1 > # ifup eth1 > #ip -o link | grep eth1 > 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff > # ifdown eth0 > # ifup eth0 > #ip -o link | grep eth0 > 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff > > So, the question is how long alway will take when bring up the interface? > 3 seconds as expection or no? if not, is that a firmware issure or driver > issue? > > Thanks, > Joe > > > > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-18 16:08 ` Benjamin Li @ 2010-08-18 20:44 ` Mitchell Erblich 2010-08-18 21:25 ` Rick Jones 2010-08-19 1:13 ` Joe Jin 1 sibling, 1 reply; 8+ messages in thread From: Mitchell Erblich @ 2010-08-18 20:44 UTC (permalink / raw) To: Benjamin Li Cc: Joe Jin, Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org On Aug 18, 2010, at 9:08 AM, Benjamin Li wrote: > Hi Joe, > > Just a couple of quick questions: > > 1. It is possible that autonegotiation could take in the orders of > seconds depending on the remote partner. No. It happens in ms. Mitchell Erblich > Could you describe the remote > partner? > > 2. Are there iSCSI offload connections? With iSCSI there is > additionial overhead to cleanup/setup iSCSI connections on the > chip/firmware when ifup'ing/ifdown'ing a network interface. > > 3. Could you provide the kernel logs during this up/down test? I > wanted to see if there was anything interesting in the kernel logs. > > Thanks again. > > -Ben > > > On Wed, 2010-08-18 at 07:34 -0700, Joe Jin wrote: >> Hi Michael, >> >> We try boning two broadcom 5709 nics with active-backup mode, we try to >> down/up one of the nics, I found some ping packets loss as below: >> # ifdown eth0 >> # ifup eth0 <-- no pkg loss >> # ifdown eth1 >> # ifup eth1 <-- 2 pkg loss >> # ifdown eth0 >> # ifup eth0 <-- 10 pkg loss >> # ifdown eth1 >> # ifup eth1 <-- no pkg loss >> # ifdown eth0 >> # ifup eth0 <-- 10 pkg loss >> >> module bond parameter are: >> "mode=1 miimon=100 primary=eth0 downdelay=100 updelay=100" >> >> we have tried the latest driver(2.0.8c) of broadcom and Redhat el5.5 >> official driver(2.0.2), result are same. >> >> I checked and debugged the driver found it caused by take long time(~3s) >> to bring up the device. here are link state when call if up: >> # ifdown eth0 >> # ifup eth0 >> #ip -o link | grep eth0 >> 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff >> # ifdown eth1 >> # ifup eth1 >> #ip -o link | grep eth1 >> 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff >> # ifdown eth0 >> # ifup eth0 >> #ip -o link | grep eth0 >> 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff >> # ifdown eth1 >> # ifup eth1 >> #ip -o link | grep eth1 >> 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff >> # ifdown eth0 >> # ifup eth0 >> #ip -o link | grep eth0 >> 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000\ link/ether 00:26:55:4f:1f:18 brd ff:ff:ff:ff:ff:ff >> >> So, the question is how long alway will take when bring up the interface? >> 3 seconds as expection or no? if not, is that a firmware issure or driver >> issue? >> >> Thanks, >> Joe >> >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe netdev" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-18 20:44 ` Mitchell Erblich @ 2010-08-18 21:25 ` Rick Jones 2010-08-18 22:19 ` Ben Hutchings 0 siblings, 1 reply; 8+ messages in thread From: Rick Jones @ 2010-08-18 21:25 UTC (permalink / raw) To: Mitchell Erblich Cc: Benjamin Li, Joe Jin, Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org Mitchell Erblich wrote: > On Aug 18, 2010, at 9:08 AM, Benjamin Li wrote: > > >>Hi Joe, >> >>Just a couple of quick questions: >> >>1. It is possible that autonegotiation could take in the orders of >>seconds depending on the remote partner. > > > No. It happens in ms. > > Mitchell Erblich Both may be correct, depending on the context. The IEEE specs likely mandate that autonegotiation take no longer than N ms, but there may indeed be kit out there that takes rather longer... rick jones ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-18 21:25 ` Rick Jones @ 2010-08-18 22:19 ` Ben Hutchings 0 siblings, 0 replies; 8+ messages in thread From: Ben Hutchings @ 2010-08-18 22:19 UTC (permalink / raw) To: Rick Jones Cc: Mitchell Erblich, Benjamin Li, Joe Jin, Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org On Wed, 2010-08-18 at 14:25 -0700, Rick Jones wrote: > Mitchell Erblich wrote: > > On Aug 18, 2010, at 9:08 AM, Benjamin Li wrote: > > > > > >>Hi Joe, > >> > >>Just a couple of quick questions: > >> > >>1. It is possible that autonegotiation could take in the orders of > >>seconds depending on the remote partner. > > > > > > No. It happens in ms. > > > > Mitchell Erblich > > Both may be correct, depending on the context. The IEEE specs likely mandate > that autonegotiation take no longer than N ms, but there may indeed be kit out > there that takes rather longer... Autonegotiation per se should not take very long, but 1000BASE-T or 10GBASE-T also require the PHYs to sync clocks, detect and cancel echoes, and so on. I believe that that link training normally takes a few seconds after AN is done. Ben. -- Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-18 16:08 ` Benjamin Li 2010-08-18 20:44 ` Mitchell Erblich @ 2010-08-19 1:13 ` Joe Jin 2010-08-19 8:11 ` Joe Jin 1 sibling, 1 reply; 8+ messages in thread From: Joe Jin @ 2010-08-19 1:13 UTC (permalink / raw) To: Benjamin Li Cc: Joe Jin, Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org Hi Benjamin, Thanks for your reply, see the answers in the lines. On 2010-08-18 09:08, Benjamin Li wrote: > Hi Joe, > > Just a couple of quick questions: > > 1. It is possible that autonegotiation could take in the orders of > seconds depending on the remote partner. Could you describe the remote > partner? I dont think it caused that. >From system log(see below) could saw brought up the interface took around 3 seconds. I have add some debugging codes in the drivers and could saw 3 times poll the devices and all the state of device is link down. Also, from system could found during bootup, brought all devices took less than 1 seconds, but ifdown/ifup take long times. > > 2. Are there iSCSI offload connections? With iSCSI there is > additionial overhead to cleanup/setup iSCSI connections on the > chip/firmware when ifup'ing/ifdown'ing a network interface. I'll confirm it later, but from system log saw have initialized iSCSI module, it maybe cause it. > > 3. Could you provide the kernel logs during this up/down test? I > wanted to see if there was anything interesting in the kernel logs. > Here are kernel log: System log of bootup ~~~~~~~~~~~~~~~~~~~~ Jul 19 11:23:28 bond-test kernel: Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008) Jul 19 11:23:28 bond-test kernel: bonding: Warning: updelay (150) is not a multiple of miimon (100), updelay rounded to 100 ms Jul 19 11:23:28 bond-test kernel: bonding: MII link monitoring set to 100 ms Jul 19 11:23:28 bond-test kernel: ADDRCONF(NETDEV_UP): bond0: link is not ready Jul 19 11:23:28 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:23:28 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:23:28 bond-test kernel: iscsi: registered transport (bcm570x-020000) Jul 19 11:23:28 bond-test kernel: bnx2i: netif=eth0, iscsi=bcm570x-020000 Jul 19 11:23:28 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:23:28 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:23:28 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:23:28 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:23:28 bond-test kernel: iscsi: registered transport (bcm570x-020001) Jul 19 11:23:28 bond-test kernel: bnx2i: netif=eth1, iscsi=bcm570x-020001 Jul 19 11:23:28 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:23:28 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:23:28 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:23:28 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:23:28 bond-test kernel: bonding: bond0: first active interface up! Jul 19 11:23:28 bond-test kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready Jul 19 11:23:28 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:23:28 bond-test kernel: bonding: bond0: making interface eth0 the new active one. System log of the testcase ~~~~~~~~~~~~~~~~~~~~~~~~~~ Jul 19 11:25:41 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:25:41 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:25:41 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:25:41 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:25:45 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:25:45 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:25:45 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:25:45 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:25:47 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:25:47 bond-test kernel: bonding: bond0: releasing active interface eth1 Jul 19 11:25:48 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:25:48 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 0 ms. Jul 19 11:25:48 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:25:48 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:25:53 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:25:53 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:25:53 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:25:53 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:25:55 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:25:55 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:25:55 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:25:56 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:25:56 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. Jul 19 11:25:56 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:25:56 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:25:59 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:25:59 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:25:59 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:25:59 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:26:02 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:26:02 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:26:02 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:26:02 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:26:27 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:26:27 bond-test kernel: bonding: bond0: releasing backup interface eth1 Jul 19 11:26:29 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:26:29 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:26:29 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:26:29 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:26:32 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:26:32 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. Jul 19 11:26:32 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:26:33 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:26:33 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:26:33 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:26:33 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:26:36 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:26:36 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:26:36 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:26:36 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:26:38 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:26:39 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:26:39 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:26:39 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:27:07 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:27:07 bond-test kernel: bonding: bond0: releasing backup interface eth1 Jul 19 11:27:09 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:27:09 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:27:09 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:27:09 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:27:12 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:27:12 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. Jul 19 11:27:12 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:27:12 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:27:12 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:27:12 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:27:12 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:27:15 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:27:15 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:27:15 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:27:15 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:27:18 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:27:18 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:27:18 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:27:18 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:27:22 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:27:22 bond-test kernel: bonding: bond0: releasing backup interface eth1 Jul 19 11:27:24 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:27:24 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:27:24 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:27:24 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:27:26 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:27:26 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. Jul 19 11:27:26 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:27:27 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:27:27 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:27:27 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:27:27 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:27:30 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:27:30 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:27:30 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:27:30 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:27:33 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:27:33 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:27:33 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:27:33 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:27:52 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:27:52 bond-test kernel: bonding: bond0: releasing backup interface eth1 Jul 19 11:27:54 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:27:54 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:27:54 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:27:54 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:27:56 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:27:56 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. Jul 19 11:27:56 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:28:05 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:28:05 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:28:05 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:28:05 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:28:07 bond-test ntpd[7189]: synchronized to 144.145.109.92, stratum 3 Jul 19 11:28:07 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:28:07 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:28:07 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:28:07 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:28:11 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:28:11 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:28:11 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:28:11 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:28:32 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:28:32 bond-test kernel: bonding: bond0: releasing backup interface eth1 Jul 19 11:28:34 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:28:34 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:28:34 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:28:34 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:28:37 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:28:37 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. Jul 19 11:28:37 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:28:40 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:28:40 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:28:40 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:28:40 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:28:43 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:28:43 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:28:43 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:28:43 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:28:45 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:28:45 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:28:45 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:28:45 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:28:45 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:28:45 bond-test kernel: bonding: bond0: releasing backup interface eth1 Jul 19 11:28:48 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:28:48 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:28:48 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:28:48 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:28:51 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:28:51 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. Jul 19 11:28:51 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:28:53 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:28:53 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:28:53 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:28:53 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:28:57 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:28:57 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:28:57 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:28:57 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:28:59 bond-test kernel: bonding: bond0: Removing slave eth1 Jul 19 11:28:59 bond-test kernel: bonding: bond0: releasing active interface eth1 Jul 19 11:29:00 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:29:00 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 0 ms. Jul 19 11:29:00 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Jul 19 11:29:00 bond-test kernel: bonding: bond0: making interface eth0 the new active one. Jul 19 11:29:13 bond-test kernel: bonding: bond0: Adding slave eth1. Jul 19 11:29:13 bond-test kernel: bnx2: eth1: using MSI Jul 19 11:29:13 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed Jul 19 11:29:13 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. Jul 19 11:29:15 bond-test kernel: bonding: bond0: Removing slave eth0 Jul 19 11:29:15 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. Jul 19 11:29:15 bond-test kernel: bonding: bond0: releasing active interface eth0 Jul 19 11:29:15 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:29:16 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. Jul 19 11:29:16 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. Jul 19 11:29:16 bond-test kernel: bonding: bond0: making interface eth1 the new active one. Jul 19 11:29:20 bond-test kernel: bonding: bond0: Adding slave eth0. Jul 19 11:29:20 bond-test kernel: bnx2: eth0: using MSI Jul 19 11:29:20 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed Jul 19 11:29:20 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. Jul 19 11:29:22 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON Jul 19 11:29:22 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. Jul 19 11:29:22 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. Thanks, Joe ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-19 1:13 ` Joe Jin @ 2010-08-19 8:11 ` Joe Jin 2010-08-23 14:21 ` Joe Jin 0 siblings, 1 reply; 8+ messages in thread From: Joe Jin @ 2010-08-19 8:11 UTC (permalink / raw) To: Joe Jin Cc: Benjamin Li, Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org Hi, Just confirmed it, there was not iscsi configured on the server. Thanks, Joe On 2010-08-19 09:13, Joe Jin wrote: > Hi Benjamin, > > Thanks for your reply, see the answers in the lines. > > On 2010-08-18 09:08, Benjamin Li wrote: > > Hi Joe, > > > > Just a couple of quick questions: > > > > 1. It is possible that autonegotiation could take in the orders of > > seconds depending on the remote partner. Could you describe the remote > > partner? > > I dont think it caused that. > >From system log(see below) could saw brought up the interface took > around 3 seconds. I have add some debugging codes in the drivers and > could saw 3 times poll the devices and all the state of device is > link down. > > Also, from system could found during bootup, brought all devices > took less than 1 seconds, but ifdown/ifup take long times. > > > > > 2. Are there iSCSI offload connections? With iSCSI there is > > additionial overhead to cleanup/setup iSCSI connections on the > > chip/firmware when ifup'ing/ifdown'ing a network interface. > > I'll confirm it later, but from system log saw have initialized > iSCSI module, it maybe cause it. > > > > > 3. Could you provide the kernel logs during this up/down test? I > > wanted to see if there was anything interesting in the kernel logs. > > > > Here are kernel log: > > > System log of bootup > ~~~~~~~~~~~~~~~~~~~~ > Jul 19 11:23:28 bond-test kernel: Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008) > Jul 19 11:23:28 bond-test kernel: bonding: Warning: updelay (150) is not a multiple of miimon (100), updelay rounded to 100 ms > Jul 19 11:23:28 bond-test kernel: bonding: MII link monitoring set to 100 ms > Jul 19 11:23:28 bond-test kernel: ADDRCONF(NETDEV_UP): bond0: link is not ready > Jul 19 11:23:28 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:23:28 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:23:28 bond-test kernel: iscsi: registered transport (bcm570x-020000) > Jul 19 11:23:28 bond-test kernel: bnx2i: netif=eth0, iscsi=bcm570x-020000 > Jul 19 11:23:28 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:23:28 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:23:28 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:23:28 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:23:28 bond-test kernel: iscsi: registered transport (bcm570x-020001) > Jul 19 11:23:28 bond-test kernel: bnx2i: netif=eth1, iscsi=bcm570x-020001 > Jul 19 11:23:28 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:23:28 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:23:28 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:23:28 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:23:28 bond-test kernel: bonding: bond0: first active interface up! > Jul 19 11:23:28 bond-test kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready > Jul 19 11:23:28 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:23:28 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > > System log of the testcase > ~~~~~~~~~~~~~~~~~~~~~~~~~~ > Jul 19 11:25:41 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:25:41 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:25:41 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:25:41 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:25:45 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:25:45 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:25:45 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:25:45 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:25:47 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:25:47 bond-test kernel: bonding: bond0: releasing active interface eth1 > Jul 19 11:25:48 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:25:48 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 0 ms. > Jul 19 11:25:48 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:25:48 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:25:53 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:25:53 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:25:53 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:25:53 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:25:55 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:25:55 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:25:55 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:25:56 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:25:56 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. > Jul 19 11:25:56 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:25:56 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:25:59 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:25:59 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:25:59 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:25:59 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:26:02 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:26:02 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:26:02 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:26:02 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:26:27 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:26:27 bond-test kernel: bonding: bond0: releasing backup interface eth1 > Jul 19 11:26:29 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:26:29 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:26:29 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:26:29 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:26:32 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:26:32 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > Jul 19 11:26:32 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:26:33 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:26:33 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:26:33 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:26:33 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:26:36 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:26:36 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:26:36 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:26:36 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:26:38 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:26:39 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:26:39 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:26:39 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:27:07 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:27:07 bond-test kernel: bonding: bond0: releasing backup interface eth1 > Jul 19 11:27:09 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:27:09 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:27:09 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:27:09 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:27:12 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:27:12 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > Jul 19 11:27:12 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:27:12 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:27:12 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:27:12 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:27:12 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:27:15 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:27:15 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:27:15 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:27:15 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:27:18 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:27:18 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:27:18 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:27:18 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:27:22 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:27:22 bond-test kernel: bonding: bond0: releasing backup interface eth1 > Jul 19 11:27:24 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:27:24 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:27:24 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:27:24 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:27:26 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:27:26 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > Jul 19 11:27:26 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:27:27 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:27:27 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:27:27 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:27:27 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:27:30 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:27:30 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:27:30 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:27:30 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:27:33 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:27:33 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:27:33 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:27:33 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:27:52 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:27:52 bond-test kernel: bonding: bond0: releasing backup interface eth1 > Jul 19 11:27:54 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:27:54 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:27:54 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:27:54 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:27:56 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:27:56 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > Jul 19 11:27:56 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:28:05 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:28:05 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:28:05 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:28:05 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:28:07 bond-test ntpd[7189]: synchronized to 144.145.109.92, stratum 3 > Jul 19 11:28:07 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:28:07 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:28:07 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:28:07 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:28:11 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:28:11 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:28:11 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:28:11 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:28:32 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:28:32 bond-test kernel: bonding: bond0: releasing backup interface eth1 > Jul 19 11:28:34 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:28:34 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:28:34 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:28:34 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:28:37 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:28:37 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > Jul 19 11:28:37 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:28:40 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:28:40 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:28:40 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:28:40 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:28:43 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:28:43 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:28:43 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:28:43 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:28:45 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:28:45 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:28:45 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:28:45 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:28:45 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:28:45 bond-test kernel: bonding: bond0: releasing backup interface eth1 > Jul 19 11:28:48 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:28:48 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:28:48 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:28:48 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:28:51 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:28:51 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > Jul 19 11:28:51 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:28:53 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:28:53 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:28:53 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:28:53 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:28:57 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:28:57 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:28:57 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:28:57 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:28:59 bond-test kernel: bonding: bond0: Removing slave eth1 > Jul 19 11:28:59 bond-test kernel: bonding: bond0: releasing active interface eth1 > Jul 19 11:29:00 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:29:00 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 0 ms. > Jul 19 11:29:00 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > Jul 19 11:29:00 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > Jul 19 11:29:13 bond-test kernel: bonding: bond0: Adding slave eth1. > Jul 19 11:29:13 bond-test kernel: bnx2: eth1: using MSI > Jul 19 11:29:13 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > Jul 19 11:29:13 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > Jul 19 11:29:15 bond-test kernel: bonding: bond0: Removing slave eth0 > Jul 19 11:29:15 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > Jul 19 11:29:15 bond-test kernel: bonding: bond0: releasing active interface eth0 > Jul 19 11:29:15 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:29:16 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. > Jul 19 11:29:16 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > Jul 19 11:29:16 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > Jul 19 11:29:20 bond-test kernel: bonding: bond0: Adding slave eth0. > Jul 19 11:29:20 bond-test kernel: bnx2: eth0: using MSI > Jul 19 11:29:20 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > Jul 19 11:29:20 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > Jul 19 11:29:22 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > Jul 19 11:29:22 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > Jul 19 11:29:22 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > > > Thanks, > Joe -- Joe Jin | Team Leader, Software Development | +8610.8278.6295 ORACLE | Linux and Virtualization Incubator Building 2-A ZPark | Beijing China, 100094 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Broadcom 5709 take long time to bring up 2010-08-19 8:11 ` Joe Jin @ 2010-08-23 14:21 ` Joe Jin 0 siblings, 0 replies; 8+ messages in thread From: Joe Jin @ 2010-08-23 14:21 UTC (permalink / raw) To: Joe Jin Cc: Benjamin Li, Michael Chan, terry.liu@oracle.com, netdev@vger.kernel.org Again, I moved cnic.ko and bnx2i.ko to other directory, still bnx2 device came back took 3 seconds. Any suggestions is appreciate! Joe On 2010-08-19 16:11, Joe Jin wrote: > Hi, > > Just confirmed it, there was not iscsi configured on the server. > > Thanks, > Joe > > > On 2010-08-19 09:13, Joe Jin wrote: > > Hi Benjamin, > > > > Thanks for your reply, see the answers in the lines. > > > > On 2010-08-18 09:08, Benjamin Li wrote: > > > Hi Joe, > > > > > > Just a couple of quick questions: > > > > > > 1. It is possible that autonegotiation could take in the orders of > > > seconds depending on the remote partner. Could you describe the remote > > > partner? > > > > I dont think it caused that. > > >From system log(see below) could saw brought up the interface took > > around 3 seconds. I have add some debugging codes in the drivers and > > could saw 3 times poll the devices and all the state of device is > > link down. > > > > Also, from system could found during bootup, brought all devices > > took less than 1 seconds, but ifdown/ifup take long times. > > > > > > > > 2. Are there iSCSI offload connections? With iSCSI there is > > > additionial overhead to cleanup/setup iSCSI connections on the > > > chip/firmware when ifup'ing/ifdown'ing a network interface. > > > > I'll confirm it later, but from system log saw have initialized > > iSCSI module, it maybe cause it. > > > > > > > > 3. Could you provide the kernel logs during this up/down test? I > > > wanted to see if there was anything interesting in the kernel logs. > > > > > > > Here are kernel log: > > > > > > System log of bootup > > ~~~~~~~~~~~~~~~~~~~~ > > Jul 19 11:23:28 bond-test kernel: Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008) > > Jul 19 11:23:28 bond-test kernel: bonding: Warning: updelay (150) is not a multiple of miimon (100), updelay rounded to 100 ms > > Jul 19 11:23:28 bond-test kernel: bonding: MII link monitoring set to 100 ms > > Jul 19 11:23:28 bond-test kernel: ADDRCONF(NETDEV_UP): bond0: link is not ready > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:23:28 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:23:28 bond-test kernel: iscsi: registered transport (bcm570x-020000) > > Jul 19 11:23:28 bond-test kernel: bnx2i: netif=eth0, iscsi=bcm570x-020000 > > Jul 19 11:23:28 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:23:28 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:23:28 bond-test kernel: iscsi: registered transport (bcm570x-020001) > > Jul 19 11:23:28 bond-test kernel: bnx2i: netif=eth1, iscsi=bcm570x-020001 > > Jul 19 11:23:28 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:23:28 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: first active interface up! > > Jul 19 11:23:28 bond-test kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready > > Jul 19 11:23:28 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:23:28 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > > > > > System log of the testcase > > ~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Jul 19 11:25:41 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:25:41 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:25:41 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:25:41 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:25:45 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:25:45 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:25:45 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:25:45 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:25:47 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:25:47 bond-test kernel: bonding: bond0: releasing active interface eth1 > > Jul 19 11:25:48 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:25:48 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 0 ms. > > Jul 19 11:25:48 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:25:48 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:25:53 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:25:53 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:25:53 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:25:53 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:25:55 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:25:55 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:25:55 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:25:56 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:25:56 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. > > Jul 19 11:25:56 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:25:56 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:25:59 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:25:59 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:25:59 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:25:59 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:26:02 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:26:02 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:26:02 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:26:02 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:26:27 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:26:27 bond-test kernel: bonding: bond0: releasing backup interface eth1 > > Jul 19 11:26:29 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:26:29 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:26:29 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:26:29 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:26:32 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:26:32 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > > Jul 19 11:26:32 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:26:33 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:26:33 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:26:33 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:26:33 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:26:36 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:26:36 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:26:36 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:26:36 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:26:38 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:26:39 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:26:39 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:26:39 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:27:07 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:27:07 bond-test kernel: bonding: bond0: releasing backup interface eth1 > > Jul 19 11:27:09 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:27:09 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:27:09 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:27:09 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:27:12 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:27:12 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > > Jul 19 11:27:12 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:27:12 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:27:12 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:27:12 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:27:12 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:27:15 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:27:15 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:27:15 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:27:15 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:27:18 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:27:18 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:27:18 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:27:18 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:27:22 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:27:22 bond-test kernel: bonding: bond0: releasing backup interface eth1 > > Jul 19 11:27:24 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:27:24 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:27:24 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:27:24 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:27:26 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:27:26 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > > Jul 19 11:27:26 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:27:27 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:27:27 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:27:27 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:27:27 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:27:30 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:27:30 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:27:30 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:27:30 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:27:33 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:27:33 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:27:33 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:27:33 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:27:52 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:27:52 bond-test kernel: bonding: bond0: releasing backup interface eth1 > > Jul 19 11:27:54 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:27:54 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:27:54 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:27:54 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:27:56 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:27:56 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > > Jul 19 11:27:56 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:28:05 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:28:05 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:28:05 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:28:05 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:28:07 bond-test ntpd[7189]: synchronized to 144.145.109.92, stratum 3 > > Jul 19 11:28:07 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:28:07 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:28:07 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:28:07 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:28:11 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:28:11 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:28:11 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:28:11 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:28:32 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:28:32 bond-test kernel: bonding: bond0: releasing backup interface eth1 > > Jul 19 11:28:34 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:28:34 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:28:34 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:28:34 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:28:37 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:28:37 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > > Jul 19 11:28:37 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:28:40 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:28:40 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:28:40 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:28:40 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:28:43 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:28:43 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:28:43 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:28:43 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:28:45 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:28:45 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:28:45 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:28:45 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:28:45 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:28:45 bond-test kernel: bonding: bond0: releasing backup interface eth1 > > Jul 19 11:28:48 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:28:48 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:28:48 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:28:48 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:28:51 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:28:51 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 100 ms. > > Jul 19 11:28:51 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:28:53 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:28:53 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:28:53 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:28:53 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:28:57 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:28:57 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:28:57 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:28:57 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:28:59 bond-test kernel: bonding: bond0: Removing slave eth1 > > Jul 19 11:28:59 bond-test kernel: bonding: bond0: releasing active interface eth1 > > Jul 19 11:29:00 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:29:00 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 0 ms. > > Jul 19 11:29:00 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > Jul 19 11:29:00 bond-test kernel: bonding: bond0: making interface eth0 the new active one. > > Jul 19 11:29:13 bond-test kernel: bonding: bond0: Adding slave eth1. > > Jul 19 11:29:13 bond-test kernel: bnx2: eth1: using MSI > > Jul 19 11:29:13 bond-test kernel: bnx2i [02:00.01]: ISCSI_INIT passed > > Jul 19 11:29:13 bond-test kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link. > > Jul 19 11:29:15 bond-test kernel: bonding: bond0: Removing slave eth0 > > Jul 19 11:29:15 bond-test kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:26:55:4F:1F:18 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. > > Jul 19 11:29:15 bond-test kernel: bonding: bond0: releasing active interface eth0 > > Jul 19 11:29:15 bond-test kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:29:16 bond-test kernel: bonding: bond0: link status up for interface eth1, enabling it in 0 ms. > > Jul 19 11:29:16 bond-test kernel: bonding: bond0: link status definitely up for interface eth1. > > Jul 19 11:29:16 bond-test kernel: bonding: bond0: making interface eth1 the new active one. > > Jul 19 11:29:20 bond-test kernel: bonding: bond0: Adding slave eth0. > > Jul 19 11:29:20 bond-test kernel: bnx2: eth0: using MSI > > Jul 19 11:29:20 bond-test kernel: bnx2i [02:00.00]: ISCSI_INIT passed > > Jul 19 11:29:20 bond-test kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link. > > Jul 19 11:29:22 bond-test kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON > > Jul 19 11:29:22 bond-test kernel: bonding: bond0: link status up for interface eth0, enabling it in 100 ms. > > Jul 19 11:29:22 bond-test kernel: bonding: bond0: link status definitely up for interface eth0. > > > > > > > > Thanks, > > Joe > > -- > Joe Jin | Team Leader, Software Development | +8610.8278.6295 > ORACLE | Linux and Virtualization > Incubator Building 2-A ZPark | Beijing China, 100094 -- Joe Jin | Team Leader, Software Development | +8610.8278.6295 ORACLE | Linux and Virtualization Incubator Building 2-A ZPark | Beijing China, 100094 ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-08-23 14:22 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-08-18 14:34 Broadcom 5709 take long time to bring up Joe Jin 2010-08-18 16:08 ` Benjamin Li 2010-08-18 20:44 ` Mitchell Erblich 2010-08-18 21:25 ` Rick Jones 2010-08-18 22:19 ` Ben Hutchings 2010-08-19 1:13 ` Joe Jin 2010-08-19 8:11 ` Joe Jin 2010-08-23 14:21 ` Joe Jin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).