[BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Lucas Nussbaum]

Hi,

We use PXE with Broadcom NetXtreme cards. After a soft reboot (using the
"reboot" command), the system cannot get an IP address using DHCP. On
the console, a MAC address of 00 00 00 00 00 00 is shown. When rebooting
with "reboot -f" or with the reset button, everything works as expected.

This used to work, and broke between 2.6.16 and 2.6.17. Using git bissect,
I could trace this back to that commit:
commit bc1c756741b065cfebf850e4164c0e2aae9d527f
Author: Michael Chan <mchan@broadcom.com>
Date:   Mon Mar 20 17:48:03 2006 -0800
    [TG3]: Support shutdown WoL.

During boot, the following messages are displayed:
Broadcom NetXtreme Gigabit Ethernet Boot Agent v2.2.8
[...]
Broadcom UNDI, PXE-2.1 (build 082) v2.2.8
[...]
CLIENT MAC ADDR: 00 10 18 01 E5 2F GUID: 44454C4C 4800 1052 8032
B9C04F53304A

After a soft reboot, the last line is changed to:
CLIENT MAC ADDR: 00 00 00 00 00 00 GUID: 44454C4C 4800 1052 8032
B9C04F53304A

lspci -v for the card:
02:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02)
        Subsystem: Broadcom Corporation NetXtreme BCM5703 1000Base-T
        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 177
        Memory at fcf00000 (64-bit, non-prefetchable) [size=64K]
        Capabilities: [40] PCI-X non-bridge device
        Capabilities: [48] Power Management version 2
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+ Queue=0/3 Enable-

Thank you,
-- 
| Lucas Nussbaum                        PhD student |
| lucas.nussbaum@imag.fr        LIG / Projet MESCAL |
| jabber: lucas@nussbaum.fr    +33 (0)6 64 71 41 65 |
| homepage:        http://www-id.imag.fr/~nussbaum/ |

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Michael Chan]

Lucas Nussbaum wrote:

> This used to work, and broke between 2.6.16 and 2.6.17. Using 
> git bissect,
> I could trace this back to that commit:
> commit bc1c756741b065cfebf850e4164c0e2aae9d527f
> Author: Michael Chan <mchan@broadcom.com>
> Date:   Mon Mar 20 17:48:03 2006 -0800
>     [TG3]: Support shutdown WoL.

This may be caused by bugs in early versions of bootcode or
PXE code.  When tg3 powers down the PHY during shutdown, the
MAC address will become zero in the MAC address register.  The
PXE code or bootcode needs to fetch the MAC address again from
The NVRAM.

Can you also send me ethtool -i eth0 which will provide the
bootcode version?  What machine are you using?  Thanks.

> 
> During boot, the following messages are displayed:
> Broadcom NetXtreme Gigabit Ethernet Boot Agent v2.2.8
> [...]
> Broadcom UNDI, PXE-2.1 (build 082) v2.2.8
> [...]
> CLIENT MAC ADDR: 00 10 18 01 E5 2F GUID: 44454C4C 4800 1052 8032
> B9C04F53304A
> 
> After a soft reboot, the last line is changed to:
> CLIENT MAC ADDR: 00 00 00 00 00 00 GUID: 44454C4C 4800 1052 8032
> B9C04F53304A
> 
> lspci -v for the card:
> 02:02.0 Ethernet controller: Broadcom Corporation NetXtreme 
> BCM5703X Gigabit Ethernet (rev 02)
>         Subsystem: Broadcom Corporation NetXtreme BCM5703 1000Base-T
>         Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 177
>         Memory at fcf00000 (64-bit, non-prefetchable) [size=64K]
>         Capabilities: [40] PCI-X non-bridge device
>         Capabilities: [48] Power Management version 2
>         Capabilities: [50] Vital Product Data
>         Capabilities: [58] Message Signalled Interrupts: 
> Mask- 64bit+ Queue=0/3 Enable-
> 
> Thank you,
> -- 
> | Lucas Nussbaum                        PhD student |
> | lucas.nussbaum@imag.fr        LIG / Projet MESCAL |
> | jabber: lucas@nussbaum.fr    +33 (0)6 64 71 41 65 |
> | homepage:        http://www-id.imag.fr/~nussbaum/ |
> 
> 

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Lucas Nussbaum]

On 13/09/07 at 08:15 -0700, Michael Chan wrote:
> Lucas Nussbaum wrote:
> 
> > This used to work, and broke between 2.6.16 and 2.6.17. Using 
> > git bissect,
> > I could trace this back to that commit:
> > commit bc1c756741b065cfebf850e4164c0e2aae9d527f
> > Author: Michael Chan <mchan@broadcom.com>
> > Date:   Mon Mar 20 17:48:03 2006 -0800
> >     [TG3]: Support shutdown WoL.
> 
> This may be caused by bugs in early versions of bootcode or
> PXE code.  When tg3 powers down the PHY during shutdown, the
> MAC address will become zero in the MAC address register.  The
> PXE code or bootcode needs to fetch the MAC address again from
> The NVRAM.
> 
> Can you also send me ethtool -i eth0 which will provide the
> bootcode version?  What machine are you using?  Thanks.

# ethtool -i eth0
driver: tg3
version: 3.65
firmware-version: 5703-v2.21a
bus-info: 0000:02:02.0

What do you mean by "what machine" ? The systems are Dell PowerEdge
1600SC, but the NICs were bought separately AFAIK.
-- 
| Lucas Nussbaum                        PhD student |
| lucas.nussbaum@imag.fr        LIG / Projet MESCAL |
| jabber: lucas@nussbaum.fr    +33 (0)6 64 71 41 65 |
| homepage:        http://www-id.imag.fr/~nussbaum/ |

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Michael Chan]

On Thu, 2007-09-13 at 17:41 +0200, Lucas Nussbaum wrote:

> # ethtool -i eth0
> driver: tg3
> version: 3.65
> firmware-version: 5703-v2.21a
> bus-info: 0000:02:02.0

The firmware is quite old and needs to be upgraded to fix the problem.
I'll have someone contact you to get it upgraded.

> 
> What do you mean by "what machine" ? The systems are Dell PowerEdge
> 1600SC, but the NICs were bought separately AFAIK.

I assumed the device was on-board which would normally require a BIOS
upgrade.  For NICs, it's easier.

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Lucas Nussbaum]

On 13/09/07 at 11:05 -0700, Michael Chan wrote:
> On Thu, 2007-09-13 at 17:41 +0200, Lucas Nussbaum wrote:
> 
> > # ethtool -i eth0
> > driver: tg3
> > version: 3.65
> > firmware-version: 5703-v2.21a
> > bus-info: 0000:02:02.0
> 
> The firmware is quite old and needs to be upgraded to fix the problem.
> I'll have someone contact you to get it upgraded.

Erm, Wouldn't it be possible to print a warning when the driver loads,
saying that the firmware is outdated ?
-- 
| Lucas Nussbaum                        PhD student |
| lucas.nussbaum@imag.fr        LIG / Projet MESCAL |
| jabber: lucas@nussbaum.fr    +33 (0)6 64 71 41 65 |
| homepage:        http://www-id.imag.fr/~nussbaum/ |

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Michael Chan]

On Thu, 2007-09-13 at 21:28 +0200, Lucas Nussbaum wrote:

> Erm, Wouldn't it be possible to print a warning when the driver loads,
> saying that the firmware is outdated ?

It's possible, but would require the driver to parse the version string.
The driver currently reports the version string for information and for
the human to parse it.

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Ingo Oeser]

Michael Chan schrieb:
> On Thu, 2007-09-13 at 21:28 +0200, Lucas Nussbaum wrote:
> > Erm, Wouldn't it be possible to print a warning when the driver loads,
> > saying that the firmware is outdated ?
> 
> It's possible, but would require the driver to parse the version string.
> The driver currently reports the version string for information and for
> the human to parse it.

Yes, but most humans don't know all valid firmware versions of their 
components.

Is it enough to parse the first number in the firmware via simple_strtoul()?

Then we could do this (pseudo-kernel-code :-)):


firmare_version = simple_strtoul(version_string, NULL, 0);

if (firmware_version < oldest_supported) {
	printk(KERN_WARNING "Please upgrade the firmware (you have %s, we need at least %d)\n", 
		version_string, oldest_supported);
} else if (firmware_version > newest_supported) {
	printk(KERN_INFO "Firmware version %s (latest supported: %d). You might need a newer driver.\n", 
		version_string, oldest_supported);
} else if (firmware_version != newest_supported) {
	dprintk("Firmware version %s (latest supported: %d). You might consider a firmware upgrade.\n", 
		version_string, oldest_supported);
}

Maybe that mechanism should be global somewhere? Having this kind of information available
would help system maintenance in heterogenous hardware environments.

Best regards

Ingo Oeser

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Michael Chan]

On Fri, 2007-09-14 at 10:14 +0200, Ingo Oeser wrote:
> Michael Chan schrieb:
> > On Thu, 2007-09-13 at 21:28 +0200, Lucas Nussbaum wrote:
> > > Erm, Wouldn't it be possible to print a warning when the driver loads,
> > > saying that the firmware is outdated ?
> > 
> > It's possible, but would require the driver to parse the version string.
> > The driver currently reports the version string for information and for
> > the human to parse it.
> 
> Yes, but most humans don't know all valid firmware versions of their 
> components.
> 
> Is it enough to parse the first number in the firmware via simple_strtoul()?

No, it's not.  We need to check the number after the '.' and possibly an
alphabet at the end as well.  Worse yet, the version may be different
for different chips.

We'll see if we can come up with a simple way to detect the problem.

Re: [BUG] tg3 cannot do PXE (loses MAC address) after soft reboot

[Ingo Oeser]

Michael Chan schrieb:
> On Fri, 2007-09-14 at 10:14 +0200, Ingo Oeser wrote:
> > Is it enough to parse the first number in the firmware via simple_strtoul()?
> 
> No, it's not.  We need to check the number after the '.' and possibly an
> alphabet at the end as well.  Worse yet, the version may be different
> for different chips.

Ah, I see. I thought the first number is some constantly increasing
internal version number from your SCM[1] and the other is the 
"marketing release number".

Thank you for explaining this. And that different version for different
chips issue make it really hard. I can see that now.
 
> We'll see if we can come up with a simple way to detect the problem.

That would help your users (like the customers of my employer).
Admins and system vendors will love you for that :-)


Best Regards

Ingo Oeser

[1] Source Control Management e.g. cvs, subversion, git