r8189 mac address changes persist across reboot (was: duplicate MAC

r8189 mac address changes persist across reboot (was: duplicate MAC

[Alan Jenkins]

On 8/23/09, marty <marty@goodoldmarty.com> wrote:
> Greg KH wrote:
>> > On Tue, Aug 18, 2009 at 04:27:04PM -0400, marty wrote:
>>> >> I got trouble...
>>> >> (duplicate MAC addresses)
>> >
>> > That's a bug in your hardware, have you asked your manufacturer to
>> > resolve this for you?  That violates the ethernet spec...
>
> I have resolved that problem as of today. I found this was caused
> by the software I had been using. If a hardware issue remains, it is moot.
>
> The bonding driver/utilities normally sets the bond address to the MAC of
> the
> first NIC. But it also set the MAC of the slave (eth3) to the MAC of the
> first
> NIC. This persists through reboots so that is how my MACs got duplicated.
>
> Resetting the MAC corrected those problems and everything works fine now.
>
> Marty B.

Hmm.  <Documentation/networking/bonding.txt>

"To restore your slaves' MAC addresses, you need to detach them
from the bond (`ifenslave -d bond0 eth0'). The bonding driver will
then restore the MAC addresses that the slaves had before they were
enslaved."

In which case one would hope that a clean shutdown would restore the
MAC addresses in the same way as ifenslave -d.

I found that snippet via Google, which pointed me to the 2.4 version
of the same document [1].  It suggests that your hardware or driver is
a little unusual.

"To restore your slaves' MAC addresses, you need to detach them
from the bond (`ifenslave -d bond0 eth0'), set them down
(`ifconfig eth0 down'), unload the drivers (`rmmod 3c59x', for
example) and reload them to get the MAC addresses from their
eeproms."

So other drivers would not have this problem, because they always
reset the MAC address at load time.  That would explain why the the
kernel doesn't bother resetting the MAC addresses on shutdown.

Looking at r8169.c confirms this.  It doesn't appear to initialize the
MAC address register from elsewhere; it just uses the current value.
It will also report this initial value as the "permanent" MAC address,
which your report suggests is wrong.  I think your problem is a bug in
r8169.

Francois, I found a datasheet for the 8139; it was claimed to be
similar and it does indeed appear so.  The datasheet suggests that the
driver needs to provoke "auto-load" from the EEPROM at load time.
Could you please have a look at fixing this, so that MAC address
changes do not persist over a reboot?

Thanks
Alan

"1. Features

Uses 93C46 (64*16-bit EEPROM) to store resource
                    configuration, ID parameter, and VPD data
...

5. Register Descriptions
The RTL8139D(L) provides the following set of operational registers
mapped into PCI memory space or I/O space.
     Offset       R/W                Tag
           Description
     0000h         R/W               IDR0            ID Register 0,
The ID register0-5 are only permitted to read/write by
                                                     4-byte access.
Read access can be byte, word, or double word access.
                                                     The initial value
is autoloaded from EEPROM EthernetID field.

...

5.9 9346CR: 93C46 Command Register
(Offset 0050h, R/W)
This register is used for issuing commands to the RTL8139D(L). These
commands are issued by setting the corresponding bits for the
function. A warm software reset along with individual reset and
enable/disable for transmitter and receiver are provided as well.

Bit R/W Symbol                          Description
7-6 R/W EEM1-0 Operating Mode: These 2 bits select the RTL8139D(L) operating
               mode.

                 EEM1    EEM0                      Operating Mode
                    0      0    Normal (RTL8139D(L) network/host communication
                                mode)
                    0      1    Auto-load: Entering this mode will make the
                                RTL8139D(L) load the contents of 93C46
like when the
                                RSTB signal is asserted. This
auto-load operation will
                                take about 2 ms. After it is
completed, the RTL8139D(L)
                                goes back to the normal mode
automatically (EEM1                                 EEM0 = 0) and all the other registers
are reset to default
                                values."




[1] Bonding mini-howto for 2.4 kernel
<http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/bonding.txt>

Re: r8189 mac address changes persist across reboot (was: duplicate

[Ivan Vecera]

Alan Jenkins napsal(a):
> On 8/23/09, marty <marty@goodoldmarty.com> wrote:
>> Greg KH wrote:
>>>> On Tue, Aug 18, 2009 at 04:27:04PM -0400, marty wrote:
>>>>>> I got trouble...
>>>>>> (duplicate MAC addresses)
>>>> That's a bug in your hardware, have you asked your manufacturer to
>>>> resolve this for you?  That violates the ethernet spec...
>> I have resolved that problem as of today. I found this was caused
>> by the software I had been using. If a hardware issue remains, it is moot.
>>
>> The bonding driver/utilities normally sets the bond address to the MAC of
>> the
>> first NIC. But it also set the MAC of the slave (eth3) to the MAC of the
>> first
>> NIC. This persists through reboots so that is how my MACs got duplicated.
>>
>> Resetting the MAC corrected those problems and everything works fine now.
>>
>> Marty B.
> 
> Hmm.  <Documentation/networking/bonding.txt>
> 
> "To restore your slaves' MAC addresses, you need to detach them
> from the bond (`ifenslave -d bond0 eth0'). The bonding driver will
> then restore the MAC addresses that the slaves had before they were
> enslaved."
> 
> In which case one would hope that a clean shutdown would restore the
> MAC addresses in the same way as ifenslave -d.
> 
> I found that snippet via Google, which pointed me to the 2.4 version
> of the same document [1].  It suggests that your hardware or driver is
> a little unusual.
> 
> "To restore your slaves' MAC addresses, you need to detach them
> from the bond (`ifenslave -d bond0 eth0'), set them down
> (`ifconfig eth0 down'), unload the drivers (`rmmod 3c59x', for
> example) and reload them to get the MAC addresses from their
> eeproms."
> 
> So other drivers would not have this problem, because they always
> reset the MAC address at load time.  That would explain why the the
> kernel doesn't bother resetting the MAC addresses on shutdown.
> 
> Looking at r8169.c confirms this.  It doesn't appear to initialize the
> MAC address register from elsewhere; it just uses the current value.
> It will also report this initial value as the "permanent" MAC address,
> which your report suggests is wrong.  I think your problem is a bug in
> r8169.
> 
> Francois, I found a datasheet for the 8139; it was claimed to be
> similar and it does indeed appear so.  The datasheet suggests that the
> driver needs to provoke "auto-load" from the EEPROM at load time.
> Could you please have a look at fixing this, so that MAC address
> changes do not persist over a reboot?
> 
Using auto-loading method is not possible for all new Realtek products
(mainly for PCI-E chipsets). I asked the Realtek engineers and this
is the answer:
Q: Is it possible to use HW register at offset 50h to reload the MAC
   address from EEPROM or somewhere else? I mean the usage of Auto-load
   mode (bits EEM1=0 and EEM0=1 in 50h HW reg).
A: Current new LANs don't load mac address throught autoload command.
   You need to read it out from external eeprom or internal efuse then
   put it back to ID0~ID5.

So you need to read the MAC address from the EEPROM and then write it into
ID0-ID5 registers. I already created the patch that initializes the MAC
address from EEPROM but there were some issues with this patch so it was
reverted. Mikael reported that the MAC address from its adapter (on Thecus
n2100) is read only partly (first 3 bytes were correct but the rest were
zeros). Later we found that the MAC address is correct and there are really
3 correct bytes + 3 zeros in EEPROM. The Thecus n2100 system probably uses
only these 3 bytes and the remaining 3 bytes fills in by itself (they are
probably stored somewhere in the firmware).

I have tested my patch with several different realtek NICs without any
problem but what should we do with embedded system like the n2100 that
initializes the MAC in other way.

There are 2 possibilities:
1) There could be an additional module param to enable/disable the MAC
   initialization
2) The MAC address read from EEPROM will be checked against the current
   MAC address in ID0-5 registers. And the current MAC will be replaced
   by the one from EEPROM only if the first 3 bytes (OUI part) are
   different.

Francois, what do you think about it?

Regards,
Ivan

Re: r8189 mac address changes persist across reboot (was: duplicate

[Alan Jenkins]

On 9/1/09, Ivan Vecera <ivecera@redhat.com> wrote:
> Alan Jenkins napsal(a):
>> On 8/23/09, marty <marty@goodoldmarty.com> wrote:
>>> Greg KH wrote:
>>>>> On Tue, Aug 18, 2009 at 04:27:04PM -0400, marty wrote:
>>>>>>> I got trouble...
>>>>>>> (duplicate MAC addresses)
>>>>> That's a bug in your hardware, have you asked your manufacturer to
>>>>> resolve this for you?  That violates the ethernet spec...
>>> I have resolved that problem as of today. I found this was caused
>>> by the software I had been using. If a hardware issue remains, it is
>>> moot.
>>>
>>> The bonding driver/utilities normally sets the bond address to the MAC of
>>> the
>>> first NIC. But it also set the MAC of the slave (eth3) to the MAC of the
>>> first
>>> NIC. This persists through reboots so that is how my MACs got duplicated.
>>>
>>> Resetting the MAC corrected those problems and everything works fine now.
>>>
>>> Marty B.
>>
>> Hmm.  <Documentation/networking/bonding.txt>
>>
>> "To restore your slaves' MAC addresses, you need to detach them
>> from the bond (`ifenslave -d bond0 eth0'). The bonding driver will
>> then restore the MAC addresses that the slaves had before they were
>> enslaved."
>>
>> In which case one would hope that a clean shutdown would restore the
>> MAC addresses in the same way as ifenslave -d.
>>
>> I found that snippet via Google, which pointed me to the 2.4 version
>> of the same document [1].  It suggests that your hardware or driver is
>> a little unusual.
>>
>> "To restore your slaves' MAC addresses, you need to detach them
>> from the bond (`ifenslave -d bond0 eth0'), set them down
>> (`ifconfig eth0 down'), unload the drivers (`rmmod 3c59x', for
>> example) and reload them to get the MAC addresses from their
>> eeproms."
>>
>> So other drivers would not have this problem, because they always
>> reset the MAC address at load time.  That would explain why the the
>> kernel doesn't bother resetting the MAC addresses on shutdown.
>>
>> Looking at r8169.c confirms this.  It doesn't appear to initialize the
>> MAC address register from elsewhere; it just uses the current value.
>> It will also report this initial value as the "permanent" MAC address,
>> which your report suggests is wrong.  I think your problem is a bug in
>> r8169.
>>
>> Francois, I found a datasheet for the 8139; it was claimed to be
>> similar and it does indeed appear so.  The datasheet suggests that the
>> driver needs to provoke "auto-load" from the EEPROM at load time.
>> Could you please have a look at fixing this, so that MAC address
>> changes do not persist over a reboot?
>>
> Using auto-loading method is not possible for all new Realtek products
> (mainly for PCI-E chipsets).

:-(

> I asked the Realtek engineers and this
> is the answer:
>
> Q: Is it possible to use HW register at offset 50h to reload the MAC
>    address from EEPROM or somewhere else? I mean the usage of Auto-load
>    mode (bits EEM1=0 and EEM0=1 in 50h HW reg).
> A: Current new LANs don't load mac address throught autoload command.
>    You need to read it out from external eeprom or internal efuse then
>    put it back to ID0~ID5.
>
> So you need to read the MAC address from the EEPROM and then write it into
> ID0-ID5 registers. I already created the patch that initializes the MAC
> address from EEPROM but there were some issues with this patch so it was
> reverted. Mikael reported that the MAC address from its adapter (on Thecus
> n2100) is read only partly (first 3 bytes were correct but the rest were
> zeros). Later we found that the MAC address is correct and there are really
> 3 correct bytes + 3 zeros in EEPROM. The Thecus n2100 system probably uses
> only these 3 bytes and the remaining 3 bytes fills in by itself (they are
> probably stored somewhere in the firmware).
>
> I have tested my patch with several different realtek NICs without any
> problem but what should we do with embedded system like the n2100 that
> initializes the MAC in other way.
>
> There are 2 possibilities:
> 1) There could be an additional module param to enable/disable the MAC
>    initialization
> 2) The MAC address read from EEPROM will be checked against the current
>    MAC address in ID0-5 registers. And the current MAC will be replaced
>    by the one from EEPROM only if the first 3 bytes (OUI part) are
>    different.
>
> Francois, what do you think about it?

I'm not sure about 2).  It won't help for the bonding case if you're
using two NICs with the same OUI.  On the other hand, If you change
the OUI, it won't work correctly on the n2100 system.

If you decide to enable 1) by default, here's my suggestion towards a
warning message when the MAC address is changed.   It can be omitted
when the current address already matches the EEPROM.

"Replaced initial MAC xx:xx:xx:xx:xx:xx with MAC xx:xx:xx:xx:xx:xx from EEPROM"
"If the initial MAC was intentionally set by embedded firmware, try <option>."
"Ignore this warning if the address is correct and you use MAC
changing software (e.g. bonding or bridging)"

Alan

Re: r8189 mac address changes persist across reboot (was:

[Sergey Vlasov]

[-- Attachment #1: Type: text/plain, Size: 3231 bytes --]

On Tue, Sep 01, 2009 at 09:58:59AM +0200, Ivan Vecera wrote:
> Alan Jenkins napsal(a):
[...]
> > Looking at r8169.c confirms this.  It doesn't appear to initialize the
> > MAC address register from elsewhere; it just uses the current value.
> > It will also report this initial value as the "permanent" MAC address,
> > which your report suggests is wrong.  I think your problem is a bug in
> > r8169.
> > 
> > Francois, I found a datasheet for the 8139; it was claimed to be
> > similar and it does indeed appear so.  The datasheet suggests that the
> > driver needs to provoke "auto-load" from the EEPROM at load time.
> > Could you please have a look at fixing this, so that MAC address
> > changes do not persist over a reboot?
> > 
> Using auto-loading method is not possible for all new Realtek products
> (mainly for PCI-E chipsets). I asked the Realtek engineers and this
> is the answer:
> Q: Is it possible to use HW register at offset 50h to reload the MAC
>    address from EEPROM or somewhere else? I mean the usage of Auto-load
>    mode (bits EEM1=0 and EEM0=1 in 50h HW reg).
> A: Current new LANs don't load mac address throught autoload command.
>    You need to read it out from external eeprom or internal efuse then
>    put it back to ID0~ID5.
> 
> So you need to read the MAC address from the EEPROM and then write it into
> ID0-ID5 registers. I already created the patch that initializes the MAC
> address from EEPROM but there were some issues with this patch so it was
> reverted. Mikael reported that the MAC address from its adapter (on Thecus
> n2100) is read only partly (first 3 bytes were correct but the rest were
> zeros). Later we found that the MAC address is correct and there are really
> 3 correct bytes + 3 zeros in EEPROM. The Thecus n2100 system probably uses
> only these 3 bytes and the remaining 3 bytes fills in by itself (they are
> probably stored somewhere in the firmware).

Unfortunately, this kind of crap (crucial information such as MAC
addresses stored in places known only to some proprietary firmware) is
too common with recent devices (e.g., forcedeth has the same problem
even on PCs).

> I have tested my patch with several different realtek NICs without any
> problem but what should we do with embedded system like the n2100 that
> initializes the MAC in other way.
> 
> There are 2 possibilities:
> 1) There could be an additional module param to enable/disable the MAC
>    initialization
> 2) The MAC address read from EEPROM will be checked against the current
>    MAC address in ID0-5 registers. And the current MAC will be replaced
>    by the one from EEPROM only if the first 3 bytes (OUI part) are
>    different.

 3) Try the same solution as forcedeth does - save original contents
    of MAC address registers in rtl8169_init_one() (we already have
    perm_addr available for this; forcedeth uses a separate variable
    due to its "reversed MAC" brain damage), then restore MAC to its
    initial state in rtl8169_remove_one() (to handle module reload)
    and rtl_shutdown() (for soft reboot or kexec), and hope that on an
    unexpected hard reboot the firmware will reinit the chip properly.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]