Realtek 8168D: no active link (2.6.29.2)

Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi

I am currently testing a Gigabyte EX58-UD4P mainboard which is equipped with
an onboard Realtek RTL-8168D network controller.  The kernel version is
2.6.29.2.  The r8169 module is loaded correctly, identifies the chip as an
8168D and seemingly is happy.  However, after loading the module the link
remains dead.  The green and orange status LEDs remain off at all times.

It doesn't seem to matter whether the remote end of the network line is
plugged into a switch when the computer is booted.  Repeatedly removing and
inserting the r8169 module also doesn't help.  In short, it seems impossible
to get a functioning network link with the 8168D.

Support for the 8168D appears relatively recent given related patch activity
around October 2008, so I'm guessing it's still a work in progress.  I would
be happy to test patches and run other tests if it will help make the
8168D-based chipsets work better.  Alternatively, if there's a known
workaround to get the 8168D working now I would be interested to try it.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Francois Romieu]

Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
[...]
> I am currently testing a Gigabyte EX58-UD4P mainboard which is equipped with
> an onboard Realtek RTL-8168D network controller.  The kernel version is
> 2.6.29.2.  The r8169 module is loaded correctly, identifies the chip as an
> 8168D and seemingly is happy.  However, after loading the module the link
> remains dead.  The green and orange status LEDs remain off at all times.

Which XID does it display in dmesg when the module is inserted and which
behavior do you experience with version 8.011.00 of Realtek's r8168 driver ?

-- 
Ueimor

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi 

> [...]
> > I am currently testing a Gigabyte EX58-UD4P mainboard which is equipped with
> > an onboard Realtek RTL-8168D network controller.  The kernel version is
> > 2.6.29.2.  The r8169 module is loaded correctly, identifies the chip as an
> > 8168D and seemingly is happy.  However, after loading the module the link
> > remains dead.  The green and orange status LEDs remain off at all times.
> 
> Which XID does it display in dmesg when the module is inserted and which
> behavior do you experience with version 8.011.00 of Realtek's r8168 driver ?

Ok, FYI the machine concerned is now running 2.6.30-rc5.  As far as I can
tell there is no change with this kernel version (I will double-check in a
few moments time and follow up if there are changes).  On initialisation the
2.6.30-r5 r8169 driver gives

  r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
  r8169 0000:08:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
  r8169 0000:08:00.0: setting latency timer to 64
  r8169 0000:08:00.0: irq 33 for MSI/MSI-X
  eth1: RTL8168d/8111d at 0xf91be000, 00:24:1d:15:5d:a9, XID 281000c0 IRQ 33

I then downloaded Realtek's 8.012.00 driver, it being the latest release.
Some internal API changes in 2.6.30-rc mean that this doesn't compile, but
it was trivial to fix these (for the benefit of those trawling the archives
the diff is at the end of this email).  With this done I get this when the
module is inserted (having removed the previously loaded r8169 driver):

  r8168 0000:08:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
  r8168 0000:08:00.0: setting latency timer to 64
  r8168 0000:08:00.0: irq 33 for MSI/MSI-X
  eth1 (r8168): not using net_device_ops yet
  eth1: Identified chip type is 'RTL8168D/8111D'.
  eth1: RTL8168B/8111B at 0xfcb1e000, 00:24:1d:15:5d:a9, IRQ 33
  r8168: eth1: link down
  r8168: eth1: link up

The "link up" occurred as soon as I plugged a live network cable into the
card.  Under the kernel's r8169 driver I never saw any "link up" messages.

Regards
  jonathan

diff -ruP r8168-8.012.00-orig/src/r8168.h r8168-8.012.00-new/src/r8168.h
--- r8168-8.012.00-orig/src/r8168.h	2009-04-20 17:37:23.000000000 +0930
+++ r8168-8.012.00-new/src/r8168.h	2009-05-14 20:57:45.166102000 +0930
@@ -37,11 +37,13 @@
 #define CHECKSUM_PARTIAL CHECKSUM_HW
 #endif
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,30)
 #ifndef IRQ_HANDLED
 #define irqreturn_t void
 #define IRQ_HANDLED
 #define IRQ_NONE
 #endif
+#endif
 
 #ifndef HAVE_FREE_NETDEV
 #define free_netdev(x)	kfree(x)
@@ -255,10 +257,14 @@
 	#define RTL_NETIF_RX_COMPLETE(dev, napi)		netif_rx_complete(dev, napi)
 	#define RTL_NETIF_RX_SCHEDULE_PREP(dev, napi)		netif_rx_schedule_prep(dev, napi)
 	#define __RTL_NETIF_RX_SCHEDULE(dev, napi)		__netif_rx_schedule(dev, napi)
- #else
+ #elif LINUX_VERSION_CODE == KERNEL_VERSION(2,6,29)
 	#define RTL_NETIF_RX_COMPLETE(dev, napi)		netif_rx_complete(napi)
 	#define RTL_NETIF_RX_SCHEDULE_PREP(dev, napi)		netif_rx_schedule_prep(napi)
 	#define __RTL_NETIF_RX_SCHEDULE(dev, napi)		__netif_rx_schedule(napi)
+ #else
+	#define RTL_NETIF_RX_COMPLETE(dev, napi)		napi_complete(napi)
+	#define RTL_NETIF_RX_SCHEDULE_PREP(dev, napi)		napi_schedule_prep(napi)
+	#define __RTL_NETIF_RX_SCHEDULE(dev, napi)		__napi_schedule(napi)
  #endif
 	#define RTL_NAPI_RETURN_VALUE work_done
 	#define RTL_NAPI_ENABLE(dev, napi)			napi_enable(napi)

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi again

> Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
> [...]
> > I am currently testing a Gigabyte EX58-UD4P mainboard which is equipped with
> > an onboard Realtek RTL-8168D network controller.  The kernel version is
> > 2.6.29.2.  The r8169 module is loaded correctly, identifies the chip as an
> > 8168D and seemingly is happy.  However, after loading the module the link
> > remains dead.  The green and orange status LEDs remain off at all times.
> 
> Which XID does it display in dmesg when the module is inserted and which
> behavior do you experience with version 8.011.00 of Realtek's r8168 driver ?

Following up on my previous email, it seems that the stock kernel r8169
driver is now working with the card in this machine ever since I tried the
Realtek driver version 8.012.00.  It's almost as if the Realtek driver
"unlocked" something on the chip (eg: activated the PHY), the status of
which is remembered across reboots and hard powercycles.

For instance, before using the Realtek driver the green "link" LED would
never light - not during BIOS POST, Linux boot or after the respective
ethernet interface was brought up.  Now, having used the Realtek driver
once, the green "link" LED lights even during BIOS POST.  I then tested this
by removing power from the machine completely for about a minute.  When
power was restored and the PC was turned on, the green "link" LED came on
almost immediately.

Note that all this is with 2.6.30-rc5.

I'll test again tomorrow after the machine has been off for several hours.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[David Dillow]

On Thu, 2009-05-14 at 21:38 +0930, Jonathan Woithe wrote:
> Hi again
>
> Following up on my previous email, it seems that the stock kernel r8169
> driver is now working with the card in this machine ever since I tried the
> Realtek driver version 8.012.00.  It's almost as if the Realtek driver
> "unlocked" something on the chip (eg: activated the PHY), the status of
> which is remembered across reboots and hard powercycles.

It's worth noting that a power cycle may not clear everything out, as
there is likely standby power being supplied even with you have the
machine off. You can be sure by disconnecting the mains power to the
power supply for 30 seconds or so -- I've seen some motherboards with
large caps that take a while to drain.

It would be interesting to see if your problems come back after doing
this. Also, if you have another machine on the gigabit network, it would
be interesting to do 'nc -l 9000 > /dev/null' and 'nc target 9000
< /dev/zero' on the other and see if a ping between the machine
eventually stops working. This reliably hangs my 8168d card at home, but
it comes back with an ifdown/ifup sequence.

Dave

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi

> On Thu, 2009-05-14 at 21:38 +0930, Jonathan Woithe wrote:
> > Hi again
> >
> > Following up on my previous email, it seems that the stock kernel r8169
> > driver is now working with the card in this machine ever since I tried the
> > Realtek driver version 8.012.00.  It's almost as if the Realtek driver
> > "unlocked" something on the chip (eg: activated the PHY), the status of
> > which is remembered across reboots and hard powercycles.
> 
> It's worth noting that a power cycle may not clear everything out, as
> there is likely standby power being supplied even with you have the
> machine off.

Absolutely, that's why I went on to say that I did what I call a "hard power
cycle", which involves removing mains power from the PC for a minute or so. 
As previously advised, the adapter kept working even after this.

> You can be sure by disconnecting the mains power to the power supply for
> 30 seconds or so -- I've seen some motherboards with large caps that take
> a while to drain.

Indeed.  To follow up on this, as per my earlier email I left the machine
disconnected from the mains all night - so it was truly "off" for about 9
hours.  Just before coming to work I flicked it on and the "link" LED lit
almost immediately.  So whatever the Realtek driver did, the resulting
status would appear to be stored in some kind of non-volatile memory.  While
it would appear that this has worked around the problem for me, it would be
nice if the mainline driver could do this itself.

> It would be interesting to see if your problems come back after doing
> this.

As per my previous email and the above extended test, the problems do not
seem to return - there doesn't appear to be any way to make the problem
return in fact (at least under Linux).

> Also, if you have another machine on the gigabit network, it would
> be interesting to do 'nc -l 9000 > /dev/null' and 'nc target 9000
> < /dev/zero' on the other and see if a ping between the machine
> eventually stops working. This reliably hangs my 8168d card at home, but
> it comes back with an ifdown/ifup sequence.

This is obviously a separate issue but I will test this when I get home
tonight and report back.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Francois Romieu]

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]

Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
[...]
> Indeed.  To follow up on this, as per my earlier email I left the machine
> disconnected from the mains all night - so it was truly "off" for about 9
> hours.  Just before coming to work I flicked it on and the "link" LED lit
> almost immediately.  So whatever the Realtek driver did, the resulting
> status would appear to be stored in some kind of non-volatile memory.  While
> it would appear that this has worked around the problem for me, it would be
> nice if the mainline driver could do this itself.

The attached patch is not complete but can you give it a try and check if
it makes a difference ?

Thanks in advance.

-- 
Ueimor

[-- Attachment #2: r8169-8168d.patch.gz --]
[-- Type: application/x-gzip, Size: 4045 bytes --]

Re: Realtek 8168D: no active link (2.6.29.2)

[David Dillow]

On Sun, 2009-05-17 at 23:57 +0200, Francois Romieu wrote:
> Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
> [...]
> > Indeed.  To follow up on this, as per my earlier email I left the machine
> > disconnected from the mains all night - so it was truly "off" for about 9
> > hours.  Just before coming to work I flicked it on and the "link" LED lit
> > almost immediately.  So whatever the Realtek driver did, the resulting
> > status would appear to be stored in some kind of non-volatile memory.  While
> > it would appear that this has worked around the problem for me, it would be
> > nice if the mainline driver could do this itself.
> 
> The attached patch is not complete but can you give it a try and check if
> it makes a difference ?
> 
> Thanks in advance.

While it looks like Jonathan and I are chasing two different problems,
would your patch make any difference to my XID 281000c0 locking up? It
looks like the init sequence for MAC_VER_25 changed, but I don't know if
it would help my issue.

Dave

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

> Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
> [...]
> > Indeed.  To follow up on this, as per my earlier email I left the machine
> > disconnected from the mains all night - so it was truly "off" for about 9
> > hours.  Just before coming to work I flicked it on and the "link" LED lit
> > almost immediately.  So whatever the Realtek driver did, the resulting
> > status would appear to be stored in some kind of non-volatile memory.  While
> > it would appear that this has worked around the problem for me, it would be
> > nice if the mainline driver could do this itself.
> 
> The attached patch is not complete but can you give it a try and check if
> it makes a difference ?

Will do.  Given that at present the chip appears to be bringing up the link
more or less as soon as it's powered up (having run the Realtek's driver
once last week), what behavioural changes should I see?  The patch appears
to change the initialisation sequence which may bring the link to life
without having to run the driver from Realtek.  It doesn't appear to change
the shutdown behaviour though, so presumedly the link status in the PHY will
remain.  Since my chip is already set to have a live link (and that this is
persistent across power cycles), what changes might I look out for?

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Francois Romieu]

Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
[...]
> Will do.  Given that at present the chip appears to be bringing up the link
> more or less as soon as it's powered up (having run the Realtek's driver
> once last week), what behavioural changes should I see ?

A complete lack of regression would be a good start. :o)

-- 
Ueimor

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi 

> Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
> [...]
> > Will do.  Given that at present the chip appears to be bringing up the link
> > more or less as soon as it's powered up (having run the Realtek's driver
> > once last week), what behavioural changes should I see ?
> 
> A complete lack of regression would be a good start. :o)

That's hard for me to judge since for me it's never worked.  Anyway, as
reported in an earlier post to the list, the patch you sent seems to send
the PHY into 1 Gbps mode.  Realtek's driver made the interface come alive
but only at 100 Mbps.  So in that respect the patch seems good.

After sending the post last night I tested the behaviour across a hard power
cycle.  In short, the interface seems to come up in 100 Mbps mode on power
up, and then switches to 1 Gbps once the patched Linux 8169 driver loads. 
So it seems Realtek's driver has a side effect of storing some kind of
default mode into the device.  The patch however does seem to allow Linux to
gain control over this and set things up to its liking.

As I mentioned, if there are other tests you want me to do just drop me a
line.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Francois Romieu]

Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
[...]
> As I mentioned, if there are other tests you want me to do just drop me a
> line.

You may see if you can force the link settings with the patches #2 and #3 at
http://userweb.kernel.org/~romieu/r8169/2.6.30-rc6/20090519/

Thanks.

-- 
Ueimor

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi

> Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
> [...]
> > As I mentioned, if there are other tests you want me to do just drop me a
> > line.
> 
> You may see if you can force the link settings with the patches #2 and #3 at
> http://userweb.kernel.org/~romieu/r8169/2.6.30-rc6/20090519/

Sorry I haven't gotten to this yet - been caught up with other things.  I
hope to get to it over the weekend some time.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi

Sorry for the delay in getting this done.

> Jonathan Woithe <jwoithe@physics.adelaide.edu.au> :
> [...]
> > As I mentioned, if there are other tests you want me to do just drop me a
> > line.
> 
> You may see if you can force the link settings with the patches #2 and #3 at
> http://userweb.kernel.org/~romieu/r8169/2.6.30-rc6/20090519/

As far as I can tell, with these 3 patches applied against 2.6.30-rc6 I can
force the link settings:

  ethtool -s eth0 autoneg off speed 10
    - 10 Mbps speed set

  ethtool -s eth0 autoneg off speed 100
    - 100 Mbps speed achieved

  ethtool -s eth0 autoneg on speed 1000
    - 1000 Mbps speed attained

The duplex setting might not be having an effect though since my switch's
FDx (full duplex) LED was extinguished for both the 10 and 100 Mbps cases
above.  Explicitly setting the duplex flag using

  ethtool -s eth0 autoneg off speed 100 duplex full

didn't make any difference - the FDx LED remained off.  For completeness I
also tried

  ethtool -s eth0 autoneg off speed 100 duplex half

and this caused no change compared to the "duplex full" case above. 
Restoring autoneg made the FDx LED return (along with 1000 Mbps speed).

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Benny Amorsen]

Jonathan Woithe <jwoithe@physics.adelaide.edu.au> writes:

>   ethtool -s eth0 autoneg off speed 10
>     - 10 Mbps speed set
>
>   ethtool -s eth0 autoneg off speed 100
>     - 100 Mbps speed achieved
>
>   ethtool -s eth0 autoneg on speed 1000
>     - 1000 Mbps speed attained
>
> The duplex setting might not be having an effect though since my switch's
> FDx (full duplex) LED was extinguished for both the 10 and 100 Mbps cases
> above.  Explicitly setting the duplex flag using
>
>   ethtool -s eth0 autoneg off speed 100 duplex full
>
> didn't make any difference - the FDx LED remained off.

You turned auto negotiation off, and you are surprised that the switch
doesn't autonegotiate full duplex?


/Benny

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

> >   ethtool -s eth0 autoneg off speed 10
> >     - 10 Mbps speed set
> >
> >   ethtool -s eth0 autoneg off speed 100
> >     - 100 Mbps speed achieved
> >
> >   ethtool -s eth0 autoneg on speed 1000
> >     - 1000 Mbps speed attained
> >
> > The duplex setting might not be having an effect though since my switch's
> > FDx (full duplex) LED was extinguished for both the 10 and 100 Mbps cases
> > above.  Explicitly setting the duplex flag using
> >
> >   ethtool -s eth0 autoneg off speed 100 duplex full
> >
> > didn't make any difference - the FDx LED remained off.
> 
> You turned auto negotiation off, and you are surprised that the switch
> doesn't autonegotiate full duplex?

I'm not surprised one way or the other since I have no idea what to expect
when doing these tests on this card with the patches applied.  Given that
ethtool has a "duplex" option I naively thought that it controlled the
card's duplex setting.  I interprete your comment to mean that duplex can't
be controlled like this and it requires autonegotiation in order to active
full duplex mode.  If that's the case then so be it - I'm only reporting
what I did and what effect it had on the card.

FYI, turning off autonegotiation was the only way I had of forcing the
card's speed.  If autoneg was left on the "speed 10" and "speed 100" options
made no difference - the card would always configure for 1000 Mbps.

Furthermore, in order to attain 1000 Mbps it was necessary to turn autoneg
on - with it off, "speed 1000" would result in 100 Mbps only.  However, I
seem to recall reading somewhere that 1000 Mbps required autoneg so I'm not
all that surprised with that observation.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Benny Amorsen]

tor, 28 05 2009 kl. 09:39 +0930, skrev Jonathan Woithe:

> I'm not surprised one way or the other since I have no idea what to expect
> when doing these tests on this card with the patches applied.  Given that
> ethtool has a "duplex" option I naively thought that it controlled the
> card's duplex setting.

It does. The card is set to full duplex when you force full duplex.

> I interprete your comment to mean that duplex can't
> be controlled like this and it requires autonegotiation in order to active
> full duplex mode.  If that's the case then so be it - I'm only reporting
> what I did and what effect it had on the card.

No, you're reporting which effect it had on the SWITCH. The switch
detected that autonegotiation failed, so it picked half duplex mode.
Card full duplex, switch half duplex, recipe for disaster.

> FYI, turning off autonegotiation was the only way I had of forcing the
> card's speed.  If autoneg was left on the "speed 10" and "speed 100" options
> made no difference - the card would always configure for 1000 Mbps.

Correct. If you want to influence autonegotiation, you need to do
something like:

ethtool -s eth1 advertise 0x00F

This allows 10Mbps and 100Mbps with half or full duplex, but nothing
faster.

> Furthermore, in order to attain 1000 Mbps it was necessary to turn autoneg
> on - with it off, "speed 1000" would result in 100 Mbps only.  However, I
> seem to recall reading somewhere that 1000 Mbps required autoneg so I'm not
> all that surprised with that observation.

That is correct.

I can recommend man ethtool.


/Benny

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

> > I interprete your comment to mean that duplex can't
> > be controlled like this and it requires autonegotiation in order to active
> > full duplex mode.  If that's the case then so be it - I'm only reporting
> > what I did and what effect it had on the card.
> 
> No, you're reporting which effect it had on the SWITCH. The switch
> detected that autonegotiation failed, so it picked half duplex mode.
> Card full duplex, switch half duplex, recipe for disaster.

<shrug> I'm reporting what I could observe.  Whether that's the card, the
switch or the combination of both is for others more knowledgable in these
matters to determine.

So, all this aside, were the previously reported results useful for the
purposes they were requested for?  Or do I need to run additional tests?  If
so, what tests, exactly (since I'm not sure exactly what it is we are aiming
to test here)?

> I can recommend man ethtool.

Yes, if time allowed I would have gone through it earlier.  When I get some
spare time I'll check it out.

Regards
  jonathan

Re: Realtek 8168D: no active link (2.6.29.2)

[Jonathan Woithe]

Hi Ueimor

> > Indeed.  To follow up on this, as per my earlier email I left the machine
> > disconnected from the mains all night - so it was truly "off" for about 9
> > hours.  Just before coming to work I flicked it on and the "link" LED lit
> > almost immediately.  So whatever the Realtek driver did, the resulting
> > status would appear to be stored in some kind of non-volatile memory.  While
> > it would appear that this has worked around the problem for me, it would be
> > nice if the mainline driver could do this itself.
> 
> The attached patch is not complete but can you give it a try and check if
> it makes a difference ?

Ok, I've tried this against 2.6.30-rc5.  The practical upshot is that it has
had a positive effect.  It turns out that the Realtek driver had enabled the
interface only at 100 Mbps (it's a new mainboard and I didn't fully
appreciate the significance of the speed LED colour).  This patch allows
Linux to set the PHY to 1 Gbps.  The interface itself has had limited
testing tonight, but first impressions are excellent.  Thanks for this.

If you need further patches tested just drop me a line.

Regards
  jonathan