sun7i-dwmac: link detection failure with 1000Mbit parters

sun7i-dwmac: link detection failure with 1000Mbit parters

[Priit Laes]

Heya!

I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
detection fails consistently with certain 1000Mbit partners (for example Huawei
B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
but the same hardware works properly with certain other link partners (100Mbit GL AR150
for example).

(Just need to test with another 1000Mbit switch at the office).

I first thought it could be a regression, but I went from current master to as far back
as 5.2.0-rc6 where it was still broken.

Failure is basically following:

[   10.971485] sun7i-dwmac 1c50000.ethernet eth0: PHY [stmmac-0:01] driver [Generic PHY]
[   10.980841] sun7i-dwmac 1c50000.ethernet eth0: No Safety Features support found
[   10.988291] sun7i-dwmac 1c50000.ethernet eth0: RX IPC Checksum Offload disabled
[   10.995694] sun7i-dwmac 1c50000.ethernet eth0: No MAC Management Counters available
[   11.003381] sun7i-dwmac 1c50000.ethernet eth0: PTP not supported by HW
[   11.009927] sun7i-dwmac 1c50000.ethernet eth0: configuring for phy/rgmii link mode
... link and activity leds go blank ...
... remove and replug and link is detected again ...
[   19.371894] sun7i-dwmac 1c50000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

Ethtool output in case link is down:
[snip]
	Supported ports: [ TP MII ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Half 1000baseT/Full 
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Half 1000baseT/Full 
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: Unknown!
	Duplex: Unknown! (255)
	Port: MII
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: d
	Wake-on: d
	Current message level: 0x0000003f (63)
			       drv probe link timer ifdown ifup
	Link detected: no
[/snip]

And ethtool output in case cable is removed and replugged:
[snip]
ethtool eth0
Settings for eth0:
	...cut...
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
	                                     100baseT/Half 100baseT/Full 
	                                     1000baseT/Full 
	Link partner advertised pause frame use: Symmetric
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 1000Mb/s
	Duplex: Full
	Port: MII
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: d
	Wake-on: d
	Current message level: 0x0000003f (63)
			       drv probe link timer ifdown ifup
	Link detected: yes
[/snip]


With 100Mbit link partner (GL Inet AR150), the link is pulled up almost
immediately:
[   15.531754] sun7i-dwmac 1c50000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx

[snip]
Settings for eth0:
	... cut ...
	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
	                                     100baseT/Half 100baseT/Full 
	Link partner advertised pause frame use: Symmetric Receive-only
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 100Mb/s
	Duplex: Full
	Port: MII
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: d
	Wake-on: d
	Current message level: 0x0000003f (63)
			       drv probe link timer ifdown ifup
	Link detected: yes
[/snip]

RE: sun7i-dwmac: link detection failure with 1000Mbit parters

[Jose Abreu]

++ Florian, Andrew, Heiner, Russell

Can you please attach your dmesg log ? PHYLINK provides some useful 
debug logs.

From: Priit Laes <plaes@plaes.org>
Date: Oct/30/2019, 20:21:17 (UTC+00:00)

> Heya!
> 
> I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
> detection fails consistently with certain 1000Mbit partners (for example Huawei
> B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
> but the same hardware works properly with certain other link partners (100Mbit GL AR150
> for example).
> 
> (Just need to test with another 1000Mbit switch at the office).
> 
> I first thought it could be a regression, but I went from current master to as far back
> as 5.2.0-rc6 where it was still broken.
> 
> Failure is basically following:
> 
> [   10.971485] sun7i-dwmac 1c50000.ethernet eth0: PHY [stmmac-0:01] driver [Generic PHY]
> [   10.980841] sun7i-dwmac 1c50000.ethernet eth0: No Safety Features support found
> [   10.988291] sun7i-dwmac 1c50000.ethernet eth0: RX IPC Checksum Offload disabled
> [   10.995694] sun7i-dwmac 1c50000.ethernet eth0: No MAC Management Counters available
> [   11.003381] sun7i-dwmac 1c50000.ethernet eth0: PTP not supported by HW
> [   11.009927] sun7i-dwmac 1c50000.ethernet eth0: configuring for phy/rgmii link mode
> ... link and activity leds go blank ...
> ... remove and replug and link is detected again ...
> [   19.371894] sun7i-dwmac 1c50000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
> 
> Ethtool output in case link is down:
> [snip]
> 	Supported ports: [ TP MII ]
> 	Supported link modes:   10baseT/Half 10baseT/Full 
> 	                        100baseT/Half 100baseT/Full 
> 	                        1000baseT/Half 1000baseT/Full 
> 	Supported pause frame use: Symmetric Receive-only
> 	Supports auto-negotiation: Yes
> 	Supported FEC modes: Not reported
> 	Advertised link modes:  10baseT/Half 10baseT/Full 
> 	                        100baseT/Half 100baseT/Full 
> 	                        1000baseT/Half 1000baseT/Full 
> 	Advertised pause frame use: Symmetric Receive-only
> 	Advertised auto-negotiation: Yes
> 	Advertised FEC modes: Not reported
> 	Speed: Unknown!
> 	Duplex: Unknown! (255)
> 	Port: MII
> 	PHYAD: 1
> 	Transceiver: internal
> 	Auto-negotiation: on
> 	Supports Wake-on: d
> 	Wake-on: d
> 	Current message level: 0x0000003f (63)
> 			       drv probe link timer ifdown ifup
> 	Link detected: no
> [/snip]
> 
> And ethtool output in case cable is removed and replugged:
> [snip]
> ethtool eth0
> Settings for eth0:
> 	...cut...
> 	Advertised pause frame use: Symmetric Receive-only
> 	Advertised auto-negotiation: Yes
> 	Advertised FEC modes: Not reported
> 	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
> 	                                     100baseT/Half 100baseT/Full 
> 	                                     1000baseT/Full 
> 	Link partner advertised pause frame use: Symmetric
> 	Link partner advertised auto-negotiation: Yes
> 	Link partner advertised FEC modes: Not reported
> 	Speed: 1000Mb/s
> 	Duplex: Full
> 	Port: MII
> 	PHYAD: 1
> 	Transceiver: internal
> 	Auto-negotiation: on
> 	Supports Wake-on: d
> 	Wake-on: d
> 	Current message level: 0x0000003f (63)
> 			       drv probe link timer ifdown ifup
> 	Link detected: yes
> [/snip]
> 
> 
> With 100Mbit link partner (GL Inet AR150), the link is pulled up almost
> immediately:
> [   15.531754] sun7i-dwmac 1c50000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
> 
> [snip]
> Settings for eth0:
> 	... cut ...
> 	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
> 	                                     100baseT/Half 100baseT/Full 
> 	Link partner advertised pause frame use: Symmetric Receive-only
> 	Link partner advertised auto-negotiation: Yes
> 	Link partner advertised FEC modes: Not reported
> 	Speed: 100Mb/s
> 	Duplex: Full
> 	Port: MII
> 	PHYAD: 1
> 	Transceiver: internal
> 	Auto-negotiation: on
> 	Supports Wake-on: d
> 	Wake-on: d
> 	Current message level: 0x0000003f (63)
> 			       drv probe link timer ifdown ifup
> 	Link detected: yes
> [/snip]


---
Thanks,
Jose Miguel Abreu

Re: sun7i-dwmac: link detection failure with 1000Mbit parters

[Russell King - ARM Linux admin]

On Thu, Oct 31, 2019 at 08:39:06AM +0000, Jose Abreu wrote:
> ++ Florian, Andrew, Heiner, Russell
> 
> Can you please attach your dmesg log ? PHYLINK provides some useful 
> debug logs.
> 
> From: Priit Laes <plaes@plaes.org>
> Date: Oct/30/2019, 20:21:17 (UTC+00:00)
> 
> > Heya!
> > 
> > I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
> > detection fails consistently with certain 1000Mbit partners (for example Huawei
> > B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
> > but the same hardware works properly with certain other link partners (100Mbit GL AR150
> > for example).
> > 
> > (Just need to test with another 1000Mbit switch at the office).
> > 
> > I first thought it could be a regression, but I went from current master to as far back
> > as 5.2.0-rc6 where it was still broken.

The stmmac conversion to phylink was v5.3-rc1, so that's likely not the
issue if v5.2-rc6 also exhibits this behaviour.

My guess is that the problem lies in phylib, especially as the link LEDs
go off when the link is configured.  I notice that it's using the
generic PHY driver rather than a specific driver.

  mii-diag -v eth0

would be useful to see for the case where the link has failed, without
replugging the ethernet cable.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

Re: sun7i-dwmac: link detection failure with 1000Mbit parters

[Priit Laes]

On Thu, Oct 31, 2019 at 10:38:41AM +0000, Russell King - ARM Linux admin wrote:
> On Thu, Oct 31, 2019 at 08:39:06AM +0000, Jose Abreu wrote:
> > ++ Florian, Andrew, Heiner, Russell
> > 
> > Can you please attach your dmesg log ? PHYLINK provides some useful 
> > debug logs.
> > 
> > From: Priit Laes <plaes@plaes.org>
> > Date: Oct/30/2019, 20:21:17 (UTC+00:00)
> > 
> > > Heya!
> > > 
> > > I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
> > > detection fails consistently with certain 1000Mbit partners (for example Huawei
> > > B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
> > > but the same hardware works properly with certain other link partners (100Mbit GL AR150
> > > for example).
> > > 
> > > (Just need to test with another 1000Mbit switch at the office).
> > > 
> > > I first thought it could be a regression, but I went from current master to as far back
> > > as 5.2.0-rc6 where it was still broken.
> 
> The stmmac conversion to phylink was v5.3-rc1, so that's likely not the
> issue if v5.2-rc6 also exhibits this behaviour.
> 
> My guess is that the problem lies in phylib, especially as the link LEDs
> go off when the link is configured.  I notice that it's using the
> generic PHY driver rather than a specific driver.

Yup, it turned out to be a phy-related issue - I was using generic PHY
driver, but the board is using Micrel KSZ9031 which has some quirks that
MICREL_PHY=y managed to work around.

> 
>   mii-diag -v eth0
> 
> would be useful to see for the case where the link has failed, without
> replugging the ethernet cable.

mii-diag seems to be quite an useful tool, but unfortunately has not been
packaged anymore on newer distro releases like Debian stable and latest
Ubuntu LTS.

> 
> -- 
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> According to speedtest.net: 11.9Mbps down 500kbps up

Re: sun7i-dwmac: link detection failure with 1000Mbit parters

[Russell King - ARM Linux admin]

On Fri, Nov 01, 2019 at 09:49:20AM +0000, Priit Laes wrote:
> On Thu, Oct 31, 2019 at 10:38:41AM +0000, Russell King - ARM Linux admin wrote:
> >   mii-diag -v eth0
> > 
> > would be useful to see for the case where the link has failed, without
> > replugging the ethernet cable.
> 
> mii-diag seems to be quite an useful tool, but unfortunately has not been
> packaged anymore on newer distro releases like Debian stable and latest
> Ubuntu LTS.

which is unfortunate because it's the best tool for debugging PHY
related problems.  mii-tool doesn't dump registers, and dumping the
PHY registers is extremely useful.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

Re: sun7i-dwmac: link detection failure with 1000Mbit parters

[Andrew Lunn]

On Wed, Oct 30, 2019 at 08:21:17PM +0000, Priit Laes wrote:
> Heya!
> 
> I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
> detection fails consistently with certain 1000Mbit partners (for example Huawei
> B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
> but the same hardware works properly with certain other link partners (100Mbit GL AR150
> for example).

Hi Pritt

What PHY is used? And what happens if you use the specific PHY driver,
not the generic PHY driver?

    Andrew

Re: sun7i-dwmac: link detection failure with 1000Mbit parters

[Andrew Lunn]

On Thu, Oct 31, 2019 at 02:04:22PM +0100, Andrew Lunn wrote:
> On Wed, Oct 30, 2019 at 08:21:17PM +0000, Priit Laes wrote:
> > Heya!
> > 
> > I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
> > detection fails consistently with certain 1000Mbit partners (for example Huawei
> > B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
> > but the same hardware works properly with certain other link partners (100Mbit GL AR150
> > for example).
> 
> Hi Pritt
> 
> What PHY is used? And what happens if you use the specific PHY driver,
> not the generic PHY driver?

Schematics of the board are here:

https://github.com/OLIMEX/OLINUXINO/blob/master/HARDWARE/A20-OLinuXino-LIME2/1.%20Latest%20hardware%20revision/A20-OLinuXino-Lime2_Rev_K2_COLOR.pdf

So it has a KSZ9031. The micrel driver supports that device. And there
is a patch which might be relevant:

ommit 3aed3e2a143c9619f4c8d0a3b8fe74d7d3d79c93
Author: Antoine Tenart <antoine.tenart@bootlin.com>
Date:   Tue Apr 16 12:10:20 2019 +0200

    net: phy: micrel: add Asym Pause workaround
    
    The Micrel KSZ9031 PHY may fail to establish a link when the Asymmetric
    Pause capability is set. This issue is described in a Silicon Errata
    (DS80000691D or DS80000692D), which advises to always disable the
    capability. This patch implements the workaround by defining a KSZ9031
    specific get_feature callback to force the Asymmetric Pause capability
    bit to be cleared.
    
    This fixes issues where the link would not come up at boot time, or when
    the Asym Pause bit was set later on.
    
    Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
    Reviewed-by: Andrew Lunn <andrew@lunn.ch>
    Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
    Signed-off-by: David S. Miller <davem@davemloft.net>


Please test using the Micrel PHY driver and see if that solves your
problem.

	Andrew

Re: sun7i-dwmac: link detection failure with 1000Mbit parters

[Priit Laes]

On Thu, Oct 31, 2019 at 02:14:04PM +0100, Andrew Lunn wrote:
> On Thu, Oct 31, 2019 at 02:04:22PM +0100, Andrew Lunn wrote:
> > On Wed, Oct 30, 2019 at 08:21:17PM +0000, Priit Laes wrote:
> > > Heya!
> > > 
> > > I have noticed that with sun7i-dwmac driver (OLinuxino Lime2 eMMC), link
> > > detection fails consistently with certain 1000Mbit partners (for example Huawei
> > > B525s-23a 4g modem ethernet outputs and RTL8153-based USB3.0 ethernet dongle),
> > > but the same hardware works properly with certain other link partners (100Mbit GL AR150
> > > for example).
> > 
> > Hi Pritt
> > 
> > What PHY is used? And what happens if you use the specific PHY driver,
> > not the generic PHY driver?
> 
> Schematics of the board are here:
> 
> https://github.com/OLIMEX/OLINUXINO/blob/master/HARDWARE/A20-OLinuXino-LIME2/1.%20Latest%20hardware%20revision/A20-OLinuXino-Lime2_Rev_K2_COLOR.pdf
> 
> So it has a KSZ9031. The micrel driver supports that device. And there
> is a patch which might be relevant:
> 
> commit 3aed3e2a143c9619f4c8d0a3b8fe74d7d3d79c93
> Author: Antoine Tenart <antoine.tenart@bootlin.com>
...
> Please test using the Micrel PHY driver and see if that solves your
> problem.

Thanks, CONFIG_MICREL_PHY=y helped!

> 
> 	Andrew