Clause 73 and USXGMII

Clause 73 and USXGMII

[Jose Abreu]

Hello,

I've some sample code for Clause 73 support using Synopsys based XPCS 
but I would like to clarify some things that I noticed.

I'm using USXGMII as interface and a single SERDES that operates at 10G 
rate but MAC side is working at 2.5G. Maximum available bandwidth is 
therefore 2.5Gbps.

So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
the autoneg abilities to 2.5G max then it never finishes:
# ethtool enp4s0
Settings for enp4s0:
	Supported ports: [ ]
	Supported link modes:   1000baseKX/Full 
	                        2500baseX/Full 
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  1000baseKX/Full 
	                        2500baseX/Full 
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: Unknown!
	Duplex: Unknown! (255)
	Port: MII
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: ug
	Wake-on: d
	Current message level: 0x0000003f (63)
			       drv probe link timer ifdown ifup
	Link detected: no

When I do not limit autoneg and I say that maximum limit is 10G then I 
get Link Up and autoneg finishes with this outcome:
# ethtool enp4s0
Settings for enp4s0:
	Supported ports: [ ]
	Supported link modes:   1000baseKX/Full 
	                        2500baseX/Full 
	                        10000baseKX4/Full 
	                        10000baseKR/Full 
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  1000baseKX/Full 
	                        2500baseX/Full 
	                        10000baseKX4/Full 
	                        10000baseKR/Full 
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  1000baseKX/Full 
	                                     2500baseX/Full 
	                                     10000baseKX4/Full 
	                                     10000baseKR/Full 
	Link partner advertised pause frame use: Symmetric Receive-only
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 2500Mb/s
	Duplex: Full
	Port: MII <- Never mind this, it's a SW issue
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: ug
	Wake-on: d
	Current message level: 0x0000003f (63)
			       drv probe link timer ifdown ifup
	Link detected: yes

I was expecting that, as MAC side is limited to 2.5G, I should set in 
phylink the correct capabilities and then outcome of autoneg would only 
have up to 2.5G modes. Am I wrong ?

---
Thanks,
Jose Miguel Abreu

RE: Clause 73 and USXGMII

[Jose Abreu]

++ PHY Experts

From: Jose Abreu <joabreu@synopsys.com>
Date: Aug/07/2019, 16:46:23 (UTC+00:00)

> Hello,
> 
> I've some sample code for Clause 73 support using Synopsys based XPCS 
> but I would like to clarify some things that I noticed.
> 
> I'm using USXGMII as interface and a single SERDES that operates at 10G 
> rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> therefore 2.5Gbps.
> 
> So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> the autoneg abilities to 2.5G max then it never finishes:
> # ethtool enp4s0
> Settings for enp4s0:
> 	Supported ports: [ ]
> 	Supported link modes:   1000baseKX/Full 
> 	                        2500baseX/Full 
> 	Supported pause frame use: Symmetric Receive-only
> 	Supports auto-negotiation: Yes
> 	Supported FEC modes: Not reported
> 	Advertised link modes:  1000baseKX/Full 
> 	                        2500baseX/Full 
> 	Advertised pause frame use: Symmetric Receive-only
> 	Advertised auto-negotiation: Yes
> 	Advertised FEC modes: Not reported
> 	Speed: Unknown!
> 	Duplex: Unknown! (255)
> 	Port: MII
> 	PHYAD: 0
> 	Transceiver: internal
> 	Auto-negotiation: on
> 	Supports Wake-on: ug
> 	Wake-on: d
> 	Current message level: 0x0000003f (63)
> 			       drv probe link timer ifdown ifup
> 	Link detected: no
> 
> When I do not limit autoneg and I say that maximum limit is 10G then I 
> get Link Up and autoneg finishes with this outcome:
> # ethtool enp4s0
> Settings for enp4s0:
> 	Supported ports: [ ]
> 	Supported link modes:   1000baseKX/Full 
> 	                        2500baseX/Full 
> 	                        10000baseKX4/Full 
> 	                        10000baseKR/Full 
> 	Supported pause frame use: Symmetric Receive-only
> 	Supports auto-negotiation: Yes
> 	Supported FEC modes: Not reported
> 	Advertised link modes:  1000baseKX/Full 
> 	                        2500baseX/Full 
> 	                        10000baseKX4/Full 
> 	                        10000baseKR/Full 
> 	Advertised pause frame use: Symmetric Receive-only
> 	Advertised auto-negotiation: Yes
> 	Advertised FEC modes: Not reported
> 	Link partner advertised link modes:  1000baseKX/Full 
> 	                                     2500baseX/Full 
> 	                                     10000baseKX4/Full 
> 	                                     10000baseKR/Full 
> 	Link partner advertised pause frame use: Symmetric Receive-only
> 	Link partner advertised auto-negotiation: Yes
> 	Link partner advertised FEC modes: Not reported
> 	Speed: 2500Mb/s
> 	Duplex: Full
> 	Port: MII <- Never mind this, it's a SW issue
> 	PHYAD: 0
> 	Transceiver: internal
> 	Auto-negotiation: on
> 	Supports Wake-on: ug
> 	Wake-on: d
> 	Current message level: 0x0000003f (63)
> 			       drv probe link timer ifdown ifup
> 	Link detected: yes
> 
> I was expecting that, as MAC side is limited to 2.5G, I should set in 
> phylink the correct capabilities and then outcome of autoneg would only 
> have up to 2.5G modes. Am I wrong ?
> 
> ---
> Thanks,
> Jose Miguel Abreu


---
Thanks,
Jose Miguel Abreu

Re: Clause 73 and USXGMII

[Russell King - ARM Linux admin]

Hi,

Have you tried enabling debug mode in phylink (add #define DEBUG at the
top of the file) ?

Thanks.

On Thu, Aug 08, 2019 at 08:17:29AM +0000, Jose Abreu wrote:
> ++ PHY Experts
> 
> From: Jose Abreu <joabreu@synopsys.com>
> Date: Aug/07/2019, 16:46:23 (UTC+00:00)
> 
> > Hello,
> > 
> > I've some sample code for Clause 73 support using Synopsys based XPCS 
> > but I would like to clarify some things that I noticed.
> > 
> > I'm using USXGMII as interface and a single SERDES that operates at 10G 
> > rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> > therefore 2.5Gbps.
> > 
> > So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> > the autoneg abilities to 2.5G max then it never finishes:
> > # ethtool enp4s0
> > Settings for enp4s0:
> > 	Supported ports: [ ]
> > 	Supported link modes:   1000baseKX/Full 
> > 	                        2500baseX/Full 
> > 	Supported pause frame use: Symmetric Receive-only
> > 	Supports auto-negotiation: Yes
> > 	Supported FEC modes: Not reported
> > 	Advertised link modes:  1000baseKX/Full 
> > 	                        2500baseX/Full 
> > 	Advertised pause frame use: Symmetric Receive-only
> > 	Advertised auto-negotiation: Yes
> > 	Advertised FEC modes: Not reported
> > 	Speed: Unknown!
> > 	Duplex: Unknown! (255)
> > 	Port: MII
> > 	PHYAD: 0
> > 	Transceiver: internal
> > 	Auto-negotiation: on
> > 	Supports Wake-on: ug
> > 	Wake-on: d
> > 	Current message level: 0x0000003f (63)
> > 			       drv probe link timer ifdown ifup
> > 	Link detected: no
> > 
> > When I do not limit autoneg and I say that maximum limit is 10G then I 
> > get Link Up and autoneg finishes with this outcome:
> > # ethtool enp4s0
> > Settings for enp4s0:
> > 	Supported ports: [ ]
> > 	Supported link modes:   1000baseKX/Full 
> > 	                        2500baseX/Full 
> > 	                        10000baseKX4/Full 
> > 	                        10000baseKR/Full 
> > 	Supported pause frame use: Symmetric Receive-only
> > 	Supports auto-negotiation: Yes
> > 	Supported FEC modes: Not reported
> > 	Advertised link modes:  1000baseKX/Full 
> > 	                        2500baseX/Full 
> > 	                        10000baseKX4/Full 
> > 	                        10000baseKR/Full 
> > 	Advertised pause frame use: Symmetric Receive-only
> > 	Advertised auto-negotiation: Yes
> > 	Advertised FEC modes: Not reported
> > 	Link partner advertised link modes:  1000baseKX/Full 
> > 	                                     2500baseX/Full 
> > 	                                     10000baseKX4/Full 
> > 	                                     10000baseKR/Full 
> > 	Link partner advertised pause frame use: Symmetric Receive-only
> > 	Link partner advertised auto-negotiation: Yes
> > 	Link partner advertised FEC modes: Not reported
> > 	Speed: 2500Mb/s
> > 	Duplex: Full
> > 	Port: MII <- Never mind this, it's a SW issue
> > 	PHYAD: 0
> > 	Transceiver: internal
> > 	Auto-negotiation: on
> > 	Supports Wake-on: ug
> > 	Wake-on: d
> > 	Current message level: 0x0000003f (63)
> > 			       drv probe link timer ifdown ifup
> > 	Link detected: yes
> > 
> > I was expecting that, as MAC side is limited to 2.5G, I should set in 
> > phylink the correct capabilities and then outcome of autoneg would only 
> > have up to 2.5G modes. Am I wrong ?
> > 
> > ---
> > Thanks,
> > Jose Miguel Abreu
> 
> 
> ---
> Thanks,
> Jose Miguel Abreu
> 

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

RE: Clause 73 and USXGMII

[Jose Abreu]

From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
Date: Aug/08/2019, 09:26:26 (UTC+00:00)

> Hi,
> 
> Have you tried enabling debug mode in phylink (add #define DEBUG at the
> top of the file) ?

Yes:

[ With > 2.5G modes removed ]
# dmesg | grep -i phy
libphy: stmmac: probed
stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
00,00000000,0002e040 advertising 00,00000000,0002e040
stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
link=0 an=1
stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown

[ Without any limit ]
# dmesg | grep -i phy
libphy: stmmac: probed
stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
00,00000000,000ee040 advertising 00,00000000,000ee040
stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
link=0 an=1
stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
an=0

I'm thinking on whether this can be related with USXGMII. As link is 
operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
should always be 10G ?

> On Thu, Aug 08, 2019 at 08:17:29AM +0000, Jose Abreu wrote:
> > ++ PHY Experts
> > 
> > From: Jose Abreu <joabreu@synopsys.com>
> > Date: Aug/07/2019, 16:46:23 (UTC+00:00)
> > 
> > > Hello,
> > > 
> > > I've some sample code for Clause 73 support using Synopsys based XPCS 
> > > but I would like to clarify some things that I noticed.
> > > 
> > > I'm using USXGMII as interface and a single SERDES that operates at 10G 
> > > rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> > > therefore 2.5Gbps.
> > > 
> > > So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> > > the autoneg abilities to 2.5G max then it never finishes:
> > > # ethtool enp4s0
> > > Settings for enp4s0:
> > > 	Supported ports: [ ]
> > > 	Supported link modes:   1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	Supported pause frame use: Symmetric Receive-only
> > > 	Supports auto-negotiation: Yes
> > > 	Supported FEC modes: Not reported
> > > 	Advertised link modes:  1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	Advertised pause frame use: Symmetric Receive-only
> > > 	Advertised auto-negotiation: Yes
> > > 	Advertised FEC modes: Not reported
> > > 	Speed: Unknown!
> > > 	Duplex: Unknown! (255)
> > > 	Port: MII
> > > 	PHYAD: 0
> > > 	Transceiver: internal
> > > 	Auto-negotiation: on
> > > 	Supports Wake-on: ug
> > > 	Wake-on: d
> > > 	Current message level: 0x0000003f (63)
> > > 			       drv probe link timer ifdown ifup
> > > 	Link detected: no
> > > 
> > > When I do not limit autoneg and I say that maximum limit is 10G then I 
> > > get Link Up and autoneg finishes with this outcome:
> > > # ethtool enp4s0
> > > Settings for enp4s0:
> > > 	Supported ports: [ ]
> > > 	Supported link modes:   1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	                        10000baseKX4/Full 
> > > 	                        10000baseKR/Full 
> > > 	Supported pause frame use: Symmetric Receive-only
> > > 	Supports auto-negotiation: Yes
> > > 	Supported FEC modes: Not reported
> > > 	Advertised link modes:  1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	                        10000baseKX4/Full 
> > > 	                        10000baseKR/Full 
> > > 	Advertised pause frame use: Symmetric Receive-only
> > > 	Advertised auto-negotiation: Yes
> > > 	Advertised FEC modes: Not reported
> > > 	Link partner advertised link modes:  1000baseKX/Full 
> > > 	                                     2500baseX/Full 
> > > 	                                     10000baseKX4/Full 
> > > 	                                     10000baseKR/Full 
> > > 	Link partner advertised pause frame use: Symmetric Receive-only
> > > 	Link partner advertised auto-negotiation: Yes
> > > 	Link partner advertised FEC modes: Not reported
> > > 	Speed: 2500Mb/s
> > > 	Duplex: Full
> > > 	Port: MII <- Never mind this, it's a SW issue
> > > 	PHYAD: 0
> > > 	Transceiver: internal
> > > 	Auto-negotiation: on
> > > 	Supports Wake-on: ug
> > > 	Wake-on: d
> > > 	Current message level: 0x0000003f (63)
> > > 			       drv probe link timer ifdown ifup
> > > 	Link detected: yes
> > > 
> > > I was expecting that, as MAC side is limited to 2.5G, I should set in 
> > > phylink the correct capabilities and then outcome of autoneg would only 
> > > have up to 2.5G modes. Am I wrong ?
> > > 
> > > ---
> > > Thanks,
> > > Jose Miguel Abreu
> > 
> > 
> > ---
> > Thanks,
> > Jose Miguel Abreu
> > 
> 
> -- 
> RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=1MdSlPrmzsMMCJbbLcDYTNuPq1njfusBRjcRz3UD4Dg&s=_30hwSYkGf9DfyCG48mnh7lXP8iiULXpfAP_6agUJno&e= 
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> According to speedtest.net: 11.9Mbps down 500kbps up


---
Thanks,
Jose Miguel Abreu

Re: Clause 73 and USXGMII

[Russell King - ARM Linux admin]

On Thu, Aug 08, 2019 at 09:02:57AM +0000, Jose Abreu wrote:
> From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> Date: Aug/08/2019, 09:26:26 (UTC+00:00)
> 
> > Hi,
> > 
> > Have you tried enabling debug mode in phylink (add #define DEBUG at the
> > top of the file) ?
> 
> Yes:
> 
> [ With > 2.5G modes removed ]
> # dmesg | grep -i phy
> libphy: stmmac: probed
> stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> 00,00000000,0002e040 advertising 00,00000000,0002e040
> stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
> link=0 an=1
> stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown

This shows that the PHY isn't reporting that the link came up.  Did
the PHY negotiate link?  If so, why isn't it reporting that the link
came up?  Maybe something is mis-programming the capability bits in
the PHY?  Maybe disabling the 10G speeds disables everything faster
than 1G?

> [ Without any limit ]
> # dmesg | grep -i phy
> libphy: stmmac: probed
> stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> 00,00000000,000ee040 advertising 00,00000000,000ee040
> stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
> link=0 an=1
> stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
> stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
> an=0
> 
> I'm thinking on whether this can be related with USXGMII. As link is 
> operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
> should always be 10G ?

As I understand USXGMII (which isn't very well, because the spec isn't
available) I believe that it operates in a similar way to SGMII where
data is replicated the appropriate number of times to achieve the link
speed.  So, the USXGMII link always operates at a bit rate equivalent
to 10G, but data is replicated twice for 5G, four times for 2.5G, ten
times for 1G, etc.

I notice that you don't say that you support any copper speeds, which
brings up the question about what the PHY's media is...

> > On Thu, Aug 08, 2019 at 08:17:29AM +0000, Jose Abreu wrote:
> > > ++ PHY Experts
> > > 
> > > From: Jose Abreu <joabreu@synopsys.com>
> > > Date: Aug/07/2019, 16:46:23 (UTC+00:00)
> > > 
> > > > Hello,
> > > > 
> > > > I've some sample code for Clause 73 support using Synopsys based XPCS 
> > > > but I would like to clarify some things that I noticed.
> > > > 
> > > > I'm using USXGMII as interface and a single SERDES that operates at 10G 
> > > > rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> > > > therefore 2.5Gbps.
> > > > 
> > > > So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> > > > the autoneg abilities to 2.5G max then it never finishes:
> > > > # ethtool enp4s0
> > > > Settings for enp4s0:
> > > > 	Supported ports: [ ]
> > > > 	Supported link modes:   1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	Supported pause frame use: Symmetric Receive-only
> > > > 	Supports auto-negotiation: Yes
> > > > 	Supported FEC modes: Not reported
> > > > 	Advertised link modes:  1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	Advertised pause frame use: Symmetric Receive-only
> > > > 	Advertised auto-negotiation: Yes
> > > > 	Advertised FEC modes: Not reported
> > > > 	Speed: Unknown!
> > > > 	Duplex: Unknown! (255)
> > > > 	Port: MII
> > > > 	PHYAD: 0
> > > > 	Transceiver: internal
> > > > 	Auto-negotiation: on
> > > > 	Supports Wake-on: ug
> > > > 	Wake-on: d
> > > > 	Current message level: 0x0000003f (63)
> > > > 			       drv probe link timer ifdown ifup
> > > > 	Link detected: no
> > > > 
> > > > When I do not limit autoneg and I say that maximum limit is 10G then I 
> > > > get Link Up and autoneg finishes with this outcome:
> > > > # ethtool enp4s0
> > > > Settings for enp4s0:
> > > > 	Supported ports: [ ]
> > > > 	Supported link modes:   1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	                        10000baseKX4/Full 
> > > > 	                        10000baseKR/Full 
> > > > 	Supported pause frame use: Symmetric Receive-only
> > > > 	Supports auto-negotiation: Yes
> > > > 	Supported FEC modes: Not reported
> > > > 	Advertised link modes:  1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	                        10000baseKX4/Full 
> > > > 	                        10000baseKR/Full 
> > > > 	Advertised pause frame use: Symmetric Receive-only
> > > > 	Advertised auto-negotiation: Yes
> > > > 	Advertised FEC modes: Not reported
> > > > 	Link partner advertised link modes:  1000baseKX/Full 
> > > > 	                                     2500baseX/Full 
> > > > 	                                     10000baseKX4/Full 
> > > > 	                                     10000baseKR/Full 
> > > > 	Link partner advertised pause frame use: Symmetric Receive-only
> > > > 	Link partner advertised auto-negotiation: Yes
> > > > 	Link partner advertised FEC modes: Not reported
> > > > 	Speed: 2500Mb/s
> > > > 	Duplex: Full
> > > > 	Port: MII <- Never mind this, it's a SW issue
> > > > 	PHYAD: 0
> > > > 	Transceiver: internal
> > > > 	Auto-negotiation: on
> > > > 	Supports Wake-on: ug
> > > > 	Wake-on: d
> > > > 	Current message level: 0x0000003f (63)
> > > > 			       drv probe link timer ifdown ifup
> > > > 	Link detected: yes
> > > > 
> > > > I was expecting that, as MAC side is limited to 2.5G, I should set in 
> > > > phylink the correct capabilities and then outcome of autoneg would only 
> > > > have up to 2.5G modes. Am I wrong ?
> > > > 
> > > > ---
> > > > Thanks,
> > > > Jose Miguel Abreu
> > > 
> > > 
> > > ---
> > > Thanks,
> > > Jose Miguel Abreu
> > > 
> > 
> > -- 
> > RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=1MdSlPrmzsMMCJbbLcDYTNuPq1njfusBRjcRz3UD4Dg&s=_30hwSYkGf9DfyCG48mnh7lXP8iiULXpfAP_6agUJno&e= 
> > FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> > According to speedtest.net: 11.9Mbps down 500kbps up
> 
> 
> ---
> Thanks,
> Jose Miguel Abreu
> 

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

RE: Clause 73 and USXGMII

[Jose Abreu]

From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
Date: Aug/08/2019, 10:23:13 (UTC+00:00)

> On Thu, Aug 08, 2019 at 09:02:57AM +0000, Jose Abreu wrote:
> > From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> > Date: Aug/08/2019, 09:26:26 (UTC+00:00)
> > 
> > > Hi,
> > > 
> > > Have you tried enabling debug mode in phylink (add #define DEBUG at the
> > > top of the file) ?
> > 
> > Yes:
> > 
> > [ With > 2.5G modes removed ]
> > # dmesg | grep -i phy
> > libphy: stmmac: probed
> > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > 00,00000000,0002e040 advertising 00,00000000,0002e040
> > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
> > link=0 an=1
> > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> 
> This shows that the PHY isn't reporting that the link came up.  Did
> the PHY negotiate link?  If so, why isn't it reporting that the link
> came up?  Maybe something is mis-programming the capability bits in
> the PHY?  Maybe disabling the 10G speeds disables everything faster
> than 1G?

Autoneg was started but never finishes and disabling 10G modes is 
causing autoneg to fail.

> 
> > [ Without any limit ]
> > # dmesg | grep -i phy
> > libphy: stmmac: probed
> > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > 00,00000000,000ee040 advertising 00,00000000,000ee040
> > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
> > link=0 an=1
> > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
> > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
> > an=0
> > 
> > I'm thinking on whether this can be related with USXGMII. As link is 
> > operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
> > should always be 10G ?
> 
> As I understand USXGMII (which isn't very well, because the spec isn't
> available) I believe that it operates in a similar way to SGMII where
> data is replicated the appropriate number of times to achieve the link
> speed.  So, the USXGMII link always operates at a bit rate equivalent
> to 10G, but data is replicated twice for 5G, four times for 2.5G, ten
> times for 1G, etc.
> 
> I notice that you don't say that you support any copper speeds, which
> brings up the question about what the PHY's media is...

I just added the speeds that XPCS supports within Clause 73 
specification:
Technology Ability field. Indicates the supported technologies:
	A0: When this bit is set to 1, the 1000BASE-KX technology is supported
	A1: When this bit is set to 1, the 10GBASE-KX4 technology is supported
	A2: When this bit is set to 1, the 10GBASE-KR technology is supported
	A11: When this bit is set to 1, the 2.5GBASE-KX technology is supported
	A12: When this bit is set to 1, the 5GBASE-KR technology is supported

And, within USXGMII, XPCS supports the following:
	Single Port: 10G-SXGMII, 5G-SXGMII, 2.5G-SXGMII
	Dual Port: 10G-DXGMII, 5G-DXGMII
	Quad Port: 10G-XGMII

My HW is currently fixed for USXGMII at 2.5G.

> 
> > > On Thu, Aug 08, 2019 at 08:17:29AM +0000, Jose Abreu wrote:
> > > > ++ PHY Experts
> > > > 
> > > > From: Jose Abreu <joabreu@synopsys.com>
> > > > Date: Aug/07/2019, 16:46:23 (UTC+00:00)
> > > > 
> > > > > Hello,
> > > > > 
> > > > > I've some sample code for Clause 73 support using Synopsys based XPCS 
> > > > > but I would like to clarify some things that I noticed.
> > > > > 
> > > > > I'm using USXGMII as interface and a single SERDES that operates at 10G 
> > > > > rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> > > > > therefore 2.5Gbps.
> > > > > 
> > > > > So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> > > > > the autoneg abilities to 2.5G max then it never finishes:
> > > > > # ethtool enp4s0
> > > > > Settings for enp4s0:
> > > > > 	Supported ports: [ ]
> > > > > 	Supported link modes:   1000baseKX/Full 
> > > > > 	                        2500baseX/Full 
> > > > > 	Supported pause frame use: Symmetric Receive-only
> > > > > 	Supports auto-negotiation: Yes
> > > > > 	Supported FEC modes: Not reported
> > > > > 	Advertised link modes:  1000baseKX/Full 
> > > > > 	                        2500baseX/Full 
> > > > > 	Advertised pause frame use: Symmetric Receive-only
> > > > > 	Advertised auto-negotiation: Yes
> > > > > 	Advertised FEC modes: Not reported
> > > > > 	Speed: Unknown!
> > > > > 	Duplex: Unknown! (255)
> > > > > 	Port: MII
> > > > > 	PHYAD: 0
> > > > > 	Transceiver: internal
> > > > > 	Auto-negotiation: on
> > > > > 	Supports Wake-on: ug
> > > > > 	Wake-on: d
> > > > > 	Current message level: 0x0000003f (63)
> > > > > 			       drv probe link timer ifdown ifup
> > > > > 	Link detected: no
> > > > > 
> > > > > When I do not limit autoneg and I say that maximum limit is 10G then I 
> > > > > get Link Up and autoneg finishes with this outcome:
> > > > > # ethtool enp4s0
> > > > > Settings for enp4s0:
> > > > > 	Supported ports: [ ]
> > > > > 	Supported link modes:   1000baseKX/Full 
> > > > > 	                        2500baseX/Full 
> > > > > 	                        10000baseKX4/Full 
> > > > > 	                        10000baseKR/Full 
> > > > > 	Supported pause frame use: Symmetric Receive-only
> > > > > 	Supports auto-negotiation: Yes
> > > > > 	Supported FEC modes: Not reported
> > > > > 	Advertised link modes:  1000baseKX/Full 
> > > > > 	                        2500baseX/Full 
> > > > > 	                        10000baseKX4/Full 
> > > > > 	                        10000baseKR/Full 
> > > > > 	Advertised pause frame use: Symmetric Receive-only
> > > > > 	Advertised auto-negotiation: Yes
> > > > > 	Advertised FEC modes: Not reported
> > > > > 	Link partner advertised link modes:  1000baseKX/Full 
> > > > > 	                                     2500baseX/Full 
> > > > > 	                                     10000baseKX4/Full 
> > > > > 	                                     10000baseKR/Full 
> > > > > 	Link partner advertised pause frame use: Symmetric Receive-only
> > > > > 	Link partner advertised auto-negotiation: Yes
> > > > > 	Link partner advertised FEC modes: Not reported
> > > > > 	Speed: 2500Mb/s
> > > > > 	Duplex: Full
> > > > > 	Port: MII <- Never mind this, it's a SW issue
> > > > > 	PHYAD: 0
> > > > > 	Transceiver: internal
> > > > > 	Auto-negotiation: on
> > > > > 	Supports Wake-on: ug
> > > > > 	Wake-on: d
> > > > > 	Current message level: 0x0000003f (63)
> > > > > 			       drv probe link timer ifdown ifup
> > > > > 	Link detected: yes
> > > > > 
> > > > > I was expecting that, as MAC side is limited to 2.5G, I should set in 
> > > > > phylink the correct capabilities and then outcome of autoneg would only 
> > > > > have up to 2.5G modes. Am I wrong ?
> > > > > 
> > > > > ---
> > > > > Thanks,
> > > > > Jose Miguel Abreu
> > > > 
> > > > 
> > > > ---
> > > > Thanks,
> > > > Jose Miguel Abreu
> > > > 
> > > 
> > > -- 
> > > RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=1MdSlPrmzsMMCJbbLcDYTNuPq1njfusBRjcRz3UD4Dg&s=_30hwSYkGf9DfyCG48mnh7lXP8iiULXpfAP_6agUJno&e= 
> > > FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> > > According to speedtest.net: 11.9Mbps down 500kbps up
> > 
> > 
> > ---
> > Thanks,
> > Jose Miguel Abreu
> > 
> 
> -- 
> RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=d_Og5QaTJOl1WoLi43ZAlCMajHnZp4mGg8Npwlaa2pk&s=bs6ws6HmZNKiutYWGFPy1ztnEQBuhtWyjiE0Hr1_URo&e= 
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> According to speedtest.net: 11.9Mbps down 500kbps up


---
Thanks,
Jose Miguel Abreu

Re: Clause 73 and USXGMII

[Russell King - ARM Linux admin]

On Thu, Aug 08, 2019 at 11:45:29AM +0000, Jose Abreu wrote:
> From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> Date: Aug/08/2019, 10:23:13 (UTC+00:00)
> 
> > On Thu, Aug 08, 2019 at 09:02:57AM +0000, Jose Abreu wrote:
> > > From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> > > Date: Aug/08/2019, 09:26:26 (UTC+00:00)
> > > 
> > > > Hi,
> > > > 
> > > > Have you tried enabling debug mode in phylink (add #define DEBUG at the
> > > > top of the file) ?
> > > 
> > > Yes:
> > > 
> > > [ With > 2.5G modes removed ]
> > > # dmesg | grep -i phy
> > > libphy: stmmac: probed
> > > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > > 00,00000000,0002e040 advertising 00,00000000,0002e040
> > > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
> > > link=0 an=1
> > > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > 
> > This shows that the PHY isn't reporting that the link came up.  Did
> > the PHY negotiate link?  If so, why isn't it reporting that the link
> > came up?  Maybe something is mis-programming the capability bits in
> > the PHY?  Maybe disabling the 10G speeds disables everything faster
> > than 1G?
> 
> Autoneg was started but never finishes and disabling 10G modes is 
> causing autoneg to fail.
> 
> > 
> > > [ Without any limit ]
> > > # dmesg | grep -i phy
> > > libphy: stmmac: probed
> > > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > > 00,00000000,000ee040 advertising 00,00000000,000ee040
> > > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
> > > link=0 an=1
> > > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > > stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
> > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
> > > an=0
> > > 
> > > I'm thinking on whether this can be related with USXGMII. As link is 
> > > operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
> > > should always be 10G ?
> > 
> > As I understand USXGMII (which isn't very well, because the spec isn't
> > available) I believe that it operates in a similar way to SGMII where
> > data is replicated the appropriate number of times to achieve the link
> > speed.  So, the USXGMII link always operates at a bit rate equivalent
> > to 10G, but data is replicated twice for 5G, four times for 2.5G, ten
> > times for 1G, etc.
> > 
> > I notice that you don't say that you support any copper speeds, which
> > brings up the question about what the PHY's media is...
> 
> I just added the speeds that XPCS supports within Clause 73 
> specification:
> Technology Ability field. Indicates the supported technologies:
> 	A0: When this bit is set to 1, the 1000BASE-KX technology is supported
> 	A1: When this bit is set to 1, the 10GBASE-KX4 technology is supported
> 	A2: When this bit is set to 1, the 10GBASE-KR technology is supported
> 	A11: When this bit is set to 1, the 2.5GBASE-KX technology is supported
> 	A12: When this bit is set to 1, the 5GBASE-KR technology is supported
> 
> And, within USXGMII, XPCS supports the following:
> 	Single Port: 10G-SXGMII, 5G-SXGMII, 2.5G-SXGMII
> 	Dual Port: 10G-DXGMII, 5G-DXGMII
> 	Quad Port: 10G-XGMII
> 
> My HW is currently fixed for USXGMII at 2.5G.

So what do you actually have?

+-----+              +----------+
| STM |    USXGMII   | Synopsis |   ????????
| MAC | <----------> |   PHY    | <----------> ????
|     |     link     |          |
+-----+              +----------+ (media side)

Does the above refer to what the STM MAC and Synopsis PHY support for
the USXGMII link?  What about the media side?

If you just tell phylink what the USXGMII part is capable of, there's
no way for the media side to do anything unless it also supports the
capabilities that the USXGMII link supports.

phylink doesn't do any kind of translation of capabilities of the
MAC-PHY link to media capabilities; this is why the documentation for
the validate callback has this note:

 * Note that the PHY may be able to transform from one connection
 * technology to another, so, eg, don't clear 1000BaseX just
 * because the MAC is unable to BaseX mode. This is more about
 * clearing unsupported speeds and duplex settings.

To put it another way - if the link between the MAC and PHY supports
10G speed, the MAC should indicate that _all_ 10G ethtool link modes
that support 10G speed are set in the supported mask.  If the link
supports 1G speeds, then all 1G ethtool link modes that can be
supported irrespective of technology should be set.  This is because
the capabilities of the overall setup is the logical union of the
capabilities of each device in the setup.

So, if, say, the USXGMII link can support 2.5Gbps, and the PHY
supports copper media at 2.5Gbps via the NBASE-T specifications,
then the system as a whole can support
ETHTOOL_LINK_MODE_2500baseT_Full_BIT.  If the MAC decides to clear
that bit, then the system can't support 2.5Gbps even if the PHY
does.

Maybe what we need to do with phylink is move away from exposing
ethtool link modes to the MAC validate callback, and instead
devise a way for the MAC to indicate merely the speeds and duplexes
it supports without any of the technology stuff getting in the way.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

RE: Clause 73 and USXGMII

[Jose Abreu]

From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
Date: Aug/08/2019, 13:09:03 (UTC+00:00)

> On Thu, Aug 08, 2019 at 11:45:29AM +0000, Jose Abreu wrote:
> > From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> > Date: Aug/08/2019, 10:23:13 (UTC+00:00)
> > 
> > > On Thu, Aug 08, 2019 at 09:02:57AM +0000, Jose Abreu wrote:
> > > > From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> > > > Date: Aug/08/2019, 09:26:26 (UTC+00:00)
> > > > 
> > > > > Hi,
> > > > > 
> > > > > Have you tried enabling debug mode in phylink (add #define DEBUG at the
> > > > > top of the file) ?
> > > > 
> > > > Yes:
> > > > 
> > > > [ With > 2.5G modes removed ]
> > > > # dmesg | grep -i phy
> > > > libphy: stmmac: probed
> > > > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > > > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > > > 00,00000000,0002e040 advertising 00,00000000,0002e040
> > > > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
> > > > link=0 an=1
> > > > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > > 
> > > This shows that the PHY isn't reporting that the link came up.  Did
> > > the PHY negotiate link?  If so, why isn't it reporting that the link
> > > came up?  Maybe something is mis-programming the capability bits in
> > > the PHY?  Maybe disabling the 10G speeds disables everything faster
> > > than 1G?
> > 
> > Autoneg was started but never finishes and disabling 10G modes is 
> > causing autoneg to fail.
> > 
> > > 
> > > > [ Without any limit ]
> > > > # dmesg | grep -i phy
> > > > libphy: stmmac: probed
> > > > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > > > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > > > 00,00000000,000ee040 advertising 00,00000000,000ee040
> > > > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
> > > > link=0 an=1
> > > > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > > > stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
> > > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > > mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
> > > > an=0
> > > > 
> > > > I'm thinking on whether this can be related with USXGMII. As link is 
> > > > operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
> > > > should always be 10G ?
> > > 
> > > As I understand USXGMII (which isn't very well, because the spec isn't
> > > available) I believe that it operates in a similar way to SGMII where
> > > data is replicated the appropriate number of times to achieve the link
> > > speed.  So, the USXGMII link always operates at a bit rate equivalent
> > > to 10G, but data is replicated twice for 5G, four times for 2.5G, ten
> > > times for 1G, etc.
> > > 
> > > I notice that you don't say that you support any copper speeds, which
> > > brings up the question about what the PHY's media is...
> > 
> > I just added the speeds that XPCS supports within Clause 73 
> > specification:
> > Technology Ability field. Indicates the supported technologies:
> > 	A0: When this bit is set to 1, the 1000BASE-KX technology is supported
> > 	A1: When this bit is set to 1, the 10GBASE-KX4 technology is supported
> > 	A2: When this bit is set to 1, the 10GBASE-KR technology is supported
> > 	A11: When this bit is set to 1, the 2.5GBASE-KX technology is supported
> > 	A12: When this bit is set to 1, the 5GBASE-KR technology is supported
> > 
> > And, within USXGMII, XPCS supports the following:
> > 	Single Port: 10G-SXGMII, 5G-SXGMII, 2.5G-SXGMII
> > 	Dual Port: 10G-DXGMII, 5G-DXGMII
> > 	Quad Port: 10G-XGMII
> > 
> > My HW is currently fixed for USXGMII at 2.5G.
> 
> So what do you actually have?
> 
> +-----+              +----------+
> | STM |    USXGMII   | Synopsis |   ????????
> | MAC | <----------> |   PHY    | <----------> ????
> |     |     link     |          |
> +-----+              +----------+ (media side)
> 
> Does the above refer to what the STM MAC and Synopsis PHY support for
> the USXGMII link?  What about the media side?

This is my setup:

XGMAC <-> XPCS <-> Xilinx SERDES <-> SFP

I'm not sure about the media side. I use a passive SFP cable if that 
helps ...

> 
> If you just tell phylink what the USXGMII part is capable of, there's
> no way for the media side to do anything unless it also supports the
> capabilities that the USXGMII link supports.
> 
> phylink doesn't do any kind of translation of capabilities of the
> MAC-PHY link to media capabilities; this is why the documentation for
> the validate callback has this note:
> 
>  * Note that the PHY may be able to transform from one connection
>  * technology to another, so, eg, don't clear 1000BaseX just
>  * because the MAC is unable to BaseX mode. This is more about
>  * clearing unsupported speeds and duplex settings.
> 
> To put it another way - if the link between the MAC and PHY supports
> 10G speed, the MAC should indicate that _all_ 10G ethtool link modes
> that support 10G speed are set in the supported mask.  If the link
> supports 1G speeds, then all 1G ethtool link modes that can be
> supported irrespective of technology should be set.  This is because
> the capabilities of the overall setup is the logical union of the
> capabilities of each device in the setup.
> 
> So, if, say, the USXGMII link can support 2.5Gbps, and the PHY
> supports copper media at 2.5Gbps via the NBASE-T specifications,
> then the system as a whole can support
> ETHTOOL_LINK_MODE_2500baseT_Full_BIT.  If the MAC decides to clear
> that bit, then the system can't support 2.5Gbps even if the PHY
> does.
> 
> Maybe what we need to do with phylink is move away from exposing
> ethtool link modes to the MAC validate callback, and instead
> devise a way for the MAC to indicate merely the speeds and duplexes
> it supports without any of the technology stuff getting in the way.
> 
> -- 
> RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=WXDOl4mhp4Xwg4de0KTKH9RNulfBLbkA6MQGZ1G9_FI&s=DbL6asfMKlMoCCeLhWoeBLrUQ0FSIrLkJCKoVbKVEQg&e= 
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> According to speedtest.net: 11.9Mbps down 500kbps up


---
Thanks,
Jose Miguel Abreu

Re: Clause 73 and USXGMII

[Russell King - ARM Linux admin]

On Fri, Aug 09, 2019 at 06:42:39PM +0000, Jose Abreu wrote:
> From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> Date: Aug/08/2019, 13:09:03 (UTC+00:00)
> 
> > On Thu, Aug 08, 2019 at 11:45:29AM +0000, Jose Abreu wrote:
> > > From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> > > Date: Aug/08/2019, 10:23:13 (UTC+00:00)
> > > 
> > > > On Thu, Aug 08, 2019 at 09:02:57AM +0000, Jose Abreu wrote:
> > > > > From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> > > > > Date: Aug/08/2019, 09:26:26 (UTC+00:00)
> > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > Have you tried enabling debug mode in phylink (add #define DEBUG at the
> > > > > > top of the file) ?
> > > > > 
> > > > > Yes:
> > > > > 
> > > > > [ With > 2.5G modes removed ]
> > > > > # dmesg | grep -i phy
> > > > > libphy: stmmac: probed
> > > > > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > > > > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > > > > 00,00000000,0002e040 advertising 00,00000000,0002e040
> > > > > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > > > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > > > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
> > > > > link=0 an=1
> > > > > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > > > 
> > > > This shows that the PHY isn't reporting that the link came up.  Did
> > > > the PHY negotiate link?  If so, why isn't it reporting that the link
> > > > came up?  Maybe something is mis-programming the capability bits in
> > > > the PHY?  Maybe disabling the 10G speeds disables everything faster
> > > > than 1G?
> > > 
> > > Autoneg was started but never finishes and disabling 10G modes is 
> > > causing autoneg to fail.
> > > 
> > > > 
> > > > > [ Without any limit ]
> > > > > # dmesg | grep -i phy
> > > > > libphy: stmmac: probed
> > > > > stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> > > > > stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> > > > > 00,00000000,000ee040 advertising 00,00000000,000ee040
> > > > > stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> > > > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > > > mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
> > > > > link=0 an=1
> > > > > stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> > > > > stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
> > > > > stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> > > > > mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
> > > > > an=0
> > > > > 
> > > > > I'm thinking on whether this can be related with USXGMII. As link is 
> > > > > operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
> > > > > should always be 10G ?
> > > > 
> > > > As I understand USXGMII (which isn't very well, because the spec isn't
> > > > available) I believe that it operates in a similar way to SGMII where
> > > > data is replicated the appropriate number of times to achieve the link
> > > > speed.  So, the USXGMII link always operates at a bit rate equivalent
> > > > to 10G, but data is replicated twice for 5G, four times for 2.5G, ten
> > > > times for 1G, etc.
> > > > 
> > > > I notice that you don't say that you support any copper speeds, which
> > > > brings up the question about what the PHY's media is...
> > > 
> > > I just added the speeds that XPCS supports within Clause 73 
> > > specification:
> > > Technology Ability field. Indicates the supported technologies:
> > > 	A0: When this bit is set to 1, the 1000BASE-KX technology is supported
> > > 	A1: When this bit is set to 1, the 10GBASE-KX4 technology is supported
> > > 	A2: When this bit is set to 1, the 10GBASE-KR technology is supported
> > > 	A11: When this bit is set to 1, the 2.5GBASE-KX technology is supported
> > > 	A12: When this bit is set to 1, the 5GBASE-KR technology is supported
> > > 
> > > And, within USXGMII, XPCS supports the following:
> > > 	Single Port: 10G-SXGMII, 5G-SXGMII, 2.5G-SXGMII
> > > 	Dual Port: 10G-DXGMII, 5G-DXGMII
> > > 	Quad Port: 10G-XGMII
> > > 
> > > My HW is currently fixed for USXGMII at 2.5G.
> > 
> > So what do you actually have?
> > 
> > +-----+              +----------+
> > | STM |    USXGMII   | Synopsis |   ????????
> > | MAC | <----------> |   PHY    | <----------> ????
> > |     |     link     |          |
> > +-----+              +----------+ (media side)
> > 
> > Does the above refer to what the STM MAC and Synopsis PHY support for
> > the USXGMII link?  What about the media side?
> 
> This is my setup:
> 
> XGMAC <-> XPCS <-> Xilinx SERDES <-> SFP
> 
> I'm not sure about the media side. I use a passive SFP cable if that 
> helps ...

And at the other end of the passive SFP cable?  I guess some kind of
standard networking device.

If the SFP is running at 10G, the protocol to the SFP is expected to be
10GBASE-R - which is fixed at 10G speed.  Unless the Xilinx serdes
(which you previously stated was a PHY) is capable of speed conversion
between 2.5G to the XPCS over USXGMII and 10G to the SFP, then I don't
see how you can expect this setup to work - at least not in a standard
environment.

In any case, I don't think it is phylink causing your problems. Without
knowing more about the "phy", what the Xilinx serdes is capable of, and
how that is being programmed, I don't see how I can provide further
help.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up