Lynx 10G SerDes Driver on my kernel

Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

Hello Vladimir,

I want to use your Lynx 10G SerDes driver (plus fman and fsl_guts) on 
our OpenWRT kernel (6.6.110) and don't know how to integrate it properly.

I only found the driver in the NXP kernel[1]. Did you send patch to a 
linux kernel mailing list that I may apply?

Kind regards, Tanjeff


[1] https://github.com/nxp-imx/linux-imx.git


-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Vladimir Oltean]

Hi Tanjeff,

On Tue, Dec 02, 2025 at 12:12:26PM +0100, Tanjeff Moos wrote:
> Hello Vladimir,
> 
> I want to use your Lynx 10G SerDes driver (plus fman and fsl_guts) on
> our OpenWRT kernel (6.6.110) and don't know how to integrate it properly.
> 
> I only found the driver in the NXP kernel[1]. Did you send patch to a
> linux kernel mailing list that I may apply?
> 
> Kind regards, Tanjeff
> 
> 
> [1] https://github.com/nxp-imx/linux-imx.git
>

(I'm unsure why you CCed linux-phy, but OK)

Picking the right base kernel seems to be of utmost importance. From
what I know, OpenWRT maintain their own kernel version and don't use
vendor kernels (they import patches from vendor kernels as .patch files,
however).

I see this commit in OpenWRT:

commit 6942ca412124a1f322e19e2289a59ed8e27b9c24
Author: Pawel Dembicki <paweldembicki@gmail.com>
Date:   Sat Nov 1 23:36:36 2025 +0100

    layerscape: switch to Linux 6.12

    Linux 6.12 has been validated on Layerscape platforms, including NXP
    reference boards and multiple custom designs. Tests covered system
    boot, networking, storage, and common peripherals, with no regressions
    observed.

    Enable 6.12 as the default to continue broader testing and integration.

    Signed-off-by: Pawel Dembicki <paweldembicki@gmail.com>
    Link: https://github.com/openwrt/openwrt/pull/20614
    Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>

To integrate the Lynx 10G driver, you'll need to do that on your own as
patches. The correct git tree used by Layerscape boards in Linux Factory is:
https://github.com/nxp-qoriq/linux/tree/lf-6.12.34-2.1.0

My strong recommendation, assuming you want something stable, is to
integrate the NXP Linux Factory patches, which you'll receive some
official NXP support for.

The rest of the email is just for the sake of full transparency. I will
be talking about my development work, which I strongly recommend that
you do not integrate into OpenWrt at this stage.

The SerDes patches (both for the 10G and the 28G Lynx) are being
constantly reworked in an attempt to get as much upstream as possible.
Since the 10G and the 28G Lynx SerDes blocks are very similar, they
should share a lot of code. For that reason, adding a new driver for the
10G Lynx depends on some changes being made in the 28G Lynx.

I am keeping a personal development branch on top of linux-next here:
https://github.com/vladimiroltean/linux/commits/net-phy-upstreaming
*However* I modify it frequently, *and* I only retest those
modifications rigorously when I need to (i.e. when I need to submit some
patches upstream). I am doing this because any time I am making a change
as a result of a review comment, it has to propagate during rebase like
a shock wave through a huge stack of dishes.

So if you pick up the patches from this branch today, there is no
guarantee you will find them in the same state next week.

The bottleneck is on the patches being accepted into linux-phy.
Currently a group of 15 patches from this development branch have been
submitted to the mailing list a week ago:
https://lore.kernel.org/linux-phy/20251125114847.804961-1-vladimir.oltean@nxp.com/
and that's about it. There are 22 more linux-phy patches on top of that
to be accepted until mainline gains some support for the 10G Lynx. And
that isn't considering:
- device trees, which are also waiting for the acceptance of the driver
- networking consumer driver changes (I'm moving SerDes handling into
  the Lynx PCS driver, in order for not just dpaa-eth to benefit from
  it, but also enetc, felix, dpaa2)

Out of curiosity, for what platform and what use case do you need the patches?

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

Hi Vladimir,

Am 02.12.25 um 17:29 schrieb Vladimir Oltean:
> Hi Tanjeff,
> 
> On Tue, Dec 02, 2025 at 12:12:26PM +0100, Tanjeff Moos wrote:
>> Hello Vladimir,
>>
>> I want to use your Lynx 10G SerDes driver (plus fman and fsl_guts) on
>> our OpenWRT kernel (6.6.110) and don't know how to integrate it properly.
>>
>> I only found the driver in the NXP kernel[1]. Did you send patch to a
>> linux kernel mailing list that I may apply?
>>
>> Kind regards, Tanjeff
>>
>>
>> [1] https://github.com/nxp-imx/linux-imx.git
>>
> 
> (I'm unsure why you CCed linux-phy, but OK)
> 
> Picking the right base kernel seems to be of utmost importance. From
> what I know, OpenWRT maintain their own kernel version and don't use
> vendor kernels (they import patches from vendor kernels as .patch files,
> however).
> 
> I see this commit in OpenWRT:
> 
> commit 6942ca412124a1f322e19e2289a59ed8e27b9c24
> Author: Pawel Dembicki <paweldembicki@gmail.com>
> Date:   Sat Nov 1 23:36:36 2025 +0100
> 
>      layerscape: switch to Linux 6.12
> 
>      Linux 6.12 has been validated on Layerscape platforms, including NXP
>      reference boards and multiple custom designs. Tests covered system
>      boot, networking, storage, and common peripherals, with no regressions
>      observed.
> 
>      Enable 6.12 as the default to continue broader testing and integration.
> 
>      Signed-off-by: Pawel Dembicki <paweldembicki@gmail.com>
>      Link: https://github.com/openwrt/openwrt/pull/20614
>      Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This commit is on the main branch, but not on a release branch. We base 
our OpenWRT build on the stable version v24.10.4, which still uses 
kernel v6.6 for layerscape.

Nevertheless I will consider updating the kernel.

> To integrate the Lynx 10G driver, you'll need to do that on your own as
> patches. The correct git tree used by Layerscape boards in Linux Factory is:
> https://github.com/nxp-qoriq/linux/tree/lf-6.12.34-2.1.0
>
> My strong recommendation, assuming you want something stable, is to
> integrate the NXP Linux Factory patches, which you'll receive some
> official NXP support for.
> 
> The rest of the email is just for the sake of full transparency. I will
> be talking about my development work, which I strongly recommend that
> you do not integrate into OpenWrt at this stage.
> 
> The SerDes patches (both for the 10G and the 28G Lynx) are being
> constantly reworked in an attempt to get as much upstream as possible.
> Since the 10G and the 28G Lynx SerDes blocks are very similar, they
> should share a lot of code. For that reason, adding a new driver for the
> 10G Lynx depends on some changes being made in the 28G Lynx.
> 
> I am keeping a personal development branch on top of linux-next here:
> https://github.com/vladimiroltean/linux/commits/net-phy-upstreaming
> *However* I modify it frequently, *and* I only retest those
> modifications rigorously when I need to (i.e. when I need to submit some
> patches upstream). I am doing this because any time I am making a change
> as a result of a review comment, it has to propagate during rebase like
> a shock wave through a huge stack of dishes.
> 
> So if you pick up the patches from this branch today, there is no
> guarantee you will find them in the same state next week.
> 
> The bottleneck is on the patches being accepted into linux-phy.
> Currently a group of 15 patches from this development branch have been
> submitted to the mailing list a week ago:
> https://lore.kernel.org/linux-phy/20251125114847.804961-1-vladimir.oltean@nxp.com/
> and that's about it. There are 22 more linux-phy patches on top of that
> to be accepted until mainline gains some support for the 10G Lynx. And
> that isn't considering:
> - device trees, which are also waiting for the acceptance of the driver
> - networking consumer driver changes (I'm moving SerDes handling into
>    the Lynx PCS driver, in order for not just dpaa-eth to benefit from
>    it, but also enetc, felix, dpaa2)
> 
> Out of curiosity, for what platform and what use case do you need the patches?
We build WLAN access points for railway (i.e. passenger WLAN). We design 
the hardware ourselves. Older models are still based on PowerPC QorIQ 
(T1024 and others), but the most recent model uses a Layerscape LS1046A. 
This is the one I work with.

We use OpenWRT with our own additions. We build binary images for all 
devices (PowerPC and ARM) from the same OpenWRT tree, using different 
configuration files. Currently, all devices use the same kernel version 
(v6.6.110 + OpenWRT patches + our patches).

Speaking of the device I'm working with, we connected two Broadcom 
BCM84891L PHYs (100M / 1G / 2.5G / 5G / 10G) to MAC9 and MAC10. The PHY 
supports different protocols for different speeds. The only protocol 
supporting all speeds is USXGMII, but this is not supported by LS1046A. 
Therefore we use SGMII (for 100M / 1G), 2500Base-X (for 2.5G) and XFI 
(for 2.5G / 5G / 10G). Yes, we use different protocols for 2.5G; see below.

So far I didn't manage to switch the LS1046A SerDes between SGMII and 
XFI protocols without changing the RCW. However, I can switch between 
SGMII and 2500Base-X at runtime. Therefore we provide different variants 
of the device:

- "Slow" Ethernet (100M / 1G / 2.5G), switching between SGMII and
   2500Base-X at runtime. Here we use the BCM54991L PHY (only up to 2.5G,
   otherwise identical), but this is not relevant for my problem.

- "Fast" Ethernet (2.5G / 5G / 10G), using XFI + rate limiting in the
   MAC.

I patched fman and fman_memac to alter SerDes settings when the PHY 
reports speed changes. This is rude, but worked...

I created the patches for kernel 5.15.167, and they do not apply on 
v6.6.110. Instead of reworking my patch, I'd rather use your work, which 
is much cleaner. Also I hope that it supports switching between SGMII 
and XFI, which would allow our "fast" variant to support all speeds from 
100M to 10G.

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Vladimir Oltean]

On Wed, Dec 03, 2025 at 12:41:16PM +0100, Tanjeff Moos wrote:
> I created the patches for kernel 5.15.167, and they do not apply on
> v6.6.110. Instead of reworking my patch, I'd rather use your work, which
> is much cleaner. Also I hope that it supports switching between SGMII
> and XFI, which would allow our "fast" variant to support all speeds from
> 100M to 10G.

The intention, based on hardware capabilities, is for the SerDes driver
to read the PLL clock net frequencies at probe time, and build for each
of the 2 PLLs a list of supported SerDes protocols.

The phy_validate() API function is supposed to return which SerDes protocols
can be used on a particular lane. Phylink in particular uses this
information to build its phylink_config :: supported_interfaces map.

In the SerDes, each lane can be reconfigured to use the clock net
provided by PLLF or PLLS ("F" = first, "S" - second), further divided or
multiplied by a configurable power of 2 factor.

Furthermore, each lane may support only a subset of SerDes protocols
that its PLL selection can support (depending which protocol converters
are instantiated on it).

For LS1046A, lanes 2 and 3 have all the required protocol converters to
support LANE_MODE_1000BASEX_SGMII, LANE_MODE_2500BASEX and LANE_MODE_10GBASER:
https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c#L774-L790

but there is an incompatibility between your use case and the PLL frequencies.
Each SerDes protocol requires a different clock net frequency:
- LANE_MODE_1000BASEX_SGMII requires PLLnCR0_FRATE_5G
- LANE_MODE_2500BASEX requires PLLnCR0_FRATE_3_125G
- LANE_MODE_10GBASER requires PLLnCR0_FRATE_5_15625G
https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c#L2273-L2298

The problem is that 2 PLLs can't provide 3 distinct clock net frequencies.
So you can either support 1G and 2.5G, or 1G and 10G, or 2.5G and 10G.

Reading your message carefully, it seems that you aren't requesting the
"fast" Ethernet device to support the 2.5G media speed via LANE_MODE_2500BASEX,
because that isn't going to be possible.

You are just requesting to support the 10G, 5G and 2.5G media speeds via
LANE_MODE_10GBASER (with pause-based rate adaptation in the Ethernet PHY),
and the 1G and 100M media speeds via LANE_MODE_1000BASEX_SGMII.
That second case *is* going to be possible, provided that your PLL clock
net frequencies are properly provisioned.

Look at the output from lynx_10g_pll_dump() to avoid any unpleasant
surprises. It should tell you that PLLF supports LANE_MODE_1000BASEX_SGMII,
and PLLS supports LANE_MODE_10GBASER. If it doesn't, you need to make
changes to your hardware design.

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

On 12/3/25 13:26, Vladimir Oltean wrote:
> On Wed, Dec 03, 2025 at 12:41:16PM +0100, Tanjeff Moos wrote:
>> I created the patches for kernel 5.15.167, and they do not apply on
>> v6.6.110. Instead of reworking my patch, I'd rather use your work, which
>> is much cleaner. Also I hope that it supports switching between SGMII
>> and XFI, which would allow our "fast" variant to support all speeds from
>> 100M to 10G.
> 
> The intention, based on hardware capabilities, is for the SerDes driver
> to read the PLL clock net frequencies at probe time, and build for each
> of the 2 PLLs a list of supported SerDes protocols.
> 
> The phy_validate() API function is supposed to return which SerDes protocols
> can be used on a particular lane. Phylink in particular uses this
> information to build its phylink_config :: supported_interfaces map.
> 
> In the SerDes, each lane can be reconfigured to use the clock net
> provided by PLLF or PLLS ("F" = first, "S" - second), further divided or
> multiplied by a configurable power of 2 factor.
> 
> Furthermore, each lane may support only a subset of SerDes protocols
> that its PLL selection can support (depending which protocol converters
> are instantiated on it).
> 
> For LS1046A, lanes 2 and 3 have all the required protocol converters to
> support LANE_MODE_1000BASEX_SGMII, LANE_MODE_2500BASEX and LANE_MODE_10GBASER:
> https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c#L774-L790
>
> but there is an incompatibility between your use case and the PLL frequencies.
> Each SerDes protocol requires a different clock net frequency:
> - LANE_MODE_1000BASEX_SGMII requires PLLnCR0_FRATE_5G
> - LANE_MODE_2500BASEX requires PLLnCR0_FRATE_3_125G
> - LANE_MODE_10GBASER requires PLLnCR0_FRATE_5_15625G
> https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c#L2273-L2298
According to LS1046ARM [1] SGMII requires 1.25 Gbps. I don't understand 
where the 5 GHz comes from. I believe I am missing some knowledge here.

> The problem is that 2 PLLs can't provide 3 distinct clock net frequencies.
> So you can either support 1G and 2.5G, or 1G and 10G, or 2.5G and 10G.
According to table 31-4 "Valid SerDes RCW Encodings and Reference 
Clocks" (page 1919):
- SGMII can be used with PLL=100 MHz or PLL=125 MHZ
- 2.5G SGMII can be used with PLL=125 MHz or PLL=156.25 MHZ
- XFI can be used with PLL=156.25 MHZ
My clocks receive the following frequencies (on all devices, "fast" and 
"slow"):
- PLL 1: 125 MHz
- PLL 2: 156.25 MHz
By choosing the right PLL I can use SGMII, 2.5G SGMII or XFI, which I 
actually do. It is possible to freely assign PLL 1 or PLL 2 to any lane 
using LNxGCR0[RPLL_LES].

> Reading your message carefully, it seems that you aren't requesting the
> "fast" Ethernet device to support the 2.5G media speed via LANE_MODE_2500BASEX,
> because that isn't going to be possible.
Correct.

> You are just requesting to support the 10G, 5G and 2.5G media speeds via
> LANE_MODE_10GBASER (with pause-based rate adaptation in the Ethernet PHY),
> and the 1G and 100M media speeds via LANE_MODE_1000BASEX_SGMII.
Yes, that is my minimum target. I hoped that the "fast" device can 
support all speeds, but it's not mandatory.

> That second case *is* going to be possible, provided that your PLL clock
> net frequencies are properly provisioned.
>
> Look at the output from lynx_10g_pll_dump() to avoid any unpleasant
> surprises. It should tell you that PLLF supports LANE_MODE_1000BASEX_SGMII,
> and PLLS supports LANE_MODE_10GBASER. If it doesn't, you need to make
> changes to your hardware design.
I will, thank you.

Kind regards, Tanjeff

[1] LS1046A Reference Manual, LS1046ARM, Rev. 3, 08/2021

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Vladimir Oltean]

On Wed, Dec 03, 2025 at 05:25:46PM +0100, Tanjeff Moos wrote:
> > Each SerDes protocol requires a different clock net frequency:
> > - LANE_MODE_1000BASEX_SGMII requires PLLnCR0_FRATE_5G
> > - LANE_MODE_2500BASEX requires PLLnCR0_FRATE_3_125G
> > - LANE_MODE_10GBASER requires PLLnCR0_FRATE_5_15625G
> > https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c#L2273-L2298
> 
> According to LS1046ARM [1] SGMII requires 1.25 Gbps. I don't understand
> where the 5 GHz comes from. I believe I am missing some knowledge here.

The clock net is an internal frequency provided by the PLL for lanes.
It is given by PLLnCR0[FRATE_SEL]. I said that the lane can further
divide or multiply it using a power of 2 factor.

Indeed SGMII wants a 1.25 Gbps baud rate, and it gets that by dividing
the 5 GHz clock net provided by the PLL (PLLnCR0_FRATE_5G) by 4 (it uses
RAT_SEL_QUARTER for LNaGCR0_TRAT_SEL and LNaGCR0_RRAT_SEL).

Because we _know_ that lanes can divide the 5 GHz clock net frequency by
4 to get 1.25, we say that a PLL which uses PLLnCR0_FRATE_5G is good for
SGMII.

> > The problem is that 2 PLLs can't provide 3 distinct clock net frequencies.
> > So you can either support 1G and 2.5G, or 1G and 10G, or 2.5G and 10G.
> 
> According to table 31-4 "Valid SerDes RCW Encodings and Reference Clocks"
> (page 1919):
> - SGMII can be used with PLL=100 MHz or PLL=125 MHZ
> - 2.5G SGMII can be used with PLL=125 MHz or PLL=156.25 MHZ
> - XFI can be used with PLL=156.25 MHZ
> My clocks receive the following frequencies (on all devices, "fast" and
> "slow"):
> - PLL 1: 125 MHz
> - PLL 2: 156.25 MHz
> By choosing the right PLL I can use SGMII, 2.5G SGMII or XFI, which I
> actually do. It is possible to freely assign PLL 1 or PLL 2 to any lane
> using LNxGCR0[RPLL_LES].

Oof, this is hard to explain, let me try. The short summary is: you can
use the 125 MHz PLL refclk for both SGMII and 2.5G SGMII, but not at the
same time.

The clock net frequency is derived from the PLL reference clock, but
they are not one and the same thing. Their relationship is:
- From one reference clock you can provide one of two clock net
  frequencies.
- One clock net frequency can be provided from different reference clock
  frequencies.

The entity that loads the RCW configures the SerDes PLLs in this way.

It knows the SerDes protocols per lane, and their respective mappings to
a PLL or the other, through RCW[SRDS_PRTCL_S1].

1. It looks at PLLF. Based on which SerDes protocols are mapped to it, it
programs the clock net frequency (PLLnCR0[FRATE_SEL]) at the right value
for those protocols. It is a *fact* that SGMII requires an FRATE_SEL of
5 GHz, 2500Base-X of 3.125 GHz, etc etc.

2. Then, the RCW loader asks itself - how do I obtain this PLLnCR0[FRATE_SEL]?

As you say: SGMII (aka the 5 GHz clock net frequency) can be obtained
from 100 MHz or 125 MHz reference clock. This information sits in the
RCW[SRDS_PLL_REF_CLK_SEL_S1] field. If 1, it will try to obtain the
clock net from the higher frequency for that protocol. If 0, from the
lower reference clock frequency.

So applied to the particular case of SGMII from step 1, you are telling
it in the RCW to generate 5 GHz clock net from the higher reference
clock, aka 125 MHz. This translates into a write to PLLnCR0[RFCLK_SEL]
of 001b (aka 125 MHz).

This is all to say - when you use SGMII from the 125 MHz refclk, you
have to use RCW[SRDS_PLL_REF_CLK_SEL_S1]=1, and when you use 2500Base-X
from the same refclk, you have to set RCW[SRDS_PLL_REF_CLK_SEL_S1]=0.

What happens under the hood is that these 2 settings result in different
clock net frequencies produced by that PLL, incompatible with each other.
You can either produce one, or the other, at any given time.

In Linux we don't change the PLL refclk and clock net selection that was
made by the RCW. Doing so would mean powering down all lanes that use
that PLL, and unbinding and rebinding the Ethernet driver (because the
supported_interfaces mask will change, and the phylink instances need to
be destroyed and re-created to pick up that change). It is a very
invasive operation.

> > You are just requesting to support the 10G, 5G and 2.5G media speeds via
> > LANE_MODE_10GBASER (with pause-based rate adaptation in the Ethernet PHY),
> > and the 1G and 100M media speeds via LANE_MODE_1000BASEX_SGMII.
> 
> Yes, that is my minimum target. I hoped that the "fast" device can support
> all speeds, but it's not mandatory.

What do you mean "can support all speeds", do you mean the media side
speeds? By switching from 10GBase-R to SGMII it would support them all,
wouldn't it? Above I'm just re-explaining why you can't support 3 SerDes
protocols which require different clock net frequencies when architecturally
we have just 2 PLLs.

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

On 12/3/25 19:00, Vladimir Oltean wrote:
> On Wed, Dec 03, 2025 at 05:25:46PM +0100, Tanjeff Moos wrote:
>>> Each SerDes protocol requires a different clock net frequency:
>>> - LANE_MODE_1000BASEX_SGMII requires PLLnCR0_FRATE_5G
>>> - LANE_MODE_2500BASEX requires PLLnCR0_FRATE_3_125G
>>> - LANE_MODE_10GBASER requires PLLnCR0_FRATE_5_15625G
>>> https://urldefense.com/v3/__https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c*L2273-L2298__;Iw!!I9LPvj3b!C2MhsRe43DSJeXGT0xpGpqZ8TOnWlFtW1VY2DPka6a9H7t5QWgCMneu8mRvQGeR1ZcaSQidiREblHfFkXcKclJkqX_1ePg$ 
>>
>> According to LS1046ARM [1] SGMII requires 1.25 Gbps. I don't understand
>> where the 5 GHz comes from. I believe I am missing some knowledge here.
> 
> The clock net is an internal frequency provided by the PLL for lanes.
> It is given by PLLnCR0[FRATE_SEL]. I said that the lane can further
> divide or multiply it using a power of 2 factor.
> 
> Indeed SGMII wants a 1.25 Gbps baud rate, and it gets that by dividing
> the 5 GHz clock net provided by the PLL (PLLnCR0_FRATE_5G) by 4 (it uses
> RAT_SEL_QUARTER for LNaGCR0_TRAT_SEL and LNaGCR0_RRAT_SEL).
> 
> Because we _know_ that lanes can divide the 5 GHz clock net frequency by
> 4 to get 1.25, we say that a PLL which uses PLLnCR0_FRATE_5G is good for
> SGMII.
> 
>>> The problem is that 2 PLLs can't provide 3 distinct clock net frequencies.
>>> So you can either support 1G and 2.5G, or 1G and 10G, or 2.5G and 10G.
1G and 10G is sufficient. 1G (SGMII) also supports 100M. 10G (XFI) also supports
2.5G and 5G via rate limiting.

>>
>> According to table 31-4 "Valid SerDes RCW Encodings and Reference Clocks"
>> (page 1919):
>> - SGMII can be used with PLL=100 MHz or PLL=125 MHZ
>> - 2.5G SGMII can be used with PLL=125 MHz or PLL=156.25 MHZ
>> - XFI can be used with PLL=156.25 MHZ
>> My clocks receive the following frequencies (on all devices, "fast" and
>> "slow"):
>> - PLL 1: 125 MHz
>> - PLL 2: 156.25 MHz
>> By choosing the right PLL I can use SGMII, 2.5G SGMII or XFI, which I
>> actually do. It is possible to freely assign PLL 1 or PLL 2 to any lane
>> using LNxGCR0[RPLL_LES].
> 
> Oof, this is hard to explain, let me try. The short summary is: you can
> use the 125 MHz PLL refclk for both SGMII and 2.5G SGMII, but not at the
> same time.
> 
> The clock net frequency is derived from the PLL reference clock, but
> they are not one and the same thing. Their relationship is:
> - From one reference clock you can provide one of two clock net
>   frequencies.
> - One clock net frequency can be provided from different reference clock
>   frequencies.
> 
> The entity that loads the RCW configures the SerDes PLLs in this way.
> 
> It knows the SerDes protocols per lane, and their respective mappings to
> a PLL or the other, through RCW[SRDS_PRTCL_S1].
> 
> 1. It looks at PLLF. Based on which SerDes protocols are mapped to it, it
> programs the clock net frequency (PLLnCR0[FRATE_SEL]) at the right value
> for those protocols. It is a *fact* that SGMII requires an FRATE_SEL of
> 5 GHz, 2500Base-X of 3.125 GHz, etc etc.
> 
> 2. Then, the RCW loader asks itself - how do I obtain this PLLnCR0[FRATE_SEL]?
> 
> As you say: SGMII (aka the 5 GHz clock net frequency) can be obtained
> from 100 MHz or 125 MHz reference clock. This information sits in the
> RCW[SRDS_PLL_REF_CLK_SEL_S1] field. If 1, it will try to obtain the
> clock net from the higher frequency for that protocol. If 0, from the
> lower reference clock frequency.
> 
> So applied to the particular case of SGMII from step 1, you are telling
> it in the RCW to generate 5 GHz clock net from the higher reference
> clock, aka 125 MHz. This translates into a write to PLLnCR0[RFCLK_SEL]
> of 001b (aka 125 MHz).
> 
> This is all to say - when you use SGMII from the 125 MHz refclk, you
> have to use RCW[SRDS_PLL_REF_CLK_SEL_S1]=1, and when you use 2500Base-X
> from the same refclk, you have to set RCW[SRDS_PLL_REF_CLK_SEL_S1]=0.
> 
> What happens under the hood is that these 2 settings result in different
> clock net frequencies produced by that PLL, incompatible with each other.
> You can either produce one, or the other, at any given time.
Wow, this was an elaborate explanation which pushed my understanding one level
up. Thank you for investing your time!

For clarity, I like to re-state my PLL configuration:

- "slow" devices:
  - RCW[SRDS_PRTCL_S1] = 0x2233
  - RCW[SRDS_PLL_REF_CLK_SEL_S1] = 0b11
  - PLL 1:     5 GHz (derived from external 125 MHz)
  - PLL 2: 3.125 GHz (derived from external 156.25 MHz)

- "fast" devices:
  - RCW[SRDS_PRTCL_S1] = 0x1133
  - RCW[SRDS_PLL_REF_CLK_SEL_S1] = 0b11
  - PLL 1:       5 GHz (derived from external 125 MHz)
  - PLL 2: 5.15625 GHz (derived from external 156.25 MHz)

My patch never reconfigures the PLLs or their clock net frequencies. It 
assigns lanes to PLL 1 (for SGMII) or PLL 2 (for 2.5SGMII). This is only
done on "slow" devices. The "fast" devices always use XFI (PLL 2).

> 
> In Linux we don't change the PLL refclk and clock net selection that was
> made by the RCW. Doing so would mean powering down all lanes that use
> that PLL, and unbinding and rebinding the Ethernet driver (because the
> supported_interfaces mask will change, and the phylink instances need to
> be destroyed and re-created to pick up that change). It is a very
> invasive operation.
The lynx-10g driver seems to alter PLL assignment in lynx_10g_lane_set_pll().
That means that a lane could be operated in SGMII or 2.5SGMII mode by assigning
the correct PLL (given that the PLLs are properly pre-configured via RCW).

But it also means that in theory I could configure PLL1 = 5 GHz and PLL2 = 5.15625 GHz
and then switch between SGMII (100M/1G) and XFI (2.5G / 5G / 10G via rate limiting).
This way all media speeds would be possible.

However, switching to another PLL isn't enough, I also have to switch the lane's protocol.
This worked for SGMII and 2.5SGMII, but I never managed to switch between SGMII to XFI.

> 
>>> You are just requesting to support the 10G, 5G and 2.5G media speeds via
>>> LANE_MODE_10GBASER (with pause-based rate adaptation in the Ethernet PHY),
>>> and the 1G and 100M media speeds via LANE_MODE_1000BASEX_SGMII.
>>
>> Yes, that is my minimum target. I hoped that the "fast" device can support
>> all speeds, but it's not mandatory.
> 
> What do you mean "can support all speeds", do you mean the media side
> speeds?
Yes, indeed.

> By switching from 10GBase-R to SGMII it would support them all,
> wouldn't it?
I don't think so. SGMII does only support 100M and 1G. Tell me if I'm wrong.

> Above I'm just re-explaining why you can't support 3 SerDes
> protocols which require different clock net frequencies when architecturally
> we have just 2 PLLs.


-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Vladimir Oltean]

On Thu, Dec 04, 2025 at 08:01:10PM +0100, Tanjeff Moos wrote:
> For clarity, I like to re-state my PLL configuration:
> 
> - "slow" devices:
>   - RCW[SRDS_PRTCL_S1] = 0x2233
>   - RCW[SRDS_PLL_REF_CLK_SEL_S1] = 0b11
>   - PLL 1:     5 GHz (derived from external 125 MHz)
>   - PLL 2: 3.125 GHz (derived from external 156.25 MHz)
> 
> - "fast" devices:
>   - RCW[SRDS_PRTCL_S1] = 0x1133
>   - RCW[SRDS_PLL_REF_CLK_SEL_S1] = 0b11
>   - PLL 1:       5 GHz (derived from external 125 MHz)
>   - PLL 2: 5.15625 GHz (derived from external 156.25 MHz)
> 
> My patch never reconfigures the PLLs or their clock net frequencies. It 
> assigns lanes to PLL 1 (for SGMII) or PLL 2 (for 2.5SGMII). This is only
> done on "slow" devices. The "fast" devices always use XFI (PLL 2).

And the lynx-10g kernel driver does the same as your patch, plus more
(below).

> > In Linux we don't change the PLL refclk and clock net selection that was
> > made by the RCW. Doing so would mean powering down all lanes that use
> > that PLL, and unbinding and rebinding the Ethernet driver (because the
> > supported_interfaces mask will change, and the phylink instances need to
> > be destroyed and re-created to pick up that change). It is a very
> > invasive operation.
> The lynx-10g driver seems to alter PLL assignment in lynx_10g_lane_set_pll().

Yes.

> That means that a lane could be operated in SGMII or 2.5SGMII mode by assigning
> the correct PLL (given that the PLLs are properly pre-configured via RCW).

Yes.

> But it also means that in theory I could configure PLL1 = 5 GHz and PLL2 = 5.15625 GHz
> and then switch between SGMII (100M/1G) and XFI (2.5G / 5G / 10G via rate limiting).
> This way all media speeds would be possible.

Yes.

> However, switching to another PLL isn't enough, I also have to switch the lane's protocol.

Yes.

> This worked for SGMII and 2.5SGMII, but I never managed to switch between SGMII to XFI.

Yes, the SoC architecture on devices with the 10G Lynx didn't initially
target the use case of switching between SGMII and XFI, so you won't find
documentation on how to do that. We've since validated that the use case
works on LS1088A, LS2088A and LS1046A by implementing a procedure called
"RCW override", where you use undocumented SoC features in drivers/soc/fsl/guts.c
to essentially change the RCW[SRDS_PRTCL_S1] value to something different
than what was originally loaded at boot time. We've tested the feature
on the above SoCs and it is seamlessly integrated in the lynx-10g driver
(it is not invasive, the user is essentially unaware that this is taking
place).

As mentioned earlier, lynx-10g is in our upstreaming queue, but it may
take a while to reach it. You can get the driver with the RCW override
support from Linux Factory, though.

> >>> You are just requesting to support the 10G, 5G and 2.5G media speeds via
> >>> LANE_MODE_10GBASER (with pause-based rate adaptation in the Ethernet PHY),
> >>> and the 1G and 100M media speeds via LANE_MODE_1000BASEX_SGMII.
> >>
> >> Yes, that is my minimum target. I hoped that the "fast" device can support
> >> all speeds, but it's not mandatory.
> > 
> > What do you mean "can support all speeds", do you mean the media side
> > speeds?
> Yes, indeed.
> 
> > By switching from 10GBase-R to SGMII it would support them all,
> > wouldn't it?
> I don't think so. SGMII does only support 100M and 1G. Tell me if I'm wrong.

I hope I didn't confuse you by naming XFI "10GBase-R". This is what the
10GbE SerDes protocol should truly be called - XFI is the name of an
electrical standard.

All I meant to say here was that by using XFI for the >1G media speeds
and SGMII for the <=1G ones, and dynamically switching SerDes protocols
each time your Broadcom PHY changes phydev->interface as a result of a
media side auto-negotiation process, you should be able to support them
all. Basically, the lynx_pcs_config() function, called by phylink on
major protocol reconfig paths, calls phy_set_mode_ext(lynx->serdes[i],
PHY_MODE_ETHERNET, ifmode), and this is what triggers the SerDes
protocol change.

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

On 12/4/25 20:34, Vladimir Oltean wrote:
> On Thu, Dec 04, 2025 at 08:01:10PM +0100, Tanjeff Moos wrote:
>> This worked for SGMII and 2.5SGMII, but I never managed to switch between SGMII to XFI.
> 
> Yes, the SoC architecture on devices with the 10G Lynx didn't initially
> target the use case of switching between SGMII and XFI, so you won't find
> documentation on how to do that. We've since validated that the use case
> works on LS1088A, LS2088A and LS1046A by implementing a procedure called
> "RCW override", where you use undocumented SoC features in drivers/soc/fsl/guts.c
> to essentially change the RCW[SRDS_PRTCL_S1] value to something different
> than what was originally loaded at boot time. We've tested the feature
> on the above SoCs and it is seamlessly integrated in the lynx-10g driver
> (it is not invasive, the user is essentially unaware that this is taking
> place).
Sounds promising!

> 
> As mentioned earlier, lynx-10g is in our upstreaming queue, but it may
> take a while to reach it. You can get the driver with the RCW override
> support from Linux Factory, though.
I'm still unsure how to get it. Probably I'm completely off track...

I tried flexbuild (https://github.com/nxp/flexbuild) to get sources, but
didn't find the patches.

I also tried 
repo init -u https://github.com/nxp-imx/imx-manifest -b imx-linux-walnascar -m imx-6.12.34-2.1.0.xml
to get the Yocto repo (LF is yocto-based, right?). The "IMX" looks suspicious,
since we are talking about qoriq, but I didn't find LF for qoriq.

The repo https://github.com/nxp-qoriq/linux.git has a branch net/phy which was
merged with commit 0229b2500383. So I will try to "extract" this branch with 
"git format-patch" so I can apply its commits to my OpenWRT-Repo.

BTW, I'm still working on getting my board running again after I upgraded to
kernel 6.12.34 (by merging the corresponding OpenWRT commit).

>>> By switching from 10GBase-R to SGMII it would support them all,
>>> wouldn't it?
>> I don't think so. SGMII does only support 100M and 1G. Tell me if I'm wrong.
> 
> I hope I didn't confuse you by naming XFI "10GBase-R". This is what the
> 10GbE SerDes protocol should truly be called - XFI is the name of an
> electrical standard.
I did grasp the meaning. But yeah, the terms are confusing, even data sheets mix 
them up. It took me some time to understand that my PHY does no SGMII (i.e. no
in-band autoneg) despite its manual claiming that it does.

Anyhow, thanks for clarifying it.

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Vladimir Oltean]

On Fri, Dec 05, 2025 at 12:38:17PM +0100, Tanjeff Moos wrote:
> > As mentioned earlier, lynx-10g is in our upstreaming queue, but it may
> > take a while to reach it. You can get the driver with the RCW override
> > support from Linux Factory, though.
> I'm still unsure how to get it. Probably I'm completely off track...
> 
> I tried flexbuild (https://github.com/nxp/flexbuild) to get sources, but
> didn't find the patches.
> 
> I also tried 
> repo init -u https://github.com/nxp-imx/imx-manifest -b imx-linux-walnascar -m imx-6.12.34-2.1.0.xml
> to get the Yocto repo (LF is yocto-based, right?). The "IMX" looks suspicious,
> since we are talking about qoriq, but I didn't find LF for qoriq.
> 
> The repo https://github.com/nxp-qoriq/linux.git has a branch net/phy which was
> merged with commit 0229b2500383. So I will try to "extract" this branch with 
> "git format-patch" so I can apply its commits to my OpenWRT-Repo.
> 
> BTW, I'm still working on getting my board running again after I upgraded to
> kernel 6.12.34 (by merging the corresponding OpenWRT commit).

To my knowledge, LF is just the kernel component. It is a common
development process between i.MX and Layerscape. You don't need the user
space BSP distribution (Yocto, Flexbuild, whatever).

The nxp-qoriq/linux has tags indicating LF releases. The latest is
lf-6.12.49-2.2.0, which is what you should be looking at. You can first
try to run this kernel directly, and then see using "git log" what you
need to pick up from:
- drivers/net/pcs/pcs-lynx.c
- drivers/net/ethernet/freescale/fman/
- drivers/phy/freescale/phy-fsl-lynx-10g.c
- drivers/soc/fsl/guts.c
- include/linux/phy/phy-fsl-lynx.h
- Documentation/devicetree/bindings/phy/fsl,lynx-10g.yaml

The truth is I don't have something ready-made to give you either, and
some assembly will be required. The Linux Factory changes were made in
chronological order, and it took me some time to disentangle the base
SerDes patches from the backplane link training work (which require
their own clause 72/73 API which I also had to introduce), and the
resulting linearized changes are in my personal net-phy-upstreaming
branch. If you want to take a look there, you can, but you are On Your
Own in terms of figuring out what you need and integrating these changes
into OpenWrt sadly.

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

On 12/5/25 13:44, Vladimir Oltean wrote:
> The latest is
> lf-6.12.49-2.2.0, which is what you should be looking at. 
On Dec 02 you wrote:

    To integrate the Lynx 10G driver, you'll need to do that on your own as
    patches. The correct git tree used by Layerscape boards in Linux Factory is:
    https://github.com/nxp-qoriq/linux/tree/lf-6.12.34-2.1.0

But since lf-6.12.49-2.2.0 is newer I rather choose this one.

> The truth is I don't have something ready-made to give you either, and
> some assembly will be required. The Linux Factory changes were made in
> chronological order, and it took me some time to disentangle the base
> SerDes patches from the backplane link training work (which require
> their own clause 72/73 API which I also had to introduce), and the
> resulting linearized changes are in my personal net-phy-upstreaming
> branch. If you want to take a look there, you can, but you are On Your
> Own in terms of figuring out what you need and integrating these changes
> into OpenWrt sadly.
I'll give it a try. Thanks a lot for your hints! I'll come back to you if I have questions :)

Regards, Tanjeff

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

Hello Vladimir,

I wish you a great year 2026!

Meanwhile I ported some of your code and I got my networking partially 
running. Details follow, but first I have a question.


== Question ==

The lynx-10g driver disables the managed lanes in lynx_10g_init(), and 
they are never enabled afterwards. Probably I didn't port the associated 
code, but can't find where it is.

I modified lynx_10g_init() to leave the lane enabled, which solves the 
problem, but this is a dirty hack.

Question: How is the lane supposed to be enabled?


== Status of my work ==

In case you are interested I provide some details here.

I ported only part of your code (tag lf-6.12.49-2.2.0), mainly the 
lynx-10g driver plus some changes you did in pcs-lynx.c, phy-core.c, 
guts.c, the dpaa drivers plus their associated header files. I 
deliberately omitted code dealing with c72 autonegotiation (for 
backplane), as I don't need it. I'm still working with kernel 6.6.119 
plus OpenWRT patches and cannot easily upgrade.

So far, I have a 2.5 GbE PHY and can run the following modes:
- 100M : SGMII
- 1G:    SGMII
- 2.5G:  2500BaseX

Switching forth and back the modes works seamlessly (with my dirty hack 
to power on the lane!). But I didn't test RCW rewriting as it is not 
needed in this scenario.

Kind regards so far, Tanjeff



-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Vladimir Oltean]

On Thu, Feb 05, 2026 at 05:47:12PM +0100, Tanjeff Moos wrote:
> Question: How is the lane supposed to be enabled?

By a phy_init() and phy_power_on() sequence.

In current mainline this is done by the fman_memac driver:
https://elixir.bootlin.com/linux/v6.18.6/source/drivers/net/ethernet/freescale/fman/fman_memac.c#L526-L545

In LF I've moved the phy_init()
https://github.com/nxp-qoriq/linux/blob/lf-6.12.y/drivers/net/pcs/pcs-lynx.c#L563
and phy_power_on()
https://github.com/nxp-qoriq/linux/blob/lf-6.12.y/drivers/net/pcs/pcs-lynx.c#L435
to the Lynx PCS driver to simplify MAC drivers a little bit, since they
all have to go through the same steps (the Lynx PCS is present in dpaa1,
dpaa2, enetc, felix switch).

-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy

Re: Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

Am 08.02.26 um 14:42 schrieb Vladimir Oltean:
> In LF I've moved the phy_init()
> https://github.com/nxp-qoriq/linux/blob/lf-6.12.y/drivers/net/pcs/pcs-lynx.c#L563
> and phy_power_on()
> https://github.com/nxp-qoriq/linux/blob/lf-6.12.y/drivers/net/pcs/pcs-lynx.c#L435
> to the Lynx PCS driver to simplify MAC drivers a little bit, since they
> all have to go through the same steps (the Lynx PCS is present in dpaa1,
> dpaa2, enetc, felix switch).
Found it, thanks. I missed this code while porting.

Meanwhile I can confirm that all speeds work now, from 100M to 10G. The 
lynx-10g driver switches between sgmii and 10-BaseR on my "fast" 
devices. On my "slow" devices it properly switches between sgmii and 
2500-BaseX. All works well, thank you for your extensive support on that!

Kind regards, Tanjeff


-- 
linux-phy mailing list
linux-phy@lists.infradead.org
https://lists.infradead.org/mailman/listinfo/linux-phy