[RFC] mdio demux multiplexer driver

[RFC] mdio demux multiplexer driver

[Jisheng Zhang]

Hi,

Assume we have the following implementation:
          ----------
          |mmio reg|
	  ----------
             |
 --------    |      -------
 | MAC0 |--MDIO-----| PHY |
 -------- |     |   -------
          |     |
 -------- |     |   -------
 | MAC1 |--     ----| PHY |
 --------           -------

Both MAC0 and MAC1 have MDIO bus masters, and tie together to
a single set of clock and data lines, which go to some PHYs. While
there's a mmio reg to control which MAC mdio master can operate
the single mdio clock and data lines, so only one MAC can operate
the mdio clock and data lines.

We also need to fully support three use cases: only MAC0 + PHY is used
on board; only MAC1 + PHY is used on board; MAC0 + MAC1 + PHYs are
all used.

How can this be modeled in current mdio? It looks like a kind of demux,
so my initial plan is implement a mdio-demux driver, which firstly
switch mdio master then call the master's corresponding ops such as
read, write, read_c45, write_c45 and so on. What do you think about
this RFC? Any comment/suggestion is appreciated!

thanks

Re: [RFC] mdio demux multiplexer driver

[Andrew Lunn]

On Wed, Aug 13, 2025 at 08:58:06AM +0800, Jisheng Zhang wrote:
> Hi,
> 
> Assume we have the following implementation:
>           ----------
>           |mmio reg|
> 	  ----------
>              |
>  --------    |      -------
>  | MAC0 |--MDIO-----| PHY |
>  -------- |     |   -------
>           |     |
>  -------- |     |   -------
>  | MAC1 |--     ----| PHY |
>  --------           -------
> 
> Both MAC0 and MAC1 have MDIO bus masters, and tie together to
> a single set of clock and data lines, which go to some PHYs. While
> there's a mmio reg to control which MAC mdio master can operate
> the single mdio clock and data lines, so only one MAC can operate
> the mdio clock and data lines.

Where is the SoC boundary? Are the PHYs all external? So there is a
single MDIO bus connected to the outside word? And all the PHYs are on
that external MDIO bus?

> We also need to fully support three use cases: only MAC0 + PHY is used
> on board; only MAC1 + PHY is used on board; MAC0 + MAC1 + PHYs are
> all used.

Linux does not care where a PHY is connected. A PHY is on "some" MDIO
bus. It could be one associated to the MAC, it could be on the MDIO
bus of some other MAC, it could be a GPIO bit banging MDIO bus. It
could be an MDIO bus on its own, not associated to anything. The MAC
DT node has phy-handle pointing to wherever the PHY is.

So, why not KISS. Hard code the MMIO reg so MAC0 is connected to the
PHYs, and MAC1 just uses a phy-handle pointing to the PHYs on MAC0s
MDIO bus.

	Andrew

Re: [RFC] mdio demux multiplexer driver

[Jisheng Zhang]

On Fri, Aug 15, 2025 at 05:51:41AM +0200, Andrew Lunn wrote:
> On Wed, Aug 13, 2025 at 08:58:06AM +0800, Jisheng Zhang wrote:
> > Hi,
> > 
> > Assume we have the following implementation:
> >           ----------
> >           |mmio reg|
> > 	  ----------
> >              |
> >  --------    |      -------
> >  | MAC0 |--MDIO-----| PHY |
> >  -------- |     |   -------
> >           |     |
> >  -------- |     |   -------
> >  | MAC1 |--     ----| PHY |
> >  --------           -------
> > 
> > Both MAC0 and MAC1 have MDIO bus masters, and tie together to
> > a single set of clock and data lines, which go to some PHYs. While
> > there's a mmio reg to control which MAC mdio master can operate
> > the single mdio clock and data lines, so only one MAC can operate
> > the mdio clock and data lines.
> 
> Where is the SoC boundary? Are the PHYs all external? So there is a
> single MDIO bus connected to the outside word? And all the PHYs are on
> that external MDIO bus?

Hi Andrew,

Thank you very much for the comments.

Yep, only one single MDIO bus connected to outside world, all the PHYs
are external and soldered on the board.

> 
> > We also need to fully support three use cases: only MAC0 + PHY is used
> > on board; only MAC1 + PHY is used on board; MAC0 + MAC1 + PHYs are
> > all used.
> 
> Linux does not care where a PHY is connected. A PHY is on "some" MDIO
> bus. It could be one associated to the MAC, it could be on the MDIO
> bus of some other MAC, it could be a GPIO bit banging MDIO bus. It
> could be an MDIO bus on its own, not associated to anything. The MAC
> DT node has phy-handle pointing to wherever the PHY is.
> 
> So, why not KISS. Hard code the MMIO reg so MAC0 is connected to the
> PHYs, and MAC1 just uses a phy-handle pointing to the PHYs on MAC0s
> MDIO bus.

Previously, I went with this solution. But then I met an issue -- who
does the harcoding? bootloader or linux kernel?

Bootloader? Bootloader guys says bootloader only needs to setup linux
bootenv such as DDR, necessary plls/clks, then read kernel Image from
external storage to memory and jump to it. mdio demux setting? No,
it's OS responsibility to setup it.

Linux? This is what this email thread ask for comment -- How does
linux model this?

Another issue is: the hardcoding maybe different on different boards.
E.g If only MAC1 is used, we need to hardcode the MMIO reg to let
MAC1 MDIO master own the single MDIO DATA and CLK line.

Thanks

Re: [RFC] mdio demux multiplexer driver

[Andrew Lunn]

> > So, why not KISS. Hard code the MMIO reg so MAC0 is connected to the
> > PHYs, and MAC1 just uses a phy-handle pointing to the PHYs on MAC0s
> > MDIO bus.
> 
> Previously, I went with this solution. But then I met an issue -- who
> does the harcoding? bootloader or linux kernel?

Linux. The bootloader can do it as well, e.g. for TFTP booting, but
Linux should not rely on the bootloader.

> Linux? This is what this email thread ask for comment -- How does
> linux model this?
> 
> Another issue is: the hardcoding maybe different on different boards.
> E.g If only MAC1 is used, we need to hardcode the MMIO reg to let
> MAC1 MDIO master own the single MDIO DATA and CLK line.

Both Ethernet and MDIO hardware exists, even if it is not used. I
would separate the drivers. Have a MAC driver and an MDIO driver. List
them as separate entities in DT. Always probe the MDIO0 driver. It can
set the MUX registers. Don't probe MDIO1 driver. Even if it does
probe, you know MDIO0 is one to be used, so MDIO1 can still set the
MUX to point to MDIO0.

How messy is the address space? Are the MDIO registers in the middle
of the MAC registers? Is this a standard, off the shelf MAC/MDIO IP?
stmmac? Or something currently without a driver? If you are dealing
with a driver which already exists, it gets a bit messy.

	Andrew

Re: [RFC] mdio demux multiplexer driver

[Jisheng Zhang]

On Sat, Aug 16, 2025 at 04:52:25PM +0200, Andrew Lunn wrote:
> > > So, why not KISS. Hard code the MMIO reg so MAC0 is connected to the
> > > PHYs, and MAC1 just uses a phy-handle pointing to the PHYs on MAC0s
> > > MDIO bus.
> > 
> > Previously, I went with this solution. But then I met an issue -- who
> > does the harcoding? bootloader or linux kernel?
> 
> Linux. The bootloader can do it as well, e.g. for TFTP booting, but
> Linux should not rely on the bootloader.

+1

> 
> > Linux? This is what this email thread ask for comment -- How does
> > linux model this?
> > 
> > Another issue is: the hardcoding maybe different on different boards.
> > E.g If only MAC1 is used, we need to hardcode the MMIO reg to let
> > MAC1 MDIO master own the single MDIO DATA and CLK line.
> 
> Both Ethernet and MDIO hardware exists, even if it is not used. I
> would separate the drivers. Have a MAC driver and an MDIO driver. List
> them as separate entities in DT. Always probe the MDIO0 driver. It can
> set the MUX registers. Don't probe MDIO1 driver. Even if it does
> probe, you know MDIO0 is one to be used, so MDIO1 can still set the
> MUX to point to MDIO0.
> 
> How messy is the address space? Are the MDIO registers in the middle
> of the MAC registers? Is this a standard, off the shelf MAC/MDIO IP?
> stmmac? Or something currently without a driver? If you are dealing

stmmac :(  And the MMIO reg doesn't sit together with MAC IP's.
As can be seen, the stmmac mdio registers sit in the middle of the
MAC regs. And current stmmac still tries to register a mdio driver for
the MDIO bus master. And to be honest, it's not the stmmac make things
messy, but the two MDIO masters sharing the single clk and data lines
makes the mess. Modeling the mmio as a demux seems a just so so but
not perfect solution.

> with a driver which already exists, it gets a bit messy.

Re: [RFC] mdio demux multiplexer driver

[Russell King (Oracle)]

On Mon, Aug 18, 2025 at 11:39:35PM +0800, Jisheng Zhang wrote:
> stmmac :(  And the MMIO reg doesn't sit together with MAC IP's.
> As can be seen, the stmmac mdio registers sit in the middle of the
> MAC regs. And current stmmac still tries to register a mdio driver for
> the MDIO bus master. And to be honest, it's not the stmmac make things
> messy, but the two MDIO masters sharing the single clk and data lines
> makes the mess. Modeling the mmio as a demux seems a just so so but
> not perfect solution.

So, let's say we have four devices, stmmac0, stmmac1, stmmac2 and
stmmac3. We decide that we will access the PHYs via stmmac0.

In DT, we describe the four PHYs under stmmac0's node, with labels
for each. We then reference the lables using phy-handle in each
of the stmmac device nodes.

One of the issues with stmmac is that the driver will successfully
probe without the PHYs being present, because the driver only looks
for PHYs when the netdev is adminsitratively brought up. The problem
here is that there is no "EPROBE_DEFER" mechanism available in this
path. Returning any error code goes straight back to e.g. userspace
and it's up to userspace to decide what to do.

Commands like "ip link" will just report the error and fail.

I don't know how programs like systemd's mega-suite of programs deal
would with any errors - would they retry or declare the interface to
have failed.

While I can see the advantage of a demux driver, this problem remains
whether we have a demux or not, unless somehow the probe of stmmac is
influenced by the presence or not of the demux driver.

Remember of course that the demux driver has to cope with locking to
ensure two transactions can't happen at the same time (it needs a
common lock somehow shared between each user).

If we can't solve these, then one might as well go with all PHYs on
one MDIO bus and keep it relatively simple.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Re: [RFC] mdio demux multiplexer driver

[Andrew Lunn]

> > Both Ethernet and MDIO hardware exists, even if it is not used. I
> > would separate the drivers. Have a MAC driver and an MDIO driver. List
> > them as separate entities in DT. Always probe the MDIO0 driver. It can
> > set the MUX registers. Don't probe MDIO1 driver. Even if it does
> > probe, you know MDIO0 is one to be used, so MDIO1 can still set the
> > MUX to point to MDIO0.
> > 
> > How messy is the address space? Are the MDIO registers in the middle
> > of the MAC registers? Is this a standard, off the shelf MAC/MDIO IP?
> > stmmac? Or something currently without a driver? If you are dealing
> 
> stmmac :(  And the MMIO reg doesn't sit together with MAC IP's.
> As can be seen, the stmmac mdio registers sit in the middle of the
> MAC regs. And current stmmac still tries to register a mdio driver for
> the MDIO bus master. And to be honest, it's not the stmmac make things
> messy, but the two MDIO masters sharing the single clk and data lines
> makes the mess. Modeling the mmio as a demux seems a just so so but
> not perfect solution.

So you only really have problems when MAC0 is not used. Are there any
boards actually designed that way? If there are no such boards at the
moment, you can delay handling that until later.

When later arrives, i would probably look at refactoring the stmmac
MDIO code into a library. The stmmac driver can use the library, and
you can add a new MDIO only driver around the library for when MAC0 is
not used, but you need the MDIO bus.

For the moment, you can add setting the MUX register in the stmmac
glue driver.

	Andrew

Re: [RFC] mdio demux multiplexer driver

[Jisheng Zhang]

On Mon, Aug 18, 2025 at 06:27:33PM +0200, Andrew Lunn wrote:
> > > Both Ethernet and MDIO hardware exists, even if it is not used. I
> > > would separate the drivers. Have a MAC driver and an MDIO driver. List
> > > them as separate entities in DT. Always probe the MDIO0 driver. It can
> > > set the MUX registers. Don't probe MDIO1 driver. Even if it does
> > > probe, you know MDIO0 is one to be used, so MDIO1 can still set the
> > > MUX to point to MDIO0.
> > > 
> > > How messy is the address space? Are the MDIO registers in the middle
> > > of the MAC registers? Is this a standard, off the shelf MAC/MDIO IP?
> > > stmmac? Or something currently without a driver? If you are dealing
> > 
> > stmmac :(  And the MMIO reg doesn't sit together with MAC IP's.
> > As can be seen, the stmmac mdio registers sit in the middle of the
> > MAC regs. And current stmmac still tries to register a mdio driver for
> > the MDIO bus master. And to be honest, it's not the stmmac make things
> > messy, but the two MDIO masters sharing the single clk and data lines
> > makes the mess. Modeling the mmio as a demux seems a just so so but
> > not perfect solution.
> 
> So you only really have problems when MAC0 is not used. Are there any
> boards actually designed that way? If there are no such boards at the
> moment, you can delay handling that until later.
> 
> When later arrives, i would probably look at refactoring the stmmac
> MDIO code into a library. The stmmac driver can use the library, and
> you can add a new MDIO only driver around the library for when MAC0 is
> not used, but you need the MDIO bus.
> 
> For the moment, you can add setting the MUX register in the stmmac
> glue driver.

Hi Andrew, Russell

Thanks a lot for your useful inputs. I will check again and try the
demux and compare it with different solutions.

Thanks a lot!