Re: Multi-PHYs and multiple-ports bonding support

[Maxime Chevallier <maxime.chevallier@bootlin.com>]

Hello Andrew,

On Mon, 17 Oct 2022 15:03:59 +0200
Andrew Lunn <andrew@lunn.ch> wrote:

> On Mon, Oct 17, 2022 at 10:24:49AM +0100, Russell King (Oracle) wrote:
> > On Mon, Oct 17, 2022 at 10:51:00AM +0200, Maxime Chevallier wrote:  
> > > 2) Changes in Phylink
> > > 
> > > This might be the tricky part, as we need to track several ports,
> > > possibly connected to different PHYs, to get their state. For
> > > now, I haven't prototyped any of this yet.  
> > 
> > The problem is _way_ larger than phylink. It's a fundamental
> > throughout the net layer that there is one-PHY to one-MAC
> > relationship. Phylink just adopts this because it is the
> > established norm, and trying to fix it is rather rediculous without
> > a use case.
> > 
> > See code such as the ethtool code, where the MAC and associated
> > layers are# entirely bypassed with all the PHY-accessing ethtool
> > commands and the commands are passed directly to phylib for the PHY
> > registered against the netdev.  
> 
> We probably need to model the MII MUX. We can then have netdev->phydev
> and netdev->sfp_bus point to the MUX, which then defers to the
> currently active PHY/SFP for backwards compatibility. Additionally,
> for netlink ethtool, we can add a new property which allows a specific
> PHY/SFP hanging off the MUX to be addressed.

That's a good idea ! I find it pretty elegant indeed, and would be the
right place to implement the switching logic too.

> Modeling the MUX probably helps us with the overall architecture.  As
> Maxime described, there are at least two different architectures: the
> MUX is between the MAC and the PHYs, and the MUX is inside the PHY
> between the host interface and the line interfaces. There are at least
> 4 PHYs like this.
> 
> We also have Russells problem of two PHYs on one path. It would be
> nice to solve that at the same time, which the additional identifier
> attribute should help solve.
> 
> I would probably start this work from the uAPI. How does the uAPI
> work?

From the doc of struct ethtool_link_setting, there seems to be an
attempt to support that already :

"* Some hardware interfaces may have multiple PHYs and/or physical
 * connectors fitted or do not allow the driver to detect which are
 * fitted.  For these interfaces @port and/or @phy_address may be
 * writable, possibly dependent on @autoneg being %AUTONEG_DISABLE.
 * Otherwise, attempts to write different values may be ignored or
 * rejected.
"

https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/ethtool.h#L2047

However, this doesn't allow to enumerate clearly the interfaces
available, and it relies on the mdio address + port. This doesn't
address the chained PHYs as we don't have a clear view of the topology
(but do we need to ?)

I like very much the concept of having a way to address the interfaces
or the parts of the link chain.

Could we introduce a new ethtool cmd, allowing to enumerate the
ports and discover the topology, and another one to get an equivalent
of the ethtool_link_settings for each block in the chain ?

I'll try to prototype a few things to get a clearer picture...

Thanks a lot for your input,

Maxime

> 	Andrew