Re: [RFC] RJ45 to SFP auto-sensing and switching in mux-ed single-mac devices (XOR RJ/SFP)

[Maxime Chevallier <maxime.chevallier@bootlin.com>]

Hello everyone,

On Mon, 4 Sep 2023 00:51:05 +0200
Andrew Lunn <andrew@lunn.ch> wrote:

> > To solve that sanely, every PHY-based ethtool probably needs a way
> > to specify which PHY the command is intended for, but then there's
> > the question of how userspace users react to that - because it's
> > likely more than just modifying the ethtool utility, ethtool
> > commands are probably used from many programs.  
> 
> This idea of extending ethtool with a PHY ID has discussed last
> year. It helps solve some of the problems discussed here. You can then
> enumerate all the PHYs connected to a MAC, and operate on each PHY
> independently.
> 
> https://lore.kernel.org/netdev/20221017105100.0cb33490@pc-8.home/

Indeed and I'm actively working on this right now, I have an RFC series
that'll be sent during the week, I'll make sure to CC everyone.

As stated this isn't an easy problem, my course of action is the
following to address this : 

 - First allow addressing individual PHYs attached to the same netdev,
without taking the mux into consideration for now. There are already
cases where several PHYs are attached to one MAC, which is when we have
a PHY between the MAC and SFP port, and a PHY in the SFP module.

As Russell said, there are some ethtool operations today that target
PHYs (cable testing, plca, but more importantly maybe timestamping).
With PHY addressing, we could imagine using the SFP's PHY for
timestamping.

The RFC will be strictly about that, adding the ability to list PHYs
(including th ones in SFP modules), get information on them, but the
main part really is about that id, that we can use in subsequent
commands. I'm also adding a netlink notification upon PHY
hotplugging/removal.

For the actual muxing my current idea is to better model the PHY port,
and allow userspace to pick which port to use (or auto-switch). The
reasonning is that there are a lot of topologies that lead to this
situation : 

 - Your case, with a real mux, switchign between 1 PHY and 1 SFP port :

                 /-- PHY -- RJ45 (8P8C)
   MAC --- mux --|
                 \-- SFP ( -- PHY ) -- RJ45/Fiber

 - PHYs that have an internal mux (the 88x3310 for example, or some
ports of the 88e6390X switch) :

                /-- RJ45
   MAC -- PHY --|
                \-- SFP -- RJ45/Fiber


 - Finally we have products in the wild using a pure-software mux :

        /-- PHY -- RJ45
  MAC --|
        \-- PHY -- RJ45

(muxing is done by putting one of the 2 PHYs in isolate mode).

I think for userspace, it would be better to directly configure which
front-facing port they want to see being used, and the current
representation of PORT_TP/PORT_FIBRE/etc. doesn't give enough details
about that. So I would add the phy_port enumeration (using the PHY id
previously introduced, each PHY would report the ports they have), then
muxing capability. I think we would have a pretty good overview
of the real topology at that point.

This is challenging, and will probably take quite some iterations to
get it right, as there are lots of things to consider, but hopefully 
as this subject is gaining traction (there are already a few people
interested in supporting such a thing) we can make it work.

Thanks,

Maxime


> 	Andrew
>