Re: ethtool settings and SFP modules with PHYs

[Maxime Chevallier <maxime.chevallier@bootlin.com>]

Hello Daniel,

On Mon, 16 Sep 2024 16:36:47 +0100
Daniel Golle <daniel@makrotopia.org> wrote:

> Hi,
> 
> I'm wondering how (or rahter: when?) one is supposed to apply ethtool
> settings, such as modifying advertisement of speed, duplex, ..., with
> SFP modules containing a PHY.
> 
> My first approach was to try catching the event of the PHY being
> attached and then re-applying ethtool settings[1]. As there isn't a
> dedicated event for that, I found that IFF_UP && !IFF_LOWER_UP is as
> close as it gets.
> 
> However, that doesn't go well with some PHY drivers and the result seems
> to depend on a race condition.
> 
> Simply ignoring the supported link modes and assuming the kernel would
> filter them also doesn't work as also the advertised modes get reset
> every time the SFP module is removed or inserted.
> 
> Do you think it would make sense to keep the user selection of
> advertised modes for each networking device accross removal or insertion
> of an SFP module?
> 
> The user selection would by default select all known link modes, using
> ethtool (ioctl or nl) would modify it, while the actually advertised
> modes would always be the intersection of user-selected modes and
> supported modes.

The problem I see is that the modes can change completely depending on
the SFP module that's inserted. If say, you plug a Copper module,
filter advertising to 100BaseT, unplug it, then plug a Fiber module,
you end-up with nothing in the intersection.

Same goes for speed limitation. You can plug a copper SFP, limit the
speed to 100M, switch it with a Fiber SFP that does 1000BaseX, and here
100Mbps isn't possible, and you have an invalid setting.

> 
> Alternatively we could of course also introduce a dedicated NETLINK_ROUTE
> event which fires exactly one time once a new is PHY attached.

That could be done. We have netlink messages in ethtool that report the
PHYs existing on the link (ETHTOOL_A_PHY_GET), that's new and still in
net-next. There's no notification yet, but this is something doable,
adding a netlink notification to indicate that a new PHY was attached.

While this doesn't exist yet, you can take a look at the recent
phy_link_topology work, that allows you to list the PHYs attached to a
netdev, including the ones in SFP modules [1].

A workaround for you can be to wait until the SFP PHY show up in
ethtool --show-phys ethX (it's parent SFP bus name will be populated,
you can filter on that), then re-apply your settings. You'll need a
recent ethtool [2].

A notification would indeed be better, and is something I can prototype
quickly. I was hesitating to add that, but as you show interest in
this, I'm OK to move forward on that :)

[1]: https://lore.kernel.org/netdev/20240821151009.1681151-1-maxime.chevallier@bootlin.com/
[2]: https://git.kernel.org/pub/scm/network/ethtool/ethtool.git/commit/?h=next&id=b3ee7c0fa87032dec614dcc716c08a3b77d80fb0

Thanks,

Maxime