Re: Disappearance of network PHYs

[Vladimir Oltean <vladimir.oltean@nxp.com>]

On Wed, Mar 11, 2026 at 07:17:18PM +0100, Andrew Lunn wrote:
> > One other thing could be to rely on phylink to handle that ? It's the
> > one part of the net stack that already handles PHY devices suddenly
> > disappearing, with the SFP case. It has all the logic in place to
> > maintain the netdev's phydev pointer without the MAC driver having to
> > deal with that.
> 
> We might want to consider the different use cases.
> 
> 1) Hot pluggable hardware, and it disappears. As you say, phylink can
> handle this.
> 
> 2) Root unbinds the device, while it is in use, for hardware which is
> only cold pluggable? Cannot we just consider this a foot gun?
> Exploding is O.K?
> 
> Is there a good use case for root unbinding the PHY driver? We should
> only add complexity if there is a good use case.
> 
>    Andrew

I do have a customer specifically targetting equipment for explosive
zones (EX-A, EX-P), which has components powered from independent power
domains that can be administratively powered down and taken out for
servicing, and then plugged in.

The requirements that were formulated to me are:
- it is known ahead of time, typically by a few seconds in advance, that
  a power domain will be turned off.
- the rest of the system must continue to run with no interruption
- the system must pick up the components after being powered back up

Their system is a ZynqMP with 3 SJA1105 switches forming a single DSA
tree.
i.  2 of the 3 switches are in EX areas, and the main switch (in the main
    power domain) must continue to run when the EX area is powered down
ii. one on-board PHY of the main switch, with an M12 connector, is also
    part of an EX area and can be powered down independently

There are isolators on the *GMII and SPI lines, pull ups for MDIO, so
electrically the boards can handle the missing components, but software
is a bit limited, and I had to figure out what is missing.

First off, the customer is OK with unbinding the driver from a device
when the power is about to go down. Linux mostly* makes no access to the
registers of unbound devices, so you can do anything to them in that state.
It also cleanly fails to probe if you try that and there's no power
applied (because probing typically accesses registers, which would fail),
operation which you can retry later with well defined semantics.
So, binding/unbinding was chosen as the simplest tool that gets the job
done.

The DSA assumptions preventing use case i. from working are that a tree
is only set up when complete (all switches present) and torn down when the
first switch goes missing. There is no "degraded" state - I had to add it,
and rewrite DSA tree probing such that each switch contributes independently
with just its bits, rather than the tree doing it all centrally.

For use case ii, the phylib code is actually mostly fine already. You
can unbind a PHY driver as long as the MAC hasn't connected to it, and
nothing bad will happen, the PHY state machine doesn't run, ethtool
operations don't pass through to the PHY, etc. The problem is again in
DSA, which connects to the PHY at probe time and disconnects at remove.
I've moved these operations to ndo_open() and ndo_stop(), and as long as
I run "ip link set swp0 down" prior to unbinding the PHY driver, things
are again fine.

*actually the EX area may be unpowered even at boot time, and in that
case, being that MDIO is an enumerated bus, the kernel will fail to
create a device for the PHY in the EX area. That is completely distinct
from having a device with no driver.

We solved this by disabling PHY enumeration by using an
"ethernet-phy-idAAAA.BBBB" style compatible string.  This tells the
kernel the PHY is always present, so it will create a device for it
without trying to access any PHY ID register. When we apply power and
then run phy_connect(), it works.

My involvement with Wei's device link patch is actually just that I want
to proactively protect this use case from breaking (and to upstream it).
Of course a device link between the PHY in the EX area and the main
switch would throw a wrench in the whole use case ii. This makes it a
bit fragile, given the fact that, as explained, device links seem pretty
common sense for single-port NICs. So it is in my best interest that we
have a clear model where unbinding the driver from a PHY doesn't "explode"
(no reference to EX areas intended), even in uncontrolled situations
where the netdev isn't carefully disconnected from the PHY first.