Re: mv88e6240 configuration broken for B850v3

["Russell King (Oracle)" <linux@armlinux.org.uk>]

On Mon, Dec 06, 2021 at 10:01:11PM +0200, Vladimir Oltean wrote:
> On Mon, Dec 06, 2021 at 08:53:46PM +0100, Andrew Lunn wrote:
> > So i suspect something is missing when phylink sometime later does
> > bring the interface up. It is not fully undoing what this down
> > does. Maybe enable the dev_dbg() in mv88e6xxx_port_set_link() and see
> > what value it has in both the good and bad case?
> 
> Andrew, here is mv88e6xxx_mac_link_down():
> 
> 	if (((!mv88e6xxx_phy_is_internal(ds, port) &&
> 	      !mv88e6xxx_port_ppu_updates(chip, port)) ||
> 	     mode == MLO_AN_FIXED) && ops->port_sync_link)
> 		err = ops->port_sync_link(chip, port, mode, false);
> 
> and here is mv88e6xxx_mac_link_up():
> 
> 	if ((!mv88e6xxx_phy_is_internal(ds, port) &&
> 	     !mv88e6xxx_port_ppu_updates(chip, port)) ||
> 	    mode == MLO_AN_FIXED) {
> 		(...)
> 		if (ops->port_sync_link)
> 			err = ops->port_sync_link(chip, port, mode, true);
> 
> This is the CPU port from Martyn's device tree:
> 
> 	port@4 {
> 		reg = <4>;
> 		label = "cpu";
> 		ethernet = <&switch_nic>;
> 		phy-handle = <&switchphy4>;
> 	};
> 
> It has an internal PHY, so mv88e6xxx_phy_is_internal() will return true.
> True negated is false, so the AND with the other PPU condition is always
> false. BUT: the logic is: "force the link IF it doesn't have an internal
> PHY OR it is a fixed link".
> 
> DSA fabricates a mv88e6xxx_mac_link_down call with MLO_AN_FIXED. So
> ->port_sync_link is called with false even if the PHY is internal, due
> to the right hand operand to the || operator.
> 
> Then the real phylink, not the impersonator, comes along and calls
> mv88e6xxx_mac_link_up with MLO_AN_PHY. The same check is now not
> satisfied, because the input data has changed!
> 
> If we're going to impersonate phylink we could at least provide the same
> arguments as phylink will.

What is going on here in terms of impersonation is entirely reasonable.

The only things in this respect that phylink guarantees are:

1) The MAC/PCS configuration will not be substantially reconfigured
   unless a call to mac_link_down() was made if a call to mac_link_up()
   was previously made.

2) The arguments to mac_link_down() will be the same as the preceeding
   mac_link_up() call - in other words, the "mode" and "interface".

Phylink does *not* guarantee that a call to mac_link_up() or
mac_config() will have the same "mode" as a preceeding call to
mac_link_down(), in the same way that "interface" is not guaranteed.
This has been true for as long as we've had SFPs that need to switch
between MLO_AN_INBAND and MLO_AN_PHY - e.g. because the PHY doesn't
supply in-band information.

So, this has uncovered a latent bug in the Marvell DSA code - and
that is that mac_config() needs to take care of the forcing state
after completing its configuration as I suggested in my previous
reply.

There is also the question whether the automatic fetching of PHY
status information by the hardware should be regarded as a form of
in-band by phylink, even though it isn't true in-band - but from
the software point of view, the PPU's automatic fetching is not
materially different from what happens with SGMII.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!