Re: [net-next RFC PATCH v2 0/8] Add support for qca8k mdio rw in Ethernet packet

[Vladimir Oltean <olteanv@gmail.com>]

On Wed, Dec 08, 2021 at 04:40:32AM +0100, Ansuel Smith wrote:
> I still have to find a solution to a slowdown problem and this is where
> I would love to get some hint.
> Currently I still didn't find a good way to understand when the tagger
> starts to accept packets and because of this the initial setup is slow
> as every completion timeouts. Am I missing something or is there a way
> to check for this?
> After the initial slowdown, as soon as the cpu port is ready and starts
> to accept packet, every transaction is near instant and no completion
> timeouts.

My guess is that the problem with the initial slowdown is that you try
to use the Ethernet based register access before things are set up:
before the master is up and ready, before the switch is minimally set
up, etc.

I think what this Ethernet-based register access technique needs to be
more reliable is a notification about the DSA master going up or down.
Otherwise it won't be very efficient at all, to wait for every single
Ethernet access attempt to time out before attempting a direct MDIO
access.

But there are some problems with offering a "master_going_up/master_going_down"
set of callbacks. Specifically, we could easily hook into the NETDEV_PRE_UP/
NETDEV_GOING_DOWN netdev notifiers and transform these into DSA switch
API calls. The goal would be for the qca8k tagger to mark the
Ethernet-based register access method as available/unavailable, and in
the regmap implementation, to use that or the other. DSA would then also
be responsible for calling "master_going_up" when the switch ports and
master are sufficiently initialized that traffic should be possible.
But that first "master_going_up" notification is in fact the most
problematic one, because we may not receive a NETDEV_PRE_UP event,
because the DSA master may already be up when we probe our switch tree.
This would be a bit finicky to get right. We may, for instance, hold
rtnl_lock for the entirety of dsa_tree_setup_master(). This will block
potentially concurrent netdevice notifiers handled by dsa_slave_nb.
And while holding rtnl_lock() and immediately after each dsa_master_setup(),
we may check whether master->flags & IFF_UP is true, and if it is,
synthesize a call to ds->ops->master_going_up(). We also need to do the
reverse in dsa_tree_teardown_master().