Loopback, PRBS and link topo enumeration

Loopback, PRBS and link topo enumeration

[Maxime Chevallier]

Hi everyone,

This week is the Netdev Conference in Rome, and at this occasion, we've
had a small chat with Oleksij Rempel, Björn Töpel and Alexander Duyck
about the ongoing efforts for PRBS and Loopback support, that I'd like
to summarize so that people who didn't attend can jump-in on the topic.

This topic has also been brought-up at the Netconf conference, which
Jakub summarized well during the Netdev event.

There's a good discussion going on about the PRBS topic lead by folks at
Intel here [1], I don't want to hijack or replace this discussion, as it's
on a good track to define a uAPI for PRBS setups.

[1] : https://lore.kernel.org/netdev/SN7PR11MB810923BBDEC398E860099150FF1B2@SN7PR11MB8109.namprd11.prod.outlook.com/

The topic we discussed was about expressing as part of the uAPI the location
of the PRBS and loopback configuration in the components of a NIC.

Loopback in particular, and to some extent PRBS, can be configured in
different blocks of an Ethernet interface, and it would be best if we had
an ability to:

 - List the different components of an interface down to the 802.3 level :
   PCS, PMA, PMD, RS, etc.
   
 - List the capabilities of these blocks :
   - Loopback ability: local, remote, none, both
   - PRBS ability: availability, supported algorithms
   - Any other stats, pkt gen, etc. that we could use at a low-level
   
 - List the "packages" (maybe "domain" or "component" is a better name ?) and
   their interconnections :
   - Are these PCS/PMA/PMD grouped into the same HW block i.e. an ethernet PHY,
     a MAC-side block, some SerDes (in the Generic PHY sense) driven by
     linux or firmware, etc.
     
  - List the lanes available
     
A proposed approach would be to decompose this into separate ethnl commands :

 - A set of commands to report the HW layout to the user :
  ethtool --interface-capabilities ?
  
  it would return a list of the above mentionned blocks, packages, as well
  as an identifier for each block, something like what Björn has proposed for
  loopback [2] :

[2] : https://lore.kernel.org/netdev/20260325145022.2607545-4-bjorn@kernel.org/

"  
Each loopback point is described by a nested entry attribute
containing:

 - component  where in the path (MAC, PHY, MODULE)
 - name       subsystem label, e.g. "cmis-host" or "cmis-media"
 - id         optional instance selector (e.g. PHY id, port id)
 - depth      ordering index within a component (0 = first/only)
 - supported  bitmask of supported directions
 - direction  LOCAL, REMOTE, or 0 (disabled)  
"

We'd remove the "direction" part, but "component" is same as "domain",
name could be an enum instead (PMA / PMD / RS / FEC / PCS ?) or we keep
it as a string, id should be renamed "component_id", and we would return
an extra 'id', unique per entry, that we would use for the actual
PRBS/Loopback command, here as the 'block' parameter.

   
   ethtool --set-loopback eth0 block <XXX> type local
   
   ethtool --phy-test eth0 block <XXX> lane 0 tx-pattern prbs7
   
In terms of kAPI, this would go as an ethtool ops to the MAC, that would
then dispact enumeration to phylink, phylib, the local firmware, the generic
phy (with a new API for that), etc. Oleksij mentionned that we'll have to
also deal with DSA at some point, to configure loopback on the switch-side of
the conduit.
 
 (Extra idea that wasn't discussed, adding it as I'm writing :)
Or we could have the netdev/phy/phy_device/phylink/sfp register their
components to the phy_link_topology struct that we would extend, that way
we would only to the block enumeration at netdev bringup.
 
As you can see, we didn't really discuss the actual 'how' for PRBS /
loopback feature, but rather the 'where do we configure this' part.


  Another approach was mentionned, that is to do all that outside of net,
as a dedicated "diag" framework where each component (generic PHY, phy_device,
mac, etc.) would register prbs/loopback/etc as new devices in a new diag
framework, a bit like hwmon, and the control knobs would be in sysfs or
directly in "/dev".

We however may face complex issues having to sync the netdev
state with the ongoing diag, i.e. making sure the netdev becomes unavailable
when we have a diag in progress, and that may be hard to do.

Björn's work on loopback IMO is a good starting point for the topo enumeration,
I don't think this impacts much the ongoing PRBS discussions as you're on 
a good track defining the uAPI :)

Thanks everyone,

Maxime