Re: [net-next V3 15/15] Documentation: networking: Add description for multi-pf netdev

[Jakub Kicinski <kuba@kernel.org>]

On Mon, 19 Feb 2024 17:26:36 +0200 Tariq Toukan wrote:
> > There are multiple devlink instances, right?  
> 
> Right.

Just to be clear I'm asking you questions about things which need to 
be covered by the doc :)

> > In that case we should call out that there may be more than one.
> >   
> 
> We are combining the PFs in the netdev level.
> I did not focus on the parts that we do not touch.

Sure but one of the goals here is to drive convergence.
So if another vendor is on the fence let's nudge them towards the same
decision.

> That's why I didn't mention the sysfs for example, until you asked.
> 
> For example, irqns for the two PFs are still reachable as they used to, 
> under two distinct paths:
> ll /sys/bus/pci/devices/0000\:08\:00.0/msi_irqs/
> ll /sys/bus/pci/devices/0000\:09\:00.0/msi_irqs/
> 
> >> +Currently the sysfs is kept untouched, letting the netdev sysfs point to its primary PF.
> >> +Enhancing sysfs to reflect the actual topology is to be discussed and contributed separately.  
> > 
> > I don't anticipate it to be particularly hard, let's not merge
> > half-baked code and force users to grow workarounds that are hard
> > to remove.
> 
> Changing sysfs to expose queues from multiple PFs under one path might 
> be misleading and break backward compatibility. IMO it should come as an 
> extension to the existing entries.

I don't know what "multiple PFs under one path" means, links in VFs are
one to one, right? :)

> Anyway, the interesting info exposed in sysfs is now available through 
> the netdev genl.

Right, that's true.

Greg, we have a feature here where a single device of class net has
multiple "bus parents". We used to have one attr under class net
(device) which is a link to the bus parent. Now we either need to add
more or not bother with the linking of the whole device. Is there any
precedent / preference for solving this from the device model
perspective?

> Now, is this sysfs part integral to the feature? IMO, no. This in-driver 
> feature is large enough to be completed in stages and not as a one shot.

It's not a question of size and/or implementing everything.
What I want to make sure is that you surveyed the known user space
implementations sufficiently to know what looks at those links,
and perhaps ethtool -i.
Perhaps the answer is indeed "nothing much will care" and given
we can link IRQs correctly we put that as a conclusion in the doc.

Saying "sysfs is coming soon" is not adding much information :(

> > Also could you add examples of how the queue and napis look when listed
> > via the netdev genl on these devices?
> >   
> 
> Sure. Example for a 24-cores system:

Could you reconfigure to 5 channels to make the output asymmetric and
shorter and include the example in the doc?