Re: [net-next 10/15] net/mlx5e: Let channels be SD-aware

[Jakub Kicinski <kuba@kernel.org>]

On Tue, 9 Jan 2024 16:15:50 +0200 Gal Pressman wrote:
> >> I'm confused, how are RX queues related to XPS?  
> > 
> > Separate sentence, perhaps I should be more verbose..  
> 
> Sorry, yes, your understanding is correct.
> If a packet is received on RQ 0 then it is from PF 0, RQ 1 came from PF
> 1, etc. Though this is all from the same wire/port.
> 
> You can enable arfs for example, which will make sure that packets that
> are destined to a certain CPU will be received by the PF that is closer
> to it.

Got it.

> >> XPS shouldn't be affected, we just make sure that whatever queue XPS
> >> chose will go out through the "right" PF.  
> > 
> > But you said "correct" to queue 0 going to PF 0 and queue 1 to PF 1.
> > The queue IDs in my question refer to the queue mapping form the stacks
> > perspective. If user wants to send everything to queue 0 will it use
> > both PFs?  
> 
> If all traffic is transmitted through queue 0, it will go out from PF 0
> (the PF that is closer to CPU 0 numa).

Okay, but earlier you said: "whatever queue XPS chose will go out
through the "right" PF." - which I read as PF will be chosen based
on CPU locality regardless of XPS logic.

If queue 0 => PF 0, then user has to set up XPS to make CPUs from NUMA
node which has PF 0 use even number queues, and PF 1 to use odd number
queues. Correct?

> >> So for example, XPS will choose a queue according to the CPU, and the
> >> driver will make sure that packets transmitted from this SQ are going
> >> out through the PF closer to that NUMA.  
> > 
> > Sounds like queue 0 is duplicated in both PFs, then?  
> 
> Depends on how you look at it, each PF has X queues, the netdev has 2X
> queues.

I'm asking how it looks from the user perspective, to be clear.
From above I gather than the answer is no - queue 0 maps directly 
to PF 0 / queue 0, nothing on PF 1 will ever see traffic of queue 0.

> >> Can you share a link please?  
> > 
> > commit a90d56049acc45802f67cd7d4c058ac45b1bc26f  
> 
> Thanks, will take a look.
> 
> >> All the logic is internal to the driver, so I expect it to be fine, but
> >> I'd like to double check.
> > 
> > Herm, "internal to the driver" is a bit of a landmine. It will be fine
> > for iperf testing but real users will want to configure the NIC.
> 
> What kind of configuration are you thinking of?

Well, I was hoping you'd do the legwork and show how user configuration
logic has to be augmented for all relevant stack features to work with
multi-PF devices. I can list the APIs that come to mind while writing
this email, but that won't be exhaustive :(