Re: [RFC PATCH v3 net-next] Documentation: devlink: Add devlink-sd

[Michal Swiatkowski <michal.swiatkowski@linux.intel.com>]

On Thu, Feb 01, 2024 at 06:00:54AM -0800, William Tu wrote:
> 
> On 1/31/24 6:23 PM, Samudrala, Sridhar wrote:
> > External email: Use caution opening links or attachments
> > 
> > 
> > On 1/31/2024 5:17 PM, Jakub Kicinski wrote:
> > > On Wed, 31 Jan 2024 15:02:58 -0800 William Tu wrote:
> > > > > I just did a grep on METADATA_HW_PORT_MUX and assumed bnxt,
> > > > > ice and nfp
> > > > > all do buffer sharing. You're saying you mux Tx queues but not Rx
> > > > > queues? Or I need to actually read the code instead of grepping? :)
> > > > 
> > > > I guess bnxt, ice, nfp are doing tx buffer sharing?
> > > 
> > > I'm not familiar with ice. I'm 90% sure bnxt shares both Rx and Tx.
> > > I'm 99.9% sure nfp does.
> > 
> > In ice, all the VF representor netdevs share a VSI(TX/RX queues). UL/PF
> > netdev has its own VSI and TX/RX queues. But there is patch from Michal
> > under review that is going to simplify the design with a single VSI and
> > all the VF representor netdevs and UL/PF netdev will be sharing the
> > TX/RX queues in switchdev mode.
> > 
> Thank you!
> 
> Reading the ice code, ice_eswitch_remap_rings_to_vectors(), it is setting up
> tx/rx rings for each reps.
> 
> "Each port representor will have dedicated 1 Tx/Rx ring pair, so number of
> rings pair is equal to
>  number of VFs."
> 
> So after Michal's patch, representors will share TX/RX queues of uplink-pf?
> 
> 

Yeah, right, we though about solution like in mlx5, but we can easily
get queues shortage in ice. We need to allow representor to share the
queues. The easiest solution was to move to sharing queues with PF like
(I think so) nfp and few other vendors do.

> > Does mlx5 has separate TX/RX queues for each of its representor netdevs?
> > 
> Yes, in mlx5e_rep_open calls mlx5e_open_locked, which will create TX/RX
> queues like typical mlx5 device.
> 
> Each representor can set it TX/RX queues by using ethtool -L
>

I am a little out of context here. Do you allow also sharing queues
between representors? API for sharing descriptors sounds great, but in
ice we also have queues shortage, because of that we want to use PF
queues instead.

> > > It'd be great if you could do the due diligence rather than guessing
> > > given that you're proposing uAPI extension :(
> > > 
> Working on it!
> 
> Thanks
> 
> William
> 
>

Thaks,
Michal