Re: htb offload on vlan (mlx5)

[Maxim Mikityanskiy <maxtram95@gmail.com>]

On Mon, Mar 06, 2023 at 02:59:40PM +0100, Stanisław Czech wrote:
> 06.03.2023  10:35, Maxim Mikityanskiy wrote:
> > That's expected, vlan_features doesn't contain NETIF_F_HW_TC, and I
> > think that's the case for all drivers. Regarding HTB offload, I don't
> > think the current implementation in mlx5e can be easily modified to
> > support being attached to a VLAN only, because the current
> > implementation relies on objects created globally in the NIC.
> > 
> > CCed Nvidia folks in case they have more comments.
> > 
> 
> Thank you for you answer Maxim... I tried to use SR IOV and use the HTB
> offload functionality on the VF
> but it's not possible either:
> 
> ethtool -K enp1s0np0 hw-tc-offload  on
> echo 7 > /sys/class/infiniband/mlx5_0/device/mlx5_num_vfs
> ethtool -K enp1s0f7v6 hw-tc-offload  on
> 
> ip l s dev enp1s0np0 name eth0
> ip l s dev eth0 vf 6 vlan 4
> 
> and I see in
> ethtool -k eth0
> hw-tc-offload: on
> 
> but still:
> Error: mlx5_core: Missing QoS capabilities. Try disabling SRIOV or use a
> supported device.
> 
> So I guess there is no way to use HTB offloading anywhere else than on the
> PF device itself...

Yes, as the error message suggests, when SRIOV is enabled, the firmware
doesn't expose the needed capabilities for HTB offload. That means these
two features aren't compatible at the moment, and there is nothing the
driver could do, because the limitation comes from the firmware side.

> 
> Anyway, maybe using multiple VFS to support multiple VLANs (single VF for
> single vlan) would
> be more efficent than simple vlans on PF interface (regarding qdisc lock
> problem) ?

You mean with non-offloaded HTB? You might try, but there will still be
the lock contention issue in case of multiple queues. There will be
multiple locks, though (one per VF), which might alleviate the
contention, but there are too many variables to guess without actually
testing it. It also depends on how many VLANs you have, because each VF
has its memory footprint. It also may be worth looking at SFs, which are
lighter than VFs.

> I would like to utilize more CPU cores as the vlans on a single PF interface
> use only a single
> cpu core ( the 100% ksoftirqd problem)
> 
> Could this be some workaround?
> 
> 
> Greetings,
> *Stanisław Czech*
>