Re: [PATCH net-next 13/13] devlink, ice: add MSIX vectors as devlink resource

[Jiri Pirko <jiri@nvidia.com>]

Mon, Nov 14, 2022 at 05:03:43PM CET, piotr.raczynski@intel.com wrote:
>On Mon, Nov 14, 2022 at 04:28:38PM +0100, Jiri Pirko wrote:
>> Mon, Nov 14, 2022 at 01:57:55PM CET, michal.swiatkowski@linux.intel.com wrote:
>> >From: Michal Kubiak <michal.kubiak@intel.com>
>> >
>> >Implement devlink resource to control how many MSI-X vectors are
>> >used for eth, VF and RDMA. Show misc MSI-X as read only.
>> >
>> >This is first approach to control the mix of resources managed
>> >by ice driver. This commit registers number of available MSI-X
>> >as devlink resource and also add specific resources for eth, vf and RDMA.
>> >
>> >Also, make those resources generic.
>> >
>> >$ devlink resource show pci/0000:31:00.0
>> >  name msix size 1024 occ 172 unit entry dpipe_tables none
>> 
>> 
>> So, 1024 is the total vector count available in your hw?
>> 
>
>For this particular device and physical function, yes.
>
>
>> 
>> >    resources:
>> >      name msix_misc size 4 unit entry dpipe_tables none
>> 
>> What's misc? Why you don't show occupancy for it? Yet, it seems to be
>> accounted in the total (172)
>> 
>> Also, drop the "msix_" prefix from all, you already have parent called
>> "msix".
>
>misc interrupts are for miscellaneous purposes like communication with
>Firmware or other control plane interrupts (if any).
>
>> 
>> 
>> >      name msix_eth size 92 occ 92 unit entry size_min 1 size_max
>> 
>> Why "size_min is not 0 here?
>
>Thanks, actually 0 would mean disable the eth, default, netdev at all.
>It could be done, however not implemented in this patchset. But for
>cases when the default port is not needed at all, it seems like a good
>idea.

So the behaviour you describe is, if you set number of vectors to 0, the
driver won't instantiate netdev ? That sounds quite odd. In that case,
the minimal value should really be 1. The same goes to rdma.


>
>> 
>> 
>> >	128 size_gran 1 dpipe_tables none
>> >      name msix_vf size 128 occ 0 unit entry size_min 0 size_max
>> >	1020 size_gran 1 dpipe_tables none
>> >      name msix_rdma size 76 occ 76 unit entry size_min 0 size_max
>> 
>> Okay, this means that for eth and rdma, the vectors are fully used, no
>> VF is instanciated?
>
>Yes, in this driver implementation, both eth and rdma will most probably
>be always fully utilized, but the moment you change the size and execute
>`devlink reload` then they will reconfigure with new values.
>
>The VF allocation here is the maximum number of interrupt vectors that
>can be assigned to actually created VFs. If so, then occ shows how many
>are actually utilized by the VFs.
>
>> 
>> 
>> 
>> >	132 size_gran 1 dpipe_tables none
>> >
>> >example commands:
>> >$ devlink resource set pci/0000:31:00.0 path msix/msix_eth size 16
>> >$ devlink resource set pci/0000:31:00.0 path msix/msix_vf size 512
>> >$ devlink dev reload pci/0000:31:00.0