Re: [PATCH mlx5-next v6 0/4] Dynamically assign MSI-X vectors count

Alexander Duyck
2021-02-09T21:31:05Z

On Tue, Feb 9, 2021 at 5:34 AM Leon Romanovsky <leon@kernel.org> wrote:
>
> From: Leon Romanovsky <leonro@nvidia.com>

<snip>

> --------------------------------------------------------------------
> Hi,
>
> The number of MSI-X vectors is PCI property visible through lspci, that
> field is read-only and configured by the device.
>
> The static assignment of an amount of MSI-X vectors doesn't allow utilize
> the newly created VF because it is not known to the device the future load
> and configuration where that VF will be used.
>
> The VFs are created on the hypervisor and forwarded to the VMs that have
> different properties (for example number of CPUs).
>
> To overcome the inefficiency in the spread of such MSI-X vectors, we
> allow the kernel to instruct the device with the needed number of such
> vectors, before VF is initialized and bounded to the driver.
>
> Before this series:
> [root@server ~]# lspci -vs 0000:08:00.2
> 08:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
> ....
> Capabilities: [9c] MSI-X: Enable- Count=12 Masked-
>
> Configuration script:
> 1. Start fresh
> echo 0 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs
> modprobe -q -r mlx5_ib mlx5_core
> 2. Ensure that driver doesn't run and it is safe to change MSI-X
> echo 0 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_drivers_autoprobe
> 3. Load driver for the PF
> modprobe mlx5_core
> 4. Configure one of the VFs with new number
> echo 2 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs
> echo 21 > /sys/bus/pci/devices/0000\:08\:00.2/sriov_vf_msix_count
>
> After this series:
> [root@server ~]# lspci -vs 0000:08:00.2
> 08:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
> ....
> Capabilities: [9c] MSI-X: Enable- Count=21 Masked-
>
> Thanks
>
> Leon Romanovsky (4):
> PCI: Add sysfs callback to allow MSI-X table size change of SR-IOV VFs
> net/mlx5: Add dynamic MSI-X capabilities bits
> net/mlx5: Dynamically assign MSI-X vectors count
> net/mlx5: Allow to the users to configure number of MSI-X vectors
>
> Documentation/ABI/testing/sysfs-bus-pci | 28 ++++
> .../net/ethernet/mellanox/mlx5/core/main.c | 17 ++
> .../ethernet/mellanox/mlx5/core/mlx5_core.h | 27 ++++
> .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 72 +++++++++
> .../net/ethernet/mellanox/mlx5/core/sriov.c | 58 ++++++-
> drivers/pci/iov.c | 153 ++++++++++++++++++
> include/linux/mlx5/mlx5_ifc.h | 11 +-
> include/linux/pci.h | 12 ++
> 8 files changed, 375 insertions(+), 3 deletions(-)
>

This seems much improved from the last time I reviewed the patch set.
I am good with the drop of the folder in favor of using "sriov" in the
naming of the fields.

For the series:
Reviewed-by: Alexander Duyck <alexanderduyck@fb.com>