[PATCH net-next v4 00/14] sfc: add vDPA support for EF100 devices

Gautam Dawar
2023-04-07T08:10:51Z

Hi All,

This series adds the vdpa support for EF100 devices.
For now, only a network class of vdpa device is supported and
they can be created only on a VF. Each EF100 VF can have one
of the three function personalities (EF100, vDPA & None) at
any time with EF100 being the default. A VF's function personality
is changed to vDPA while creating the vdpa device using vdpa tool.

A vDPA management device is created per VF to allow selection of
the desired VF for vDPA device creation. The MAC address for the
target net device must be set either by specifying at the vdpa
device creation time via the `mac` parameter of the `vdpa dev add`
command or should be specified as the hardware address of the virtual
function using `devlink port function set hw_addr` command before
creating the vdpa device with the former taking precedence.

To use with vhost-vdpa, QEMU version 6.1.0 or later must be used
as it fixes the incorrect feature negotiation (vhost-vdpa backend)
without which VIRTIO_F_IN_ORDER feature bit is negotiated but not
honored when using the guest kernel virtio driver.

Changes since v3:

- Removed the patch v3 13/13 which was included mistakenly in the series
- Fixed build error and warning on patches 4 and 6, reported by kernel
test robot.

Changes since v2:

- Introduced vdpa state EF100_VDPA_STATE_SUSPENDED to avoid updating
vdpa device config space after device is suspended during VM LM
- Removed the masking off of features not supported by SVQ implementation
in QEMU for Live Migration. This in-turn imposes the restriction of using
QEMU version 6.1.0 and above with vhost-vdpa
- Used IS_ENABLED(CONFIG_SFC_VDPA) to replace #ifdef CONFIG_SFC_VDPA,
wherever possible
- Updated the values for EF100_VRING_XXX_CONFIGURED macros to use the
initial bits (0, 1, 2 and 3)
- Used __maybe_unused in ef100_probe_vf() to avoid #ifdef for conditional
compilation
- Fixed possible uninitialized return code from ef100_vdpa_delete_filter()
- Avoided use of goto and else at a couple of places in filters handling
- Replaced switch statement with single case with if statement in a couple
of functions in mcdi_vdpa.c
- Updated patch 4 commit description to explain the need of refactoring
around efx_ef100_update_tso_features()

Changes since v1:

- To ensure isolation between DMA initiated by userspace (guest OS)
and the host MCDI buffer, ummap VF's MCDI DMA buffer and use PF's
IOMMU domain instead for executing vDPA VF's MCDI commands.
- As a result of above change, it is no more necessary to check for
MCDI buffer's IOVA range overlap with the guest buffers. Accordingly,
the DMA config operations and the rbtree/list implementation to store
IOVA mappings have been dropped.
- Support vDPA only if running Firmware supports CLIENT_CMD_VF_PROXY
capability.
- Added .suspend config operation and updated get_vq_state/set_vq_state
to support Live Migration. Also, features VIRTIO_F_ORDER_PLATFORM and
VIRTIO_F_IN_ORDER have been masked off in get_device_features() to
allow Live Migration as QEMU SVQ doesn't support them yet.
- Removed the minimum version (v6.1.0) requirement of QEMU as
VIRTIO_F_IN_ORDER is not exposed
- Fetch the vdpa device MAC address from the underlying VF hw_addr (if
set via `devlink port function set hw_addr` command)
- Removed the mandatory requirement of specifying mac address while
creating vdpa device
- Moved create_vring_ctx() and get_doorbell_offset() in dev_add()
- Moved IRQ allocation at the time of vring creation
- Merged vring_created member of struct ef100_vdpa_vring_info as one
of the flags in vring_state
- Simplified .set_status() implementation
- Removed un-necessary vdpa_state checks against
EF100_VDPA_STATE_INITIALIZED
- Removed userspace triggerable warning in kick_vq()
- Updated year 2023 in copyright banner of new files

Gautam Dawar (14):
sfc: add function personality support for EF100 devices
sfc: implement MCDI interface for vDPA operations
sfc: update MCDI headers for CLIENT_CMD_VF_PROXY capability bit
sfc: evaluate vdpa support based on FW capability CLIENT_CMD_VF_PROXY
sfc: implement init and fini functions for vDPA personality
sfc: implement vDPA management device operations
sfc: implement vdpa device config operations
sfc: implement vdpa vring config operations
sfc: implement device status related vdpa config operations
sfc: implement filters for receiving traffic
sfc: use PF's IOMMU domain for running VF's MCDI commands
sfc: unmap VF's MCDI buffer when switching to vDPA mode
sfc: update vdpa device MAC address
sfc: register the vDPA device

drivers/net/ethernet/sfc/Kconfig | 8 +
drivers/net/ethernet/sfc/Makefile | 1 +
drivers/net/ethernet/sfc/ef10.c | 2 +-
drivers/net/ethernet/sfc/ef100.c | 8 +-
drivers/net/ethernet/sfc/ef100_netdev.c | 26 +-
drivers/net/ethernet/sfc/ef100_nic.c | 186 +-
drivers/net/ethernet/sfc/ef100_nic.h | 29 +-
drivers/net/ethernet/sfc/ef100_vdpa.c | 548 +++
drivers/net/ethernet/sfc/ef100_vdpa.h | 225 ++
drivers/net/ethernet/sfc/ef100_vdpa_ops.c | 793 ++++
drivers/net/ethernet/sfc/mcdi.c | 108 +-
drivers/net/ethernet/sfc/mcdi.h | 13 +-
drivers/net/ethernet/sfc/mcdi_filters.c | 51 +-
drivers/net/ethernet/sfc/mcdi_functions.c | 9 +-
drivers/net/ethernet/sfc/mcdi_functions.h | 3 +-
drivers/net/ethernet/sfc/mcdi_pcol.h | 4391 ++++++++++++++++++++-
drivers/net/ethernet/sfc/mcdi_vdpa.c | 251 ++
drivers/net/ethernet/sfc/mcdi_vdpa.h | 83 +
drivers/net/ethernet/sfc/net_driver.h | 21 +
drivers/net/ethernet/sfc/ptp.c | 4 +-
20 files changed, 6582 insertions(+), 178 deletions(-)
create mode 100644 drivers/net/ethernet/sfc/ef100_vdpa.c
create mode 100644 drivers/net/ethernet/sfc/ef100_vdpa.h
create mode 100644 drivers/net/ethernet/sfc/ef100_vdpa_ops.c
create mode 100644 drivers/net/ethernet/sfc/mcdi_vdpa.c
create mode 100644 drivers/net/ethernet/sfc/mcdi_vdpa.h

--
2.30.1