[PATCH 00/11] mlx5 support for VFIO self test

Jason Gunthorpe
2026-05-01T00:08:48Z

Add an mlx5 driver to VFIO self test. This is largely a remix of the
existing VFIO mlx5 driver in rdma-core. It uses an RDMA loopback QP
to issue RDMA WRITE operations which effectively perform memory
copies using DMA. Since mlx5 has a stable programming ABI this
should work on devices from CX5 to current HW. The device FW must
support the QP loopback configuration.

Also support send_msi by arming completion events of the RDMA WRITE
to trigger MSI delivery.

mlx5 device startup is very complex and most of this code is just
booting the device, with a smaller amount for operating the QP.

This entire series was coded by Claude Code in about 4 days. It
used about 4.5M output tokens, 30 individual sessions and 5600 lines
of AI-generated .md files. I spent an annoying amount of time
de-slopping and cleaning its work product to make it presentable.
However, previous VFIO drivers have taken on the order of 1-2
months to write, so getting one in a week is pretty remarkable.

For those interested, the flow I used was broadly a prompt sequence
sort of like:

- Hey Claude, go look at the falcon series, VFIO self test, the
mlx5 driver, rdma-core and some PDF documentation and make a
plan to put mlx5 under the selftest.
- Write an rdma-core application using the built-in VFIO provider
that can do the required memcpy operations that vfio selftests
wants.
(This resulted in a 1k loc C file that compiled and ran the
first time but had a few bugs related to device programming
that the AI resolved.)
- Replace the rdma-core components with open-coded versions to
create a fully stand-alone program that does the DMA memcpy.
- Review and audit the thing.
[Pause and de-slop it]
- Make it work on a PF too (this is surprisingly hard!).
[Move to a kernel tree and copy all the .md files and .c program
it made]
- Hey Claude, look at all this stuff and make a broad plan to
actually build a VFIO self test.
- Here is my 1 sentence advice on what each patch should look
like, make a detailed plan to make a patch for every one.
[Pause and polish the patch plans]
- Execute plan X then commit it [pause and de-slop each patch,
repeat].
[Review and final polish]

It is based on a tree with the falcon series applied.

Jason Gunthorpe (11):
net/mlx5: Add IFC structures for CQE and WQE
net/mlx5: Move HW constant groups from device.h/cq.h to mlx5_ifc.h
net/mlx5: Extract MLX5_SET/GET macros into mlx5_ifc_macros.h
net/mlx5: Add ONCE and MMIO accessor variants to mlx5_ifc_macros.h
selftests: Add additional kernel functions to tools/include/
selftests: Fix arm64 IO barriers to match kernel
vfio: selftests: Allow drivers to specify required region size
vfio: selftests: Add dev_dbg
vfio: selftests: Add mlx5 driver - HW init and command interface
vfio: selftests: Add mlx5 driver - data path and memcpy ops
vfio: selftests: mlx5 driver - add send_msi support

include/linux/mlx5/cq.h | 10 -
include/linux/mlx5/device.h | 231 +-
include/linux/mlx5/mlx5_ifc.h | 178 ++
include/linux/mlx5/mlx5_ifc_macros.h | 185 ++
tools/arch/arm64/include/asm/barrier.h | 18 +
tools/arch/x86/include/asm/barrier.h | 5 +
tools/include/asm-generic/io.h | 28 +
tools/include/asm/barrier.h | 8 +
tools/include/linux/stddef.h | 10 +
.../selftests/vfio/lib/drivers/mlx5/mlx5.c | 1920 +++++++++++++++++
.../selftests/vfio/lib/drivers/mlx5/mlx5_hw.h | 114 +
.../vfio/lib/drivers/mlx5/mlx5_ifc.h | 1 +
.../vfio/lib/drivers/mlx5/mlx5_ifc_fpga.h | 1 +
.../vfio/lib/drivers/mlx5/mlx5_ifc_macros.h | 1 +
.../lib/include/libvfio/vfio_pci_device.h | 6 +
.../lib/include/libvfio/vfio_pci_driver.h | 3 +
tools/testing/selftests/vfio/lib/libvfio.mk | 1 +
.../selftests/vfio/lib/vfio_pci_driver.c | 2 +
.../selftests/vfio/vfio_pci_driver_test.c | 3 +-
19 files changed, 2485 insertions(+), 240 deletions(-)
create mode 100644 include/linux/mlx5/mlx5_ifc_macros.h
create mode 100644 tools/include/linux/stddef.h
create mode 100644 tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5.c
create mode 100644 tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_hw.h
create mode 120000 tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_ifc.h
create mode 120000 tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_ifc_fpga.h
create mode 120000 tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_ifc_macros.h

base-commit: 75b2e40c376951e2174f08c871389955253d3f5e
--
2.43.0