[PATCH net-next v6 0/7] devmem/io_uring: allow more flexibility for ZC DMA devices

[PATCH net-next v6 0/7] devmem/io_uring: allow more flexibility for ZC DMA devices

[Dragos Tatulea]

For TCP zerocopy rx (io_uring, devmem), there is an assumption that the
parent device can do DMA. However that is not always the case:
- Scalable Function netdevs [1] have the DMA device in the grandparent.
- For Multi-PF netdevs [2] queues can be associated to different DMA
  devices.

The series adds an API for getting the DMA device for a netdev queue.
Drivers that have special requirements can implement the newly added
queue management op. Otherwise the parent will still be used as before.

This series continues with switching to this API for io_uring zcrx and
devmem and adds a ndo_queue_dma_dev op for mlx5.

The last part of the series changes devmem rx bind to get the DMA device
per queue and blocks the case when multiple queues use different DMA
devices. The tx bind is left as is.

[1] Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst
[2] Documentation/networking/multi-pf-netdev.rst

Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>

---
Changes sice v5 [6]:
- Added NL_SET_BAD_ATTR for incorrect rq idx. (patch 6)

Changes sice v4 [5]:
- Dropped EXPORT_SYMBOL of netdev_queue_get_dma_dev() (patch 1).
- Fixed nits, typos and line length issues.

Changes sice v3 [4]:
- Moved ndo_queue_get_dma_dev() from header to own file (patch 1).
- Used rel_num_rx_queues for queue bitmap (patch 6).
- Allocate zeroed bitmap (patch 6).
- Validate queue index (patch 6).
- Dropped rxq_dma_dev check (patch 7).
- Fixed incorrect handling of extack message on bad dma dev (patch 7).
- Added conflicting queues in error message (patch 7).
- Dropped RFC status as feedback was mostly positive.

Changes sice v2 [3]:
- Downgraded to RFC status until consensus is reached.
- Implemented more generic approach as discussed during
  v2 review.
- Refactor devmem to get DMA device for multiple rx queues for
  multi PF netdev support.
- Renamed series with a more generic name.

Changes since v1 [2]:
- Dropped the Fixes tag.
- Added more documentation as requeseted.
- Renamed the patch title to better reflect its purpose.

Changes since RFC [1]:
- Upgraded from RFC status.
- Dropped driver specific bits for generic solution.
- Implemented single patch as a fix as requested in RFC.
- Handling of multi-PF netdevs will be handled in a subsequent patch
  series.

[1] RFC: https://lore.kernel.org/all/20250702172433.1738947-2-dtatulea@nvidia.com/
[2]  v1: https://lore.kernel.org/all/20250709124059.516095-2-dtatulea@nvidia.com/
[3]  v2: https://lore.kernel.org/all/20250711092634.2733340-2-dtatulea@nvidia.com/
[4]  v3: https://lore.kernel.org/all/20250815110401.2254214-2-dtatulea@nvidia.com/
[5]  v4: https://lore.kernel.org/all/20250820171214.3597901-1-dtatulea@nvidia.com/
[6]  v5: https://lore.kernel.org/all/20250825063655.583454-1-dtatulea@nvidia.com/

---

Dragos Tatulea (7):
  queue_api: add support for fetching per queue DMA dev
  io_uring/zcrx: add support for custom DMA devices
  net: devmem: get netdev DMA device via new API
  net/mlx5e: add op for getting netdev DMA device
  net: devmem: pull out dma_dev out of net_devmem_bind_dmabuf
  net: devmem: pre-read requested rx queues during bind
  net: devmem: allow binding on rx queues with same DMA devices

 .../net/ethernet/mellanox/mlx5/core/en_main.c |  24 ++++
 include/net/netdev_queues.h                   |   7 +
 io_uring/zcrx.c                               |   3 +-
 net/core/Makefile                             |   1 +
 net/core/devmem.c                             |   8 +-
 net/core/devmem.h                             |   2 +
 net/core/netdev-genl.c                        | 122 +++++++++++++-----
 net/core/netdev_queues.c                      |  27 ++++
 8 files changed, 163 insertions(+), 31 deletions(-)
 create mode 100644 net/core/netdev_queues.c

-- 
2.50.1

[PATCH net-next v6 2/7] io_uring/zcrx: add support for custom DMA devices

[Dragos Tatulea]

Use the new API for getting a DMA device for a specific netdev queue.

This patch will allow io_uring zero-copy rx to work with devices
where the DMA device is not stored in the parent device. mlx5 SFs
are an example of such a device.

Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
---
 io_uring/zcrx.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c
index e5ff49f3425e..319eddfd30e0 100644
--- a/io_uring/zcrx.c
+++ b/io_uring/zcrx.c
@@ -12,6 +12,7 @@
 #include <net/page_pool/helpers.h>
 #include <net/page_pool/memory_provider.h>
 #include <net/netlink.h>
+#include <net/netdev_queues.h>
 #include <net/netdev_rx_queue.h>
 #include <net/tcp.h>
 #include <net/rps.h>
@@ -599,7 +600,7 @@ int io_register_zcrx_ifq(struct io_ring_ctx *ctx,
 		goto err;
 	}
 
-	ifq->dev = ifq->netdev->dev.parent;
+	ifq->dev = netdev_queue_get_dma_dev(ifq->netdev, ifq->if_rxq);
 	if (!ifq->dev) {
 		ret = -EOPNOTSUPP;
 		goto err;
-- 
2.50.1

Re: [PATCH net-next v6 0/7] devmem/io_uring: allow more flexibility for ZC DMA devices

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Wed, 27 Aug 2025 17:39:54 +0300 you wrote:
> For TCP zerocopy rx (io_uring, devmem), there is an assumption that the
> parent device can do DMA. However that is not always the case:
> - Scalable Function netdevs [1] have the DMA device in the grandparent.
> - For Multi-PF netdevs [2] queues can be associated to different DMA
>   devices.
> 
> The series adds an API for getting the DMA device for a netdev queue.
> Drivers that have special requirements can implement the newly added
> queue management op. Otherwise the parent will still be used as before.
> 
> [...]

Here is the summary with links:
  - [net-next,v6,1/7] queue_api: add support for fetching per queue DMA dev
    https://git.kernel.org/netdev/net-next/c/13d8e05adf9d
  - [net-next,v6,2/7] io_uring/zcrx: add support for custom DMA devices
    https://git.kernel.org/netdev/net-next/c/59b8b32ac8d4
  - [net-next,v6,3/7] net: devmem: get netdev DMA device via new API
    https://git.kernel.org/netdev/net-next/c/7c7e94603a76
  - [net-next,v6,4/7] net/mlx5e: add op for getting netdev DMA device
    https://git.kernel.org/netdev/net-next/c/f1debf1a2ef4
  - [net-next,v6,5/7] net: devmem: pull out dma_dev out of net_devmem_bind_dmabuf
    https://git.kernel.org/netdev/net-next/c/512c88fb0e88
  - [net-next,v6,6/7] net: devmem: pre-read requested rx queues during bind
    https://git.kernel.org/netdev/net-next/c/1b416902cd25
  - [net-next,v6,7/7] net: devmem: allow binding on rx queues with same DMA devices
    https://git.kernel.org/netdev/net-next/c/b8aab4bb9585

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html