[PATCH RFC 0/3] RDMA: Post small ULP command messages inline

[PATCH RFC 0/3] RDMA: Post small ULP command messages inline

[Andrei-Alexandru Bleortu]

The iSER initiator, the SRP initiator and the RTRS client each send a
small per-command message by reference, on a QP that asks for no
inline data. The HCA therefore has to DMA-read the message from host
memory before it can put the command on the wire.

Each patch asks for a little inline data when creating the QP and
creates it without inline data if the device refuses. SRP asks for
116 bytes and tries smaller sizes before none. The message goes out with
IB_SEND_INLINE when it fits, with the SGE pointing at the buffer's
kernel virtual address. The messages are:

  iSER  PDUs without a data segment (76 bytes of headers)
  SRP   SRP_CMD IUs without immediate data
  RTRS  read requests, about 60 bytes for RNBD

I measured them on two Xeon Platinum 8360Y hosts with ConnectX-6 Dx
(RoCEv2, 100 Gb/s), with the patches backported to an Ubuntu 6.8 kernel
(same added and removed lines). Every QP was granted 120 bytes of inline
data. Inlining lowered fio's mean 4 KiB random-read latency at queue
depth 1 (io_uring, null_blk target, fio's lat, which includes
submission) by:

  iSER, PDU posted alone              0.479 us (95% CI 0.270-0.689)
  iSER, behind a registration WR      0.701 us (0.375-1.026)
  RNBD over RTRS                      0.348 us (0.309-0.387)
  SRP                                 0.383 us (0.320-0.446)

Each is the mean of 15 paired runs, 10 for SRP. The second iSER line is
with always_register=Y. Each patch inlines the message even when
registration WRs go ahead of it in the same call, since it still saved
time there. Tell me if you would rather inline only when it is posted
alone. Before going further I would like to know whether you want these.

I am sending the same change for the NVMe/RDMA host's command capsule to
linux-nvme. An earlier version of it lowered fio's mean 4 KiB
random-read latency at queue depth 1 by 0.998 us (95% CI 0.943-1.049)
between two AMD EPYC 7452 hosts with ConnectX-5 Ex. Between two Xeon
Platinum 8360Y hosts with ConnectX-6 Dx it fell by 0.589 us
(0.558-0.618). Both used RoCEv2 at 100 Gb/s. I could not find earlier
discussion of inline sends for these ULPs on linux-rdma.

All three build with W=1 on rdma for-next without new warnings, and
the backports ran on the hosts above. RTRS also ran in a QEMU guest
with rdma_rxe, where an RNBD client and server on one host mapped a
null_blk device. 8 MiB of 4 KiB O_DIRECT writes and reads compared
equal, and 2050 of the 2051 read requests went inline. With rxe changed
to refuse inline data, QP creation fell back, every request went by
reference and the data still compared equal.

I wrote these patches with an LLM coding assistant (Claude Code),
which also drafted the commit messages and ran the build and the VM
test.

Andrei-Alexandru Bleortu (3):
  IB/iser: Post header-only PDUs inline
  RDMA/srp: Post command IUs inline
  RDMA/rtrs: Post client read requests inline

 drivers/infiniband/ulp/iser/iscsi_iser.h |   2 +
 drivers/infiniband/ulp/iser/iser_verbs.c |  29 ++++++-
 drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  17 +++-
 drivers/infiniband/ulp/rtrs/rtrs-pri.h   |  10 ++-
 drivers/infiniband/ulp/rtrs/rtrs-srv.c   |   2 +-
 drivers/infiniband/ulp/rtrs/rtrs.c       |  22 +++--
 drivers/infiniband/ulp/srp/ib_srp.c      | 102 ++++++++++++++++++++---
 drivers/infiniband/ulp/srp/ib_srp.h      |   9 ++
 8 files changed, 170 insertions(+), 23 deletions(-)


base-commit: 978361dd2ccff357d4fe6aac6ce057d1bfae8312
-- 
2.56.0

[PATCH RFC 1/3] IB/iser: Post header-only PDUs inline

[Andrei-Alexandru Bleortu]

iser_post_send() posts every PDU by reference, and the QP asks for no
inline data. Before the HCA can send a SCSI command it therefore has
to DMA-read the 76 bytes of iSER and iSCSI headers from host memory,
and that read is on the latency path of every command.

Ask for ISER_HEADERS_LEN bytes of inline data when creating the QP,
and create the QP without inline data if the device refuses. Post a
PDU without a data segment with IB_SEND_INLINE when it fits in the
inline size the device granted. PDUs with a data segment are posted
by reference as before.

Providers copy inline data from sg_list[].addr as a kernel virtual
address. The inline path therefore points tx_sg[0] at the headers'
virtual address, and the by-reference path now sets it to the DMA
address on every post. The mapping is still synced and unmapped through
tx_desc->dma_addr.

Assisted-by: LLM
Signed-off-by: Andrei-Alexandru Bleortu <me@andrei-z.com>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h |  2 ++
 drivers/infiniband/ulp/iser/iser_verbs.c | 29 ++++++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 1d7ac24c4c00..ebd994d425a3 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -362,6 +362,7 @@ struct iser_fr_pool {
  * @device:              reference to iser device
  * @fr_pool:             connection fast registration pool
  * @pi_support:          Indicate device T10-PI support
+ * @max_inline_data:     inline data size granted to the QP
  * @reg_cqe:             completion handler
  */
 struct ib_conn {
@@ -372,6 +373,7 @@ struct ib_conn {
 	struct iser_device          *device;
 	struct iser_fr_pool          fr_pool;
 	bool			     pi_support;
+	u32			     max_inline_data;
 	struct ib_cqe		     reg_cqe;
 };
 
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index ab2a27d23478..6319d7fd17cf 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -230,7 +230,7 @@ static int iser_create_ib_conn_res(struct ib_conn *ib_conn)
 	struct iser_conn *iser_conn = to_iser_conn(ib_conn);
 	struct iser_device	*device;
 	struct ib_device	*ib_dev;
-	struct ib_qp_init_attr	init_attr;
+	struct ib_qp_init_attr	init_attr, attr;
 	int			ret = -ENOMEM;
 	unsigned int max_send_wr, cq_size;
 
@@ -271,11 +271,23 @@ static int iser_create_ib_conn_res(struct ib_conn *ib_conn)
 		init_attr.create_flags |= IB_QP_CREATE_INTEGRITY_EN;
 	iser_conn->max_cmds = ISER_GET_MAX_XMIT_CMDS(max_send_wr - 1);
 
-	ret = rdma_create_qp(ib_conn->cma_id, device->pd, &init_attr);
+	/*
+	 * Ask for enough inline data to post the iSER and iSCSI headers
+	 * inline (see iser_post_send()).  If the device cannot provide it,
+	 * create the QP without inline data.
+	 */
+	attr = init_attr;
+	attr.cap.max_inline_data = ISER_HEADERS_LEN;
+	ret = rdma_create_qp(ib_conn->cma_id, device->pd, &attr);
+	if (ret) {
+		attr = init_attr;
+		ret = rdma_create_qp(ib_conn->cma_id, device->pd, &attr);
+	}
 	if (ret)
 		goto out_err;
 
 	ib_conn->qp = ib_conn->cma_id->qp;
+	ib_conn->max_inline_data = attr.cap.max_inline_data;
 	iser_info("setting conn %p cma_id %p qp %p max_send_wr %d\n", ib_conn,
 		  ib_conn->cma_id, ib_conn->cma_id->qp, max_send_wr);
 	return ret;
@@ -870,6 +882,19 @@ int iser_post_send(struct ib_conn *ib_conn, struct iser_tx_desc *tx_desc)
 	wr->opcode = IB_WR_SEND;
 	wr->send_flags = IB_SEND_SIGNALED;
 
+	/*
+	 * Post a header-only PDU inline if it fits in the granted inline
+	 * size, so the HCA does not have to DMA-read it from host memory.
+	 * The CPU copies inline data from a kernel virtual address, so point
+	 * the SGE at the headers.
+	 */
+	tx_desc->tx_sg[0].addr = tx_desc->dma_addr;
+	if (tx_desc->num_sge == 1 &&
+	    tx_desc->tx_sg[0].length <= ib_conn->max_inline_data) {
+		tx_desc->tx_sg[0].addr = (uintptr_t)&tx_desc->iser_header;
+		wr->send_flags |= IB_SEND_INLINE;
+	}
+
 	if (tx_desc->inv_wr.next)
 		first_wr = &tx_desc->inv_wr;
 	else if (tx_desc->reg_wr.wr.next)
-- 
2.56.0

Re: [PATCH RFC 1/3] IB/iser: Post header-only PDUs inline

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] iser: DMA mapping leak on iser_send_data_out() error path
--

commit 9c1d70cc4e6b5a103b601d5536559ca76fc30e86
Author: Andrei-Alexandru Bleortu <me@andrei-z.com>

IB/iser: Post header-only PDUs inline

iser_post_send() posts every PDU by reference, and the QP asks for no
inline data. Before the HCA can send a SCSI command it therefore has
to DMA-read the 76 bytes of iSER and iSCSI headers from host memory,
and that read is on the latency path of every command.

> diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
> index ab2a27d23478a..6319d7fd17cf1 100644
> --- a/drivers/infiniband/ulp/iser/iser_verbs.c
> +++ b/drivers/infiniband/ulp/iser/iser_verbs.c
[ ... ]
> @@ -870,6 +882,19 @@ int iser_post_send(struct ib_conn *ib_conn, struct iser_tx_desc *tx_desc)
>  	wr->opcode = IB_WR_SEND;
>  	wr->send_flags = IB_SEND_SIGNALED;
>  
> +	/*
> +	 * Post a header-only PDU inline if it fits in the granted inline
> +	 * size, so the HCA does not have to DMA-read it from host memory.
> +	 * The CPU copies inline data from a kernel virtual address, so point
> +	 * the SGE at the headers.
> +	 */
> +	tx_desc->tx_sg[0].addr = tx_desc->dma_addr;
> +	if (tx_desc->num_sge == 1 &&
> +	    tx_desc->tx_sg[0].length <= ib_conn->max_inline_data) {
> +		tx_desc->tx_sg[0].addr = (uintptr_t)&tx_desc->iser_header;
> +		wr->send_flags |= IB_SEND_INLINE;
> +	}
> +
>  	if (tx_desc->inv_wr.next)
>  		first_wr = &tx_desc->inv_wr;
>  	else if (tx_desc->reg_wr.wr.next)

[Severity: Medium]
If iser_post_send() returns an error here (such as when ib_post_send() fails),
iser_send_data_out() handles the error by jumping to send_data_out_error:

drivers/infiniband/ulp/iser/iser_initiator.c:iser_send_data_out() {
...
	err = iser_post_send(&iser_conn->ib_conn, tx_desc);
	if (!err)
		return 0;

send_data_out_error:
	kmem_cache_free(ig.desc_cache, tx_desc);
	iser_err("conn %p failed err %d\n", conn, err);
	return err;
}

When tx_desc was initialized earlier in iser_initialize_task_headers(), a DMA
mapping was created.

Does freeing tx_desc without calling ib_dma_unmap_single() cause a permanent
leak of the IOMMU DMA mapping?

This could potentially exhaust IOMMU space or cause memory corruption if the
address is reused.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261015125118.2966247-1-me@andrei-z.com?part=1

Re: [PATCH RFC 1/3] IB/iser: Post header-only PDUs inline

[Andrei-Alexandru Bleortu]

On Thu, Oct 15, 2026, sashiko-bot@kernel.org wrote:
> Does freeing tx_desc without calling ib_dma_unmap_single() cause a permanent
> leak of the IOMMU DMA mapping?

The error path the review points at predates this patch. The patch
only changes how iser_post_send() fills tx_sg[0] and the send flags;
iser_send_data_out() and iser_initialize_task_headers() are as on
rdma for-next. A Data-Out PDU has two SGEs, so it is never posted
inline, and every unmap uses tx_desc->dma_addr, which the patch does
not touch.

The leak looks real, though: once iser_initialize_task_headers() has
mapped the headers, the offset check and a failed post both free
tx_desc without unmapping it. I can send a separate fix for that.

Re: [PATCH RFC 1/3] IB/iser: Post header-only PDUs inline

[Andrei-Alexandru Bleortu]

Li RongQing already posted this fix:
https://lore.kernel.org/all/20260921033949.1803-1-lirongqing@baidu.com/
I will leave it to that thread.

[PATCH RFC 2/3] RDMA/srp: Post command IUs inline

[Andrei-Alexandru Bleortu]

srp_post_send() posts every information unit by reference, and the QP
asks for no inline data. Before the HCA can send an SRP_CMD it
therefore has to DMA-read the IU from host memory, and that read is
on the latency path of every command.

Ask for SRP_MAX_INLINE_DATA bytes of inline data when creating the QP,
enough for an SRP_CMD with an additional CDB and an indirect table of
two descriptors (116 bytes). If the device refuses, ask for less. Intel
irdma has a fixed limit of 216, 101 or 48 bytes, depending on the
generation, and Alibaba erdma one of 96. On those devices, found by
vendor ID, try these limits first, then step down 16 bytes at a time to
none. libfabric's verbs provider also probes the inline limit by trial
QP creation. The RDMA/CM and IB/CM QP creation paths move into
srp_create_qp() so the retries cover both. Post an IU without immediate
data with IB_SEND_INLINE when it fits in the inline size the device
granted.

Providers copy inline data from sg_list[].addr as a kernel virtual
address, so the inline path points iu->sge[0] at iu->buf.
srp_post_send() resets it to iu->dma on every post, and iu->dma keeps
the mapping.

Assisted-by: LLM
Signed-off-by: Andrei-Alexandru Bleortu <me@andrei-z.com>
---
 drivers/infiniband/ulp/srp/ib_srp.c | 102 ++++++++++++++++++++++++----
 drivers/infiniband/ulp/srp/ib_srp.h |   9 +++
 2 files changed, 99 insertions(+), 12 deletions(-)

diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 955f36efeebd..caac6960bc1d 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -43,6 +43,7 @@
 #include <linux/jiffies.h>
 #include <linux/lockdep.h>
 #include <linux/inet.h>
+#include <linux/pci_ids.h>
 #include <net/net_namespace.h>
 #include <rdma/ib_cache.h>
 
@@ -522,6 +523,66 @@ static void srp_destroy_qp(struct srp_rdma_ch *ch)
 	ib_destroy_qp(ch->qp);
 }
 
+#define SRP_INLINE_STEP	16
+
+/*
+ * The verbs API cannot report a device's inline data limit. Some drivers
+ * have a fixed one: Intel irdma 216, 101 or 48 depending on the
+ * generation, Alibaba erdma 96. Try those first, by vendor ID, then step
+ * down SRP_INLINE_STEP bytes at a time to zero. libfabric's verbs
+ * provider also probes the limit (vrb_find_max_inline()).
+ */
+static const struct {
+	u32	vendor_id;
+	u32	max_inline;
+} srp_inline_limits[] = {
+	{ PCI_VENDOR_ID_INTEL, 216 },
+	{ PCI_VENDOR_ID_INTEL, 101 },
+	{ PCI_VENDOR_ID_INTEL, 48 },
+	{ PCI_VENDOR_ID_ALIBABA, 96 },
+};
+
+/*
+ * Inline data size to ask for: on the first try, @size capped at the
+ * vendor's first known limit; after a refusal, the vendor's next smaller
+ * known limit, else one step smaller.
+ */
+static u32 srp_inline_size(u32 vendor_id, u32 size, bool refused)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(srp_inline_limits); i++)
+		if (srp_inline_limits[i].vendor_id == vendor_id &&
+		    (!refused || srp_inline_limits[i].max_inline < size))
+			return min(srp_inline_limits[i].max_inline, size);
+	return refused ? size - min(size, SRP_INLINE_STEP) : size;
+}
+
+static int srp_create_qp(struct srp_rdma_ch *ch,
+			 struct ib_qp_init_attr *init_attr, struct ib_qp **qpp)
+{
+	struct srp_target_port *target = ch->target;
+	struct srp_device *dev = target->srp_host->srp_dev;
+	struct ib_qp *qp;
+	int ret;
+
+	if (target->using_rdma_cm) {
+		ret = rdma_create_qp(ch->rdma_cm.cm_id, dev->pd, init_attr);
+		qp = ch->rdma_cm.cm_id->qp;
+	} else {
+		qp = ib_create_qp(dev->pd, init_attr);
+		if (!IS_ERR(qp)) {
+			ret = srp_init_ib_qp(target, qp);
+			if (ret)
+				ib_destroy_qp(qp);
+		} else {
+			ret = PTR_ERR(qp);
+		}
+	}
+	*qpp = qp;
+	return ret;
+}
+
 static int srp_create_ch_ib(struct srp_rdma_ch *ch)
 {
 	struct srp_target_port *target = ch->target;
@@ -529,8 +590,10 @@ static int srp_create_ch_ib(struct srp_rdma_ch *ch)
 	const struct ib_device_attr *attr = &dev->dev->attrs;
 	struct ib_qp_init_attr *init_attr;
 	struct ib_cq *recv_cq, *send_cq;
+	struct ib_qp_cap cap;
 	struct ib_qp *qp;
 	struct srp_fr_pool *fr_pool = NULL;
+	u32 inline_size;
 	const int m = 1 + dev->use_fast_reg * target->mr_per_cmd * 2;
 	int ret;
 
@@ -565,18 +628,21 @@ static int srp_create_ch_ib(struct srp_rdma_ch *ch)
 
 	ch->max_imm_sge = min(init_attr->cap.max_send_sge - 1U, 255U);
 
-	if (target->using_rdma_cm) {
-		ret = rdma_create_qp(ch->rdma_cm.cm_id, dev->pd, init_attr);
-		qp = ch->rdma_cm.cm_id->qp;
-	} else {
-		qp = ib_create_qp(dev->pd, init_attr);
-		if (!IS_ERR(qp)) {
-			ret = srp_init_ib_qp(target, qp);
-			if (ret)
-				ib_destroy_qp(qp);
-		} else {
-			ret = PTR_ERR(qp);
-		}
+	/*
+	 * Ask for enough inline data to post command IUs inline (see
+	 * srp_post_send()), or for as much as the device accepts.
+	 */
+	cap = init_attr->cap;
+	inline_size = srp_inline_size(attr->vendor_id, SRP_MAX_INLINE_DATA,
+				      false);
+	for (;;) {
+		init_attr->cap = cap;
+		init_attr->cap.max_inline_data = inline_size;
+		ret = srp_create_qp(ch, init_attr, &qp);
+		if (!ret || !inline_size)
+			break;
+		inline_size = srp_inline_size(attr->vendor_id, inline_size,
+					      true);
 	}
 	if (ret) {
 		pr_err("QP creation failed for dev %s: %d\n",
@@ -604,6 +670,7 @@ static int srp_create_ch_ib(struct srp_rdma_ch *ch)
 	ch->qp = qp;
 	ch->recv_cq = recv_cq;
 	ch->send_cq = send_cq;
+	ch->max_inline_data = init_attr->cap.max_inline_data;
 
 	if (dev->use_fast_reg) {
 		if (ch->fr_pool)
@@ -1915,6 +1982,17 @@ static int srp_post_send(struct srp_rdma_ch *ch, struct srp_iu *iu, int len)
 	wr.opcode     = IB_WR_SEND;
 	wr.send_flags = IB_SEND_SIGNALED;
 
+	/*
+	 * Post an IU without immediate data inline if it fits in the granted
+	 * inline size, so the HCA does not have to DMA-read it from host
+	 * memory.  The CPU copies inline data from a kernel virtual address,
+	 * so point the SGE at iu->buf.
+	 */
+	if (iu->num_sge == 1 && len <= ch->max_inline_data) {
+		iu->sge[0].addr = (uintptr_t)iu->buf;
+		wr.send_flags |= IB_SEND_INLINE;
+	}
+
 	return ib_post_send(ch->qp, &wr, NULL);
 }
 
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index 349576ff3845..b8712e04be5e 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -74,6 +74,14 @@ enum {
 	SRP_IMM_DATA_OFFSET	= sizeof(struct srp_cmd) +
 				  SRP_MAX_ADD_CDB_LEN +
 				  sizeof(struct srp_imm_buf),
+	/*
+	 * Inline data to ask for: enough for an SRP_CMD with an additional
+	 * CDB and an indirect table of two memory descriptors.
+	 */
+	SRP_MAX_INLINE_DATA	= sizeof(struct srp_cmd) +
+				  SRP_MAX_ADD_CDB_LEN +
+				  sizeof(struct srp_indirect_buf) +
+				  2 * sizeof(struct srp_direct_buf),
 };
 
 enum {
@@ -162,6 +170,7 @@ struct srp_rdma_ch {
 	uint32_t		max_ti_iu_len;
 	u8			max_imm_sge;
 	bool			use_imm_data;
+	u32			max_inline_data;
 
 	/* Everything above this point is used in the hot path of
 	 * command processing. Try to keep them packed into cachelines.
-- 
2.56.0

Re: [PATCH RFC 2/3] RDMA/srp: Post command IUs inline

[sashiko-bot]

> srp_post_send() posts every information unit by reference, and the QP
> asks for no inline data. Before the HCA can send an SRP_CMD it
> therefore has to DMA-read the IU from host memory, and that read is
> on the latency path of every command.
> 
> Ask for SRP_MAX_INLINE_DATA bytes of inline data when creating the QP,
> enough for an SRP_CMD with an additional CDB and an indirect table of
> two descriptors (116 bytes). If the device refuses, ask for less. Intel
> irdma has a fixed limit of 216, 101 or 48 bytes, depending on the
> generation, and Alibaba erdma one of 96. On those devices, found by
> vendor ID, try these limits first, then step down 16 bytes at a time to
> none. libfabric's verbs provider also probes the inline limit by trial
> QP creation. The RDMA/CM and IB/CM QP creation paths move into
> srp_create_qp() so the retries cover both. Post an IU without immediate
> data with IB_SEND_INLINE when it fits in the inline size the device
> granted.
> 
> Providers copy inline data from sg_list[].addr as a kernel virtual
> address, so the inline path points iu->sge[0] at iu->buf.
> srp_post_send() resets it to iu->dma on every post, and iu->dma keeps
> the mapping.
> 
> Assisted-by: LLM
> Signed-off-by: Andrei-Alexandru Bleortu <me@andrei-z.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261015125118.2966247-1-me@andrei-z.com?part=2

Re: [PATCH RFC 2/3] RDMA/srp: Post command IUs inline

[Bart Van Assche]

On 10/15/26 5:51 AM, Andrei-Alexandru Bleortu wrote:
> +/*
> + * The verbs API cannot report a device's inline data limit. Some drivers
> + * have a fixed one: Intel irdma 216, 101 or 48 depending on the
> + * generation, Alibaba erdma 96. Try those first, by vendor ID, then step
> + * down SRP_INLINE_STEP bytes at a time to zero. libfabric's verbs
> + * provider also probes the limit (vrb_find_max_inline()).
> + */
> +static const struct {
> +	u32	vendor_id;
> +	u32	max_inline;
> +} srp_inline_limits[] = {
> +	{ PCI_VENDOR_ID_INTEL, 216 },
> +	{ PCI_VENDOR_ID_INTEL, 101 },
> +	{ PCI_VENDOR_ID_INTEL, 48 },
> +	{ PCI_VENDOR_ID_ALIBABA, 96 },
> +};

A table like the above makes maintenance of the RDMA subsystem harder
than necessary. Shouldn't a mechanism be added for querying this
information from the RDMA adapter?

> +	 * Ask for enough inline data to post command IUs inline (see
> +	 * srp_post_send()), or for as much as the device accepts.
> +	 */
> +	cap = init_attr->cap;
> +	inline_size = srp_inline_size(attr->vendor_id, SRP_MAX_INLINE_DATA,
> +				      false);
> +	for (;;) {
> +		init_attr->cap = cap;
> +		init_attr->cap.max_inline_data = inline_size;
> +		ret = srp_create_qp(ch, init_attr, &qp);
> +		if (!ret || !inline_size)
> +			break;
> +		inline_size = srp_inline_size(attr->vendor_id, inline_size,
> +					      true);
>   	}

What changes in the RDMA core would be needed to eliminate this loop?

Thanks,

Bart.

[PATCH RFC 3/3] RDMA/rtrs: Post client read requests inline

[Andrei-Alexandru Bleortu]

The RTRS client sends a read request as an RDMA Write with Immediate of
a small message: the user message, the RTRS header and one memory
descriptor, about 60 bytes for RNBD. The message is posted by reference
and the QP asks for no inline data, so the HCA has to DMA-read it from
host memory before sending. That read is on the latency path of every
read.

Let rtrs_cq_qp_create() take the amount of inline data to ask for.
The client asks for RTRS_CLT_MAX_INLINE_DATA, enough for a read
request with one descriptor and up to 64 bytes of user message, and
the server asks for none. If the device refuses, the QP is created
without inline data. The client posts a read request with
IB_SEND_INLINE when it fits in the inline size the device granted.
Write requests, whose message travels behind the data in the same
WR, are not changed.

Providers copy inline data from sg_list[].addr as a kernel virtual
address, so the inline path puts req->iu->buf in the on-stack SGE.

Assisted-by: LLM
Signed-off-by: Andrei-Alexandru Bleortu <me@andrei-z.com>
---
 drivers/infiniband/ulp/rtrs/rtrs-clt.c | 17 +++++++++++++++--
 drivers/infiniband/ulp/rtrs/rtrs-pri.h | 10 +++++++++-
 drivers/infiniband/ulp/rtrs/rtrs-srv.c |  2 +-
 drivers/infiniband/ulp/rtrs/rtrs.c     | 22 +++++++++++++++++-----
 4 files changed, 42 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index bf3a0377da3f..4a552e0459b2 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -489,6 +489,17 @@ static int rtrs_post_send_rdma(struct rtrs_clt_con *con,
 				      req->iu->dma_addr,
 				      req->sg_size, DMA_TO_DEVICE);
 
+	/*
+	 * Post the request inline if it fits in the granted inline size, so
+	 * the HCA does not have to DMA-read it from host memory.  The CPU
+	 * copies inline data from a kernel virtual address, so put
+	 * req->iu->buf in the SGE.
+	 */
+	if (req->sg_size <= con->c.max_inline_data) {
+		sge.addr = (uintptr_t)req->iu->buf;
+		flags |= IB_SEND_INLINE;
+	}
+
 	return rtrs_iu_post_rdma_write_imm(&con->c, req->iu, &sge, 1,
 					    rbuf->rkey, rbuf->addr + off,
 					    imm, flags, wr, NULL);
@@ -1714,11 +1725,13 @@ static int create_con_cq_qp(struct rtrs_clt_con *con)
 	if (con->c.cid >= clt_path->s.irq_con_num)
 		err = rtrs_cq_qp_create(&clt_path->s, &con->c, max_send_sge,
 					cq_vector, cq_num, max_send_wr,
-					max_recv_wr, IB_POLL_DIRECT);
+					max_recv_wr, RTRS_CLT_MAX_INLINE_DATA,
+					IB_POLL_DIRECT);
 	else
 		err = rtrs_cq_qp_create(&clt_path->s, &con->c, max_send_sge,
 					cq_vector, cq_num, max_send_wr,
-					max_recv_wr, IB_POLL_SOFTIRQ);
+					max_recv_wr, RTRS_CLT_MAX_INLINE_DATA,
+					IB_POLL_SOFTIRQ);
 	/*
 	 * In case of error we do not bother to clean previous allocations,
 	 * since destroy_con_cq_qp() must be called.
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-pri.h b/drivers/infiniband/ulp/rtrs/rtrs-pri.h
index 59529d593869..bcba80d3c214 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-pri.h
+++ b/drivers/infiniband/ulp/rtrs/rtrs-pri.h
@@ -97,6 +97,7 @@ struct rtrs_con {
 	int                     nr_cqe;
 	atomic_t		wr_cnt;
 	atomic_t		sq_wr_avail;
+	u32			max_inline_data;
 };
 
 struct rtrs_path {
@@ -286,6 +287,13 @@ struct rtrs_msg_rdma_read {
 	struct rtrs_sg_desc    desc[];
 };
 
+/*
+ * Inline data the client asks for on its QPs: enough for a read request
+ * with one memory descriptor and a user message of up to 64 bytes.
+ */
+#define RTRS_CLT_MAX_INLINE_DATA (sizeof(struct rtrs_msg_rdma_read) + \
+				  sizeof(struct rtrs_sg_desc) + 64)
+
 /**
  * struct rtrs_msg_rdma_write - Message transferred to server with RDMA-Write
  * @type:		always @RTRS_MSG_WRITE
@@ -325,7 +333,7 @@ int rtrs_post_recv_empty(struct rtrs_con *con, struct ib_cqe *cqe);
 int rtrs_cq_qp_create(struct rtrs_path *path, struct rtrs_con *con,
 		      u32 max_send_sge, int cq_vector, int nr_cqe,
 		      u32 max_send_wr, u32 max_recv_wr,
-		      enum ib_poll_context poll_ctx);
+		      u32 max_inline_data, enum ib_poll_context poll_ctx);
 void rtrs_cq_qp_destroy(struct rtrs_con *con);
 
 void rtrs_init_hb(struct rtrs_path *path, struct ib_cqe *cqe,
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c
index 5314599d29fe..30fa4c7f64dc 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c
@@ -1759,7 +1759,7 @@ static int create_con(struct rtrs_srv_path *srv_path,
 
 	/* TODO: SOFTIRQ can be faster, but be careful with softirq context */
 	err = rtrs_cq_qp_create(&srv_path->s, &con->c, 1, cq_vector, cq_num,
-				 max_send_wr, max_recv_wr,
+				 max_send_wr, max_recv_wr, 0,
 				 IB_POLL_WORKQUEUE);
 	if (err) {
 		rtrs_err(s, "rtrs_cq_qp_create(), err: %pe\n", ERR_PTR(err));
diff --git a/drivers/infiniband/ulp/rtrs/rtrs.c b/drivers/infiniband/ulp/rtrs/rtrs.c
index a642626e0ecb..08b8b1abb1d3 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs.c
@@ -253,9 +253,10 @@ static int create_cq(struct rtrs_con *con, int cq_vector, int nr_cqe,
 }
 
 static int create_qp(struct rtrs_con *con, struct ib_pd *pd,
-		     u32 max_send_wr, u32 max_recv_wr, u32 max_sge)
+		     u32 max_send_wr, u32 max_recv_wr, u32 max_sge,
+		     u32 max_inline_data)
 {
-	struct ib_qp_init_attr init_attr = {};
+	struct ib_qp_init_attr init_attr = {}, attr;
 	struct rdma_cm_id *cm_id = con->cm_id;
 	int ret;
 
@@ -271,13 +272,24 @@ static int create_qp(struct rtrs_con *con, struct ib_pd *pd,
 	init_attr.recv_cq = con->cq;
 	init_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
 
-	ret = rdma_create_qp(cm_id, pd, &init_attr);
+	/*
+	 * If the device cannot provide the requested inline data, create the
+	 * QP without it.
+	 */
+	attr = init_attr;
+	attr.cap.max_inline_data = max_inline_data;
+	ret = rdma_create_qp(cm_id, pd, &attr);
+	if (ret && max_inline_data) {
+		attr = init_attr;
+		ret = rdma_create_qp(cm_id, pd, &attr);
+	}
 	if (ret) {
 		rtrs_err(con->path, "Creating QP failed, err: %pe\n",
 			 ERR_PTR(ret));
 		return ret;
 	}
 	con->qp = cm_id->qp;
+	con->max_inline_data = attr.cap.max_inline_data;
 
 	return ret;
 }
@@ -296,7 +308,7 @@ static void destroy_cq(struct rtrs_con *con)
 int rtrs_cq_qp_create(struct rtrs_path *path, struct rtrs_con *con,
 		       u32 max_send_sge, int cq_vector, int nr_cqe,
 		       u32 max_send_wr, u32 max_recv_wr,
-		       enum ib_poll_context poll_ctx)
+		       u32 max_inline_data, enum ib_poll_context poll_ctx)
 {
 	int err;
 
@@ -305,7 +317,7 @@ int rtrs_cq_qp_create(struct rtrs_path *path, struct rtrs_con *con,
 		return err;
 
 	err = create_qp(con, path->dev->ib_pd, max_send_wr, max_recv_wr,
-			max_send_sge);
+			max_send_sge, max_inline_data);
 	if (err) {
 		destroy_cq(con);
 		return err;
-- 
2.56.0

Re: [PATCH RFC 3/3] RDMA/rtrs: Post client read requests inline

[sashiko-bot]

> The RTRS client sends a read request as an RDMA Write with Immediate of
> a small message: the user message, the RTRS header and one memory
> descriptor, about 60 bytes for RNBD. The message is posted by reference
> and the QP asks for no inline data, so the HCA has to DMA-read it from
> host memory before sending. That read is on the latency path of every
> read.
> 
> Let rtrs_cq_qp_create() take the amount of inline data to ask for.
> The client asks for RTRS_CLT_MAX_INLINE_DATA, enough for a read
> request with one descriptor and up to 64 bytes of user message, and
> the server asks for none. If the device refuses, the QP is created
> without inline data. The client posts a read request with
> IB_SEND_INLINE when it fits in the inline size the device granted.
> Write requests, whose message travels behind the data in the same
> WR, are not changed.
> 
> Providers copy inline data from sg_list[].addr as a kernel virtual
> address, so the inline path puts req->iu->buf in the on-stack SGE.
> 
> Assisted-by: LLM
> Signed-off-by: Andrei-Alexandru Bleortu <me@andrei-z.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261015125118.2966247-1-me@andrei-z.com?part=3