* [PATCH v3 0/2] RDMA/rxe: fix ADVISE_MR prefetch on non-ODP MRs
@ 2026-09-23 14:40 ` Norbert Szetei
0 siblings, 0 replies; 11+ messages in thread
From: Norbert Szetei @ 2026-09-23 14:40 UTC (permalink / raw)
To: Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky, Bob Pearson,
Daisuke Matsuda
Cc: linux-rdma, linux-kernel, Zhu Yanjun, stable, Norbert Szetei
ADVISE_MR prefetch on a plain MR runs to_ib_umem_odp() on a struct
ib_umem, giving a KASAN slab-out-of-bounds read in
ib_umem_odp_map_dma_and_lock() (splat in patch 2).
v1 checked is_odp_mr() after lookup_mr(). Leon asked for lookup_mr() to
do the check itself, off mr->access. That only holds once mr->access
cannot disagree with the umem, so patch 1 stops the two paths that
assign mr->access after registration from touching IB_ACCESS_ON_DEMAND,
and patch 2 passes the flag to lookup_mr() in both prefetch arms.
Based on for-next. In rxe_rereg_user_mr() the new check rejects the
access flags before the mr->ibmr.pd swap, where for-rc's ae36a5b609ae
("RDMA/rxe: validate access flags before swapping the MR's PD") also
moved the existing validation.
v2 -> v3:
- rebased on for-next, no functional change
v2: https://lore.kernel.org/all/20260913-rxe-advise-mr-v2-v2-0-b806c789871c@doyensec.com/
v1: https://lore.kernel.org/all/521D5E74-89E3-43C0-81C7-AC0BE52591E7@doyensec.com/
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
---
Norbert Szetei (2):
RDMA/rxe: Reject IB_ACCESS_ON_DEMAND changes after MR creation
RDMA/rxe: Reject prefetch of a non-ODP MR
drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++
drivers/infiniband/sw/rxe/rxe_odp.c | 4 ++--
drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++
3 files changed, 14 insertions(+), 2 deletions(-)
---
base-commit: 9dcacdc41083cccff74292851bf9fde999e0731a
change-id: 20260923-rxe-advise-mr-v3-355a808e851f
Best regards,
--
Norbert Szetei <norbert@doyensec.com>
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v3 1/2] RDMA/rxe: Reject IB_ACCESS_ON_DEMAND changes after MR creation
2026-09-23 14:40 ` Norbert Szetei
@ 2026-09-23 14:40 ` Norbert Szetei
-1 siblings, 0 replies; 11+ messages in thread
From: Norbert Szetei via B4 Relay @ 2026-09-23 14:40 UTC (permalink / raw)
To: Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky, Bob Pearson,
Daisuke Matsuda
Cc: linux-rdma, linux-kernel, Zhu Yanjun, stable, Norbert Szetei
From: Norbert Szetei <norbert@doyensec.com>
Whether an MR is an ODP MR is decided once, at registration time:
rxe_reg_user_mr() picks rxe_odp_mr_init_user() over rxe_mr_init_user()
based on IB_ACCESS_ON_DEMAND, and only the former builds an ib_umem_odp
via ib_umem_odp_get(). The umem cannot change type afterwards, and
is_odp_mr() reads mr->umem->is_odp.
Two paths assign mr->access after that point and can leave it
describing an MR type the umem does not have:
rxe_rereg_user_mr() with IB_MR_REREG_ACCESS overwrites mr->access with
the caller's value, and IB_ACCESS_ON_DEMAND is part of
RXE_ACCESS_SUPPORTED_MR, so userspace can set the flag on a plain MR or
clear it on an ODP MR while the umem stays what it was.
rxe_reg_fast_mr() takes mr->access from the REG_MR work request
unmasked and moves the MR to RXE_MR_STATE_VALID, on an MR that
rxe_mr_init_fast() left with a NULL umem.
Reject IB_ACCESS_ON_DEMAND in both, so mr->access carries the flag only
for an MR that has an ODP umem and the flag can be used to identify one.
Fixes: 544c7f62cf32 ("RDMA/rxe: Implement rereg_user_mr")
Cc: stable@vger.kernel.org
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
---
drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++
drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index 875eceb55fdf..2afda5154dfb 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -795,6 +795,12 @@ int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
return -EINVAL;
}
+ /* an MR with no umem is never an ODP MR */
+ if (unlikely(access & IB_ACCESS_ON_DEMAND)) {
+ rxe_dbg_mr(mr, "access = 0x%x requests ODP\n", access);
+ return -EINVAL;
+ }
+
mr->access = access;
mr->lkey = key;
mr->rkey = key;
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 96c7716057fe..21855274f63a 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -1331,6 +1331,12 @@ static struct ib_mr *rxe_rereg_user_mr(struct ib_mr *ibmr, int flags,
if (err)
return ERR_PTR(err);
+ if ((flags & IB_MR_REREG_ACCESS) &&
+ ((access ^ mr->access) & IB_ACCESS_ON_DEMAND)) {
+ rxe_err_mr(mr, "cannot change IB_ACCESS_ON_DEMAND\n");
+ return ERR_PTR(-EOPNOTSUPP);
+ }
+
if (flags & IB_MR_REREG_PD) {
rxe_put(old_pd);
rxe_get(pd);
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v3 1/2] RDMA/rxe: Reject IB_ACCESS_ON_DEMAND changes after MR creation
@ 2026-09-23 14:40 ` Norbert Szetei
0 siblings, 0 replies; 11+ messages in thread
From: Norbert Szetei @ 2026-09-23 14:40 UTC (permalink / raw)
To: Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky, Bob Pearson,
Daisuke Matsuda
Cc: linux-rdma, linux-kernel, Zhu Yanjun, stable, Norbert Szetei
Whether an MR is an ODP MR is decided once, at registration time:
rxe_reg_user_mr() picks rxe_odp_mr_init_user() over rxe_mr_init_user()
based on IB_ACCESS_ON_DEMAND, and only the former builds an ib_umem_odp
via ib_umem_odp_get(). The umem cannot change type afterwards, and
is_odp_mr() reads mr->umem->is_odp.
Two paths assign mr->access after that point and can leave it
describing an MR type the umem does not have:
rxe_rereg_user_mr() with IB_MR_REREG_ACCESS overwrites mr->access with
the caller's value, and IB_ACCESS_ON_DEMAND is part of
RXE_ACCESS_SUPPORTED_MR, so userspace can set the flag on a plain MR or
clear it on an ODP MR while the umem stays what it was.
rxe_reg_fast_mr() takes mr->access from the REG_MR work request
unmasked and moves the MR to RXE_MR_STATE_VALID, on an MR that
rxe_mr_init_fast() left with a NULL umem.
Reject IB_ACCESS_ON_DEMAND in both, so mr->access carries the flag only
for an MR that has an ODP umem and the flag can be used to identify one.
Fixes: 544c7f62cf32 ("RDMA/rxe: Implement rereg_user_mr")
Cc: stable@vger.kernel.org
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
---
drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++
drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index 875eceb55fdf..2afda5154dfb 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -795,6 +795,12 @@ int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
return -EINVAL;
}
+ /* an MR with no umem is never an ODP MR */
+ if (unlikely(access & IB_ACCESS_ON_DEMAND)) {
+ rxe_dbg_mr(mr, "access = 0x%x requests ODP\n", access);
+ return -EINVAL;
+ }
+
mr->access = access;
mr->lkey = key;
mr->rkey = key;
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 96c7716057fe..21855274f63a 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -1331,6 +1331,12 @@ static struct ib_mr *rxe_rereg_user_mr(struct ib_mr *ibmr, int flags,
if (err)
return ERR_PTR(err);
+ if ((flags & IB_MR_REREG_ACCESS) &&
+ ((access ^ mr->access) & IB_ACCESS_ON_DEMAND)) {
+ rxe_err_mr(mr, "cannot change IB_ACCESS_ON_DEMAND\n");
+ return ERR_PTR(-EOPNOTSUPP);
+ }
+
if (flags & IB_MR_REREG_PD) {
rxe_put(old_pd);
rxe_get(pd);
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH v3 1/2] RDMA/rxe: Reject IB_ACCESS_ON_DEMAND changes after MR creation
2026-09-23 14:40 ` Norbert Szetei
(?)
@ 2026-09-23 14:52 ` sashiko-bot
-1 siblings, 0 replies; 11+ messages in thread
From: sashiko-bot @ 2026-09-23 14:52 UTC (permalink / raw)
To: Norbert Szetei; +Cc: linux-rdma
> From: Norbert Szetei <norbert@doyensec.com>
>
> Whether an MR is an ODP MR is decided once, at registration time:
> rxe_reg_user_mr() picks rxe_odp_mr_init_user() over rxe_mr_init_user()
> based on IB_ACCESS_ON_DEMAND, and only the former builds an ib_umem_odp
> via ib_umem_odp_get(). The umem cannot change type afterwards, and
> is_odp_mr() reads mr->umem->is_odp.
>
> Two paths assign mr->access after that point and can leave it
> describing an MR type the umem does not have:
>
> rxe_rereg_user_mr() with IB_MR_REREG_ACCESS overwrites mr->access with
> the caller's value, and IB_ACCESS_ON_DEMAND is part of
> RXE_ACCESS_SUPPORTED_MR, so userspace can set the flag on a plain MR or
> clear it on an ODP MR while the umem stays what it was.
>
> rxe_reg_fast_mr() takes mr->access from the REG_MR work request
> unmasked and moves the MR to RXE_MR_STATE_VALID, on an MR that
> rxe_mr_init_fast() left with a NULL umem.
>
> Reject IB_ACCESS_ON_DEMAND in both, so mr->access carries the flag only
> for an MR that has an ODP umem and the flag can be used to identify one.
>
> Fixes: 544c7f62cf32 ("RDMA/rxe: Implement rereg_user_mr")
> Cc: stable@vger.kernel.org
> Signed-off-by: Norbert Szetei <norbert@doyensec.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260923-rxe-advise-mr-v3-v3-0-95e1a4077e4d@doyensec.com?part=1
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v3 1/2] RDMA/rxe: Reject IB_ACCESS_ON_DEMAND changes after MR creation
2026-09-23 14:40 ` Norbert Szetei
(?)
(?)
@ 2026-09-23 22:49 ` Zhu Yanjun
-1 siblings, 0 replies; 11+ messages in thread
From: Zhu Yanjun @ 2026-09-23 22:49 UTC (permalink / raw)
To: norbert, Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky,
Bob Pearson, Daisuke Matsuda, yanjun.zhu@linux.dev
Cc: linux-rdma, linux-kernel, stable
在 2026/9/23 7:40, Norbert Szetei via B4 Relay 写道:
> From: Norbert Szetei <norbert@doyensec.com>
>
> Whether an MR is an ODP MR is decided once, at registration time:
> rxe_reg_user_mr() picks rxe_odp_mr_init_user() over rxe_mr_init_user()
> based on IB_ACCESS_ON_DEMAND, and only the former builds an ib_umem_odp
> via ib_umem_odp_get(). The umem cannot change type afterwards, and
> is_odp_mr() reads mr->umem->is_odp.
>
> Two paths assign mr->access after that point and can leave it
> describing an MR type the umem does not have:
>
> rxe_rereg_user_mr() with IB_MR_REREG_ACCESS overwrites mr->access with
> the caller's value, and IB_ACCESS_ON_DEMAND is part of
> RXE_ACCESS_SUPPORTED_MR, so userspace can set the flag on a plain MR or
> clear it on an ODP MR while the umem stays what it was.
>
> rxe_reg_fast_mr() takes mr->access from the REG_MR work request
> unmasked and moves the MR to RXE_MR_STATE_VALID, on an MR that
> rxe_mr_init_fast() left with a NULL umem.
>
> Reject IB_ACCESS_ON_DEMAND in both, so mr->access carries the flag only
> for an MR that has an ODP umem and the flag can be used to identify one.
>
> Fixes: 544c7f62cf32 ("RDMA/rxe: Implement rereg_user_mr")
> Cc: stable@vger.kernel.org
> Signed-off-by: Norbert Szetei <norbert@doyensec.com>
I have already reviewed this commit. I am fine with this commit.
So
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Zhu Yanjun
> ---
> drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++
> drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++
> 2 files changed, 12 insertions(+)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
> index 875eceb55fdf..2afda5154dfb 100644
> --- a/drivers/infiniband/sw/rxe/rxe_mr.c
> +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
> @@ -795,6 +795,12 @@ int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
> return -EINVAL;
> }
>
> + /* an MR with no umem is never an ODP MR */
> + if (unlikely(access & IB_ACCESS_ON_DEMAND)) {
> + rxe_dbg_mr(mr, "access = 0x%x requests ODP\n", access);
> + return -EINVAL;
> + }
> +
> mr->access = access;
> mr->lkey = key;
> mr->rkey = key;
> diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
> index 96c7716057fe..21855274f63a 100644
> --- a/drivers/infiniband/sw/rxe/rxe_verbs.c
> +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
> @@ -1331,6 +1331,12 @@ static struct ib_mr *rxe_rereg_user_mr(struct ib_mr *ibmr, int flags,
> if (err)
> return ERR_PTR(err);
>
> + if ((flags & IB_MR_REREG_ACCESS) &&
> + ((access ^ mr->access) & IB_ACCESS_ON_DEMAND)) {
> + rxe_err_mr(mr, "cannot change IB_ACCESS_ON_DEMAND\n");
> + return ERR_PTR(-EOPNOTSUPP);
> + }
> +
> if (flags & IB_MR_REREG_PD) {
> rxe_put(old_pd);
> rxe_get(pd);
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 2/2] RDMA/rxe: Reject prefetch of a non-ODP MR
2026-09-23 14:40 ` Norbert Szetei
@ 2026-09-23 14:40 ` Norbert Szetei
-1 siblings, 0 replies; 11+ messages in thread
From: Norbert Szetei via B4 Relay @ 2026-09-23 14:40 UTC (permalink / raw)
To: Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky, Bob Pearson,
Daisuke Matsuda
Cc: linux-rdma, linux-kernel, Zhu Yanjun, stable, Norbert Szetei
From: Norbert Szetei <norbert@doyensec.com>
rxe_ib_advise_mr_prefetch() and rxe_ib_prefetch_sg_list() look up the MR
by lkey and hand it to rxe_odp_do_pagefault_and_lock() without checking
that it is an ODP MR. That path runs to_ib_umem_odp() on mr->umem, and
for a non-ODP MR mr->umem is a plain struct ib_umem from ib_umem_get(),
so the container_of() in to_ib_umem_odp() lands past the end of the
object and ib_umem_odp_map_dma_and_lock() reads its ib_umem_odp fields
out of bounds.
lookup_mr() validates the lkey, PD, access and state but not the MR
type, and IB_UVERBS_ADVISE_MR_ADVICE_PREFETCH is accepted for any MR.
BUG: KASAN: slab-out-of-bounds in ib_umem_odp_map_dma_and_lock+0x884/0x8a0
Read of size 8 at addr ffff88810a3ebcf0 by task advi/921
ib_umem_odp_map_dma_and_lock+0x884/0x8a0
rxe_ib_advise_mr+0x543/0xad0
ib_uverbs_handler_UVERBS_METHOD_ADVISE_MR+0x446/0x530
ib_uverbs_cmd_verbs+0x2b3c/0x3b20
ib_uverbs_ioctl+0x1e3/0x310
Allocated by task 921:
__ib_umem_get_va+0x13e/0xae0
rxe_mr_init_user+0x2ae/0xb00
rxe_reg_user_mr+0x337/0x510
The buggy address belongs to the object at ffff88810a3ebc80
which belongs to the cache kmalloc-96 of size 96
Ask lookup_mr() for IB_ACCESS_ON_DEMAND in both the synchronous and
the asynchronous prefetch arm. mr->access carries that flag only for
an MR registered as ODP, so the existing
(access & mr->access) != access test rejects a plain MR and the
prefetch fails with -EINVAL.
Fixes: 3576b0df1588 ("RDMA/rxe: Implement synchronous prefetch for ODP MRs")
Cc: stable@vger.kernel.org
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
---
drivers/infiniband/sw/rxe/rxe_odp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c
index e870efa7a0a3..f8ffd6a3219a 100644
--- a/drivers/infiniband/sw/rxe/rxe_odp.c
+++ b/drivers/infiniband/sw/rxe/rxe_odp.c
@@ -463,7 +463,7 @@ static int rxe_ib_prefetch_sg_list(struct ib_pd *ibpd,
struct rxe_mr *mr;
struct ib_umem_odp *umem_odp;
- mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE,
+ mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE | IB_ACCESS_ON_DEMAND,
sg_list[i].lkey, RXE_LOOKUP_LOCAL);
if (!mr) {
@@ -529,7 +529,7 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd,
for (i = 0; i < num_sge; ++i) {
/* Takes a reference, which will be released in the queued work */
- mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE,
+ mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE | IB_ACCESS_ON_DEMAND,
sg_list[i].lkey, RXE_LOOKUP_LOCAL);
if (!mr) {
mr = ERR_PTR(-EINVAL);
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v3 2/2] RDMA/rxe: Reject prefetch of a non-ODP MR
@ 2026-09-23 14:40 ` Norbert Szetei
0 siblings, 0 replies; 11+ messages in thread
From: Norbert Szetei @ 2026-09-23 14:40 UTC (permalink / raw)
To: Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky, Bob Pearson,
Daisuke Matsuda
Cc: linux-rdma, linux-kernel, Zhu Yanjun, stable, Norbert Szetei
rxe_ib_advise_mr_prefetch() and rxe_ib_prefetch_sg_list() look up the MR
by lkey and hand it to rxe_odp_do_pagefault_and_lock() without checking
that it is an ODP MR. That path runs to_ib_umem_odp() on mr->umem, and
for a non-ODP MR mr->umem is a plain struct ib_umem from ib_umem_get(),
so the container_of() in to_ib_umem_odp() lands past the end of the
object and ib_umem_odp_map_dma_and_lock() reads its ib_umem_odp fields
out of bounds.
lookup_mr() validates the lkey, PD, access and state but not the MR
type, and IB_UVERBS_ADVISE_MR_ADVICE_PREFETCH is accepted for any MR.
BUG: KASAN: slab-out-of-bounds in ib_umem_odp_map_dma_and_lock+0x884/0x8a0
Read of size 8 at addr ffff88810a3ebcf0 by task advi/921
ib_umem_odp_map_dma_and_lock+0x884/0x8a0
rxe_ib_advise_mr+0x543/0xad0
ib_uverbs_handler_UVERBS_METHOD_ADVISE_MR+0x446/0x530
ib_uverbs_cmd_verbs+0x2b3c/0x3b20
ib_uverbs_ioctl+0x1e3/0x310
Allocated by task 921:
__ib_umem_get_va+0x13e/0xae0
rxe_mr_init_user+0x2ae/0xb00
rxe_reg_user_mr+0x337/0x510
The buggy address belongs to the object at ffff88810a3ebc80
which belongs to the cache kmalloc-96 of size 96
Ask lookup_mr() for IB_ACCESS_ON_DEMAND in both the synchronous and
the asynchronous prefetch arm. mr->access carries that flag only for
an MR registered as ODP, so the existing
(access & mr->access) != access test rejects a plain MR and the
prefetch fails with -EINVAL.
Fixes: 3576b0df1588 ("RDMA/rxe: Implement synchronous prefetch for ODP MRs")
Cc: stable@vger.kernel.org
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
---
drivers/infiniband/sw/rxe/rxe_odp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c
index e870efa7a0a3..f8ffd6a3219a 100644
--- a/drivers/infiniband/sw/rxe/rxe_odp.c
+++ b/drivers/infiniband/sw/rxe/rxe_odp.c
@@ -463,7 +463,7 @@ static int rxe_ib_prefetch_sg_list(struct ib_pd *ibpd,
struct rxe_mr *mr;
struct ib_umem_odp *umem_odp;
- mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE,
+ mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE | IB_ACCESS_ON_DEMAND,
sg_list[i].lkey, RXE_LOOKUP_LOCAL);
if (!mr) {
@@ -529,7 +529,7 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd,
for (i = 0; i < num_sge; ++i) {
/* Takes a reference, which will be released in the queued work */
- mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE,
+ mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE | IB_ACCESS_ON_DEMAND,
sg_list[i].lkey, RXE_LOOKUP_LOCAL);
if (!mr) {
mr = ERR_PTR(-EINVAL);
--
2.55.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH v3 2/2] RDMA/rxe: Reject prefetch of a non-ODP MR
2026-09-23 14:40 ` Norbert Szetei
(?)
@ 2026-09-23 14:54 ` sashiko-bot
-1 siblings, 0 replies; 11+ messages in thread
From: sashiko-bot @ 2026-09-23 14:54 UTC (permalink / raw)
To: Norbert Szetei; +Cc: linux-rdma
> From: Norbert Szetei <norbert@doyensec.com>
>
> rxe_ib_advise_mr_prefetch() and rxe_ib_prefetch_sg_list() look up the MR
> by lkey and hand it to rxe_odp_do_pagefault_and_lock() without checking
> that it is an ODP MR. That path runs to_ib_umem_odp() on mr->umem, and
> for a non-ODP MR mr->umem is a plain struct ib_umem from ib_umem_get(),
> so the container_of() in to_ib_umem_odp() lands past the end of the
> object and ib_umem_odp_map_dma_and_lock() reads its ib_umem_odp fields
> out of bounds.
>
> lookup_mr() validates the lkey, PD, access and state but not the MR
> type, and IB_UVERBS_ADVISE_MR_ADVICE_PREFETCH is accepted for any MR.
>
> BUG: KASAN: slab-out-of-bounds in ib_umem_odp_map_dma_and_lock+0x884/0x8a0
> Read of size 8 at addr ffff88810a3ebcf0 by task advi/921
> [ ... ]
> prefetch fails with -EINVAL.
>
> Fixes: 3576b0df1588 ("RDMA/rxe: Implement synchronous prefetch for ODP MRs")
> Cc: stable@vger.kernel.org
> Signed-off-by: Norbert Szetei <norbert@doyensec.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260923-rxe-advise-mr-v3-v3-0-95e1a4077e4d@doyensec.com?part=2
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v3 2/2] RDMA/rxe: Reject prefetch of a non-ODP MR
2026-09-23 14:40 ` Norbert Szetei
(?)
(?)
@ 2026-09-23 22:50 ` Zhu Yanjun
-1 siblings, 0 replies; 11+ messages in thread
From: Zhu Yanjun @ 2026-09-23 22:50 UTC (permalink / raw)
To: norbert, Zhu Yanjun, Jason Gunthorpe, Leon Romanovsky,
Bob Pearson, Daisuke Matsuda
Cc: linux-rdma, linux-kernel, stable
在 2026/9/23 7:40, Norbert Szetei via B4 Relay 写道:
> From: Norbert Szetei <norbert@doyensec.com>
>
> rxe_ib_advise_mr_prefetch() and rxe_ib_prefetch_sg_list() look up the MR
> by lkey and hand it to rxe_odp_do_pagefault_and_lock() without checking
> that it is an ODP MR. That path runs to_ib_umem_odp() on mr->umem, and
> for a non-ODP MR mr->umem is a plain struct ib_umem from ib_umem_get(),
> so the container_of() in to_ib_umem_odp() lands past the end of the
> object and ib_umem_odp_map_dma_and_lock() reads its ib_umem_odp fields
> out of bounds.
>
> lookup_mr() validates the lkey, PD, access and state but not the MR
> type, and IB_UVERBS_ADVISE_MR_ADVICE_PREFETCH is accepted for any MR.
>
> BUG: KASAN: slab-out-of-bounds in ib_umem_odp_map_dma_and_lock+0x884/0x8a0
> Read of size 8 at addr ffff88810a3ebcf0 by task advi/921
> ib_umem_odp_map_dma_and_lock+0x884/0x8a0
> rxe_ib_advise_mr+0x543/0xad0
> ib_uverbs_handler_UVERBS_METHOD_ADVISE_MR+0x446/0x530
> ib_uverbs_cmd_verbs+0x2b3c/0x3b20
> ib_uverbs_ioctl+0x1e3/0x310
> Allocated by task 921:
> __ib_umem_get_va+0x13e/0xae0
> rxe_mr_init_user+0x2ae/0xb00
> rxe_reg_user_mr+0x337/0x510
> The buggy address belongs to the object at ffff88810a3ebc80
> which belongs to the cache kmalloc-96 of size 96
>
> Ask lookup_mr() for IB_ACCESS_ON_DEMAND in both the synchronous and
> the asynchronous prefetch arm. mr->access carries that flag only for
> an MR registered as ODP, so the existing
> (access & mr->access) != access test rejects a plain MR and the
> prefetch fails with -EINVAL.
>
> Fixes: 3576b0df1588 ("RDMA/rxe: Implement synchronous prefetch for ODP MRs")
> Cc: stable@vger.kernel.org
> Signed-off-by: Norbert Szetei <norbert@doyensec.com>
Thanks a lot.
It is fine with me.
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Zhu Yanjun
> ---
> drivers/infiniband/sw/rxe/rxe_odp.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c
> index e870efa7a0a3..f8ffd6a3219a 100644
> --- a/drivers/infiniband/sw/rxe/rxe_odp.c
> +++ b/drivers/infiniband/sw/rxe/rxe_odp.c
> @@ -463,7 +463,7 @@ static int rxe_ib_prefetch_sg_list(struct ib_pd *ibpd,
> struct rxe_mr *mr;
> struct ib_umem_odp *umem_odp;
>
> - mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE,
> + mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE | IB_ACCESS_ON_DEMAND,
> sg_list[i].lkey, RXE_LOOKUP_LOCAL);
>
> if (!mr) {
> @@ -529,7 +529,7 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd,
>
> for (i = 0; i < num_sge; ++i) {
> /* Takes a reference, which will be released in the queued work */
> - mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE,
> + mr = lookup_mr(pd, IB_ACCESS_LOCAL_WRITE | IB_ACCESS_ON_DEMAND,
> sg_list[i].lkey, RXE_LOOKUP_LOCAL);
> if (!mr) {
> mr = ERR_PTR(-EINVAL);
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 0/2] RDMA/rxe: fix ADVISE_MR prefetch on non-ODP MRs
2026-09-23 14:40 ` Norbert Szetei
` (2 preceding siblings ...)
(?)
@ 2026-09-29 18:32 ` Leon Romanovsky
-1 siblings, 0 replies; 11+ messages in thread
From: Leon Romanovsky @ 2026-09-29 18:32 UTC (permalink / raw)
To: Zhu Yanjun, Jason Gunthorpe, Bob Pearson, Daisuke Matsuda,
Norbert Szetei
Cc: linux-rdma, linux-kernel, Zhu Yanjun, stable
On Wed, 23 Sep 2026 16:40:48 +0200, Norbert Szetei wrote:
> ADVISE_MR prefetch on a plain MR runs to_ib_umem_odp() on a struct
> ib_umem, giving a KASAN slab-out-of-bounds read in
> ib_umem_odp_map_dma_and_lock() (splat in patch 2).
>
> v1 checked is_odp_mr() after lookup_mr(). Leon asked for lookup_mr() to
> do the check itself, off mr->access. That only holds once mr->access
> cannot disagree with the umem, so patch 1 stops the two paths that
> assign mr->access after registration from touching IB_ACCESS_ON_DEMAND,
> and patch 2 passes the flag to lookup_mr() in both prefetch arms.
>
> [...]
Applied, thanks!
[1/2] RDMA/rxe: Reject IB_ACCESS_ON_DEMAND changes after MR creation
https://git.kernel.org/rdma/rdma/c/2dac7006b9fb46
[2/2] RDMA/rxe: Reject prefetch of a non-ODP MR
https://git.kernel.org/rdma/rdma/c/66f18a749a3eee
Best regards,
--
Leon Romanovsky <leon@kernel.org>
^ permalink raw reply [flat|nested] 11+ messages in thread