Re: [PATCH for-next v3 2/2] RDMA/rxe: Enable asynchronous prefetch for ODP MRs

[Leon Romanovsky <leon@kernel.org>]

On Tue, May 13, 2025 at 05:04:05AM +0000, Daisuke Matsuda wrote:
> Calling ibv_advise_mr(3) with flags other than IBV_ADVISE_MR_FLAG_FLUSH
> invokes asynchronous request. It is best-effort, and thus can safely be
> deferred to the system-wide workqueue.
> 
> Signed-off-by: Daisuke Matsuda <dskmtsd@gmail.com>
> Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
> ---
>  drivers/infiniband/sw/rxe/rxe_odp.c | 84 ++++++++++++++++++++++++++++-
>  1 file changed, 82 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c
> index 4c98a02d572c..0f3b281a265f 100644
> --- a/drivers/infiniband/sw/rxe/rxe_odp.c
> +++ b/drivers/infiniband/sw/rxe/rxe_odp.c
> @@ -425,6 +425,73 @@ enum resp_states rxe_odp_do_atomic_write(struct rxe_mr *mr, u64 iova, u64 value)
>  	return RESPST_NONE;
>  }

<...>

> +static int rxe_init_prefetch_work(struct ib_pd *ibpd,
> +				  enum ib_uverbs_advise_mr_advice advice,
> +				  u32 pf_flags, struct prefetch_mr_work *work,
> +				  struct ib_sge *sg_list, u32 num_sge)

There is no need one-time called function. It can be embedded into rxe_ib_advise_mr_prefetch().

> +{

<...>

> @@ -475,6 +542,8 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd,
>  				     u32 flags, struct ib_sge *sg_list, u32 num_sge)
>  {

<...>

> +	queue_work(system_unbound_wq, &work->work);

How do you ensure that this work isn't running after RXE is destroyed?

Thanks

> +
> +	return 0;
>  }
>  
>  int rxe_ib_advise_mr(struct ib_pd *ibpd,
> -- 
> 2.43.0
>