Re: [PATCH 1/1] RDMA/rxe: Use a dedicated and robust workqueue for RXE tasks

[Leon Romanovsky <leon@kernel.org>]

On Wed, Mar 18, 2026 at 08:34:42AM -0700, Zhu Yanjun wrote:
> 
> 在 2026/3/18 7:53, Leon Romanovsky 写道:
> > On Wed, Mar 18, 2026 at 03:57:39AM +0100, Zhu Yanjun wrote:
> > > Currently, the RXE driver uses the system-wide 'system_unbound_wq' for
> > > auxiliary tasks like ODP prefetching. This can lead to interference
> > > from other system services and lacks guaranteed forward progress
> > > under memory pressure.
> > > 
> > > Currently make all the tasks queue into the driver-specific 'rxe_wq'.
> > > 
> > > Suggested-by: Leon Romanovsky <leon@kernel.org>
> > > Signed-off-by: Zhu Yanjun <yanjun.zhu@linux.dev>
> > > ---
> > >   drivers/infiniband/sw/rxe/rxe_odp.c  |  2 +-
> > >   drivers/infiniband/sw/rxe/rxe_task.c | 10 +++++++++-
> > >   drivers/infiniband/sw/rxe/rxe_task.h |  1 +
> > >   3 files changed, 11 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c
> > > index bc11b1ec59ac..98092dcc1870 100644
> > > --- a/drivers/infiniband/sw/rxe/rxe_odp.c
> > > +++ b/drivers/infiniband/sw/rxe/rxe_odp.c
> > > @@ -545,7 +545,7 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd,
> > >   		work->frags[i].mr = mr;
> > >   	}
> > > -	queue_work(system_unbound_wq, &work->work);
> > > +	rxe_queue_work(&work->work);
> > >   	return 0;
> > > diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c
> > > index f522820b950c..4385137eb4d7 100644
> > > --- a/drivers/infiniband/sw/rxe/rxe_task.c
> > > +++ b/drivers/infiniband/sw/rxe/rxe_task.c
> > > @@ -10,7 +10,8 @@ static struct workqueue_struct *rxe_wq;
> > >   int rxe_alloc_wq(void)
> > >   {
> > > -	rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND, WQ_MAX_ACTIVE);
> > > +	rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND | WQ_MEM_RECLAIM,
> > Why did you add WQ_MEM_RECLAIM flag? rxe_ib_advise_mr_prefetch() doesn't
> > perform any memory reclaim.
> 
> You are correct that rxe_ib_advise_mr_prefetch() does not directly call
> memory reclaim functions.
> 
> However, the WQ_MEM_RECLAIM flag was added to prevent circular dependencies
> during
> 
> low-memory conditions.
> 
> Since rxe handles memory regions that may be part of the storage or network
> stack,
> 
> the workqueue must be able to make progress even when the system is under
> extreme
> 
> memory pressure. Without this flag, if the kernel attempts to reclaim memory
> and that
> 
> reclaim process depends on an RDMA operation being processed by this
> workqueue,
> 
> the system could deadlock because the workqueue might be unable to spawn a
> new
> 
> worker thread.
> 
> By setting WQ_MEM_RECLAIM, we ensure that a rescuer thread is pre-allocated,
> 
> guaranteeing that prefetch and MR-related tasks can complete and allow the
> 
> memory management subsystem to finish its reclaim cycle.

Zhu,

Please avoid relying on AI when answering ML-related questions. The  
response you received is broadly correct, but it is incorrect for RXE.  
You should set the WQ_MEM_RECLAIM flag only when the workqueue handlers  
free memory. RXE does the opposite in rxe_ib_advise_mr_prefetch().

Thanks

> 
> 
> Zhu Yanjun
> 
> > 
> > Thanks
> 
> -- 
> Best Regards,
> Yanjun.Zhu
> 
>