From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-186.mta1.migadu.com (out-186.mta1.migadu.com [95.215.58.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AD173101B8 for ; Wed, 18 Mar 2026 16:11:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.186 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773850284; cv=none; b=tDLRGmPYQmLy75ZudygxYqJg/l/viiBSFARw8ucxc0O0QBWsPhBpJep4MJSlflxa8cEzasxOHHYMkwd7bV0x8PdhaFyRYdFUXVZjE3dQZnC4RR4gbIGoCpz7Z0A0Qy/4nyfeWWDnpp3nqmhmlKLTFfnsSdZH6BgySWScpxLfDyw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773850284; c=relaxed/simple; bh=sSDi+K0/0a5BYNaze/SzNRiI6cLysWJg3J25A44e/AI=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=K5gXfek1QAu5ErXHmEWPOlC3dZ+NcE3QXM+i8Kr9kIl0/9aU9Fs6BX44bl+wN8tTGM9rbcOD684h7H1glR7AZv36CgSFtZxo6eeRunBKNlA4KIzqsrwJ45d430tNvkYs0ZUmzxhm7+ZKb6yg7yxhbRStMurvLKrtQziYh2iXDHg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=AgFAqRFr; arc=none smtp.client-ip=95.215.58.186 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="AgFAqRFr" Message-ID: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1773850269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rl/DY8lrkMQtwNaASDNkaKjbaXVQe48jBBUXJp2zTkM=; b=AgFAqRFrsuc73G7zo59lRujPKErqYT+6djZ9dB6O6dS2BVAMdmUpD7muKeafdUZGExGU2p Zilge2mKW+em1RwxWWRQ3e03AmTn2xNM7HAlON/KA/YZKQY4zZ5e9ms52dPZNeWDxIr8nt uTon6321K1AbU2R6ynX+vSd8jtSL+Kk= Date: Wed, 18 Mar 2026 09:10:51 -0700 Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Subject: Re: [PATCH 1/1] RDMA/rxe: Use a dedicated and robust workqueue for RXE tasks To: Leon Romanovsky , Zhu Yanjun Cc: zyjzyj2000@gmail.com, jgg@ziepe.ca, linux-rdma@vger.kernel.org References: <20260318025739.5058-1-yanjun.zhu@linux.dev> <20260318145327.GC352386@unreal> <9963f346-cd53-4f88-bb54-642a5babb768@linux.dev> <20260318154133.GF352386@unreal> Content-Language: en-US X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: "Yanjun.Zhu" In-Reply-To: <20260318154133.GF352386@unreal> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT On 3/18/26 8:41 AM, Leon Romanovsky wrote: > On Wed, Mar 18, 2026 at 08:34:42AM -0700, Zhu Yanjun wrote: >> 在 2026/3/18 7:53, Leon Romanovsky 写道: >>> On Wed, Mar 18, 2026 at 03:57:39AM +0100, Zhu Yanjun wrote: >>>> Currently, the RXE driver uses the system-wide 'system_unbound_wq' for >>>> auxiliary tasks like ODP prefetching. This can lead to interference >>>> from other system services and lacks guaranteed forward progress >>>> under memory pressure. >>>> >>>> Currently make all the tasks queue into the driver-specific 'rxe_wq'. >>>> >>>> Suggested-by: Leon Romanovsky >>>> Signed-off-by: Zhu Yanjun >>>> --- >>>> drivers/infiniband/sw/rxe/rxe_odp.c | 2 +- >>>> drivers/infiniband/sw/rxe/rxe_task.c | 10 +++++++++- >>>> drivers/infiniband/sw/rxe/rxe_task.h | 1 + >>>> 3 files changed, 11 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c >>>> index bc11b1ec59ac..98092dcc1870 100644 >>>> --- a/drivers/infiniband/sw/rxe/rxe_odp.c >>>> +++ b/drivers/infiniband/sw/rxe/rxe_odp.c >>>> @@ -545,7 +545,7 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd, >>>> work->frags[i].mr = mr; >>>> } >>>> - queue_work(system_unbound_wq, &work->work); >>>> + rxe_queue_work(&work->work); >>>> return 0; >>>> diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c >>>> index f522820b950c..4385137eb4d7 100644 >>>> --- a/drivers/infiniband/sw/rxe/rxe_task.c >>>> +++ b/drivers/infiniband/sw/rxe/rxe_task.c >>>> @@ -10,7 +10,8 @@ static struct workqueue_struct *rxe_wq; >>>> int rxe_alloc_wq(void) >>>> { >>>> - rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND, WQ_MAX_ACTIVE); >>>> + rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND | WQ_MEM_RECLAIM, >>> Why did you add WQ_MEM_RECLAIM flag? rxe_ib_advise_mr_prefetch() doesn't >>> perform any memory reclaim. >> You are correct that rxe_ib_advise_mr_prefetch() does not directly call >> memory reclaim functions. >> >> However, the WQ_MEM_RECLAIM flag was added to prevent circular dependencies >> during >> >> low-memory conditions. >> >> Since rxe handles memory regions that may be part of the storage or network >> stack, >> >> the workqueue must be able to make progress even when the system is under >> extreme >> >> memory pressure. Without this flag, if the kernel attempts to reclaim memory >> and that >> >> reclaim process depends on an RDMA operation being processed by this >> workqueue, >> >> the system could deadlock because the workqueue might be unable to spawn a >> new >> >> worker thread. >> >> By setting WQ_MEM_RECLAIM, we ensure that a rescuer thread is pre-allocated, >> >> guaranteeing that prefetch and MR-related tasks can complete and allow the >> >> memory management subsystem to finish its reclaim cycle. > Zhu, > > Please avoid relying on AI when answering ML-related questions. The > response you received is broadly correct, but it is incorrect for RXE. > You should set the WQ_MEM_RECLAIM flag only when the workqueue handlers OK. Thanks a lot. Zhu Yanjun > free memory. RXE does the opposite in rxe_ib_advise_mr_prefetch(). > > Thanks > >> >> Zhu Yanjun >> >>> Thanks >> -- >> Best Regards, >> Yanjun.Zhu >> >>