From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 750DE32ED24; Wed, 28 Jan 2026 11:13:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769598830; cv=none; b=LWuPoZTZMRKzG29om/bpnj9r3dAYUXxrCwQ10A9d4doLFUgTLxoxKOkkh4qJthz10PO9rbAwsspABcJj2SwAmtQlzORrf4aEnFMPvarHykHMOjl7sqhFla1HPciJhQhCCoqoX8UJypTVEai1r1fwE8oJ+ExkaX+SbGAEO8SyS0k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769598830; c=relaxed/simple; bh=XhAa/knyZ6haG7FHsLWt0DcbM4mnkccj/WzXt56VTpQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oTeTdPz3FIlMC8KZepjA4CutgSr1HJ3P6/dXIrQZ50cqJOi8ELLIEfyc1lOa4CFOqHcvHpQawAJQ0Je3EiHEwoM1O5siMSoCn+fKrVwFZ/Ve+NyPnVLakEmRgxxIIdYiDchNpxoe3epa9D0QqDFU8Qx3+L5O/cK2eCkqUPf/YD4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QsOIO6cV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QsOIO6cV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E815DC4CEF1; Wed, 28 Jan 2026 11:13:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1769598830; bh=XhAa/knyZ6haG7FHsLWt0DcbM4mnkccj/WzXt56VTpQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=QsOIO6cV70uo0MZSW5kkJyDzI6RcSrMOzS5q7b7ULZKvuvWOx/IEA15z5DVAGIIJB coB0IBdwYPmvjySgO+Q5k0Jh7yt3uJdRV7gfvJoAN354mNNBcRN0nPgfT+cNOutquJ ZpegGQAj4wSMSonkLxC7IwKMy3Ey977CuGMoGbkKTiCHGIQ0NLz5TwtRb/TERR82EW gi+yJN3vH9nFSz8UetoM6ON9giYtcaD2dAs7OBryHfj7ofCebLLX0mZMO5KXNUbTV6 tJEsoktIRK3o+24kAK4i4HfY3V6Kp+kTlsfAFbeP/LXceIkOsVtbdbiUnMiqvrXZhI Zl1Y90LMpECMQ== Date: Wed, 28 Jan 2026 13:13:46 +0200 From: Leon Romanovsky To: "D. Wythe" Cc: Uladzislau Rezki , "David S. Miller" , Andrew Morton , Dust Li , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Sidraya Jayagond , Wenjia Zhang , Mahanta Jambigi , Simon Horman , Tony Lu , Wen Gu , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rdma@vger.kernel.org, linux-s390@vger.kernel.org, netdev@vger.kernel.org, oliver.yang@linux.alibaba.com Subject: Re: [PATCH net-next 2/3] mm: vmalloc: export find_vm_area() Message-ID: <20260128111346.GD12149@unreal> References: <20260123082349.42663-1-alibuda@linux.alibaba.com> <20260123082349.42663-3-alibuda@linux.alibaba.com> <20260124093505.GA98529@j66a10360.sqa.eu95> <20260124145754.GA57116@j66a10360.sqa.eu95> <20260127133417.GU13967@unreal> <20260128034558.GA126415@j66a10360.sqa.eu95> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260128034558.GA126415@j66a10360.sqa.eu95> On Wed, Jan 28, 2026 at 11:45:58AM +0800, D. Wythe wrote: > On Tue, Jan 27, 2026 at 03:34:17PM +0200, Leon Romanovsky wrote: > > On Sat, Jan 24, 2026 at 10:57:54PM +0800, D. Wythe wrote: > > > On Sat, Jan 24, 2026 at 11:48:59AM +0100, Uladzislau Rezki wrote: > > > > Hello, D. Wythe! > > > > > > > > > On Fri, Jan 23, 2026 at 07:55:17PM +0100, Uladzislau Rezki wrote: > > > > > > On Fri, Jan 23, 2026 at 04:23:48PM +0800, D. Wythe wrote: > > > > > > > find_vm_area() provides a way to find the vm_struct associated with a > > > > > > > virtual address. Export this symbol to modules so that modularized > > > > > > > subsystems can perform lookups on vmalloc addresses. > > > > > > > > > > > > > > Signed-off-by: D. Wythe > > > > > > > --- > > > > > > > mm/vmalloc.c | 1 + > > > > > > > 1 file changed, 1 insertion(+) > > > > > > > > > > > > > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > > > > > > > index ecbac900c35f..3eb9fe761c34 100644 > > > > > > > --- a/mm/vmalloc.c > > > > > > > +++ b/mm/vmalloc.c > > > > > > > @@ -3292,6 +3292,7 @@ struct vm_struct *find_vm_area(const void *addr) > > > > > > > > > > > > > > return va->vm; > > > > > > > } > > > > > > > +EXPORT_SYMBOL_GPL(find_vm_area); > > > > > > > > > > > > > This is internal. We can not just export it. > > > > > > > > > > > > -- > > > > > > Uladzislau Rezki > > > > > > > > > > Hi Uladzislau, > > > > > > > > > > Thank you for the feedback. I agree that we should avoid exposing > > > > > internal implementation details like struct vm_struct to external > > > > > subsystems. > > > > > > > > > > Following Christoph's suggestion, I'm planning to encapsulate the page > > > > > order lookup into a minimal helper instead: > > > > > > > > > > unsigned int vmalloc_page_order(const void *addr){ > > > > > struct vm_struct *vm; > > > > > vm = find_vm_area(addr); > > > > > return vm ? vm->page_order : 0; > > > > > } > > > > > EXPORT_SYMBOL_GPL(vmalloc_page_order); > > > > > > > > > > Does this approach look reasonable to you? It would keep the vm_struct > > > > > layout private while satisfying the optimization needs of SMC. > > > > > > > > > Could you please clarify why you need info about page_order? I have not > > > > looked at your second patch. > > > > > > > > Thanks! > > > > > > > > -- > > > > Uladzislau Rezki > > > > > > Hi Uladzislau, > > > > > > This stems from optimizing memory registration in SMC-R. To provide the > > > RDMA hardware with direct access to memory buffers, we must register > > > them with the NIC. During this process, the hardware generates one MTT > > > entry for each physically contiguous block. Since these hardware entries > > > are a finite and scarce resource, and SMC currently defaults to a 4KB > > > registration granularity, a single 2MB buffer consumes 512 entries. In > > > high-concurrency scenarios, this inefficiency quickly exhausts NIC > > > resources and becomes a major bottleneck for system scalability. > > > > I believe this complexity can be avoided by using the RDMA MR pool API, > > as other ULPs do, for example NVMe. > > > > Thanks > > > > Hi Leon, > > Am I correct in assuming you are suggesting mr_pool to limit the number > of MRs as a way to cap MTTE consumption? I don't see this a limit, but something that is considered standard practice to reduce MTT consumption. > > However, our goal is to maximize the total registered memory within > the MTTE limits rather than to cap it. In SMC-R, each connection > occupies a configurable, fixed-size registered buffer; consequently, > the more memory we can register, the more concurrent connections > we can support. It is not cap, but more efficient use of existing resources. > > By leveraging vmalloc_huge() and the proposed helper to increase the > page_size in ib_map_mr_sg(), each MTTE covers a much larger contiguous > physical block. This significantly reduces the total number of entries > required to map the same amount of memory, allowing us to serve more > connections under the same hardware constraints > > D. Wythe