From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA407EB64D7 for ; Mon, 26 Jun 2023 14:52:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57CC78D0002; Mon, 26 Jun 2023 10:52:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 52C9E8D0001; Mon, 26 Jun 2023 10:52:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41C8C8D0002; Mon, 26 Jun 2023 10:52:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 327658D0001 for ; Mon, 26 Jun 2023 10:52:16 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0950F80772 for ; Mon, 26 Jun 2023 14:52:15 +0000 (UTC) X-FDA: 80945189430.30.354B4B6 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id 9A4B410000B for ; Mon, 26 Jun 2023 14:52:12 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=PULst19W; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687791133; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UFSCE21v5gU7lTurZrT+QlNS55eGUt7Z7/S6IMG3+Xg=; b=3g+Fcv640c1PJbAlHD7Z870lHINei08IrFcKFlb5zFhmY//nE+OjDla6v+2OeHNeYazawI aTLxsei7k46tncRpHx7lyDs2TrZAm7BlE6fK5b3dC4932S+Kpio80Gcerzxwozagcmpu1I 44wTp62pjJi6XbLQnH44cct2iHLT7JQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687791133; a=rsa-sha256; cv=none; b=C1Se1ngIFF32wZnwu62hFAsYSF0lbHkGOyFc8H+OIMWLLRgmhjgMmVXfUCUroIg1oGChKt e4PqhburiDd9CjAPT3KnVjL47xZBsRKTJN31rhJtlLkuhuAVFCAZho6/rBiXuj4JFMkKqK MXHyWdLrr35bO2VJKiTXyAb4ilzI60M= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=PULst19W; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=UFSCE21v5gU7lTurZrT+QlNS55eGUt7Z7/S6IMG3+Xg=; b=PULst19W/HF8uHCuP/1XYnryGp sF3mRSCqrqY6jVA38EgdWtr6Z40Im8BisMDa+LiO+T7lLydhVkPmJ6NJLl+9LZAClilQoeQ1pmg/j zkv8mYrV8sNkrgoTUpndFk9fI4KKeaKKO9O0Xfh3Qp3w+EIiye4ZVsnuAw9eXnVDREQzJajs8aklE cKnEnaX9ZMDTzO6NYBesic1Ek7gpPyWcx8By1XODaoN7yjQkqis5rgbaMvP7WJlhsWHZdm8eb2IDR oVJFd/AwYsbGA/DVOf6Le27MmvLVxWsxpAP90xn5f9RPE6R1ZsV0vrH5B9QvptcGKxvfZCBt2Yvg0 r4UMm0FA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qDnZU-001oc4-Ms; Mon, 26 Jun 2023 14:52:05 +0000 Date: Mon, 26 Jun 2023 15:52:04 +0100 From: Matthew Wilcox To: Danilo Krummrich Cc: Peng Zhang , maple-tree@lists.infradead.org, linux-mm@kvack.org, Andrew Morton , "Liam R. Howlett" , linux-kernel@vger.kernel.org, Peng Zhang , David Airlie , Boris Brezillon Subject: Re: [PATCH v2 14/16] maple_tree: Refine mas_preallocate() node calculations Message-ID: References: <20230612203953.2093911-1-Liam.Howlett@oracle.com> <20230612203953.2093911-15-Liam.Howlett@oracle.com> <26d8fbcf-d34f-0a79-9d91-8c60e66f7341@redhat.com> <43ce08db-210a-fec8-51b4-351625b3cdfb@redhat.com> <57527c36-57a1-6699-b6f0-373ba895014c@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <57527c36-57a1-6699-b6f0-373ba895014c@redhat.com> X-Stat-Signature: qcezyewo71cs8rrqwc5ergh5xpfr83em X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 9A4B410000B X-Rspam-User: X-HE-Tag: 1687791132-447495 X-HE-Meta: U2FsdGVkX1+Z+ZKI20+JgYf+ghte2SahtgHc7q6wjkPgrg2UmrTUDnOdGsRAykJ93S4i5eD/d20gm4IK3NR+5QOvPN28ffciW76alLVAWXdfu8gN0aV17T/xzF0D3+G72EDdH7mNokL7WJuqaJuZcX9/g7fn5+OVgg35noQN6K3vyd0nsdv1hGdqzjCrA0HFBZyAhR8EVzlV8vR3UCQNtCkKcTPgq7Gqqr6TQEn/OGPF7KfaNDOAc43jITr6qmiIvgSgdua18w6mTAnQQdudSJ2pAMxTt+RVOKNY0Xx1iThCG3c/wpsQ2yyYe5ofE/4yob9x11y4kovHskUrkmdB9jYwVY03sfS5NVU3EwTgJc60nizZHcHsaaCbXvyZSj3o8FJrLftE5YLHYe0xMW88vGCCFYfLJ7XAzsWhL8eV/jhGMGFKJk5qAoD/ihbu+dgh73LgJ074YCjynCqm8Q7c3fxhN4SYxfXcFsBIQHNuy64/QS+HpZRS63Z7D0VmpfK2VEEY4ATlIkCkUPb1eE6o2m6uO7TO5rNNutPrdhCYuUaqAUPrM1CIFsII9MCPaVbKoybXTmT/31qFrnMCycD8uI3ZX4rDLLEYHkEKJiC1E6ovd2QydH7sw+eSJFZM6gl8xho9NtNREGiDxTCfcCD8kzJEqeOGh4z5bFseaM+RlnwfmLAXGmA7FLaEeaMATMdrgeKYWj833P7g1dIH6mLg/HLLlZvvR/Wo8XaC0Wuz/orfmnCM9VOtwijHbWdpC6725b/kas09UukhJ6zBlZia/4tCNIuxEcWiQtU9VF/er+o3izPi2LQ+fugzHVR5hXgUKyrD85Rz9NuqGMIf2aWwPmd26fscpha9gMS0yHM3QV9it6in9r1lIEK8I18zaMvnTUdQV6opJuMqGw48nVm13FB4J6C3UWvMUd2NZFLK4cbG5j0/FXce5avuEbvvYlZ9yz2TH77o+ZzoJ530w6c HAe4IeeW HMuPbgazMxiHkDxlwCwV/AuXnErubcsTdCEsrxfAkTtoBFZWEOphV5Vfgf+orZ6ZEGYrpO7uzallV2i/YYsE0r1RhFWlvBqyCYUj/EC3HvpjpdoTiXeXguT76IQSXCGUcVRHGxgkMnWM5jdE88rH45hS1umzLwlukTdd73joBGTX1/AY2ou5YlC3FAg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jun 26, 2023 at 04:27:54PM +0200, Danilo Krummrich wrote: > On 6/26/23 15:19, Matthew Wilcox wrote: > > On Mon, Jun 26, 2023 at 02:38:06AM +0200, Danilo Krummrich wrote: > > > On the other hand, unless I miss something (and if so, please let me know), > > > something is bogus with the API then. > > > > > > While the documentation of the Advanced API of the maple tree explicitly > > > claims that the user of the API is responsible for locking, this should be > > > limited to the bounds set by the maple tree implementation. Which means, the > > > user must decide for either the internal (spin-) lock or an external lock > > > (which possibly goes away in the future) and acquire and release it > > > according to the rules maple tree enforces through lockdep checks. > > > > > > Let's say one picks the internal lock. How is one supposed to ensure the > > > tree isn't modified using the internal lock with mas_preallocate()? > > > > > > Besides that, I think the documentation should definitely mention this > > > limitation and give some guidance for the locking. > > > > > > Currently, from an API perspective, I can't see how anyone not familiar with > > > the implementation details would be able to recognize this limitation. > > > > > > In terms of the GPUVA manager, unfortunately, it seems like I need to drop > > > the maple tree and go back to using a rb-tree, since it seems there is no > > > sane way doing a worst-case pre-allocation that does not suffer from this > > > limitation. > > > > I haven't been paying much attention here (too many other things going > > on), but something's wrong. > > > > First, you shouldn't need to preallocate. Preallocation is only there > > Unfortunately, I think we really have a case where we have to. Typically GPU > mappings are created in a dma-fence signalling critical path and that is > where such mappings need to be added to the maple tree. Hence, we can't do > any sleeping allocations there. OK, so there are various ways to hadle this, depending on what's appropriate for your case. The simplest is to use GFP_ATOMIC. Essentially, you're saying to the MM layer "This is too hard, let me tap into the emergency reserves". It's mildly frowned upon, so let's see if we can do better. If you know where the allocation needs to be stored, but want it to act as NULL until the time is right, you can store a ZERO entry. That will read as NULL until you store to it. A pure overwriting store will not cause any memory allocation since all the implementation has to do is change a pointer. The XArray wraps this up nicely behind an xa_reserve() API. As you're discovering, the Maple Tree API isn't fully baked yet.