From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E170DC43458 for ; Thu, 9 Jul 2026 19:58:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2740B10E0C6; Thu, 9 Jul 2026 19:58:07 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.b="HUZ0FPy6"; dkim-atps=neutral Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 413E410E0C6 for ; Thu, 9 Jul 2026 19:58:05 +0000 (UTC) Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by sea.source.kernel.org (Postfix) with ESMTP id 78EA143440; Thu, 9 Jul 2026 19:58:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A69C51F000E9; Thu, 9 Jul 2026 19:58:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783627084; bh=nASUEopdTVpbiOlaqA3bl70JASaPDBXc6RrPfU8TUhM=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=HUZ0FPy6tJ5k2FpZZk4TmtW8oNPYG0UCTaswg34FnSsVTMZ+nc6abBzQoGLAe7QpZ 4839oR2E7eU5H+0VQhcuYDxXFXmLbqe785v0OtfK8pZM64xT/vjBbY8gU6OQqxdQko 2bj2K0aisc+j7i5kyFSxJBzHYPyWTVbTuJKmXAkqp+WJvRZsZfqnl91bOFM7185CBN pNNt3ie4Xo7ripQOTTqoaN5lwefEhmBf7ncJ41sgP4urIb+UmKalmTY+FHBAjFjAbH sZHtyFtM9TSgQQsGPMBhSwsdDCYRoGd5HD7SutnqDROuHsn6T0qtpDhVLf9Zj3BAl/ ejFRs0GNI3+Yw== Date: Thu, 9 Jul 2026 21:58:01 +0200 From: Thierry Reding To: Will Deacon Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Hunter , David Airlie , Simona Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Sowjanya Komatineni , Luca Ceresoli , Mikko Perttunen , Yury Norov , Rasmus Villemoes , Russell King , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Marek Szyprowski , Robin Murphy , Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , Christian =?utf-8?B?S8O2bmln?= , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Catalin Marinas , Thierry Reding , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux.dev, linaro-mm-sig@lists.linaro.org, linux-trace-kernel@vger.kernel.org, Thierry Reding , Chun Ng Subject: Re: [PATCH v3 04/11] arm64/mm: Add set_memory_device() and set_memory_normal() Message-ID: References: <20260701-tegra-vpr-v3-0-d80f7b871bb4@nvidia.com> <20260701-tegra-vpr-v3-4-d80f7b871bb4@nvidia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="rvxsrpqt5djhs2ak" Content-Disposition: inline In-Reply-To: X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" --rvxsrpqt5djhs2ak Content-Type: text/plain; protected-headers=v1; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: [PATCH v3 04/11] arm64/mm: Add set_memory_device() and set_memory_normal() MIME-Version: 1.0 On Tue, Jul 07, 2026 at 12:27:13PM +0100, Will Deacon wrote: > On Mon, Jul 06, 2026 at 03:49:24PM +0200, Thierry Reding wrote: > > On Fri, Jul 03, 2026 at 06:13:31PM +0100, Will Deacon wrote: > > > On Thu, Jul 02, 2026 at 06:41:23PM +0200, Thierry Reding wrote: > > > > On Thu, Jul 02, 2026 at 03:46:44PM +0200, Thierry Reding wrote: > > > > > On Thu, Jul 02, 2026 at 10:18:47AM +0100, Will Deacon wrote: > > > > > > On Wed, Jul 01, 2026 at 06:08:15PM +0200, Thierry Reding wrote: > > > > > > > From: Chun Ng > > > > > > >=20 > > > > > > > Add helpers to swap PROT_NORMAL and PROT_DEVICE_nGnRnE protec= tion bits > > > > > > > on a kernel-linear-map range. > > > > > >=20 > > > > > > That sounds like a really terrible idea. Why is this necessary = and how > > > > > > does it interact with things like load_unaligned_zeropad()? > > > > >=20 > > > > > This is necessary because once the memory controller has walled o= ff the > > > > > new memory region the CPU must not access it under any circumstan= ces or > > > > > it'll cause the CPU to lock up (I think technically it'll hit an = SError > > > > > but in practice that just means it'll freeze, as far as I can tel= l). > > > > >=20 > > > > > Probably doesn't interact well at all with load_unaligned_zeropad= (). > > > > >=20 > > > > > > I think you should unmap the memory from the linear map and mem= remap() > > > > > > it instead. > > > > >=20 > > > > > Given that the memory can never be accessed by the CPU after the = memory > > > > > controller locks it down, I don't think we'll even need memremap(= ). The > > > > > only thing we really need is the sg_table we hand out via the DMA= BUFs > > > > > so that they can be used by device drivers to program their DMA e= ngines > > > > > internally. > > > > >=20 > > > > > Looking through some of the architecture code around this, should= n't we > > > > > simply be using set_memory_encrypted() and set_memory_decrypted()= for > > > > > this? While they might've been created for slightly other use-cas= es, > > > > > they seem to be doing exactly what we want (i.e. remove the page = range > > > > > from the linear mapping and flushing it, or restoring the valid b= it and > > > > > standard permissions, respectively). > > > >=20 > > > > Ah... I guess we can't do it because we're not in a realm world and= so > > > > the early checks in __set_memory_enc_dec() would return early and t= urn > > > > it into a no-op. > > > >=20 > > > > How about if I extract a common helper and provide set_memory_p() a= nd > > > > set_memory_np() in terms of those. Those are available on x86 and > > > > PowerPC as well, so fairly standard. I suppose at that point we're > > > > closer to set_memory_valid(). > > >=20 > > > Why not just call set_direct_map_invalid_noflush() + > > > flush_tlb_kernel_range() for each page? We already have APIs for this. > >=20 > > Having a "standard" helper with a fixed and documented purposed seemed > > like a preferable approach for this particular case. We also may want to > > make the driver that uses this buildable as a module, in which case we'd > > need to export these rather low-level APIs. And then there's also the > > fact that we typically call this on a rather large region of memory > > (usually something like 512 MiB), so doing it page-by-page is rather > > suboptimal. > >=20 > > > The big challenge I see with any linear map manipulation, however, is > > > that it will rely on can_set_direct_map() which likely means you need= to > > > give up some performance and/or security to make this work. Does memo= ry > > > become inaccesible dynamically at runtime? If not, the best bet would > > > be to describe it as a carveout in the DT and mark it as "no-map" so > > > we avoid mapping it in the first place. > >=20 > > VPR exists in two modes: static and resizable. For static VPR we do > > exactly that: describe it as carveout in DT with no-map and deal with it > > accordingly in the driver. Resizable VPR is for device that have small > > amounts of RAM. Content-protected video playback will in the worst case > > consume around 1.8 GiB of RAM, so we want to be able to reuse for other > > purposes when VPR is unused on those devices. In that case, the memory > > is also described as a reserved-memory region in DT, but it is marked as > > reusable so that it can be managed by CMA. > >=20 > > The resize operation is fairly slow to begin with because we need to > > stall the GPU and put it into reset before the operation, then take it > > out of reset and resume it afterwards. > >=20 > > What kind of performance impact do you expect? >=20 > You'll need to measure it, but we've seen reports of double-digit > percentage regressions in performance and power. As I said, the problem > is that you need to split the linear map to 4k page at runtime to unmap > the dynamic carveout, but that isn't something that can be done on most > CPUs. Therefore you end up having to use page-granular mappings for the > entire thing, similarly to how 'rodata_full' drives can_set_direct_map() > and the perf/power hit affects everything. The VPR has fairly large alignment restrictions (1 MiB) and we do unmap in fairly large chunks (512 MiB currently, but we can change that if it is helpful) because we really want to avoid resizing operations, so the tradeoff is between frequency of resize vs. potential memory wasted. Does that change anything with regards to performance? Thierry --rvxsrpqt5djhs2ak Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEiOrDCAFJzPfAjcif3SOs138+s6EFAmpP/UkACgkQ3SOs138+ s6GFqBAAsfNpGtxzDc/1UKAc4Vjp9eilQE6L3ZzhQhn1d3Crc+ym9vGZzyfe4lTR di2aYaWn+6XslOczfKwW58jo+D8/fP64sFF9UY3mR7U+MzuIGEF61RVgF0fFH4Vb wo6H5bLYvA2J1VhXyIv08l/Oxb+GGHMa2gRVdTbFux+csAi4ahPJVT0oS9S0sJht cmPEaDkjN8StPrx4IN3oTcj7g3LW3aDuShMtgqRrceSaIw2xvtb+rvWLvGJac/HH kNrn5Vc+pAJJigN3NZqMgpjeD3gHZDZYaKUToPXJuIOlZILR0VzdDAHxD9U+lmyD YVPGNnPi13QmxTk9KHR+Tm2aL7AxXx8TK0jIOUFuaySrgD16Oo9CTxOvsxewNGpg lGDhuvag7BCxZqQzlxbhAuGtHZS+s6OWCXleWAsKpVs4Ga/FDaAvZYCHugxIur84 mexc1/5CBcjohTAfvTLpAZGUxDXslKopeYaBZGgRJXz+iwulrGeeDpEvDH1VNo1r SkCIXoLOlq4bEKwejuGzuVzc47hLYGL+c09cCOwa73rhvVARke2v05fRowwN29Iv 1OVDKNQTsHbBZ8PNTsoaEApLvif8crc38kborIPtiuaGz4WDg7q90ogJoIhqEK2L z8Lt64Ul985C2ht8PNWLbWeUKGNX54YFmnBCCP+lgBzDRu2sc4g= =Axxn -----END PGP SIGNATURE----- --rvxsrpqt5djhs2ak--