From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB5ABC44515 for ; Fri, 17 Jul 2026 18:08:35 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4h1yZp32G8z3c89; Sat, 18 Jul 2026 04:08:34 +1000 (AEST) Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2600:3c04:e001:324:0:1991:8:25" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1784311714; cv=none; b=KJKBYEATyc82bPTCCHNrqt3BWKkqegbLahAsNmHRUq5dD18mqOrk/J0rVwhFrj4sWGpUeV0D6sZGDGzRoBbMN2JUolWuSYd3NfGiuLf4nf2e+fQq8+EC5EczV010FcEVOwgTCL8lJTgWG3XdZXwUWQhtyx2X1tWOt0Cuxu4vaz05d4zUJysHQ4kzs/YoWLFGnTcUunYBO7VvDqGok2uW028jntGhiFsgK+jcVizSooAuo74GbSXcJjHJKf385pTcb1lOxSJ490bKMsgNwoLBJDHpEvOgrvL++8DloPoBqCsFxWCXohn/9C9395Ich2DfWCxn4tR3ylcn73e4i9YdIQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1784311714; c=relaxed/relaxed; bh=gZrGl+jtrdO3STdBAPtI9YcrqokEMnK381jHPPxX4QM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y82Z1DilraC8VQnQX8XJs4x3EBdGhsNy1Il8uA4LMWqi2rtGQMfdWon4hTCVRZQswxmGcNu99ui7S7218r8Ml6Ee+W2bHIFEfaBAkj+SCEJN7KB6S9o3Hj47MmNYXL3PXMkUCJ6fRsntdRw3zCdT3JsCTA2UwmKJ/6hDDfWJIneB98wh0ZWwGi2imcKO0smJKGqfDrJO7nAZKY8RFXC0rIUEv5SUVpfhfZ2AvGHVi7+ENoT9JAgFiciYW4gP2bA0d+53tvzm2ZpAgJrQT0ZkvSz6FSmhjiKtNa11a8Z5qEucbLReREekX4aYJY7liFjPGoS4oMK3/ObNyONzp8s7sQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=Q33giytp; dkim-atps=neutral; spf=pass (client-ip=2600:3c04:e001:324:0:1991:8:25; helo=tor.source.kernel.org; envelope-from=aneesh.kumar@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=Q33giytp; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=2600:3c04:e001:324:0:1991:8:25; helo=tor.source.kernel.org; envelope-from=aneesh.kumar@kernel.org; receiver=lists.ozlabs.org) Received: from tor.source.kernel.org (tor.source.kernel.org [IPv6:2600:3c04:e001:324:0:1991:8:25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4h1yZn3xsRz3c4Y for ; Sat, 18 Jul 2026 04:08:33 +1000 (AEST) Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by tor.source.kernel.org (Postfix) with ESMTP id A2F35600DA; Fri, 17 Jul 2026 18:08:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 099D01F000E9; Fri, 17 Jul 2026 18:08:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784311711; bh=gZrGl+jtrdO3STdBAPtI9YcrqokEMnK381jHPPxX4QM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Q33giytp3Pgs15qPJ1uZX7Rj8NcL8zDMsdhdsUMfr0vCKfMSuT9Tet9T2sH4Fvtdx RVYNPIJsmev0Bviu6bDXydJJ6XRIwsRJ6+4NoLaVoFGNN8V1B9G2C6xvTEPbUYmcEx 3qjf2tvr5H6hjTN2aJ4+BAycTPse4r+/+bfvf9r3rV8LUYuvqv6lrjdSRm26w5lUsB eT/9KpcWPKmdW2ypdaJm1YEauyXbohdQePWmzwPFF+tEr+EMyeekRReFWj54UC7V1J zCQfT0waCLsxriu+oXFree30DZaF1SLCRyMJTrZUZ212KRje67qrdih8qmWBNGP5ON IaaSbzUWq9fMA== From: "Aneesh Kumar K.V (Arm)" To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Robin Murphy , Marek Szyprowski , Will Deacon , Marc Zyngier , Steven Price , Suzuki K Poulose , Catalin Marinas , Jiri Pirko , Jason Gunthorpe , Mostafa Saleh , Petr Tesarik , Alexey Kardashevskiy , Dan Williams , Xu Yilun , linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , "Christophe Leroy (CS GROUP)" , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , x86@kernel.org, Jason Gunthorpe , Jiri Pirko , Michael Kelley Subject: [PATCH v8 18/23] dma-direct: set decrypted flag for remapped DMA allocations Date: Fri, 17 Jul 2026 23:34:36 +0530 Message-ID: <20260717180442.110954-19-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260717180442.110954-1-aneesh.kumar@kernel.org> References: <20260717180442.110954-1-aneesh.kumar@kernel.org> X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Owner: List-Post: List-Archive: , List-Subscribe: , , List-Unsubscribe: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Devices that are DMA non-coherent and require a remap were skipping dma_set_decrypted(), leaving DMA buffers encrypted even when the device requires unencrypted access. Move the call after the if (remap) branch so that both the direct and remapped allocation paths correctly mark the allocation as decrypted (or fail cleanly) before use. Fix dma_direct_alloc() and dma_direct_free() to apply set_memory_*() to the linear-map alias of the backing pages instead of the remapped CPU address. Also disallow highmem pages for __DMA_ATTR_ALLOC_CC_SHARED, because highmem buffers do not provide a usable linear-map address. Reviewed-by: Jason Gunthorpe Tested-by: Jiri Pirko Tested-by: Michael Kelley Tested-by: Mostafa Saleh Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/direct.c | 56 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 12 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index fed901c0224e..f7f064323bd9 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -198,14 +198,23 @@ void *dma_direct_alloc(struct device *dev, size_t size, { bool remap = false, set_uncached = false; bool mark_mem_decrypt = false; + bool allow_highmem = true; struct page *page; void *ret; if (force_dma_unencrypted(dev)) attrs |= __DMA_ATTR_ALLOC_CC_SHARED; - if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) + if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) { + /* + * Unencrypted/shared DMA requires a linear-mapped buffer + * address to look up the PFN and set architecture-required PFN + * attributes. This is not possible with HighMem. Avoid HighMem + * allocation. + */ + allow_highmem = false; mark_mem_decrypt = true; + } size = PAGE_ALIGN(size); if (attrs & DMA_ATTR_NO_WARN) @@ -270,7 +279,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, } /* we always manually zero the memory once we are done */ - page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true); + page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, allow_highmem); if (!page) return NULL; @@ -285,6 +294,14 @@ void *dma_direct_alloc(struct device *dev, size_t size, set_uncached = false; } + if (mark_mem_decrypt) { + void *lm_addr; + + lm_addr = page_address(page); + if (set_memory_decrypted((unsigned long)lm_addr, PFN_UP(size))) + goto out_leak_pages; + } + if (remap) { pgprot_t prot = dma_pgprot(dev, PAGE_KERNEL, attrs); @@ -295,29 +312,36 @@ void *dma_direct_alloc(struct device *dev, size_t size, ret = dma_common_contiguous_remap(page, size, prot, __builtin_return_address(0)); if (!ret) - goto out_free_pages; + goto out_encrypt_pages; } else { ret = page_address(page); - if (mark_mem_decrypt && dma_set_decrypted(dev, ret, size)) - goto out_leak_pages; } memset(ret, 0, size); if (set_uncached) { + void *uncached_cpu_addr; + arch_dma_prep_coherent(page, size); - ret = arch_dma_set_uncached(ret, size); - if (IS_ERR(ret)) - goto out_encrypt_pages; + uncached_cpu_addr = arch_dma_set_uncached(ret, size); + if (IS_ERR(uncached_cpu_addr)) + goto out_free_remap_pages; + ret = uncached_cpu_addr; } *dma_handle = phys_to_dma_direct(dev, page_to_phys(page)); return ret; + +out_free_remap_pages: + if (remap) + dma_common_free_remap(ret, size); + out_encrypt_pages: - if (mark_mem_decrypt && dma_set_encrypted(dev, page_address(page), size)) - return NULL; -out_free_pages: + if (mark_mem_decrypt && + dma_set_encrypted(dev, page_address(page), size)) + goto out_leak_pages; + if (!swiotlb_free(dev, page, size)) dma_free_contiguous(dev, page, size); return NULL; @@ -380,8 +404,16 @@ void dma_direct_free(struct device *dev, size_t size, } else { if (IS_ENABLED(CONFIG_ARCH_HAS_DMA_CLEAR_UNCACHED)) arch_dma_clear_uncached(cpu_addr, size); - if (mark_mem_encrypted && dma_set_encrypted(dev, cpu_addr, size)) + } + + if (mark_mem_encrypted) { + void *lm_addr; + + lm_addr = phys_to_virt(phys); + if (set_memory_encrypted((unsigned long)lm_addr, PFN_UP(size))) { + pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n"); return; + } } if (swiotlb_pool) -- 2.43.0