From: "Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org>
To: linux-coco@lists.linux.dev
Cc: "Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Baoquan He" <baoquan.he@linux.dev>,
"Mike Rapoport" <rppt@kernel.org>,
"Pasha Tatashin" <pasha.tatashin@soleen.com>,
"Pratyush Yadav" <pratyush@kernel.org>,
"Catalin Marinas" <catalin.marinas@arm.com>,
"Christian König" <christian.koenig@amd.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Joerg Roedel" <joro@8bytes.org>, "Marc Zyngier" <maz@kernel.org>,
"Marek Szyprowski" <m.szyprowski@samsung.com>,
"Robin Murphy" <robin.murphy@arm.com>,
"Steven Price" <steven.price@arm.com>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Suzuki K Poulose" <suzuki.poulose@arm.com>,
"Thomas Gleixner" <tglx@kernel.org>,
"Will Deacon" <will@kernel.org>,
"Russell King" <linux@armlinux.org.uk>,
"Benjamin Gaignard" <benjamin.gaignard@collabora.com>,
"Brian Starkey" <Brian.Starkey@arm.com>,
"John Stultz" <jstultz@google.com>,
"Mark Rutland" <mark.rutland@arm.com>,
"Radu Rendec" <radu@rendec.net>,
"T.J. Mercier" <tjmercier@google.com>,
"Madhavan Srinivasan" <maddy@linux.ibm.com>,
"Michael Ellerman" <mpe@ellerman.id.au>,
"Nicholas Piggin" <npiggin@gmail.com>,
"Christophe Leroy" <chleroy@kernel.org>,
"Ritesh Harjani" <ritesh.list@gmail.com>,
"Shrikanth Hegde" <sshegde@linux.ibm.com>,
"Alexander Gordeev" <agordeev@linux.ibm.com>,
"Gerald Schaefer" <gerald.schaefer@linux.ibm.com>,
"Heiko Carstens" <hca@linux.ibm.com>,
"Vasily Gorbik" <gor@linux.ibm.com>,
"Christian Borntraeger" <borntraeger@linux.ibm.com>,
"Sven Schnelle" <svens@linux.ibm.com>,
"Ingo Molnar" <mingo@redhat.com>,
"Borislav Petkov" <bp@alien8.de>,
"Dave Hansen" <dave.hansen@linux.intel.com>,
x86@kernel.org, "H . Peter Anvin" <hpa@zytor.com>,
"Kiryl Shutsemau" <kas@kernel.org>,
"Rick Edgecombe" <rick.p.edgecombe@intel.com>,
"K . Y . Srinivasan" <kys@microsoft.com>,
"Haiyang Zhang" <haiyangz@microsoft.com>,
"Wei Liu" <wei.liu@kernel.org>,
"Dexuan Cui" <decui@microsoft.com>,
"Long Li" <longli@microsoft.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Vitaly Kuznetsov" <vkuznets@redhat.com>,
"Andy Lutomirski" <luto@kernel.org>,
"Peter Zijlstra" <peterz@infradead.org>,
dri-devel@lists.freedesktop.org, iommu@lists.linux.dev,
linaro-mm-sig@lists.linaro.org,
linux-arm-kernel@lists.infradead.org,
linux-media@vger.kernel.org, linux-mm@kvack.org,
kvmarm@lists.linux.dev, patches@lists.linux.dev
Subject: [PATCH v10 02/13] mm: Zero memory during shared memory transitions
Date: Mon, 16 Nov 2026 14:41:25 +0530 [thread overview]
Message-ID: <20261116091137.94252-3-aneesh.kumar@kernel.org> (raw)
In-Reply-To: <20261116091137.94252-1-aneesh.kumar@kernel.org>
Architectures need to zero memory at different points in a private-to-shared
transition. For example, pKVM must clear memory before sharing it, while Arm
CCA can only clear it after the RSI transition has completed.
Make zeroing an implicit property of set_memory_decrypted(). Each architecture
or platform performs the clear at the safe point in its transition. TDX
clears memory after the private-to-shared conversion succeeds in the finish
callback. AMD clears memory in its finish callback after the page-table
update, TLB flush, and hypervisor notification, including for SEV-SNP. This
zeros the final shared mapping without adding a flag to
set_memory_decrypted() or threading it through architecture-specific callbacks.
Keep allocations on platforms without memory encryption on the ordinary page
allocator path so the original GFP constraints, including __GFP_ZERO, remain
intact. Remove post-transition memset() calls that are now redundant and
explicitly clear allocations on paths where no transition occurs.
This intentionally makes every private-to-shared transition destructive;
callers can no longer use set_memory_decrypted() to preserve existing contents.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: x86@kernel.org
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Long Li <longli@microsoft.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
arch/arm64/mm/mem_encrypt.c | 8 +++-
arch/powerpc/platforms/pseries/svm.c | 13 ++++--
arch/s390/mm/init.c | 7 ++++
arch/x86/hyperv/hv_init.c | 6 ++-
arch/x86/include/asm/set_memory.h | 1 +
arch/x86/include/asm/vga.h | 4 +-
arch/x86/kernel/kvmclock.c | 6 +--
arch/x86/mm/mem_encrypt_amd.c | 1 +
arch/x86/mm/pat/set_memory.c | 19 ++++++++-
drivers/firmware/arm_rmm/rsi.c | 9 ++++
drivers/hv/connection.c | 41 +++++++++----------
drivers/hv/hv.c | 11 ++---
drivers/hv/hv_common.c | 2 -
drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c | 4 ++
include/linux/set_memory.h | 8 +++-
kernel/dma/direct.c | 22 ++++++----
mm/cc_shared.c | 7 +---
17 files changed, 115 insertions(+), 54 deletions(-)
diff --git a/arch/arm64/mm/mem_encrypt.c b/arch/arm64/mm/mem_encrypt.c
index ee3c0ab04384..25eb67138b28 100644
--- a/arch/arm64/mm/mem_encrypt.c
+++ b/arch/arm64/mm/mem_encrypt.c
@@ -17,6 +17,7 @@
#include <linux/compiler.h>
#include <linux/err.h>
#include <linux/mm.h>
+#include <linux/string.h>
#include <asm/mem_encrypt.h>
@@ -42,8 +43,13 @@ EXPORT_SYMBOL_GPL(set_memory_encrypted);
int set_memory_decrypted(unsigned long addr, int numpages)
{
- if (likely(!crypt_ops) || WARN_ON(!PAGE_ALIGNED(addr)))
+ if (likely(!crypt_ops)) {
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
return 0;
+ }
+
+ if (WARN_ON(!PAGE_ALIGNED(addr)))
+ return -EINVAL;
return crypt_ops->decrypt(addr, numpages);
}
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index 7a403dbd35ee..0970f1fb76d7 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -9,6 +9,7 @@
#include <linux/mm.h>
#include <linux/memblock.h>
#include <linux/mem_encrypt.h>
+#include <linux/string.h>
#include <linux/cc_platform.h>
#include <asm/machdep.h>
#include <asm/svm.h>
@@ -53,13 +54,17 @@ int set_memory_encrypted(unsigned long addr, int numpages)
int set_memory_decrypted(unsigned long addr, int numpages)
{
- if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
- return 0;
-
if (!PAGE_ALIGNED(addr))
return -EINVAL;
- uv_share_page(PHYS_PFN(__pa(addr)), numpages);
+ if (cc_platform_has(CC_ATTR_MEM_ENCRYPT))
+ uv_share_page(PHYS_PFN(__pa(addr)), numpages);
+ /*
+ * Firmware may scrub pages with nonzero values during
+ * private-to-shared conversion. Clear the final shared mapping
+ * after the transition so that callers receive zeroed memory.
+ */
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
return 0;
}
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index be7e009e7b59..fdacd9380286 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -129,6 +129,13 @@ int set_memory_encrypted(unsigned long vaddr, int numpages)
int set_memory_decrypted(unsigned long vaddr, int numpages)
{
int i;
+
+ /*
+ * uv_set_shared() exposes existing contents to the host; see the data
+ * leakage warning in share() in asm/uv.h. Clear private data first.
+ */
+ memset((void *)vaddr, 0, (size_t)numpages << PAGE_SHIFT);
+
/* make specified pages shared (swiotlb, dma_alloca) */
for (i = 0; i < numpages; ++i) {
uv_set_shared(virt_to_phys((void *)vaddr));
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index b90e6f487533..4356f4255ae9 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -156,8 +156,10 @@ static int hv_cpu_init(unsigned int cpu)
* page in non-root partition here.
*/
if (*hvp && !ms_hyperv.paravisor_present && hv_isolation_type_snp()) {
- WARN_ON_ONCE(set_memory_decrypted((unsigned long)(*hvp), 1));
- memset(*hvp, 0, PAGE_SIZE);
+ int ret;
+
+ ret = set_memory_decrypted((unsigned long)*hvp, 1);
+ WARN_ON_ONCE(ret);
}
}
diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index 39271a5ea925..dfac1dde3714 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -52,6 +52,7 @@ int set_memory_4k(unsigned long addr, int numpages);
bool set_memory_enc_stop_conversion(void);
int set_memory_encrypted(unsigned long addr, int numpages);
int set_memory_decrypted(unsigned long addr, int numpages);
+int set_memory_decrypted_nozero(unsigned long addr, int numpages);
int set_memory_np_noalias(unsigned long addr, int numpages);
int set_memory_nonglobal(unsigned long addr, int numpages);
diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index 46f9b2deab4d..e1e91dfa399d 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -14,7 +14,7 @@
* On the PC, we can just recalculate addresses and then
* access the videoram directly without any black magic.
* To support memory encryption however, we need to access
- * the videoram as decrypted memory.
+ * the videoram as decrypted memory without clearing its contents.
*/
#define VGA_MAP_MEM(x, s) \
@@ -22,7 +22,7 @@
unsigned long start = (unsigned long)phys_to_virt(x); \
\
if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) \
- set_memory_decrypted(start, (s) >> PAGE_SHIFT); \
+ set_memory_decrypted_nozero(start, (s) >> PAGE_SHIFT); \
\
start; \
})
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index cb3d0ca1fa22..3e9b7ad3dea2 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -248,7 +248,7 @@ static void __init kvmclock_init_mem(void)
* be mapped decrypted.
*/
if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
- r = set_memory_decrypted((unsigned long) hvclock_mem,
+ r = set_memory_decrypted((unsigned long)hvclock_mem,
1UL << order);
if (r) {
__free_pages(p, order);
@@ -256,9 +256,9 @@ static void __init kvmclock_init_mem(void)
pr_warn("kvmclock: set_memory_decrypted() failed. Disabling\n");
return;
}
+ } else {
+ memset(hvclock_mem, 0, PAGE_SIZE << order);
}
-
- memset(hvclock_mem, 0, PAGE_SIZE << order);
}
static int __init kvm_setup_vsyscall_timeinfo(void)
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2f8c32173972..fbcd23b745cf 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -13,6 +13,7 @@
#include <linux/dma-direct.h>
#include <linux/swiotlb.h>
#include <linux/mem_encrypt.h>
+#include <linux/string.h>
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/bitops.h>
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index b1d5c5a0ffb4..bf4466ef1220 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -2545,10 +2545,27 @@ EXPORT_SYMBOL_GPL(set_memory_encrypted);
int set_memory_decrypted(unsigned long addr, int numpages)
{
- return __set_memory_enc_dec(addr, numpages, false);
+ int ret = __set_memory_enc_dec(addr, numpages, false);
+
+ /*
+ * Firmware may scrub pages with nonzero values during private-to-shared
+ * conversion. Clear the final shared mapping after the transition so
+ * that callers receive zeroed memory.
+ */
+ if (!ret)
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(set_memory_decrypted);
+/* Device memory needs a decrypted mapping without writes to its contents. */
+int set_memory_decrypted_nozero(unsigned long addr, int numpages)
+{
+ return __set_memory_enc_dec(addr, numpages, false);
+}
+EXPORT_SYMBOL_GPL(set_memory_decrypted_nozero);
+
int set_pages_uc(struct page *page, int numpages)
{
unsigned long addr = (unsigned long)page_address(page);
diff --git a/drivers/firmware/arm_rmm/rsi.c b/drivers/firmware/arm_rmm/rsi.c
index 0fcc4e33bd92..2c1591028b0e 100644
--- a/drivers/firmware/arm_rmm/rsi.c
+++ b/drivers/firmware/arm_rmm/rsi.c
@@ -6,6 +6,7 @@
#include <linux/jump_label.h>
#include <linux/memblock.h>
#include <linux/psci.h>
+#include <linux/string.h>
#include <linux/swiotlb.h>
#include <linux/arm-rsi-cmds.h>
#include <linux/kobject.h>
@@ -146,6 +147,14 @@ static int realm_set_memory_decrypted(unsigned long addr, int numpages)
{
int ret = __set_memory_enc_dec(addr, numpages, false);
+ /*
+ * The RMM may scrub pages with nonzero values before returning them to
+ * the host. Zeroing before conversion would not guarantee zeroed shared
+ * memory, so clear the final shared mapping after the transition.
+ */
+ if (!ret)
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
+
WARN(ret, "Failed to decrypt memory, %d pages will be leaked",
numpages);
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 818667ef9c74..b25bfe2f8448 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -13,6 +13,7 @@
#include <linux/sched.h>
#include <linux/wait.h>
#include <linux/delay.h>
+#include <linux/cc_platform.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/slab.h>
@@ -263,29 +264,27 @@ int vmbus_connect(void)
goto cleanup;
}
- ret = set_memory_decrypted((unsigned long)
- vmbus_connection.monitor_pages[0], 1);
- ret |= set_memory_decrypted((unsigned long)
- vmbus_connection.monitor_pages[1], 1);
- if (ret) {
- /*
- * If set_memory_decrypted() fails, the encryption state
- * of the memory is unknown. So leak the memory instead
- * of risking returning decrypted memory to the free list.
- * For simplicity, always handle both pages the same.
- */
- vmbus_connection.monitor_pages[0] = NULL;
- vmbus_connection.monitor_pages[1] = NULL;
- goto cleanup;
+ if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
+ ret = set_memory_decrypted((unsigned long)vmbus_connection.monitor_pages[0],
+ 1);
+ ret |= set_memory_decrypted((unsigned long)vmbus_connection.monitor_pages[1],
+ 1);
+ if (ret) {
+ /*
+ * If set_memory_decrypted() fails, the encryption state
+ * of the memory is unknown. So leak the memory instead
+ * of risking returning decrypted memory to the free list.
+ * For simplicity, always handle both pages the same.
+ */
+ vmbus_connection.monitor_pages[0] = NULL;
+ vmbus_connection.monitor_pages[1] = NULL;
+ goto cleanup;
+ }
+ } else {
+ memset(vmbus_connection.monitor_pages[0], 0, HV_HYP_PAGE_SIZE);
+ memset(vmbus_connection.monitor_pages[1], 0, HV_HYP_PAGE_SIZE);
}
- /*
- * Set_memory_decrypted() will change the memory contents if
- * decryption occurs, so zero monitor pages here.
- */
- memset(vmbus_connection.monitor_pages[0], 0x00, HV_HYP_PAGE_SIZE);
- memset(vmbus_connection.monitor_pages[1], 0x00, HV_HYP_PAGE_SIZE);
-
msginfo = kzalloc(sizeof(*msginfo) +
sizeof(struct vmbus_channel_initiate_contact),
GFP_KERNEL);
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index fe50090dcc01..6e836a02910a 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -123,12 +123,13 @@ static int hv_alloc_page(void **page, bool decrypt, const char *note)
if (!*page)
return -ENOMEM;
- if (decrypt)
+ if (decrypt) {
ret = set_memory_decrypted((unsigned long)*page, 1);
- if (ret)
- goto failed;
-
- memset(*page, 0, PAGE_SIZE);
+ if (ret)
+ goto failed;
+ } else {
+ memset(*page, 0, PAGE_SIZE);
+ }
return 0;
failed:
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index df2577896271..404d2436346e 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -534,8 +534,6 @@ int hv_common_cpu_init(unsigned int cpu)
/* It may be unsafe to free 'mem' */
return ret;
}
-
- memset(mem, 0x00, pgcount * HV_HYP_PAGE_SIZE);
}
/*
diff --git a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c
index 26fe9c3f22e3..54a27b81458a 100644
--- a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c
+++ b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c
@@ -13,6 +13,7 @@
#include <linux/mem_encrypt.h>
#include <linux/mm.h>
#include <linux/pgtable.h>
+#include <linux/string.h>
#include <asm/hypervisor.h>
@@ -61,6 +62,9 @@ static int pkvm_set_memory_encrypted(unsigned long addr, int numpages)
static int pkvm_set_memory_decrypted(unsigned long addr, int numpages)
{
+ /* Sharing exposes the existing contents to the host; clear them first. */
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
+
return __set_memory_range(ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID,
addr, numpages);
}
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h
index 27c32fbabfec..09f9ea06da6f 100644
--- a/include/linux/set_memory.h
+++ b/include/linux/set_memory.h
@@ -5,6 +5,10 @@
#ifndef _LINUX_SET_MEMORY_H_
#define _LINUX_SET_MEMORY_H_
+#include <linux/string.h>
+
+#include <asm/page.h>
+
/**
* DOC: Kernel page table permissions
*
@@ -21,7 +25,8 @@
*
* When an architecture does not implement these APIs they succeed without
* doing anything, so a return value of 0 does not mean that the permissions
- * were actually changed.
+ * were actually changed. set_memory_decrypted() also clears the range,
+ * including when no encryption change is needed. It is for normal RAM only.
*
* Callers that depend on the permissions being applied must ensure that the
* architecture supports the required operation for the target addresses. The
@@ -186,6 +191,7 @@ static inline int set_memory_encrypted(unsigned long addr, int numpages)
static inline int set_memory_decrypted(unsigned long addr, int numpages)
{
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
return 0;
}
#endif /* CONFIG_ARCH_HAS_MEM_ENCRYPT */
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 25e9a86835fd..596c90e8f890 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -229,7 +229,8 @@ void *dma_direct_alloc(struct device *dev, size_t size,
if (force_dma_unencrypted(dev))
attrs |= __DMA_ATTR_ALLOC_CC_SHARED;
- if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) {
+ mark_mem_decrypt = attrs & __DMA_ATTR_ALLOC_CC_SHARED;
+ if (mark_mem_decrypt) {
/*
* Unencrypted/shared DMA requires a linear-mapped buffer
* address to look up the PFN and set architecture-required PFN
@@ -237,7 +238,6 @@ void *dma_direct_alloc(struct device *dev, size_t size,
* allocation.
*/
allow_highmem = false;
- mark_mem_decrypt = true;
}
size = PAGE_ALIGN(size);
@@ -341,7 +341,9 @@ void *dma_direct_alloc(struct device *dev, size_t size,
cpu_addr = page_address(page);
}
- memset(cpu_addr, 0, size);
+ /* Zero after remapping because the page may be in HighMem. */
+ if (!mark_mem_decrypt)
+ memset(cpu_addr, 0, size);
if (set_uncached) {
void *uncached_cpu_addr;
@@ -451,10 +453,13 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
unsigned long attrs = 0;
struct page *page;
void *cpu_addr;
+ bool mark_mem_decrypt;
if (force_dma_unencrypted(dev))
attrs |= __DMA_ATTR_ALLOC_CC_SHARED;
+ mark_mem_decrypt = attrs & __DMA_ATTR_ALLOC_CC_SHARED;
+
if ((attrs & __DMA_ATTR_ALLOC_CC_SHARED) && dma_direct_use_pool(dev, gfp))
return dma_direct_alloc_from_pool(dev, size, dma_handle,
&cpu_addr, gfp, attrs);
@@ -465,6 +470,7 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
return NULL;
cpu_addr = page_address(page);
+ mark_mem_decrypt = false;
goto setup_page;
}
@@ -473,11 +479,13 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
return NULL;
cpu_addr = page_address(page);
- if ((attrs & __DMA_ATTR_ALLOC_CC_SHARED) &&
- dma_set_decrypted(dev, cpu_addr, size))
- goto out_leak_pages;
setup_page:
- memset(cpu_addr, 0, size);
+ if (mark_mem_decrypt) {
+ if (dma_set_decrypted(dev, cpu_addr, size))
+ goto out_leak_pages;
+ } else {
+ memset(cpu_addr, 0, size);
+ }
*dma_handle = phys_to_dma_direct(dev, page_to_phys(page),
attrs & __DMA_ATTR_ALLOC_CC_SHARED);
return page;
diff --git a/mm/cc_shared.c b/mm/cc_shared.c
index 77ee65c4c35e..c9b4e3dfe06a 100644
--- a/mm/cc_shared.c
+++ b/mm/cc_shared.c
@@ -104,7 +104,6 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
struct cc_shared_layout layout;
struct page *page;
unsigned int order;
- bool zero = gfp & __GFP_ZERO;
int ret;
ret = cc_shared_calc_layout(requested, &layout);
@@ -116,7 +115,8 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
return -EINVAL;
/*
* State transitions require a linear-map address and may modify memory.
- * Allocate from low memory and defer requested zeroing until afterwards.
+ * Allocate from low memory and let the architecture place zeroing at the
+ * appropriate point in the transition.
* Shared allocations have no migration support, so they must not
* be allocated with __GFP_MOVABLE.
*/
@@ -143,9 +143,6 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
return ret;
}
- if (zero)
- memset(page_address(page), 0, layout.shared_size);
-
mem->page = page;
mem->shared_size = layout.shared_size;
return 0;
--
2.43.0
next prev parent reply other threads:[~2026-11-16 9:12 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 01/13] mm: Add an allocator for CoCo shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` Aneesh Kumar K.V (Arm) [this message]
2026-11-16 9:11 ` [PATCH v10 03/13] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 04/13] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 05/13] dma-contiguous: Derive shared alignment from DMA attributes Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 06/13] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 07/13] dma-direct: Align CoCo shared DMA allocations to the shared granule size Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 08/13] swiotlb: Align shared IO TLB pools " Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 09/13] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 10/13] dma-buf: system_heap: Limit scatterlist entries to the buffer size Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 11/13] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 12/13] swiotlb: Make rounded shared pool capacity allocatable Aneesh Kumar K.V (Arm)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116091137.94252-3-aneesh.kumar@kernel.org \
--to=aneesh.kumar@kernel.org \
--cc=Brian.Starkey@arm.com \
--cc=agordeev@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=baoquan.he@linux.dev \
--cc=benjamin.gaignard@collabora.com \
--cc=borntraeger@linux.ibm.com \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=chleroy@kernel.org \
--cc=christian.koenig@amd.com \
--cc=dave.hansen@linux.intel.com \
--cc=decui@microsoft.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=haiyangz@microsoft.com \
--cc=hca@linux.ibm.com \
--cc=hpa@zytor.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=joro@8bytes.org \
--cc=jstultz@google.com \
--cc=kas@kernel.org \
--cc=kvmarm@lists.linux.dev \
--cc=kys@microsoft.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-media@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux@armlinux.org.uk \
--cc=longli@microsoft.com \
--cc=luto@kernel.org \
--cc=m.szyprowski@samsung.com \
--cc=maddy@linux.ibm.com \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=mingo@redhat.com \
--cc=mpe@ellerman.id.au \
--cc=npiggin@gmail.com \
--cc=pasha.tatashin@soleen.com \
--cc=patches@lists.linux.dev \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=pratyush@kernel.org \
--cc=radu@rendec.net \
--cc=rick.p.edgecombe@intel.com \
--cc=ritesh.list@gmail.com \
--cc=robin.murphy@arm.com \
--cc=rppt@kernel.org \
--cc=sshegde@linux.ibm.com \
--cc=steven.price@arm.com \
--cc=sumit.semwal@linaro.org \
--cc=suzuki.poulose@arm.com \
--cc=svens@linux.ibm.com \
--cc=tglx@kernel.org \
--cc=tjmercier@google.com \
--cc=vkuznets@redhat.com \
--cc=wei.liu@kernel.org \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox