* [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory
@ 2026-11-16 9:11 Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 01/13] mm: Add an allocator for CoCo shared memory Aneesh Kumar K.V (Arm)
` (13 more replies)
0 siblings, 14 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
This series tightens the alignment requirements for buffers that are shared
between confidential-computing guests and the host, and adds a common
allocator for host-shared memory.
When a guest runs with private memory, buffers shared with the hypervisor
are not only accessed by the guest. They are also accessed by the host
kernel, and the host may manage the corresponding shared/private state at a
granularity larger than the guest page size.
Architectures such as Arm can detect incorrect accesses to Realm physical
address space PFNs through GPC faults. However, relying on that as the only
line of defence is fragile and can still lead to kernel crashes. The risk
is especially visible for shared buffers that are later mmapped into
userspace, such as guest_memfd or dma-buf backed allocations. Once
userspace can access the mapping, the kernel cannot guarantee that
applications will only touch the intended 4K region rather than the whole
host page mapped into their address space. Those userspace addresses may
also be passed back into the kernel and accessed through the linear map,
resulting in a GPC fault.
To avoid this, host-shared buffers must satisfy two constraints:
- the address must be aligned to the CoCo shared-granule size
- the size must be a multiple of that granule size
The series adds a common CoCo shared-memory layer for enforcing these
constraints. It provides shared-granule geometry and range-validation
helpers, byte-oriented private/shared transition helpers, and
alloc_cc_shared_pages(). The allocator rounds a request to the architecture
shared granule, allocates suitably aligned contiguous pages, transitions
the complete allocation to shared state, and returns the transitioned size
alongside the page.
The corresponding free helper restores the complete allocation to private
state before returning it to the buddy allocator. If private state cannot
be restored safely, the allocation is deliberately leaked rather than
returning potentially shared memory for unrelated use.
The series also makes zeroing part of private-to-shared transitions, with
each architecture clearing memory at the appropriate point. Callers must
therefore initialize shared buffers after the transition; existing contents
are not preserved.
The shared-granule size defaults to PAGE_SIZE and can be overridden by an
architecture. The arm64 CCA changes to discover the host's IPA state change
alignment are not included in this series.
GIC ITS command queues and tables and the dma-buf system_cc_shared heap
use the common allocator. DMA direct, atomic DMA pools and SWIOTLB retain
their allocation paths and use the common layout or transition helpers.
Restricted DMA pools are checked for shared-granule alignment.
Hyper-V users of set_memory_encrypted() and set_memory_decrypted() are not
changed to use cc_make_shared() by this series. Those paths are not
currently used by the arm64 CCA code path, and therefore are not part of
the arm64 CCA IPA state change alignment problem addressed here.
Notes:
* Codex assisted with implementation, review and series organization.
Changes from v9:
https://lore.kernel.org/all/20261104164555.3555725-1-aneesh.kumar@kernel.org
* Leak allocations after failed sharing in the common allocator and SWIOTLB.
* Move x86 zeroing from the AMD, TDX and Hyper-V vTOM callbacks to
set_memory_decrypted(), after a successful transition
* Add set_memory_decrypted_nozero() for VGA mappings, which must change
memory attributes without clearing device memory.
* Zero memory even when encryption is inactive in the generic, arm64 and
pSeries implementations of set_memory_decrypted().
* Move pSeries zeroing after sharing. Document why pKVM and s390 must clear
private contents before sharing
* Drop the GIC ITS VPE L1 preallocation patch from this series.
This issue predates the series and will be addressed separately.
* Add mm/cc_shared.c and include/linux/cc_shared.h to the TSM MAINTAINERS
entry.
Changes from v8:
https://lore.kernel.org/all/20260924100529.1398790-1-aneesh.kumar@kernel.org
* Rebase onto the 7.4-rc1
* Move TDX and AMD (including SEV-SNP) zeroing to their finish callbacks
so private-to-shared transitions clear the final shared mapping.
* Clear __GFP_MOVABLE for shared allocations, which cannot be migrated.
* Resolve NUMA_NO_NODE in the common ITS allocation helper to preserve
placement for all ITS callers.
* Stop SWIOTLB allocation retries when the reduced pool would fall below
the shared granule size, avoiding an allocation retry loop.
Changes from v7:
https://lore.kernel.org/all/20260921144847.501151-1-aneesh.kumar@kernel.org
* Add the following new patches:
* "irqchip/gic-v3-its: Preallocate VPE L1 tables"
* "mm: Assert CoCo shared allocations may sleep"
* "mm: Zero memory during shared memory transitions"
* Drop the arm64 RHI and shared granule size patches so that the series can
be rebased on top of upstream to enable Shashiko review.
Changes from v6:
https://lore.kernel.org/all/20260904103452.1197239-1-aneesh.kumar@kernel.org
* Add a common allocator and geometry/transition helpers for CoCo host-shared
memory.
* Convert GIC ITS, dma-direct, atomic DMA pools, and the dma-buf
system_cc_shared heap to the common allocator.
* Limit dma-buf scatterlist entries to the requested buffer size so rounded
backing is not exposed to importers.
Changes from v5:
https://lore.kernel.org/all/20260706060432.1375570-1-aneesh.kumar@kernel.org
* Rebased to latest kernel
* Drop patch arm64: realm: Move Realm memory encryption ops to RSI code
Changes from v4:
https://lore.kernel.org/all/20260427063108.909019-1-aneesh.kumar@kernel.org
* Rename the helpers to use CoCo terminology
(mem_cc_shared_granule_size() / mem_cc_align_to_shared_granule() instead of
mem_decrypt_granule_size() / mem_decrypt_align()).
* Use __DMA_ATTR_ALLOC_CC_SHARED to pass CoCo shared allocation requirements
down to CMA-based allocation helpers.
* Add validation for restricted DMA pools to reject pools that are not aligned
to the shared granule size.
* Add dma-buf system heap handling for cc-shared buffers.
* Split the previous combined DMA/SWIOTLB/ITS change into smaller subsystem
patches covering ITS, DMA direct, SWIOTLB, restricted DMA pools, dma-buf
system heap, and arm64 Realm support.
* Rework arm64 Realm support by moving Realm memory encryption ops into RSI
code and exposing the CCA shared granule size through arm64_mem_crypt_ops.
Changes from v3:
https://lore.kernel.org/all/20260309102625.2315725-1-aneesh.kumar@kernel.org
* Fix build error reported by kernel test robot <lkp@intel.com>
Changes from v2:
https://lore.kernel.org/all/20251221160920.297689-1-aneesh.kumar@kernel.org
* Rebase to latest kernel
* Consider swiotlb always decrypted and don't align when allocating from swiotlb.
Changes from v1:
* Rename the helper to mem_encrypt_align
* Improve the commit message
* Handle DMA allocations from contiguous memory
* Handle DMA allocations from the pool
* swiotlb is still considered unencrypted. Support for an encrypted swiotlb pool
is left as TODO and is independent of this series.
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joerg Roedel (AMD) <joro@8bytes.org>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Cc: Brian Starkey <Brian.Starkey@arm.com>
Cc: John Stultz <jstultz@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Radu Rendec <radu@rendec.net>
Cc: "T.J. Mercier" <tjmercier@google.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: x86@kernel.org
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Long Li <longli@microsoft.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: dri-devel@lists.freedesktop.org
Cc: iommu@lists.linux.dev
Cc: linaro-mm-sig@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-media@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: kvmarm@lists.linux.dev
Cc: patches@lists.linux.dev
Aneesh Kumar K.V (Arm) (13):
mm: Add an allocator for CoCo shared memory
mm: Zero memory during shared memory transitions
irqchip/gic-v3-its: Resolve the default NUMA node explicitly
irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory
allocator
dma-contiguous: Derive shared alignment from DMA attributes
dma-pool: Allocate CoCo atomic pools using CoCo shared memory
allocator
dma-direct: Align CoCo shared DMA allocations to the shared granule
size
swiotlb: Align shared IO TLB pools to the shared granule size
swiotlb: Reject misaligned restricted DMA pools for CoCo guests
dma-buf: system_heap: Limit scatterlist entries to the buffer size
dma-buf: system_heap: Allocate shared buffers using CoCo shared memory
allocator
swiotlb: Make rounded shared pool capacity allocatable
mm: Assert CoCo shared allocations may sleep
MAINTAINERS | 2 +
arch/arm/mm/dma-mapping.c | 5 +-
arch/arm64/mm/mem_encrypt.c | 8 +-
arch/powerpc/platforms/pseries/svm.c | 13 +-
arch/s390/mm/init.c | 7 +
arch/x86/hyperv/hv_init.c | 6 +-
arch/x86/include/asm/set_memory.h | 1 +
arch/x86/include/asm/vga.h | 4 +-
arch/x86/kernel/kvmclock.c | 6 +-
arch/x86/mm/mem_encrypt_amd.c | 1 +
arch/x86/mm/pat/set_memory.c | 19 +-
drivers/dma-buf/heaps/system_heap.c | 128 +++++-----
drivers/firmware/arm_rmm/rsi.c | 9 +
drivers/hv/connection.c | 41 ++-
drivers/hv/hv.c | 11 +-
drivers/hv/hv_common.c | 2 -
drivers/iommu/dma-iommu.c | 2 +-
drivers/irqchip/irq-gic-v3-its.c | 42 ++--
drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c | 4 +
include/linux/cc_shared.h | 39 +++
include/linux/dma-map-ops.h | 9 +-
include/linux/set_memory.h | 8 +-
kernel/dma/contiguous.c | 41 ++-
kernel/dma/direct.c | 73 ++++--
kernel/dma/ops_helpers.c | 2 +-
kernel/dma/pool.c | 27 +-
kernel/dma/swiotlb.c | 94 ++++---
kernel/kexec_file.c | 3 +-
mm/Makefile | 1 +
mm/cc_shared.c | 238 ++++++++++++++++++
30 files changed, 635 insertions(+), 211 deletions(-)
create mode 100644 include/linux/cc_shared.h
create mode 100644 mm/cc_shared.c
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
prerequisite-patch-id: 121492d71cc22b2adbfdedd668868c5a449ad764
prerequisite-patch-id: ed31fd728f0adba3d60dcd6f3bae8da1a2193bcc
--
2.43.0
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v10 01/13] mm: Add an allocator for CoCo shared memory
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 02/13] mm: Zero memory during shared memory transitions Aneesh Kumar K.V (Arm)
` (12 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
Confidential-computing guests may require memory shared with the host to
be aligned and transitioned in units larger than PAGE_SIZE. Several DMA
users need struct page-backed allocations satisfying these requirements.
Provide a common allocator instead of requiring each user to open-code
this sequence.
Add alloc_cc_shared_pages() and its node-aware variant. The allocator
rounds the requested size to the architecture's shared granule,
allocates suitably aligned contiguous pages and transitions the
complete range to shared state. It also preserves the caller's GFP
policy.
A private-to-shared transition may alter memory contents. Mask
__GFP_ZERO from the underlying allocation and when requested, clear the
complete transitioned range after cc_make_shared() succeeds.
Return the page and transitioned size so free_cc_shared_pages() can
restore the complete range to private state before freeing it. If
private state cannot be established, retain the allocation instead of
returning a possibly shared page to the buddy allocator.
Also provide the shared-granule geometry and byte-oriented transition
helpers used by the allocator and by callers managing their own backing
memory.
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
MAINTAINERS | 2 +
include/linux/cc_shared.h | 39 +++++++
mm/Makefile | 1 +
mm/cc_shared.c | 238 ++++++++++++++++++++++++++++++++++++++
4 files changed, 280 insertions(+)
create mode 100644 include/linux/cc_shared.h
create mode 100644 mm/cc_shared.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 3e626e45d432..5ce879ea2ef4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -28553,6 +28553,8 @@ F: drivers/pci/tsm/
F: drivers/virt/coco/guest/
F: drivers/virt/coco/tsm-core.c
F: include/linux/*tsm*.h
+F: include/linux/cc_shared.h
+F: mm/cc_shared.c
F: samples/tsm-mr/
TRUSTED SERVICES TEE DRIVER
diff --git a/include/linux/cc_shared.h b/include/linux/cc_shared.h
new file mode 100644
index 000000000000..5f8db7c468c5
--- /dev/null
+++ b/include/linux/cc_shared.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_CC_SHARED_H
+#define _LINUX_CC_SHARED_H
+
+#include <linux/gfp_types.h>
+#include <linux/types.h>
+
+struct page;
+
+struct cc_shared_pages {
+ struct page *page;
+ size_t shared_size;
+};
+
+struct cc_shared_layout {
+ size_t requested_size;
+ size_t shared_size;
+ size_t alignment;
+};
+
+/*
+ * Architectures may override this to return the granule used for transitions
+ * between private and shared memory. The value must be a power of two and no
+ * smaller than PAGE_SIZE.
+ */
+size_t arch_cc_shared_granule_size(void);
+
+size_t cc_shared_granule_size(void);
+int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layout);
+bool cc_shared_range_valid(phys_addr_t base, size_t size);
+int cc_make_shared(void *addr, size_t size);
+int cc_make_private(void *addr, size_t size);
+int alloc_cc_shared_pages_node(int nid, gfp_t gfp,
+ size_t requested, struct cc_shared_pages *mem);
+int alloc_cc_shared_pages(gfp_t gfp,
+ size_t requested, struct cc_shared_pages *mem);
+void free_cc_shared_pages(struct cc_shared_pages *mem);
+
+#endif /* _LINUX_CC_SHARED_H */
diff --git a/mm/Makefile b/mm/Makefile
index 2a3ec53d62ee..a712656449d1 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -57,6 +57,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
interval_tree.o list_lru.o workingset.o \
debug.o gup.o mmap_lock.o vma_init.o char-mem.o \
$(mmu-y)
+obj-y += cc_shared.o
# Give 'page_alloc' its own module-parameter namespace
page-alloc-y := page_alloc.o
diff --git a/mm/cc_shared.c b/mm/cc_shared.c
new file mode 100644
index 000000000000..77ee65c4c35e
--- /dev/null
+++ b/mm/cc_shared.c
@@ -0,0 +1,238 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2026 ARM Ltd.
+ */
+#include <linux/align.h>
+#include <linux/cc_platform.h>
+#include <linux/cc_shared.h>
+#include <linux/errno.h>
+#include <linux/export.h>
+#include <linux/gfp.h>
+#include <linux/log2.h>
+#include <linux/mm.h>
+#include <linux/mem_encrypt.h>
+#include <linux/numa.h>
+#include <linux/overflow.h>
+#include <linux/set_memory.h>
+
+size_t __weak arch_cc_shared_granule_size(void)
+{
+ return PAGE_SIZE;
+}
+
+size_t cc_shared_granule_size(void)
+{
+ size_t granule = arch_cc_shared_granule_size();
+
+ if (WARN_ON_ONCE(granule < PAGE_SIZE || !is_power_of_2(granule)))
+ return PAGE_SIZE;
+
+ return granule;
+}
+EXPORT_SYMBOL_GPL(cc_shared_granule_size);
+
+int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layout)
+{
+ size_t granule, rounded;
+
+ if (!requested || !layout)
+ return -EINVAL;
+
+ granule = cc_shared_granule_size();
+ if (check_add_overflow(requested, granule - 1, &rounded))
+ return -EOVERFLOW;
+
+ rounded = ALIGN_DOWN(rounded, granule);
+ layout->requested_size = requested;
+ layout->shared_size = rounded;
+ layout->alignment = granule;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(cc_shared_calc_layout);
+
+bool cc_shared_range_valid(phys_addr_t base, size_t size)
+{
+ size_t granule = cc_shared_granule_size();
+
+ if (!size)
+ return false;
+
+ return IS_ALIGNED(base, granule) && IS_ALIGNED(size, granule);
+}
+EXPORT_SYMBOL_GPL(cc_shared_range_valid);
+
+static int cc_validate_transition(void *addr, size_t size)
+{
+ phys_addr_t phys;
+
+ if (!addr || !size || !PAGE_ALIGNED(addr) ||
+ !virt_addr_valid(addr))
+ return -EINVAL;
+
+ phys = page_to_phys(virt_to_page(addr));
+ if (!cc_shared_range_valid(phys, size))
+ return -EINVAL;
+
+ return 0;
+}
+
+int cc_make_shared(void *addr, size_t size)
+{
+ int ret = cc_validate_transition(addr, size);
+
+ if (ret)
+ return ret;
+
+ return set_memory_decrypted((unsigned long)addr, size >> PAGE_SHIFT);
+}
+
+int cc_make_private(void *addr, size_t size)
+{
+ int ret = cc_validate_transition(addr, size);
+
+ if (ret)
+ return ret;
+
+ return set_memory_encrypted((unsigned long)addr, size >> PAGE_SHIFT);
+}
+
+static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
+ size_t requested,
+ struct cc_shared_pages *mem)
+{
+ struct cc_shared_layout layout;
+ struct page *page;
+ unsigned int order;
+ bool zero = gfp & __GFP_ZERO;
+ int ret;
+
+ ret = cc_shared_calc_layout(requested, &layout);
+ if (ret)
+ return ret;
+
+ order = get_order(layout.shared_size);
+ if (order > MAX_PAGE_ORDER)
+ return -EINVAL;
+ /*
+ * State transitions require a linear-map address and may modify memory.
+ * Allocate from low memory and defer requested zeroing until afterwards.
+ * Shared allocations have no migration support, so they must not
+ * be allocated with __GFP_MOVABLE.
+ */
+ gfp &= ~(__GFP_HIGHMEM | __GFP_ZERO | __GFP_MOVABLE);
+ if (nid == NUMA_NO_NODE)
+ page = alloc_pages(gfp, order);
+ else
+ page = alloc_pages_node(nid, gfp, order);
+ if (!page)
+ return -ENOMEM;
+
+ ret = cc_make_shared(page_address(page), layout.shared_size);
+ if (ret) {
+ /*
+ * A failed transition may leave the range partially
+ * shared, with no indication of which pages changed state.
+ * Converting the whole range back to private is not safe
+ * on architectures that cannot handle pages which are
+ * already private. Leak the allocation rather than return
+ * pages with uncertain state to the allocator.
+ */
+ pr_warn_ratelimited("leaking %zu bytes with uncertain shared state\n",
+ layout.shared_size);
+ return ret;
+ }
+
+ if (zero)
+ memset(page_address(page), 0, layout.shared_size);
+
+ mem->page = page;
+ mem->shared_size = layout.shared_size;
+ return 0;
+}
+
+/**
+ * alloc_cc_shared_pages_node - allocate memory that can be shared
+ * @nid: NUMA node from which to allocate, or %NUMA_NO_NODE
+ * @gfp: allocation flags
+ * @requested: number of bytes requested; must be nonzero
+ * @mem: storage for the allocated page and the size of the shared range
+ *
+ * Allocate at least @requested bytes and make the allocation shared when
+ * memory encryption is active. A memory-state transition requires a valid
+ * linear-map address, so such allocations never come from high memory
+ *
+ * The shared range may be rounded up to the architecture's transition
+ * granule. On success, @mem->shared_size records the actual size that was
+ * made shared and must be retained unchanged for free_cc_shared_pages().
+ * @mem is not modified on failure.
+ *
+ * Return: 0 on success, or a negative error code on failure.
+ */
+int alloc_cc_shared_pages_node(int nid, gfp_t gfp,
+ size_t requested,
+ struct cc_shared_pages *mem)
+{
+ struct page *page;
+ unsigned int order;
+
+ if (!mem || !requested)
+ return -EINVAL;
+
+ if (cc_platform_has(CC_ATTR_MEM_ENCRYPT))
+ return __alloc_cc_shared_pages_node(nid, gfp, requested, mem);
+
+ order = get_order(requested);
+ if (order > MAX_PAGE_ORDER)
+ return -EINVAL;
+
+ if (nid == NUMA_NO_NODE)
+ page = alloc_pages(gfp, order);
+ else
+ page = alloc_pages_node(nid, gfp, order);
+ if (!page)
+ return -ENOMEM;
+
+ mem->page = page;
+ mem->shared_size = requested;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(alloc_cc_shared_pages_node);
+
+/**
+ * alloc_cc_shared_pages - allocate memory that can be shared
+ * @gfp: allocation flags
+ * @requested: number of bytes requested; must be nonzero
+ * @mem: storage for the allocated page and the size of the shared range
+ *
+ * Equivalent to alloc_cc_shared_pages_node() with %NUMA_NO_NODE.
+ *
+ * Return: 0 on success, or a negative error code on failure.
+ */
+int alloc_cc_shared_pages(gfp_t gfp,
+ size_t requested, struct cc_shared_pages *mem)
+{
+ return alloc_cc_shared_pages_node(NUMA_NO_NODE, gfp, requested, mem);
+}
+EXPORT_SYMBOL_GPL(alloc_cc_shared_pages);
+
+void free_cc_shared_pages(struct cc_shared_pages *mem)
+{
+ if (!mem || !mem->page)
+ return;
+
+ if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
+ goto free_pages;
+
+ if (cc_make_private(page_address(mem->page), mem->shared_size)) {
+ pr_warn_ratelimited("leaking %zu bytes that cannot be made private\n",
+ mem->shared_size);
+ return;
+ }
+
+free_pages:
+ __free_pages(mem->page, get_order(mem->shared_size));
+ mem->page = NULL;
+ mem->shared_size = 0;
+}
+EXPORT_SYMBOL_GPL(free_cc_shared_pages);
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 02/13] mm: Zero memory during shared memory transitions
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 01/13] mm: Add an allocator for CoCo shared memory Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 13:33 ` Fuad Tabba
2026-11-16 9:11 ` [PATCH v10 03/13] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Aneesh Kumar K.V (Arm)
` (11 subsequent siblings)
13 siblings, 1 reply; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
Architectures need to zero memory at different points in a private-to-shared
transition. For example, pKVM must clear memory before sharing it, while Arm
CCA can only clear it after the RSI transition has completed.
Make zeroing an implicit property of set_memory_decrypted(). Each architecture
or platform performs the clear at the safe point in its transition. TDX
clears memory after the private-to-shared conversion succeeds in the finish
callback. AMD clears memory in its finish callback after the page-table
update, TLB flush, and hypervisor notification, including for SEV-SNP. This
zeros the final shared mapping without adding a flag to
set_memory_decrypted() or threading it through architecture-specific callbacks.
Keep allocations on platforms without memory encryption on the ordinary page
allocator path so the original GFP constraints, including __GFP_ZERO, remain
intact. Remove post-transition memset() calls that are now redundant and
explicitly clear allocations on paths where no transition occurs.
This intentionally makes every private-to-shared transition destructive;
callers can no longer use set_memory_decrypted() to preserve existing contents.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: x86@kernel.org
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Long Li <longli@microsoft.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
arch/arm64/mm/mem_encrypt.c | 8 +++-
arch/powerpc/platforms/pseries/svm.c | 13 ++++--
arch/s390/mm/init.c | 7 ++++
arch/x86/hyperv/hv_init.c | 6 ++-
arch/x86/include/asm/set_memory.h | 1 +
arch/x86/include/asm/vga.h | 4 +-
arch/x86/kernel/kvmclock.c | 6 +--
arch/x86/mm/mem_encrypt_amd.c | 1 +
arch/x86/mm/pat/set_memory.c | 19 ++++++++-
drivers/firmware/arm_rmm/rsi.c | 9 ++++
drivers/hv/connection.c | 41 +++++++++----------
drivers/hv/hv.c | 11 ++---
drivers/hv/hv_common.c | 2 -
drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c | 4 ++
include/linux/set_memory.h | 8 +++-
kernel/dma/direct.c | 22 ++++++----
mm/cc_shared.c | 7 +---
17 files changed, 115 insertions(+), 54 deletions(-)
diff --git a/arch/arm64/mm/mem_encrypt.c b/arch/arm64/mm/mem_encrypt.c
index ee3c0ab04384..25eb67138b28 100644
--- a/arch/arm64/mm/mem_encrypt.c
+++ b/arch/arm64/mm/mem_encrypt.c
@@ -17,6 +17,7 @@
#include <linux/compiler.h>
#include <linux/err.h>
#include <linux/mm.h>
+#include <linux/string.h>
#include <asm/mem_encrypt.h>
@@ -42,8 +43,13 @@ EXPORT_SYMBOL_GPL(set_memory_encrypted);
int set_memory_decrypted(unsigned long addr, int numpages)
{
- if (likely(!crypt_ops) || WARN_ON(!PAGE_ALIGNED(addr)))
+ if (likely(!crypt_ops)) {
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
return 0;
+ }
+
+ if (WARN_ON(!PAGE_ALIGNED(addr)))
+ return -EINVAL;
return crypt_ops->decrypt(addr, numpages);
}
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index 7a403dbd35ee..0970f1fb76d7 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -9,6 +9,7 @@
#include <linux/mm.h>
#include <linux/memblock.h>
#include <linux/mem_encrypt.h>
+#include <linux/string.h>
#include <linux/cc_platform.h>
#include <asm/machdep.h>
#include <asm/svm.h>
@@ -53,13 +54,17 @@ int set_memory_encrypted(unsigned long addr, int numpages)
int set_memory_decrypted(unsigned long addr, int numpages)
{
- if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
- return 0;
-
if (!PAGE_ALIGNED(addr))
return -EINVAL;
- uv_share_page(PHYS_PFN(__pa(addr)), numpages);
+ if (cc_platform_has(CC_ATTR_MEM_ENCRYPT))
+ uv_share_page(PHYS_PFN(__pa(addr)), numpages);
+ /*
+ * Firmware may scrub pages with nonzero values during
+ * private-to-shared conversion. Clear the final shared mapping
+ * after the transition so that callers receive zeroed memory.
+ */
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
return 0;
}
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index be7e009e7b59..fdacd9380286 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -129,6 +129,13 @@ int set_memory_encrypted(unsigned long vaddr, int numpages)
int set_memory_decrypted(unsigned long vaddr, int numpages)
{
int i;
+
+ /*
+ * uv_set_shared() exposes existing contents to the host; see the data
+ * leakage warning in share() in asm/uv.h. Clear private data first.
+ */
+ memset((void *)vaddr, 0, (size_t)numpages << PAGE_SHIFT);
+
/* make specified pages shared (swiotlb, dma_alloca) */
for (i = 0; i < numpages; ++i) {
uv_set_shared(virt_to_phys((void *)vaddr));
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index b90e6f487533..4356f4255ae9 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -156,8 +156,10 @@ static int hv_cpu_init(unsigned int cpu)
* page in non-root partition here.
*/
if (*hvp && !ms_hyperv.paravisor_present && hv_isolation_type_snp()) {
- WARN_ON_ONCE(set_memory_decrypted((unsigned long)(*hvp), 1));
- memset(*hvp, 0, PAGE_SIZE);
+ int ret;
+
+ ret = set_memory_decrypted((unsigned long)*hvp, 1);
+ WARN_ON_ONCE(ret);
}
}
diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index 39271a5ea925..dfac1dde3714 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -52,6 +52,7 @@ int set_memory_4k(unsigned long addr, int numpages);
bool set_memory_enc_stop_conversion(void);
int set_memory_encrypted(unsigned long addr, int numpages);
int set_memory_decrypted(unsigned long addr, int numpages);
+int set_memory_decrypted_nozero(unsigned long addr, int numpages);
int set_memory_np_noalias(unsigned long addr, int numpages);
int set_memory_nonglobal(unsigned long addr, int numpages);
diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index 46f9b2deab4d..e1e91dfa399d 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -14,7 +14,7 @@
* On the PC, we can just recalculate addresses and then
* access the videoram directly without any black magic.
* To support memory encryption however, we need to access
- * the videoram as decrypted memory.
+ * the videoram as decrypted memory without clearing its contents.
*/
#define VGA_MAP_MEM(x, s) \
@@ -22,7 +22,7 @@
unsigned long start = (unsigned long)phys_to_virt(x); \
\
if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) \
- set_memory_decrypted(start, (s) >> PAGE_SHIFT); \
+ set_memory_decrypted_nozero(start, (s) >> PAGE_SHIFT); \
\
start; \
})
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index cb3d0ca1fa22..3e9b7ad3dea2 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -248,7 +248,7 @@ static void __init kvmclock_init_mem(void)
* be mapped decrypted.
*/
if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
- r = set_memory_decrypted((unsigned long) hvclock_mem,
+ r = set_memory_decrypted((unsigned long)hvclock_mem,
1UL << order);
if (r) {
__free_pages(p, order);
@@ -256,9 +256,9 @@ static void __init kvmclock_init_mem(void)
pr_warn("kvmclock: set_memory_decrypted() failed. Disabling\n");
return;
}
+ } else {
+ memset(hvclock_mem, 0, PAGE_SIZE << order);
}
-
- memset(hvclock_mem, 0, PAGE_SIZE << order);
}
static int __init kvm_setup_vsyscall_timeinfo(void)
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
index 2f8c32173972..fbcd23b745cf 100644
--- a/arch/x86/mm/mem_encrypt_amd.c
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -13,6 +13,7 @@
#include <linux/dma-direct.h>
#include <linux/swiotlb.h>
#include <linux/mem_encrypt.h>
+#include <linux/string.h>
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/bitops.h>
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index b1d5c5a0ffb4..bf4466ef1220 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -2545,10 +2545,27 @@ EXPORT_SYMBOL_GPL(set_memory_encrypted);
int set_memory_decrypted(unsigned long addr, int numpages)
{
- return __set_memory_enc_dec(addr, numpages, false);
+ int ret = __set_memory_enc_dec(addr, numpages, false);
+
+ /*
+ * Firmware may scrub pages with nonzero values during private-to-shared
+ * conversion. Clear the final shared mapping after the transition so
+ * that callers receive zeroed memory.
+ */
+ if (!ret)
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(set_memory_decrypted);
+/* Device memory needs a decrypted mapping without writes to its contents. */
+int set_memory_decrypted_nozero(unsigned long addr, int numpages)
+{
+ return __set_memory_enc_dec(addr, numpages, false);
+}
+EXPORT_SYMBOL_GPL(set_memory_decrypted_nozero);
+
int set_pages_uc(struct page *page, int numpages)
{
unsigned long addr = (unsigned long)page_address(page);
diff --git a/drivers/firmware/arm_rmm/rsi.c b/drivers/firmware/arm_rmm/rsi.c
index 0fcc4e33bd92..2c1591028b0e 100644
--- a/drivers/firmware/arm_rmm/rsi.c
+++ b/drivers/firmware/arm_rmm/rsi.c
@@ -6,6 +6,7 @@
#include <linux/jump_label.h>
#include <linux/memblock.h>
#include <linux/psci.h>
+#include <linux/string.h>
#include <linux/swiotlb.h>
#include <linux/arm-rsi-cmds.h>
#include <linux/kobject.h>
@@ -146,6 +147,14 @@ static int realm_set_memory_decrypted(unsigned long addr, int numpages)
{
int ret = __set_memory_enc_dec(addr, numpages, false);
+ /*
+ * The RMM may scrub pages with nonzero values before returning them to
+ * the host. Zeroing before conversion would not guarantee zeroed shared
+ * memory, so clear the final shared mapping after the transition.
+ */
+ if (!ret)
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
+
WARN(ret, "Failed to decrypt memory, %d pages will be leaked",
numpages);
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 818667ef9c74..b25bfe2f8448 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -13,6 +13,7 @@
#include <linux/sched.h>
#include <linux/wait.h>
#include <linux/delay.h>
+#include <linux/cc_platform.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/slab.h>
@@ -263,29 +264,27 @@ int vmbus_connect(void)
goto cleanup;
}
- ret = set_memory_decrypted((unsigned long)
- vmbus_connection.monitor_pages[0], 1);
- ret |= set_memory_decrypted((unsigned long)
- vmbus_connection.monitor_pages[1], 1);
- if (ret) {
- /*
- * If set_memory_decrypted() fails, the encryption state
- * of the memory is unknown. So leak the memory instead
- * of risking returning decrypted memory to the free list.
- * For simplicity, always handle both pages the same.
- */
- vmbus_connection.monitor_pages[0] = NULL;
- vmbus_connection.monitor_pages[1] = NULL;
- goto cleanup;
+ if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
+ ret = set_memory_decrypted((unsigned long)vmbus_connection.monitor_pages[0],
+ 1);
+ ret |= set_memory_decrypted((unsigned long)vmbus_connection.monitor_pages[1],
+ 1);
+ if (ret) {
+ /*
+ * If set_memory_decrypted() fails, the encryption state
+ * of the memory is unknown. So leak the memory instead
+ * of risking returning decrypted memory to the free list.
+ * For simplicity, always handle both pages the same.
+ */
+ vmbus_connection.monitor_pages[0] = NULL;
+ vmbus_connection.monitor_pages[1] = NULL;
+ goto cleanup;
+ }
+ } else {
+ memset(vmbus_connection.monitor_pages[0], 0, HV_HYP_PAGE_SIZE);
+ memset(vmbus_connection.monitor_pages[1], 0, HV_HYP_PAGE_SIZE);
}
- /*
- * Set_memory_decrypted() will change the memory contents if
- * decryption occurs, so zero monitor pages here.
- */
- memset(vmbus_connection.monitor_pages[0], 0x00, HV_HYP_PAGE_SIZE);
- memset(vmbus_connection.monitor_pages[1], 0x00, HV_HYP_PAGE_SIZE);
-
msginfo = kzalloc(sizeof(*msginfo) +
sizeof(struct vmbus_channel_initiate_contact),
GFP_KERNEL);
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index fe50090dcc01..6e836a02910a 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -123,12 +123,13 @@ static int hv_alloc_page(void **page, bool decrypt, const char *note)
if (!*page)
return -ENOMEM;
- if (decrypt)
+ if (decrypt) {
ret = set_memory_decrypted((unsigned long)*page, 1);
- if (ret)
- goto failed;
-
- memset(*page, 0, PAGE_SIZE);
+ if (ret)
+ goto failed;
+ } else {
+ memset(*page, 0, PAGE_SIZE);
+ }
return 0;
failed:
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index df2577896271..404d2436346e 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -534,8 +534,6 @@ int hv_common_cpu_init(unsigned int cpu)
/* It may be unsafe to free 'mem' */
return ret;
}
-
- memset(mem, 0x00, pgcount * HV_HYP_PAGE_SIZE);
}
/*
diff --git a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c
index 26fe9c3f22e3..54a27b81458a 100644
--- a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c
+++ b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c
@@ -13,6 +13,7 @@
#include <linux/mem_encrypt.h>
#include <linux/mm.h>
#include <linux/pgtable.h>
+#include <linux/string.h>
#include <asm/hypervisor.h>
@@ -61,6 +62,9 @@ static int pkvm_set_memory_encrypted(unsigned long addr, int numpages)
static int pkvm_set_memory_decrypted(unsigned long addr, int numpages)
{
+ /* Sharing exposes the existing contents to the host; clear them first. */
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
+
return __set_memory_range(ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID,
addr, numpages);
}
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h
index 27c32fbabfec..09f9ea06da6f 100644
--- a/include/linux/set_memory.h
+++ b/include/linux/set_memory.h
@@ -5,6 +5,10 @@
#ifndef _LINUX_SET_MEMORY_H_
#define _LINUX_SET_MEMORY_H_
+#include <linux/string.h>
+
+#include <asm/page.h>
+
/**
* DOC: Kernel page table permissions
*
@@ -21,7 +25,8 @@
*
* When an architecture does not implement these APIs they succeed without
* doing anything, so a return value of 0 does not mean that the permissions
- * were actually changed.
+ * were actually changed. set_memory_decrypted() also clears the range,
+ * including when no encryption change is needed. It is for normal RAM only.
*
* Callers that depend on the permissions being applied must ensure that the
* architecture supports the required operation for the target addresses. The
@@ -186,6 +191,7 @@ static inline int set_memory_encrypted(unsigned long addr, int numpages)
static inline int set_memory_decrypted(unsigned long addr, int numpages)
{
+ memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT);
return 0;
}
#endif /* CONFIG_ARCH_HAS_MEM_ENCRYPT */
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 25e9a86835fd..596c90e8f890 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -229,7 +229,8 @@ void *dma_direct_alloc(struct device *dev, size_t size,
if (force_dma_unencrypted(dev))
attrs |= __DMA_ATTR_ALLOC_CC_SHARED;
- if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) {
+ mark_mem_decrypt = attrs & __DMA_ATTR_ALLOC_CC_SHARED;
+ if (mark_mem_decrypt) {
/*
* Unencrypted/shared DMA requires a linear-mapped buffer
* address to look up the PFN and set architecture-required PFN
@@ -237,7 +238,6 @@ void *dma_direct_alloc(struct device *dev, size_t size,
* allocation.
*/
allow_highmem = false;
- mark_mem_decrypt = true;
}
size = PAGE_ALIGN(size);
@@ -341,7 +341,9 @@ void *dma_direct_alloc(struct device *dev, size_t size,
cpu_addr = page_address(page);
}
- memset(cpu_addr, 0, size);
+ /* Zero after remapping because the page may be in HighMem. */
+ if (!mark_mem_decrypt)
+ memset(cpu_addr, 0, size);
if (set_uncached) {
void *uncached_cpu_addr;
@@ -451,10 +453,13 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
unsigned long attrs = 0;
struct page *page;
void *cpu_addr;
+ bool mark_mem_decrypt;
if (force_dma_unencrypted(dev))
attrs |= __DMA_ATTR_ALLOC_CC_SHARED;
+ mark_mem_decrypt = attrs & __DMA_ATTR_ALLOC_CC_SHARED;
+
if ((attrs & __DMA_ATTR_ALLOC_CC_SHARED) && dma_direct_use_pool(dev, gfp))
return dma_direct_alloc_from_pool(dev, size, dma_handle,
&cpu_addr, gfp, attrs);
@@ -465,6 +470,7 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
return NULL;
cpu_addr = page_address(page);
+ mark_mem_decrypt = false;
goto setup_page;
}
@@ -473,11 +479,13 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
return NULL;
cpu_addr = page_address(page);
- if ((attrs & __DMA_ATTR_ALLOC_CC_SHARED) &&
- dma_set_decrypted(dev, cpu_addr, size))
- goto out_leak_pages;
setup_page:
- memset(cpu_addr, 0, size);
+ if (mark_mem_decrypt) {
+ if (dma_set_decrypted(dev, cpu_addr, size))
+ goto out_leak_pages;
+ } else {
+ memset(cpu_addr, 0, size);
+ }
*dma_handle = phys_to_dma_direct(dev, page_to_phys(page),
attrs & __DMA_ATTR_ALLOC_CC_SHARED);
return page;
diff --git a/mm/cc_shared.c b/mm/cc_shared.c
index 77ee65c4c35e..c9b4e3dfe06a 100644
--- a/mm/cc_shared.c
+++ b/mm/cc_shared.c
@@ -104,7 +104,6 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
struct cc_shared_layout layout;
struct page *page;
unsigned int order;
- bool zero = gfp & __GFP_ZERO;
int ret;
ret = cc_shared_calc_layout(requested, &layout);
@@ -116,7 +115,8 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
return -EINVAL;
/*
* State transitions require a linear-map address and may modify memory.
- * Allocate from low memory and defer requested zeroing until afterwards.
+ * Allocate from low memory and let the architecture place zeroing at the
+ * appropriate point in the transition.
* Shared allocations have no migration support, so they must not
* be allocated with __GFP_MOVABLE.
*/
@@ -143,9 +143,6 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
return ret;
}
- if (zero)
- memset(page_address(page), 0, layout.shared_size);
-
mem->page = page;
mem->shared_size = layout.shared_size;
return 0;
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 03/13] irqchip/gic-v3-its: Resolve the default NUMA node explicitly
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 01/13] mm: Add an allocator for CoCo shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 02/13] mm: Zero memory during shared memory transitions Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 04/13] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
` (10 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
alloc_pages_node() resolves NUMA_NO_NODE to numa_mem_id() internally. In
preparation for switching ITS allocations to the CoCo shared memory
allocator, resolve the default node explicitly in
its_alloc_pages_node().
The CoCo shared memory allocator interprets NUMA_NO_NODE as a request to
apply the current task's memory policy. Resolving it to numa_mem_id() in
the common ITS allocation helper preserves the existing
nearest-memory-node placement for all callers
This patch does not change the current allocation behavior.
Cc: Marc Zyngier <maz@kernel.org>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Radu Rendec <radu@rendec.net>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/irqchip/irq-gic-v3-its.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index c952c069f937..ada5c30af41e 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -32,6 +32,7 @@
#include <linux/set_memory.h>
#include <linux/slab.h>
#include <linux/syscore_ops.h>
+#include <linux/topology.h>
#include <linux/irqchip.h>
#include <linux/irqchip/arm-gic-v3.h>
@@ -218,6 +219,9 @@ static struct page *its_alloc_pages_node(int node, gfp_t gfp,
struct page *page;
int ret = 0;
+ if (node == NUMA_NO_NODE)
+ node = numa_mem_id();
+
page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
if (!page)
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 04/13] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (2 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 03/13] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 05/13] dma-contiguous: Derive shared alignment from DMA attributes Aneesh Kumar K.V (Arm)
` (9 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
ITS command queues and tables are shared with the host in a
confidential-computing guest. The ITS allocator currently assumes that
the guest PAGE_SIZE is sufficient for both allocation alignment and
private/shared state changes.
Route ITS page allocations through alloc_cc_shared_pages_node(). Keep
the tracked table size based on the ITS-requested order. On release,
reconstruct the common allocation metadata from the original ITS order
and the architecture granule size so the same transitioned range and
buddy order are used.
Cc: Marc Zyngier <maz@kernel.org>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Radu Rendec <radu@rendec.net>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/irqchip/irq-gic-v3-its.c | 40 ++++++++++++--------------------
1 file changed, 15 insertions(+), 25 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index ada5c30af41e..a67e98cecde5 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -8,6 +8,7 @@
#include <linux/acpi_iort.h>
#include <linux/bitfield.h>
#include <linux/bitmap.h>
+#include <linux/cc_shared.h>
#include <linux/cpu.h>
#include <linux/crash_dump.h>
#include <linux/delay.h>
@@ -19,7 +20,6 @@
#include <linux/irqdomain.h>
#include <linux/list.h>
#include <linux/log2.h>
-#include <linux/mem_encrypt.h>
#include <linux/memblock.h>
#include <linux/mm.h>
#include <linux/msi.h>
@@ -29,7 +29,6 @@
#include <linux/of_pci.h>
#include <linux/of_platform.h>
#include <linux/percpu.h>
-#include <linux/set_memory.h>
#include <linux/slab.h>
#include <linux/syscore_ops.h>
#include <linux/topology.h>
@@ -216,28 +215,16 @@ static gfp_t gfp_flags_quirk;
static struct page *its_alloc_pages_node(int node, gfp_t gfp,
unsigned int order)
{
- struct page *page;
- int ret = 0;
+ struct cc_shared_pages mem;
if (node == NUMA_NO_NODE)
node = numa_mem_id();
- page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
-
- if (!page)
- return NULL;
-
- ret = set_memory_decrypted((unsigned long)page_address(page),
- 1 << order);
- /*
- * If set_memory_decrypted() fails then we don't know what state the
- * page is in, so we can't free it. Instead we leak it.
- * set_memory_decrypted() will already have WARNed.
- */
- if (ret)
+ if (alloc_cc_shared_pages_node(node, gfp | gfp_flags_quirk,
+ PAGE_SIZE << order, &mem))
return NULL;
- return page;
+ return mem.page;
}
static struct page *its_alloc_pages(gfp_t gfp, unsigned int order)
@@ -247,13 +234,15 @@ static struct page *its_alloc_pages(gfp_t gfp, unsigned int order)
static void its_free_pages(void *addr, unsigned int order)
{
- /*
- * If the memory cannot be encrypted again then we must leak the pages.
- * set_memory_encrypted() will already have WARNed.
- */
- if (set_memory_encrypted((unsigned long)addr, 1 << order))
+ struct cc_shared_layout layout;
+ struct cc_shared_pages mem;
+
+ if (WARN_ON(cc_shared_calc_layout(PAGE_SIZE << order, &layout)))
return;
- free_pages((unsigned long)addr, order);
+
+ mem.page = virt_to_page(addr);
+ mem.shared_size = layout.shared_size;
+ free_cc_shared_pages(&mem);
}
static struct gen_pool *itt_pool;
@@ -278,7 +267,8 @@ static void *itt_alloc_pool(int node, int size)
if (!page)
break;
- gen_pool_add(itt_pool, (unsigned long)page_address(page), PAGE_SIZE, node);
+ gen_pool_add(itt_pool, (unsigned long)page_address(page),
+ cc_shared_granule_size(), node);
} while (!addr);
return (void *)addr;
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 05/13] dma-contiguous: Derive shared alignment from DMA attributes
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (3 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 04/13] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 06/13] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
` (8 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
Confidential-computing shared DMA allocations require CMA backing memory
to be aligned to the architecture shared-memory granule. Passing explicit
alignment orders through the DMA allocation stack exposes this derived
constraint to callers and requires separate preferred and mandatory values.
The DMA core already records whether an allocation needs shared backing in
__DMA_ATTR_ALLOC_CC_SHARED. Pass allocation attributes to the DMA-contiguous
entry points and derive the mandatory alignment there before calling CMA.
Preserve the existing alignment clamping for ordinary allocations, while
rejecting shared allocations whose granule exceeds CONFIG_CMA_ALIGNMENT so
callers can fall back to the page allocator.
Cc: Russell King <linux@armlinux.org.uk>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: "Joerg Roedel (AMD)" <joro@8bytes.org>
Cc: Will Deacon <will@kernel.org>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
arch/arm/mm/dma-mapping.c | 5 +++--
drivers/iommu/dma-iommu.c | 2 +-
include/linux/dma-map-ops.h | 9 ++++----
kernel/dma/contiguous.c | 41 +++++++++++++++++++++++++++----------
kernel/dma/direct.c | 2 +-
kernel/dma/ops_helpers.c | 2 +-
kernel/dma/pool.c | 2 +-
kernel/kexec_file.c | 3 ++-
8 files changed, 44 insertions(+), 22 deletions(-)
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index fb2eb0ed7bb8..22888880b832 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -398,7 +398,8 @@ static void *__alloc_from_contiguous(struct device *dev, size_t size,
struct page *page;
void *ptr = NULL;
- page = dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN);
+ page = dma_alloc_from_contiguous(dev, count, order, 0,
+ gfp & __GFP_NOWARN);
if (!page)
return NULL;
@@ -866,7 +867,7 @@ static struct page **__iommu_alloc_buffer(struct device *dev, size_t size,
unsigned long order = get_order(size);
struct page *page;
- page = dma_alloc_from_contiguous(dev, count, order,
+ page = dma_alloc_from_contiguous(dev, count, order, 0,
gfp & __GFP_NOWARN);
if (!page)
goto error;
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 78742aeba932..2d2781910740 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -1646,7 +1646,7 @@ static void *iommu_dma_alloc_pages(struct device *dev, size_t size,
struct page *page = NULL;
void *cpu_addr;
- page = dma_alloc_contiguous(dev, alloc_size, gfp);
+ page = dma_alloc_contiguous(dev, alloc_size, gfp, 0);
if (!page) {
cpu_addr = alloc_pages_exact_nid(node, alloc_size, gfp);
if (!cpu_addr)
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index afaf2c12189b..7a83942a10f3 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -99,10 +99,11 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
phys_addr_t limit, struct cma **res_cma, bool fixed);
struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
- unsigned int order, bool no_warn);
+ unsigned int order, unsigned long attrs, bool no_warn);
bool dma_release_from_contiguous(struct device *dev, struct page *pages,
int count);
-struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp);
+struct page *dma_alloc_contiguous(struct device *dev, size_t size,
+ gfp_t gfp, unsigned long attrs);
void dma_free_contiguous(struct device *dev, struct page *page, size_t size);
void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size);
@@ -125,7 +126,7 @@ static inline int dma_contiguous_reserve_area(phys_addr_t size,
return -ENOSYS;
}
static inline struct page *dma_alloc_from_contiguous(struct device *dev,
- size_t count, unsigned int order, bool no_warn)
+ size_t count, unsigned int order, unsigned long attrs, bool no_warn)
{
return NULL;
}
@@ -136,7 +137,7 @@ static inline bool dma_release_from_contiguous(struct device *dev,
}
/* Use fallback alloc() and free() when CONFIG_DMA_CMA=n */
static inline struct page *dma_alloc_contiguous(struct device *dev, size_t size,
- gfp_t gfp)
+ gfp_t gfp, unsigned long attrs)
{
return NULL;
}
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index 90937dc7c627..e593ecbc64f4 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -39,6 +39,7 @@
#include <asm/page.h>
+#include <linux/cc_shared.h>
#include <linux/memblock.h>
#include <linux/err.h>
#include <linux/sizes.h>
@@ -357,19 +358,29 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
* dma_alloc_from_contiguous() - allocate pages from contiguous area
* @dev: Pointer to device for which the allocation is performed.
* @count: Requested number of pages.
- * @align: Requested alignment of pages (in PAGE_SIZE order).
+ * @align: Preferred alignment of pages (in PAGE_SIZE order).
+ * @attrs: DMA allocation attributes.
* @no_warn: Avoid printing message about failed allocation.
*
* This function allocates memory buffer for specified device. It uses
* device specific contiguous memory area if available or the default
* global one. Requires architecture specific dev_get_cma_area() helper
* function.
+ *
+ * The preferred alignment is capped at CONFIG_CMA_ALIGNMENT. The internal
+ * shared-allocation attribute requires at least the architecture shared
+ * granule alignment and fails if that exceeds the CMA alignment limit.
*/
struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
- unsigned int align, bool no_warn)
+ unsigned int align, unsigned long attrs, bool no_warn)
{
- if (align > CONFIG_CMA_ALIGNMENT)
- align = CONFIG_CMA_ALIGNMENT;
+ unsigned int required_align = 0;
+
+ if (attrs & __DMA_ATTR_ALLOC_CC_SHARED)
+ required_align = get_order(cc_shared_granule_size());
+ if (required_align > CONFIG_CMA_ALIGNMENT)
+ return NULL;
+ align = min(max(align, required_align), CONFIG_CMA_ALIGNMENT);
return cma_alloc(dev_get_cma_area(dev), count, align, no_warn);
}
@@ -390,10 +401,9 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
return cma_release(dev_get_cma_area(dev), pages, count);
}
-static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t gfp)
+static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t gfp,
+ unsigned int align)
{
- unsigned int align = min(get_order(size), CONFIG_CMA_ALIGNMENT);
-
return cma_alloc(cma, size >> PAGE_SHIFT, align, gfp & __GFP_NOWARN);
}
@@ -402,6 +412,7 @@ static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t gfp)
* @dev: Pointer to device for which the allocation is performed.
* @size: Requested allocation size.
* @gfp: Allocation flags.
+ * @attrs: DMA allocation attributes.
*
* tries to use device specific contiguous memory area if available, or it
* tries to use per-numa cma, if the allocation fails, it will fallback to
@@ -412,8 +423,11 @@ static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t gfp)
* there is no need to waste CMA pages for that kind; it also helps reduce
* fragmentations.
*/
-struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
+struct page *dma_alloc_contiguous(struct device *dev, size_t size,
+ gfp_t gfp, unsigned long attrs)
{
+ unsigned int required_align = 0;
+ unsigned int align = get_order(size);
#ifdef CONFIG_DMA_NUMA_CMA
int nid = dev_to_node(dev);
#endif
@@ -421,8 +435,13 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
/* CMA can be used only in the context which permits sleeping */
if (!gfpflags_allow_blocking(gfp))
return NULL;
+ if (attrs & __DMA_ATTR_ALLOC_CC_SHARED)
+ required_align = get_order(cc_shared_granule_size());
+ if (required_align > CONFIG_CMA_ALIGNMENT)
+ return NULL;
+ align = min(max(align, required_align), CONFIG_CMA_ALIGNMENT);
if (dev->cma_area)
- return cma_alloc_aligned(dev->cma_area, size, gfp);
+ return cma_alloc_aligned(dev->cma_area, size, gfp, align);
if (size <= PAGE_SIZE)
return NULL;
@@ -431,7 +450,7 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
struct cma *cma = dma_contiguous_numa_area[nid];
struct page *page;
if (cma) {
- page = cma_alloc_aligned(cma, size, gfp);
+ page = cma_alloc_aligned(cma, size, gfp, align);
if (page)
return page;
}
@@ -440,7 +459,7 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
if (!dma_contiguous_default_area)
return NULL;
- return cma_alloc_aligned(dma_contiguous_default_area, size, gfp);
+ return cma_alloc_aligned(dma_contiguous_default_area, size, gfp, align);
}
/**
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 596c90e8f890..7e71f96b6ffd 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -125,7 +125,7 @@ static struct page *__dma_direct_alloc_pages(struct device *dev, size_t size,
WARN_ON_ONCE(!PAGE_ALIGNED(size));
gfp |= dma_direct_optimal_gfp_mask(dev, &phys_limit);
- page = dma_alloc_contiguous(dev, size, gfp);
+ page = dma_alloc_contiguous(dev, size, gfp, 0);
if (page) {
if (dma_coherent_ok(dev, page_to_phys(page), size) &&
(allow_highmem || !PageHighMem(page)))
diff --git a/kernel/dma/ops_helpers.c b/kernel/dma/ops_helpers.c
index 5cdf3f60cf47..11453e5ee0d3 100644
--- a/kernel/dma/ops_helpers.c
+++ b/kernel/dma/ops_helpers.c
@@ -66,7 +66,7 @@ struct page *dma_common_alloc_pages(struct device *dev, size_t size,
struct page *page;
phys_addr_t phys;
- page = dma_alloc_contiguous(dev, size, gfp);
+ page = dma_alloc_contiguous(dev, size, gfp, 0);
if (!page) {
void *va = alloc_pages_exact_nid(dev_to_node(dev), size, gfp);
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index 00f422a1e896..70b7f64b17ab 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -99,7 +99,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_size,
pool_size = 1 << (PAGE_SHIFT + order);
if (cma_in_zone(gfp))
page = dma_alloc_from_contiguous(NULL, 1 << order,
- order, false);
+ order, 0, false);
if (!page)
page = alloc_pages(gfp | __GFP_NOWARN, order);
} while (!page && order-- > 0);
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index b668ce8d68ee..5bcba818f046 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -702,7 +702,8 @@ static int kexec_alloc_contig(struct kexec_buf *kbuf)
if (kbuf->image->type == KEXEC_TYPE_CRASH)
return -EPERM;
- p = dma_alloc_from_contiguous(NULL, nr_pages, get_order(kbuf->buf_align), true);
+ p = dma_alloc_from_contiguous(NULL, nr_pages,
+ get_order(kbuf->buf_align), 0, true);
if (!p)
return -ENOMEM;
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 06/13] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (4 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 05/13] dma-contiguous: Derive shared alignment from DMA attributes Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 07/13] dma-direct: Align CoCo shared DMA allocations to the shared granule size Aneesh Kumar K.V (Arm)
` (7 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
Atomic DMA allocations cannot perform a potentially sleeping
private/shared state transition at allocation time. The atomic DMA pools
avoid this by transitioning their backing allocations in a sleepable
context and suballocating memory that is already shared.
Architectures may require these transitions to use a shared granule size
larger than PAGE_SIZE. The existing fallback loop can reduce the backing
allocation below the order required by that size, producing a range that
cannot be safely transitioned.
For pools marked cc_shared, round the requested pool size up to a multiple
of the shared granule size and prevent allocation fallback below the order
required by that size. Pass __DMA_ATTR_ALLOC_CC_SHARED to
dma_alloc_from_contiguous() so it enforces the mandatory alignment. If CMA
cannot satisfy it, fall back to the buddy allocator. Non-shared pools pass
zero attributes and retain the existing CMA alignment policy.
Use the cc_make_shared() and cc_make_private() helpers to transition the
backing allocation and validate its address and size.
Individual atomic allocations may remain smaller than the shared granule
size because the backing allocation remains owned by the pool and in the
shared state.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/pool.c | 27 +++++++++++++++++++++------
1 file changed, 21 insertions(+), 6 deletions(-)
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index 70b7f64b17ab..da487c3f9dc6 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -4,12 +4,12 @@
* Copyright (C) 2020 Google LLC
*/
#include <linux/cma.h>
+#include <linux/cc_shared.h>
#include <linux/debugfs.h>
#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
#include <linux/init.h>
#include <linux/genalloc.h>
-#include <linux/set_memory.h>
#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/cc_platform.h>
@@ -85,6 +85,9 @@ static bool cma_in_zone(gfp_t gfp)
static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_size,
gfp_t gfp)
{
+ struct cc_shared_layout layout;
+ unsigned long attrs = 0;
+ unsigned int min_order = 0;
unsigned int order;
struct page *page = NULL;
bool leak_pages = false;
@@ -92,6 +95,19 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_size,
int ret = -ENOMEM;
pgprot_t prot __maybe_unused;
+ if (dma_pool->cc_shared) {
+ int err;
+
+ err = cc_shared_calc_layout(pool_size, &layout);
+ if (err)
+ return err;
+ pool_size = layout.shared_size;
+ min_order = get_order(layout.alignment);
+ if (min_order > MAX_PAGE_ORDER)
+ return -E2BIG;
+ attrs = __DMA_ATTR_ALLOC_CC_SHARED;
+ }
+
/* Cannot allocate larger than MAX_PAGE_ORDER */
order = min(get_order(pool_size), MAX_PAGE_ORDER);
@@ -99,10 +115,10 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_size,
pool_size = 1 << (PAGE_SHIFT + order);
if (cma_in_zone(gfp))
page = dma_alloc_from_contiguous(NULL, 1 << order,
- order, 0, false);
+ order, attrs, false);
if (!page)
page = alloc_pages(gfp | __GFP_NOWARN, order);
- } while (!page && order-- > 0);
+ } while (!page && order-- > min_order);
if (!page)
goto out;
@@ -126,8 +142,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_size,
* shrink so no re-encryption occurs in dma_direct_free().
*/
if (dma_pool->cc_shared) {
- ret = set_memory_decrypted((unsigned long)page_to_virt(page),
- 1 << order);
+ ret = cc_make_shared(page_to_virt(page), pool_size);
if (ret) {
leak_pages = true;
goto remove_mapping;
@@ -144,7 +159,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_size,
encrypt_mapping:
if (dma_pool->cc_shared &&
- set_memory_encrypted((unsigned long)page_to_virt(page), 1 << order))
+ cc_make_private(page_to_virt(page), pool_size))
leak_pages = true;
remove_mapping:
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 07/13] dma-direct: Align CoCo shared DMA allocations to the shared granule size
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (5 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 06/13] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 08/13] swiotlb: Align shared IO TLB pools " Aneesh Kumar K.V (Arm)
` (6 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
Use the common CoCo shared-memory geometry helpers for allocations backed
directly by CMA or the page allocator. Round the backing allocation to a
whole shared granule and pass the resolved shared-allocation attribute
through the DMA contiguous allocator so it can enforce the mandatory
alignment. Transition the complete range through the common shared/private
helpers.
Recompute the layout before freeing ordinary direct allocations so the
transition back to private memory and dma_free_contiguous() cover exactly
the range acquired by the allocation path. If restoring private state
fails, retain the existing fail-safe behavior and leak the pages rather
than returning potentially shared memory to the allocator.
This also applies the same rules to dma_direct_alloc_pages(), covering
callers which require a struct page result rather than a CPU virtual
address.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/direct.c | 51 ++++++++++++++++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 12 deletions(-)
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 7e71f96b6ffd..675ed698a3ad 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -11,10 +11,10 @@
#include <linux/scatterlist.h>
#include <linux/pfn.h>
#include <linux/vmalloc.h>
-#include <linux/set_memory.h>
#include <linux/slab.h>
#include <linux/pci-p2pdma.h>
#include <linux/cc_platform.h>
+#include <linux/cc_shared.h>
#include "direct.h"
@@ -85,7 +85,7 @@ static int dma_set_decrypted(struct device *dev, void *vaddr, size_t size)
{
int ret;
- ret = set_memory_decrypted((unsigned long)vaddr, PFN_UP(size));
+ ret = cc_make_shared(vaddr, size);
if (ret)
pr_warn_ratelimited("leaking DMA memory that can't be decrypted\n");
return ret;
@@ -95,7 +95,7 @@ static int dma_set_encrypted(struct device *dev, void *vaddr, size_t size)
{
int ret;
- ret = set_memory_encrypted((unsigned long)vaddr, PFN_UP(size));
+ ret = cc_make_private(vaddr, size);
if (ret)
pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n");
return ret;
@@ -115,7 +115,7 @@ static struct page *dma_direct_alloc_swiotlb(struct device *dev, size_t size,
}
static struct page *__dma_direct_alloc_pages(struct device *dev, size_t size,
- gfp_t gfp, bool allow_highmem)
+ gfp_t gfp, bool allow_highmem, unsigned long attrs)
{
int node = dev_to_node(dev);
struct page *page;
@@ -125,7 +125,7 @@ static struct page *__dma_direct_alloc_pages(struct device *dev, size_t size,
WARN_ON_ONCE(!PAGE_ALIGNED(size));
gfp |= dma_direct_optimal_gfp_mask(dev, &phys_limit);
- page = dma_alloc_contiguous(dev, size, gfp, 0);
+ page = dma_alloc_contiguous(dev, size, gfp, attrs);
if (page) {
if (dma_coherent_ok(dev, page_to_phys(page), size) &&
(allow_highmem || !PageHighMem(page)))
@@ -202,7 +202,7 @@ static void *dma_direct_alloc_no_mapping(struct device *dev, size_t size,
{
struct page *page;
- page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true);
+ page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true, 0);
if (!page)
return NULL;
@@ -223,6 +223,7 @@ void *dma_direct_alloc(struct device *dev, size_t size,
bool remap = false, set_uncached = false;
bool mark_mem_decrypt = false;
bool allow_highmem = true;
+ struct cc_shared_layout layout;
struct page *page;
void *cpu_addr;
@@ -302,8 +303,15 @@ void *dma_direct_alloc(struct device *dev, size_t size,
return NULL;
}
+ if (mark_mem_decrypt) {
+ if (cc_shared_calc_layout(size, &layout))
+ return NULL;
+ size = layout.shared_size;
+ }
+
/* we always manually zero the memory once we are done */
- page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, allow_highmem);
+ page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO,
+ allow_highmem, attrs);
if (!page)
return NULL;
@@ -322,7 +330,7 @@ void *dma_direct_alloc(struct device *dev, size_t size,
void *lm_addr;
lm_addr = page_address(page);
- if (set_memory_decrypted((unsigned long)lm_addr, PFN_UP(size)))
+ if (dma_set_decrypted(dev, lm_addr, size))
goto out_leak_pages;
}
@@ -381,6 +389,7 @@ void dma_direct_free(struct device *dev, size_t size,
phys_addr_t phys;
bool mark_mem_encrypted = false;
struct io_tlb_pool *swiotlb_pool;
+ struct cc_shared_layout layout;
unsigned int page_order = get_order(size);
/*
@@ -424,6 +433,12 @@ void dma_direct_free(struct device *dev, size_t size,
/* Swiotlb doesn't need a page attribute update on free */
mark_mem_encrypted = false;
+ if (mark_mem_encrypted) {
+ if (WARN_ON_ONCE(cc_shared_calc_layout(size, &layout)))
+ return;
+ size = layout.shared_size;
+ }
+
if (is_vmalloc_addr(cpu_addr)) {
vunmap(cpu_addr);
} else {
@@ -435,10 +450,8 @@ void dma_direct_free(struct device *dev, size_t size,
void *lm_addr;
lm_addr = phys_to_virt(phys);
- if (set_memory_encrypted((unsigned long)lm_addr, PFN_UP(size))) {
- pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n");
+ if (dma_set_encrypted(dev, lm_addr, size))
return;
- }
}
if (swiotlb_pool)
@@ -451,6 +464,7 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp)
{
unsigned long attrs = 0;
+ struct cc_shared_layout layout;
struct page *page;
void *cpu_addr;
bool mark_mem_decrypt;
@@ -474,7 +488,13 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
goto setup_page;
}
- page = __dma_direct_alloc_pages(dev, size, gfp, false);
+ if (mark_mem_decrypt) {
+ if (cc_shared_calc_layout(size, &layout))
+ return NULL;
+ size = layout.shared_size;
+ }
+
+ page = __dma_direct_alloc_pages(dev, size, gfp, false, attrs);
if (!page)
return NULL;
@@ -500,6 +520,7 @@ void dma_direct_free_pages(struct device *dev, size_t size,
phys_addr_t phys;
void *vaddr = page_address(page);
struct io_tlb_pool *swiotlb_pool;
+ struct cc_shared_layout layout;
/*
* if the device had requested for an unencrypted buffer,
* convert it to encrypted on free
@@ -516,6 +537,12 @@ void dma_direct_free_pages(struct device *dev, size_t size,
if (swiotlb_pool)
mark_mem_encrypted = false;
+ if (mark_mem_encrypted) {
+ if (WARN_ON_ONCE(cc_shared_calc_layout(size, &layout)))
+ return;
+ size = layout.shared_size;
+ }
+
if (mark_mem_encrypted && dma_set_encrypted(dev, vaddr, size))
return;
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 08/13] swiotlb: Align shared IO TLB pools to the shared granule size
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (6 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 07/13] dma-direct: Align CoCo shared DMA allocations to the shared granule size Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 09/13] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Aneesh Kumar K.V (Arm)
` (5 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
Align and size the early memblock pool to the common CoCo shared
granule. Use the same rounded extent when the pool is transitioned and
when it is released. This deliberately leaves the slot count unchanged:
any rounded tail belongs to the pool allocation but is not advertised as
allocatable SWIOTLB space.
Replace the default and dynamic pool set_memory calls with the helpers
so alignment validation and architecture dispatch are kept in one place.
As before, pages are intentionally leaked if their private state cannot
be restored safely.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/swiotlb.c | 53 +++++++++++++++++++++++++-------------------
1 file changed, 30 insertions(+), 23 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 2c57b49ac7c0..2bb7d83ee479 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -22,6 +22,7 @@
#include <linux/cache.h>
#include <linux/cc_platform.h>
+#include <linux/cc_shared.h>
#include <linux/ctype.h>
#include <linux/debugfs.h>
#include <linux/dma-direct.h>
@@ -367,13 +368,13 @@ void __init swiotlb_update_mem_attributes(void)
if (!mem->nslabs || mem->late_alloc)
return;
- bytes = PAGE_ALIGN(mem->nslabs << IO_TLB_SHIFT);
+
+ bytes = ALIGN(mem->nslabs << IO_TLB_SHIFT, cc_shared_granule_size());
if (io_tlb_default_mem.cc_shared) {
int ret;
- ret = set_memory_decrypted((unsigned long)mem->vaddr,
- bytes >> PAGE_SHIFT);
+ ret = cc_make_shared(mem->vaddr, bytes);
if (ret) {
pr_warn("Failed to decrypt default memory pool, disabling it\n");
swiotlb_mark_pool_used(mem);
@@ -434,8 +435,8 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs,
unsigned int flags,
int (*remap)(void *tlb, unsigned long nslabs))
{
- size_t bytes = PAGE_ALIGN(nslabs << IO_TLB_SHIFT);
void *tlb;
+ size_t bytes = ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size());
/*
* By default allocate the bounce buffer memory from low memory, but
@@ -443,9 +444,9 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs,
* memory encryption.
*/
if (flags & SWIOTLB_ANY)
- tlb = memblock_alloc(bytes, PAGE_SIZE);
+ tlb = memblock_alloc(bytes, cc_shared_granule_size());
else
- tlb = memblock_alloc_low(bytes, PAGE_SIZE);
+ tlb = memblock_alloc_low(bytes, cc_shared_granule_size());
if (!tlb) {
pr_warn("%s: Failed to allocate %zu bytes tlb structure\n",
@@ -454,7 +455,7 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs,
}
if (remap && remap(tlb, nslabs) < 0) {
- memblock_free(tlb, PAGE_ALIGN(bytes));
+ memblock_free(tlb, bytes);
pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes);
return NULL;
}
@@ -576,7 +577,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
swiotlb_adjust_nareas(num_possible_cpus());
retry:
- order = get_order(nslabs << IO_TLB_SHIFT);
+ order = get_order(ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()));
nslabs = SLABS_PER_PAGE << order;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
@@ -585,6 +586,8 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
if (vstart)
break;
order--;
+ if (order < get_order(cc_shared_granule_size()))
+ break;
nslabs = SLABS_PER_PAGE << order;
retried = true;
}
@@ -598,8 +601,13 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
free_pages((unsigned long)vstart, order);
nslabs = ALIGN(nslabs >> 1, IO_TLB_SEGSIZE);
- if (nslabs < IO_TLB_MIN_SLABS)
+ if (nslabs <= IO_TLB_MIN_SLABS)
return rc;
+
+ /* Avoid infinite loop */
+ if (WARN_ON(nslabs < (cc_shared_granule_size() >> IO_TLB_SHIFT)))
+ return rc;
+
retried = true;
goto retry;
}
@@ -624,8 +632,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
goto error_slots;
if (io_tlb_default_mem.cc_shared) {
- rc = set_memory_decrypted((unsigned long)vstart,
- (nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT);
+ rc = cc_make_shared(vstart, nslabs << IO_TLB_SHIFT);
if (rc) {
leak_pages = true;
goto error_decrypt;
@@ -665,11 +672,11 @@ void __init swiotlb_exit(void)
pr_info("tearing down default memory pool\n");
tbl_vaddr = (unsigned long)phys_to_virt(mem->start);
- tbl_size = PAGE_ALIGN(mem->end - mem->start);
+ tbl_size = ALIGN(mem->end - mem->start, cc_shared_granule_size());
slots_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), mem->nslabs));
if (io_tlb_default_mem.cc_shared) {
- if (set_memory_encrypted(tbl_vaddr, tbl_size >> PAGE_SHIFT))
+ if (cc_make_private((void *)tbl_vaddr, tbl_size))
leak_pages = true;
}
@@ -709,12 +716,15 @@ void __init swiotlb_exit(void)
static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes,
u64 phys_limit, unsigned long attrs)
{
- unsigned int order = get_order(bytes);
bool cc_shared = attrs & __DMA_ATTR_ALLOC_CC_SHARED;
+ unsigned int order;
struct page *page;
phys_addr_t paddr;
void *vaddr;
+ if (cc_shared)
+ bytes = ALIGN(bytes, cc_shared_granule_size());
+ order = get_order(bytes);
page = alloc_pages(gfp, order);
if (!page)
return NULL;
@@ -726,15 +736,10 @@ static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes,
}
vaddr = phys_to_virt(paddr);
- if (cc_shared && set_memory_decrypted((unsigned long)vaddr, PFN_UP(bytes)))
- goto error;
+ if (cc_shared && cc_make_shared(vaddr, bytes))
+ /* leak pages on error */
+ return NULL;
return page;
-
-error:
- /* Intentional leak if pages cannot be encrypted again. */
- if (cc_shared && !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes)))
- __free_pages(page, order);
- return NULL;
}
/**
@@ -805,9 +810,11 @@ static void swiotlb_free_tlb(void *vaddr, size_t bytes, bool cc_shared)
dma_free_from_pool(NULL, vaddr, bytes))
return;
+ if (cc_shared)
+ bytes = ALIGN(bytes, cc_shared_granule_size());
/* Intentional leak if pages cannot be encrypted again. */
if (!cc_shared ||
- !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes)))
+ !cc_make_private(vaddr, bytes))
__free_pages(virt_to_page(vaddr), get_order(bytes));
}
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 09/13] swiotlb: Reject misaligned restricted DMA pools for CoCo guests
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (7 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 08/13] swiotlb: Align shared IO TLB pools " Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 10/13] dma-buf: system_heap: Limit scatterlist entries to the buffer size Aneesh Kumar K.V (Arm)
` (4 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
A restricted DMA pool is a firmware-described ownership boundary. Unlike
a pool allocated by SWIOTLB itself, its base cannot be moved and its
size cannot be rounded without claiming memory outside the reserved
region.
Confidential-computing guests may require private/shared state changes
to cover units larger than PAGE_SIZE. Passing a misaligned restricted
region to set_memory_decrypted() can therefore fail.
Validate both the physical base and the complete reserved size with the
common CoCo shared-range helper before allocating SWIOTLB metadata or
performing a state transition. Perform the transition through
cc_make_shared().
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/swiotlb.c | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 2bb7d83ee479..24366038dfd3 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -39,7 +39,6 @@
#include <linux/pfn.h>
#include <linux/rculist.h>
#include <linux/scatterlist.h>
-#include <linux/set_memory.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/swiotlb.h>
@@ -2007,6 +2006,14 @@ static int rmem_swiotlb_device_init(struct reserved_mem *rmem,
return -EINVAL;
}
+ if (cc_platform_has(CC_ATTR_MEM_ENCRYPT) &&
+ !cc_shared_range_valid(rmem->base, rmem->size)) {
+ dev_err(dev,
+ "Restricted DMA pool must be aligned to %#zx bytes for memory encryption\n",
+ cc_shared_granule_size());
+ return -EINVAL;
+ }
+
/*
* Since multiple devices can share the same pool, the private data,
* io_tlb_mem struct, will be initialized by the first device attached
@@ -2040,8 +2047,8 @@ static int rmem_swiotlb_device_init(struct reserved_mem *rmem,
int ret;
mem->cc_shared = true;
- ret = set_memory_decrypted((unsigned long)phys_to_virt(rmem->base),
- rmem->size >> PAGE_SHIFT);
+ ret = cc_make_shared(phys_to_virt(rmem->base),
+ rmem->size);
if (ret) {
dev_err(dev, "Failed to decrypt restricted DMA pool\n");
kfree(pool->areas);
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 10/13] dma-buf: system_heap: Limit scatterlist entries to the buffer size
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (8 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 09/13] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 11/13] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
` (3 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
The system heap currently allocates each backing page no larger than the
remaining dma-buf length. It can therefore use the complete
compound-page size for every scatterlist entry while keeping the total
length equal to the buffer size.
Shared backing allocations may need to be rounded up to an architecture
shared granule size. A backing allocation can then be larger than the
remaining buffer length. Describing the complete allocation in the
scatterlist would incorrectly expose the rounded tail to scatterlist
consumers as part of the dma-buf.
Track the remaining buffer length while constructing the scatterlist and
limit each entry to the smaller of the compound-page size and the
remaining length. The complete backing allocation remains owned by the
heap and is still released normally.
This does not change behavior with the current allocation policy, but
prepares the heap for shared-granule-sized backing allocations.
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Cc: Brian Starkey <Brian.Starkey@arm.com>
Cc: John Stultz <jstultz@google.com>
Cc: "T.J. Mercier" <tjmercier@google.com>
Cc: "Christian König" <christian.koenig@amd.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/dma-buf/heaps/system_heap.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
index c8959eadc71d..b5b8cdf65f23 100644
--- a/drivers/dma-buf/heaps/system_heap.c
+++ b/drivers/dma-buf/heaps/system_heap.c
@@ -406,6 +406,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
struct system_heap_buffer *buffer;
DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
unsigned long size_remaining = len;
+ unsigned long sg_remaining = len;
unsigned int max_order = orders[0];
struct system_heap_priv *priv = dma_heap_get_drvdata(heap);
bool cc_shared = priv->cc_shared;
@@ -454,7 +455,11 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
sg = table->sgl;
list_for_each_entry_safe(page, tmp_page, &pages, lru) {
- sg_set_page(sg, page, page_size(page), 0);
+ unsigned long sg_len;
+
+ sg_len = min_t(unsigned long, page_size(page), sg_remaining);
+ sg_set_page(sg, page, sg_len, 0);
+ sg_remaining -= sg_len;
sg = sg_next(sg);
list_del(&page->lru);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 11/13] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (9 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 10/13] dma-buf: system_heap: Limit scatterlist entries to the buffer size Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 12/13] swiotlb: Make rounded shared pool capacity allocatable Aneesh Kumar K.V (Arm)
` (2 subsequent siblings)
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
The system_cc_shared heap currently allocates its backing pages using the
normal system-heap order policy and changes each resulting compound page to
shared state. That is unsafe when an architecture requires state changes in
units larger than PAGE_SIZE: an order-0 tail is neither sufficiently aligned
nor large enough to transition independently.
Use the common CoCo shared-page allocator for every backing allocation of the
shared heap. Preserve the existing preferred-order search and its GFP policy:
each candidate order is passed to the common allocator as a byte request, and
that allocator rounds it up when the architecture shared granule is larger.
Add __GFP_COMP for shared allocations because an order-0 candidate can
be rounded into a high-order allocation. The system heap uses
compound_order() and page_size() for accounting and release, so the
returned allocation must retain compound-page semantics.
Calculate a rounded internal backing length but retain the original
length in dma_buf::size. The preceding scatterlist-length change ensures
that the rounded tail is not included in DMA mappings or other
operations. This permits a 4 KiB request on a 64 KiB shared-granule
system without exposing the extra 60 KiB to an importer.
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Cc: Brian Starkey <Brian.Starkey@arm.com>
Cc: John Stultz <jstultz@google.com>
Cc: "T.J. Mercier" <tjmercier@google.com>
Cc: "Christian König" <christian.koenig@amd.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/dma-buf/heaps/system_heap.c | 121 ++++++++++++----------------
1 file changed, 53 insertions(+), 68 deletions(-)
diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
index b5b8cdf65f23..5970b76416ae 100644
--- a/drivers/dma-buf/heaps/system_heap.c
+++ b/drivers/dma-buf/heaps/system_heap.c
@@ -11,14 +11,13 @@
*/
#include <linux/cc_platform.h>
+#include <linux/cc_shared.h>
#include <linux/dma-buf.h>
#include <linux/dma-mapping.h>
#include <linux/dma-heap.h>
#include <linux/err.h>
#include <linux/highmem.h>
-#include <linux/mem_encrypt.h>
#include <linux/mm.h>
-#include <linux/set_memory.h>
#include <linux/module.h>
#include <linux/pgtable.h>
#include <linux/scatterlist.h>
@@ -65,34 +64,6 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
static const unsigned int orders[] = {8, 4, 0};
#define NUM_ORDERS ARRAY_SIZE(orders)
-static int system_heap_set_page_decrypted(struct page *page)
-{
- unsigned long addr = (unsigned long)page_address(page);
- unsigned int nr_pages = 1 << compound_order(page);
- int ret;
-
- ret = set_memory_decrypted(addr, nr_pages);
- if (ret)
- pr_warn_ratelimited("dma-buf system heap: failed to decrypt page at %p\n",
- page_address(page));
-
- return ret;
-}
-
-static int system_heap_set_page_encrypted(struct page *page)
-{
- unsigned long addr = (unsigned long)page_address(page);
- unsigned int nr_pages = 1 << compound_order(page);
- int ret;
-
- ret = set_memory_encrypted(addr, nr_pages);
- if (ret)
- pr_warn_ratelimited("dma-buf system heap: failed to re-encrypt page at %p, leaking memory\n",
- page_address(page));
-
- return ret;
-}
-
static int dup_sg_table(struct sg_table *from, struct sg_table *to)
{
struct scatterlist *sg, *new_sg;
@@ -337,6 +308,20 @@ static void system_heap_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
iosys_map_clear(map);
}
+static void system_heap_free_page(struct page *page, bool cc_shared)
+{
+ struct cc_shared_pages mem;
+
+ if (!cc_shared) {
+ __free_pages(page, compound_order(page));
+ return;
+ }
+
+ mem.page = page;
+ mem.shared_size = page_size(page);
+ free_cc_shared_pages(&mem);
+}
+
static void system_heap_dma_buf_release(struct dma_buf *dmabuf)
{
struct system_heap_buffer *buffer = dmabuf->priv;
@@ -345,19 +330,8 @@ static void system_heap_dma_buf_release(struct dma_buf *dmabuf)
int i;
table = &buffer->sg_table;
- for_each_sgtable_sg(table, sg, i) {
- struct page *page = sg_page(sg);
-
- /*
- * Intentionally leak pages that cannot be re-encrypted
- * to prevent shared memory from being reused.
- */
- if (cc_shared_buffer(buffer) &&
- system_heap_set_page_encrypted(page))
- continue;
-
- __free_pages(page, compound_order(page));
- }
+ for_each_sgtable_sg(table, sg, i)
+ system_heap_free_page(sg_page(sg), cc_shared_buffer(buffer));
sg_free_table(table);
kfree(buffer);
}
@@ -375,22 +349,39 @@ static const struct dma_buf_ops system_heap_buf_ops = {
.release = system_heap_dma_buf_release,
};
+static struct page *system_heap_alloc_order(unsigned int order,
+ gfp_t flags, bool cc_shared)
+{
+ struct cc_shared_pages mem;
+
+ if (!cc_shared)
+ return alloc_pages(flags, order);
+
+ /* The shared granule can raise the actual allocation order. */
+ flags |= __GFP_COMP;
+ if (alloc_cc_shared_pages(flags, PAGE_SIZE << order, &mem))
+ return NULL;
+
+ return mem.page;
+}
+
static struct page *alloc_largest_available(unsigned long size,
- unsigned int max_order)
+ unsigned int max_order, bool cc_shared)
{
struct page *page;
- int i;
gfp_t flags;
+ int i;
for (i = 0; i < NUM_ORDERS; i++) {
if (size < (PAGE_SIZE << orders[i]))
continue;
if (max_order < orders[i])
continue;
+
flags = order_flags[i];
if (mem_accounting)
flags |= __GFP_ACCOUNT;
- page = alloc_pages(flags, orders[i]);
+ page = system_heap_alloc_order(orders[i], flags, cc_shared);
if (!page)
continue;
return page;
@@ -405,6 +396,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
{
struct system_heap_buffer *buffer;
DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct cc_shared_layout layout;
unsigned long size_remaining = len;
unsigned long sg_remaining = len;
unsigned int max_order = orders[0];
@@ -417,6 +409,16 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
struct page *page, *tmp_page;
int i, ret = -ENOMEM;
+ if (cc_shared) {
+ int err;
+
+ err = cc_shared_calc_layout(len, &layout);
+ if (err)
+ return ERR_PTR(err);
+
+ size_remaining = layout.shared_size;
+ }
+
buffer = kzalloc_obj(*buffer);
if (!buffer)
return ERR_PTR(-ENOMEM);
@@ -439,7 +441,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
goto free_buffer;
}
- page = alloc_largest_available(size_remaining, max_order);
+ page = alloc_largest_available(size_remaining, max_order,
+ cc_shared);
if (!page)
goto free_buffer;
@@ -464,14 +467,6 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
list_del(&page->lru);
}
- if (cc_shared_buffer(buffer)) {
- for_each_sgtable_sg(table, sg, i) {
- ret = system_heap_set_page_decrypted(sg_page(sg));
- if (ret)
- goto free_pages;
- }
- }
-
/* create the dmabuf */
exp_info.exp_name = dma_heap_get_name(heap);
exp_info.ops = &system_heap_buf_ops;
@@ -486,22 +481,12 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
return dmabuf;
free_pages:
- for_each_sgtable_sg(table, sg, i) {
- struct page *p = sg_page(sg);
-
- /*
- * Intentionally leak pages that cannot be re-encrypted
- * to prevent shared memory from being reused.
- */
- if (cc_shared_buffer(buffer) &&
- system_heap_set_page_encrypted(p))
- continue;
- __free_pages(p, compound_order(p));
- }
+ for_each_sgtable_sg(table, sg, i)
+ system_heap_free_page(sg_page(sg), cc_shared);
sg_free_table(table);
free_buffer:
list_for_each_entry_safe(page, tmp_page, &pages, lru)
- __free_pages(page, compound_order(page));
+ system_heap_free_page(page, cc_shared);
kfree(buffer);
return ERR_PTR(ret);
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 12/13] swiotlb: Make rounded shared pool capacity allocatable
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (10 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 11/13] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 13/13] mm: Assert CoCo shared allocations may sleep Aneesh Kumar K.V (Arm)
2026-11-16 10:09 ` [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Suzuki K Poulose
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
CoCo shared memory may need to be allocated and transitioned in units larger
than the requested object. Before this change, users handled the resulting
capacity as follows:
User Rounded capacity reused
dma-buf system heap no
DMA-direct no
regular GIC tables no
small GIC ITTs yes, through a gen_pool
early SWIOTLB pool no
late SWIOTLB pool yes
persistent dynamic SWIOTLB no
transient dynamic SWIOTLB no, one mapping only
atomic DMA pools yes, through a gen_pool
restricted SWIOTLB pool no additional padding
Improve the early and persistent dynamic SWIOTLB pools. They already own
and transition backing rounded to the shared granule size, and SWIOTLB
is itself a suballocator. Advertise the rounded extent as slots, size
the slot metadata to match. This makes the extra capacity available
without reserving more backing memory.
Keep transient dynamic pools unchanged. A transient pool belongs to one
DMA mapping and is destroyed when that mapping is unmapped, so its spare
backing cannot satisfy a later request without changing the lifetime
model.
Do not attempt the same optimization for dma-buf, DMA-direct or regular
GIC objects. Those allocations have independent caller-visible sizes and
lifetimes. Reusing their padding requires a shared-granule suballocator
with reference counting, per-object mappings and accounting.
Note:
For the current 64 KiB CCA shared granule size, SWIOTLB pool sizes are
already multiples of the 256 KiB IO_TLB segment size. Consequently, the
rounding does not change any runtime values on current CCA systems. It
instead makes the code express the intended invariant that pool metadata
describes the complete shared-granule-aligned backing allocation.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/swiotlb.c | 30 ++++++++++++++++++++++++------
1 file changed, 24 insertions(+), 6 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 24366038dfd3..a245c508fb3a 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -328,6 +328,14 @@ static inline unsigned long nr_slots(u64 val)
return DIV_ROUND_UP(val, IO_TLB_SIZE);
}
+static unsigned long swiotlb_align_nslabs(unsigned long nslabs)
+{
+ unsigned long granule_nslabs;
+
+ granule_nslabs = cc_shared_granule_size() >> IO_TLB_SHIFT;
+ return ALIGN(nslabs, granule_nslabs);
+}
+
static void swiotlb_mark_pool_used(struct io_tlb_pool *pool)
{
unsigned long i;
@@ -431,11 +439,12 @@ static void add_mem_pool(struct io_tlb_mem *mem, struct io_tlb_pool *pool)
}
static void __init *swiotlb_memblock_alloc(unsigned long nslabs,
- unsigned int flags,
+ unsigned long *alloc_nslabs, unsigned int flags,
int (*remap)(void *tlb, unsigned long nslabs))
{
+ unsigned long aligned_nslabs = swiotlb_align_nslabs(nslabs);
+ size_t bytes = aligned_nslabs << IO_TLB_SHIFT;
void *tlb;
- size_t bytes = ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size());
/*
* By default allocate the bounce buffer memory from low memory, but
@@ -453,12 +462,13 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs,
return NULL;
}
- if (remap && remap(tlb, nslabs) < 0) {
+ if (remap && remap(tlb, aligned_nslabs) < 0) {
memblock_free(tlb, bytes);
pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes);
return NULL;
}
+ *alloc_nslabs = aligned_nslabs;
return tlb;
}
@@ -471,6 +481,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
{
struct io_tlb_pool *mem = &io_tlb_default_mem.defpool;
unsigned long nslabs;
+ unsigned long alloc_nslabs;
unsigned int nareas;
size_t alloc_size;
void *tlb;
@@ -495,13 +506,14 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
swiotlb_adjust_nareas(num_possible_cpus());
nslabs = default_nslabs;
- nareas = limit_nareas(default_nareas, nslabs);
- while ((tlb = swiotlb_memblock_alloc(nslabs, flags, remap)) == NULL) {
+ while ((tlb = swiotlb_memblock_alloc(nslabs, &alloc_nslabs, flags,
+ remap)) == NULL) {
if (nslabs <= IO_TLB_MIN_SLABS)
return;
nslabs = ALIGN(nslabs >> 1, IO_TLB_SEGSIZE);
- nareas = limit_nareas(nareas, nslabs);
}
+ nslabs = alloc_nslabs;
+ nareas = limit_nareas(default_nareas, nslabs);
if (default_nslabs != nslabs) {
pr_info("SWIOTLB bounce buffer size adjusted %lu -> %lu slabs",
@@ -867,6 +879,12 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
tlb_size = nslabs << IO_TLB_SHIFT;
}
+ /* Transient pools are tied to one mapping and cannot reuse padding. */
+ if (mem->cc_shared && !dev) {
+ nslabs = swiotlb_align_nslabs(nslabs);
+ tlb_size = nslabs << IO_TLB_SHIFT;
+ }
+
slot_order = get_order(array_size(sizeof(*pool->slots), nslabs));
pool->slots = (struct io_tlb_slot *)
__get_free_pages(gfp, slot_order);
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 13/13] mm: Assert CoCo shared allocations may sleep
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (11 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 12/13] swiotlb: Make rounded shared pool capacity allocatable Aneesh Kumar K.V (Arm)
@ 2026-11-16 9:11 ` Aneesh Kumar K.V (Arm)
2026-11-16 10:09 ` [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Suzuki K Poulose
13 siblings, 0 replies; 19+ messages in thread
From: Aneesh Kumar K.V (Arm) @ 2026-11-16 9:11 UTC (permalink / raw)
To: linux-coco
Cc: Aneesh Kumar K.V (Arm), Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches
On platforms with memory encryption, alloc_cc_shared_pages_node()
transitions the allocated memory to the shared state.
set_memory_decrypted() may sleep, so the transition must run from a
sleepable context.
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
mm/cc_shared.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/mm/cc_shared.c b/mm/cc_shared.c
index c9b4e3dfe06a..b30b8e78da32 100644
--- a/mm/cc_shared.c
+++ b/mm/cc_shared.c
@@ -8,6 +8,7 @@
#include <linux/errno.h>
#include <linux/export.h>
#include <linux/gfp.h>
+#include <linux/kernel.h>
#include <linux/log2.h>
#include <linux/mm.h>
#include <linux/mem_encrypt.h>
@@ -106,6 +107,8 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp,
unsigned int order;
int ret;
+ might_sleep();
+
ret = cc_shared_calc_layout(requested, &layout);
if (ret)
return ret;
--
2.43.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
` (12 preceding siblings ...)
2026-11-16 9:11 ` [PATCH v10 13/13] mm: Assert CoCo shared allocations may sleep Aneesh Kumar K.V (Arm)
@ 2026-11-16 10:09 ` Suzuki K Poulose
2026-11-16 13:22 ` Jason Gunthorpe
13 siblings, 1 reply; 19+ messages in thread
From: Suzuki K Poulose @ 2026-11-16 10:09 UTC (permalink / raw)
To: Aneesh Kumar K.V (Arm), linux-coco
Cc: Andrew Morton, Baoquan He, Mike Rapoport, Pasha Tatashin,
Pratyush Yadav, Catalin Marinas, Christian König,
Jason Gunthorpe, Joerg Roedel, Marc Zyngier, Marek Szyprowski,
Robin Murphy, Steven Price, Sumit Semwal, Thomas Gleixner,
Will Deacon, Russell King, Benjamin Gaignard, Brian Starkey,
John Stultz, Mark Rutland, Radu Rendec, T.J. Mercier,
Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
Christophe Leroy, Ritesh Harjani, Shrikanth Hegde,
Alexander Gordeev, Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H . Peter Anvin,
Kiryl Shutsemau, Rick Edgecombe, K . Y . Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Paolo Bonzini,
Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra, dri-devel,
iommu, linaro-mm-sig, linux-arm-kernel, linux-media, linux-mm,
kvmarm, patches, Alexey Kardashevskiy, Michael Roth, pankaj.gupta
Cc Michael Roth, Alexey, Pankaj Gupta
On 16/11/2026 09:11, Aneesh Kumar K.V (Arm) wrote:
>
> This series tightens the alignment requirements for buffers that are shared
> between confidential-computing guests and the host, and adds a common
> allocator for host-shared memory.
>
> When a guest runs with private memory, buffers shared with the hypervisor
> are not only accessed by the guest. They are also accessed by the host
> kernel, and the host may manage the corresponding shared/private state at a
> granularity larger than the guest page size.
>
> Architectures such as Arm can detect incorrect accesses to Realm physical
> address space PFNs through GPC faults. However, relying on that as the only
> line of defence is fragile and can still lead to kernel crashes. The risk
> is especially visible for shared buffers that are later mmapped into
> userspace, such as guest_memfd or dma-buf backed allocations. Once
> userspace can access the mapping, the kernel cannot guarantee that
> applications will only touch the intended 4K region rather than the whole
> host page mapped into their address space. Those userspace addresses may
> also be passed back into the kernel and accessed through the linear map,
> resulting in a GPC fault.
IIUC, AMD SEV-SNP also has a similar requirement for other reasons, to
align the "shared" allocations within a guest (2MB ?, as mentioned by
David Kaplan at the LPC).
Michael, Alexey,
Please could you share your thoughts on this series ?
Suzuki
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory
2026-11-16 10:09 ` [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Suzuki K Poulose
@ 2026-11-16 13:22 ` Jason Gunthorpe
0 siblings, 0 replies; 19+ messages in thread
From: Jason Gunthorpe @ 2026-11-16 13:22 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: Aneesh Kumar K.V (Arm), linux-coco, Andrew Morton, Baoquan He,
Mike Rapoport, Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Thomas Gleixner, Will Deacon, Russell King, Benjamin Gaignard,
Brian Starkey, John Stultz, Mark Rutland, Radu Rendec,
T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches, Alexey Kardashevskiy, Michael Roth,
pankaj.gupta
On Mon, Nov 16, 2026 at 10:09:35AM +0000, Suzuki K Poulose wrote:
> Cc Michael Roth, Alexey, Pankaj Gupta
>
> On 16/11/2026 09:11, Aneesh Kumar K.V (Arm) wrote:
> >
> > This series tightens the alignment requirements for buffers that are shared
> > between confidential-computing guests and the host, and adds a common
> > allocator for host-shared memory.
> >
> > When a guest runs with private memory, buffers shared with the hypervisor
> > are not only accessed by the guest. They are also accessed by the host
> > kernel, and the host may manage the corresponding shared/private state at a
> > granularity larger than the guest page size.
> >
> > Architectures such as Arm can detect incorrect accesses to Realm physical
> > address space PFNs through GPC faults. However, relying on that as the only
> > line of defence is fragile and can still lead to kernel crashes. The risk
> > is especially visible for shared buffers that are later mmapped into
> > userspace, such as guest_memfd or dma-buf backed allocations. Once
> > userspace can access the mapping, the kernel cannot guarantee that
> > applications will only touch the intended 4K region rather than the whole
> > host page mapped into their address space. Those userspace addresses may
> > also be passed back into the kernel and accessed through the linear map,
> > resulting in a GPC fault.
>
> IIUC, AMD SEV-SNP also has a similar requirement for other reasons, to
> align the "shared" allocations within a guest (2MB ?, as mentioned by
> David Kaplan at the LPC).
>
> Michael, Alexey,
>
> Please could you share your thoughts on this series ?
Yes, I think we concluded at LPC that AMD should use this series with
a 2MB setting rather than try to do their HW assisted smash/unsmash
Jason
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 02/13] mm: Zero memory during shared memory transitions
2026-11-16 9:11 ` [PATCH v10 02/13] mm: Zero memory during shared memory transitions Aneesh Kumar K.V (Arm)
@ 2026-11-16 13:33 ` Fuad Tabba
2026-11-16 13:58 ` Aneesh Kumar K.V
0 siblings, 1 reply; 19+ messages in thread
From: Fuad Tabba @ 2026-11-16 13:33 UTC (permalink / raw)
To: Aneesh Kumar K.V (Arm)
Cc: linux-coco, Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches, Vincent Donnefort
Hi Aneesh,
On Mon, 16 Nov 2026 at 09:12, Aneesh Kumar K.V (Arm)
<aneesh.kumar@kernel.org> wrote:
[...]
> Keep allocations on platforms without memory encryption on the ordinary page
> allocator path so the original GFP constraints, including __GFP_ZERO, remain
> intact. Remove post-transition memset() calls that are now redundant and
> explicitly clear allocations on paths where no transition occurs.
>
> This intentionally makes every private-to-shared transition destructive;
> callers can no longer use set_memory_decrypted() to preserve existing contents.
pKVM shares memory in place, and a guest has to be able to share
memory it has already set up, as Vincent pointed out [1]. That was one
of the primary reasons for in-place conversion in guest_memfd.
Cheers,
/fuad
[1] https://lore.kernel.org/all/avro44jUWOARFzul@google.com/
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 02/13] mm: Zero memory during shared memory transitions
2026-11-16 13:33 ` Fuad Tabba
@ 2026-11-16 13:58 ` Aneesh Kumar K.V
2026-11-16 14:02 ` Jason Gunthorpe
0 siblings, 1 reply; 19+ messages in thread
From: Aneesh Kumar K.V @ 2026-11-16 13:58 UTC (permalink / raw)
To: Fuad Tabba
Cc: linux-coco, Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Jason Gunthorpe, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches, Vincent Donnefort
Fuad Tabba <fuad.tabba@linux.dev> writes:
> Hi Aneesh,
>
> On Mon, 16 Nov 2026 at 09:12, Aneesh Kumar K.V (Arm)
> <aneesh.kumar@kernel.org> wrote:
> [...]
>> Keep allocations on platforms without memory encryption on the ordinary page
>> allocator path so the original GFP constraints, including __GFP_ZERO, remain
>> intact. Remove post-transition memset() calls that are now redundant and
>> explicitly clear allocations on paths where no transition occurs.
>>
>> This intentionally makes every private-to-shared transition destructive;
>> callers can no longer use set_memory_decrypted() to preserve existing contents.
>
> pKVM shares memory in place, and a guest has to be able to share
> memory it has already set up, as Vincent pointed out [1]. That was one
> of the primary reasons for in-place conversion in guest_memfd.
>
> Cheers,
> /fuad
>
> [1] https://lore.kernel.org/all/avro44jUWOARFzul@google.com/
Thanks for catching this. We discussed it before, but I reached the
opposite conclusion:
https://lore.kernel.org/all/yq5a5wzw2ree.fsf@kernel.org
My mistake. How do you suggest we handle this? We want to avoid zeroing
pages multiple times with different CoCo configurations. For example, to
allocate a zeroed page, we currently do:
alloc_pages(__GFP_ZERO)
set_memory_decrypted()
memset(0, PAGE_SIZE)
Should we do something like
https://lore.kernel.org/all/yq5aa4p82sqt.fsf@kernel.org/
-anees
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 02/13] mm: Zero memory during shared memory transitions
2026-11-16 13:58 ` Aneesh Kumar K.V
@ 2026-11-16 14:02 ` Jason Gunthorpe
0 siblings, 0 replies; 19+ messages in thread
From: Jason Gunthorpe @ 2026-11-16 14:02 UTC (permalink / raw)
To: Aneesh Kumar K.V
Cc: Fuad Tabba, linux-coco, Andrew Morton, Baoquan He, Mike Rapoport,
Pasha Tatashin, Pratyush Yadav, Catalin Marinas,
Christian König, Joerg Roedel, Marc Zyngier,
Marek Szyprowski, Robin Murphy, Steven Price, Sumit Semwal,
Suzuki K Poulose, Thomas Gleixner, Will Deacon, Russell King,
Benjamin Gaignard, Brian Starkey, John Stultz, Mark Rutland,
Radu Rendec, T.J. Mercier, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy, Ritesh Harjani,
Shrikanth Hegde, Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Paolo Bonzini, Vitaly Kuznetsov, Andy Lutomirski, Peter Zijlstra,
dri-devel, iommu, linaro-mm-sig, linux-arm-kernel, linux-media,
linux-mm, kvmarm, patches, Vincent Donnefort
On Mon, Nov 16, 2026 at 07:28:42PM +0530, Aneesh Kumar K.V wrote:
> Fuad Tabba <fuad.tabba@linux.dev> writes:
>
> > Hi Aneesh,
> >
> > On Mon, 16 Nov 2026 at 09:12, Aneesh Kumar K.V (Arm)
> > <aneesh.kumar@kernel.org> wrote:
> > [...]
> >> Keep allocations on platforms without memory encryption on the ordinary page
> >> allocator path so the original GFP constraints, including __GFP_ZERO, remain
> >> intact. Remove post-transition memset() calls that are now redundant and
> >> explicitly clear allocations on paths where no transition occurs.
> >>
> >> This intentionally makes every private-to-shared transition destructive;
> >> callers can no longer use set_memory_decrypted() to preserve existing contents.
> >
> > pKVM shares memory in place, and a guest has to be able to share
> > memory it has already set up, as Vincent pointed out [1]. That was one
> > of the primary reasons for in-place conversion in guest_memfd.
> >
> > Cheers,
> > /fuad
> >
> > [1] https://lore.kernel.org/all/avro44jUWOARFzul@google.com/
>
>
> Thanks for catching this. We discussed it before, but I reached the
> opposite conclusion:
>
> https://lore.kernel.org/all/yq5a5wzw2ree.fsf@kernel.org
>
> My mistake. How do you suggest we handle this? We want to avoid zeroing
> pages multiple times with different CoCo configurations. For example, to
> allocate a zeroed page, we currently do:
>
> alloc_pages(__GFP_ZERO)
> set_memory_decrypted()
> memset(0, PAGE_SIZE)
Just like you are wrapping set_memory_decrypted() in an allocator API
the user that wants to have the no-change sharing needs its own
special API too.
Then you can bury a special implementation for pkvm there.
Where is this special user in the kernel anyhow?
Jason
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-11-16 14:03 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 9:11 [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 01/13] mm: Add an allocator for CoCo shared memory Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 02/13] mm: Zero memory during shared memory transitions Aneesh Kumar K.V (Arm)
2026-11-16 13:33 ` Fuad Tabba
2026-11-16 13:58 ` Aneesh Kumar K.V
2026-11-16 14:02 ` Jason Gunthorpe
2026-11-16 9:11 ` [PATCH v10 03/13] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 04/13] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 05/13] dma-contiguous: Derive shared alignment from DMA attributes Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 06/13] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 07/13] dma-direct: Align CoCo shared DMA allocations to the shared granule size Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 08/13] swiotlb: Align shared IO TLB pools " Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 09/13] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 10/13] dma-buf: system_heap: Limit scatterlist entries to the buffer size Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 11/13] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 12/13] swiotlb: Make rounded shared pool capacity allocatable Aneesh Kumar K.V (Arm)
2026-11-16 9:11 ` [PATCH v10 13/13] mm: Assert CoCo shared allocations may sleep Aneesh Kumar K.V (Arm)
2026-11-16 10:09 ` [PATCH v10 00/13] coco: guest: Add a shared-granule allocator for host-shared memory Suzuki K Poulose
2026-11-16 13:22 ` Jason Gunthorpe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox