[PATCH 0/2] Clear shared pages after private-to-shared conversion

[PATCH 0/2] Clear shared pages after private-to-shared conversion

[Steven Price]

Arm CCA includes "Memory Encryption Contexts" (MEC) which allows the
private and shared data accessible to a guest to have different memory
encryption keys. Consequently when converting memory to shared, the
memory encryption key used to access the physical page will change.

Both the GICv3 ITS driver and the system_cc_shared dma-buf heap
currently allocate memory with __GFP_ZERO and then decrypt it. With MEC
the zeroing is done with the wrong encryption key and the data visible
after decryption may be ciphertext. The RMM is required to scrub the
data, but may perform this scrub with a different encryption key to the
eventual key that will be used for shared access.

Fix these two sites by avoiding the __GFP_ZERO during the allocation and
performing a clear_pages() call after the decryption.

Steven Price (2):
  irqchip/gic-v3-its: Zero shared pages after conversion
  dma-buf: heaps: Zero system shared heap pages after conversion

 drivers/dma-buf/heaps/system_heap.c | 14 +++++++++++---
 drivers/irqchip/irq-gic-v3-its.c    |  7 ++++++-
 2 files changed, 17 insertions(+), 4 deletions(-)

-- 
2.43.0

[PATCH 1/2] irqchip/gic-v3-its: Zero shared pages after conversion

[Steven Price]

its_alloc_pages_node() passes __GFP_ZERO to the page allocator before
calling set_memory_decrypted(). This assumes that converting a page from
private to shared preserves its contents.

For Arm CCA with MEC (Memory Encryption Contexts) the key used to access
the page will change, and so by default the visible data will change.
The host could ensure that it zeros the page, but rather than relying on
the host's behaviour it's best if the guest simply zeros after the
decryption rather than before. Specifically in this case the ITS tables
are required to be zeroed.

Mask out __GFP_ZERO from the allocation request, and do the zeroing as a
separate step after decryption.

Fixes: b08e2f42e86b ("irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor")
Signed-off-by: Steven Price <steven.price@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 6f5811aae59c..c954bbe9f4db 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -213,10 +213,12 @@ static gfp_t gfp_flags_quirk;
 static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 					 unsigned int order)
 {
+	bool want_zero = gfp & __GFP_ZERO;
 	struct page *page;
 	int ret = 0;
 
-	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
+	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
+				order);
 
 	if (!page)
 		return NULL;
@@ -231,6 +233,9 @@ static struct page *its_alloc_pages_node(int node, gfp_t gfp,
 	if (ret)
 		return NULL;
 
+	if (want_zero)
+		clear_pages(page_address(page), 1 << order);
+
 	return page;
 }
 
-- 
2.43.0

[PATCH 2/2] dma-buf: heaps: Zero system shared heap pages after conversion

[Steven Price]

The system_cc_shared heap allocates pages with __GFP_ZERO before converting
them from private to shared with set_memory_decrypted(). This assumes that
the conversion preserves the contents of the pages.

For Arm CCA with MEC (Memory Encryption Contexts) the key used to access
the page will change, and so by default the visible data will change.
The host could ensure that it zeros the page after decryption, but
rather than relying on the host's behaviour it's best if the guest
simply zeros after the decryption rather than before.

For CC shared buffers, defer zeroing until each page has been converted
successfully. For other buffers keep the existing behaviour.

Fixes: 78b30c50a7ac ("dma-buf: heaps: system: add system_cc_shared heap for explicitly shared memory")
Signed-off-by: Steven Price <steven.price@arm.com>
---
 drivers/dma-buf/heaps/system_heap.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
index c8959eadc71d..f14930904089 100644
--- a/drivers/dma-buf/heaps/system_heap.c
+++ b/drivers/dma-buf/heaps/system_heap.c
@@ -376,7 +376,8 @@ static const struct dma_buf_ops system_heap_buf_ops = {
 };
 
 static struct page *alloc_largest_available(unsigned long size,
-					    unsigned int max_order)
+					    unsigned int max_order,
+					    bool defer_zero)
 {
 	struct page *page;
 	int i;
@@ -388,6 +389,9 @@ static struct page *alloc_largest_available(unsigned long size,
 		if (max_order < orders[i])
 			continue;
 		flags = order_flags[i];
+		/* Decryption can change the contents, so clear it afterwards. */
+		if (defer_zero)
+			flags &= ~__GFP_ZERO;
 		if (mem_accounting)
 			flags |= __GFP_ACCOUNT;
 		page = alloc_pages(flags, orders[i]);
@@ -438,7 +442,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 			goto free_buffer;
 		}
 
-		page = alloc_largest_available(size_remaining, max_order);
+		page = alloc_largest_available(size_remaining, max_order,
+					       cc_shared_buffer(buffer));
 		if (!page)
 			goto free_buffer;
 
@@ -461,9 +466,12 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
 
 	if (cc_shared_buffer(buffer)) {
 		for_each_sgtable_sg(table, sg, i) {
-			ret = system_heap_set_page_decrypted(sg_page(sg));
+			page = sg_page(sg);
+			ret = system_heap_set_page_decrypted(page);
 			if (ret)
 				goto free_pages;
+
+			clear_pages(page_address(page), 1 << compound_order(page));
 		}
 	}
 
-- 
2.43.0

Re: [PATCH 0/2] Clear shared pages after private-to-shared conversion

[Jason Gunthorpe]

On Thu, Aug 20, 2026 at 11:50:23AM +0100, Steven Price wrote:
> Arm CCA includes "Memory Encryption Contexts" (MEC) which allows the
> private and shared data accessible to a guest to have different memory
> encryption keys. Consequently when converting memory to shared, the
> memory encryption key used to access the physical page will change.
> 
> Both the GICv3 ITS driver and the system_cc_shared dma-buf heap
> currently allocate memory with __GFP_ZERO and then decrypt it. With MEC
> the zeroing is done with the wrong encryption key and the data visible
> after decryption may be ciphertext. The RMM is required to scrub the
> data, but may perform this scrub with a different encryption key to the
> eventual key that will be used for shared access.
> 
> Fix these two sites by avoiding the __GFP_ZERO during the allocation and
> performing a clear_pages() call after the decryption.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

This whole set_memory_decrypted() API is awful. It really should be improved.

alloc_pages_decrypted() ?

Jason

Re: [PATCH 1/2] irqchip/gic-v3-its: Zero shared pages after conversion

[Marc Zyngier]

On Thu, 20 Aug 2026 11:50:24 +0100,
Steven Price <steven.price@arm.com> wrote:
> 
> its_alloc_pages_node() passes __GFP_ZERO to the page allocator before
> calling set_memory_decrypted(). This assumes that converting a page from
> private to shared preserves its contents.
> 
> For Arm CCA with MEC (Memory Encryption Contexts) the key used to access
> the page will change, and so by default the visible data will change.
> The host could ensure that it zeros the page, but rather than relying on
> the host's behaviour it's best if the guest simply zeros after the
> decryption rather than before. Specifically in this case the ITS tables
> are required to be zeroed.

What are the guarantees that we want to enforce post decryption?  My
recollection is that the RME firmware cleans the caches to the PoPA,
making the data immediately visible to the hypervisor. Obviously, this
isn't the case anymore, since the zeroing comes after that, and I
don't see any CMO enforcing this.

I'm concerned that this relies on undocumented behaviours that may
hold today on some undisclosed combinations of HW and hypervisors, but
that are not guaranteed at all. set_memory_decrypted() doesn't really
say anything, and I have the feeling that we may want some hypervisor
specific hook to perform the correct CMO magic. I don't think this is
required right now, but I'm not excluding anything!

> 
> Mask out __GFP_ZERO from the allocation request, and do the zeroing as a
> separate step after decryption.
> 
> Fixes: b08e2f42e86b ("irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor")
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
>  drivers/irqchip/irq-gic-v3-its.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 6f5811aae59c..c954bbe9f4db 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -213,10 +213,12 @@ static gfp_t gfp_flags_quirk;
>  static struct page *its_alloc_pages_node(int node, gfp_t gfp,
>  					 unsigned int order)
>  {
> +	bool want_zero = gfp & __GFP_ZERO;
>  	struct page *page;
>  	int ret = 0;
>  
> -	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
> +	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
> +				order);
>  
>  	if (!page)
>  		return NULL;
> @@ -231,6 +233,9 @@ static struct page *its_alloc_pages_node(int node, gfp_t gfp,
>  	if (ret)
>  		return NULL;
>  
> +	if (want_zero)
> +		clear_pages(page_address(page), 1 << order);
> +

nit: please use BIT(order), which matches the type required for
clear_pages().

But I'd really like some discussion about the CMO side of things.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 1/2] irqchip/gic-v3-its: Zero shared pages after conversion

[Steven Price]

On 20/08/2026 13:32, Marc Zyngier wrote:
> On Thu, 20 Aug 2026 11:50:24 +0100,
> Steven Price <steven.price@arm.com> wrote:
>>
>> its_alloc_pages_node() passes __GFP_ZERO to the page allocator before
>> calling set_memory_decrypted(). This assumes that converting a page from
>> private to shared preserves its contents.
>>
>> For Arm CCA with MEC (Memory Encryption Contexts) the key used to access
>> the page will change, and so by default the visible data will change.
>> The host could ensure that it zeros the page, but rather than relying on
>> the host's behaviour it's best if the guest simply zeros after the
>> decryption rather than before. Specifically in this case the ITS tables
>> are required to be zeroed.
> 
> What are the guarantees that we want to enforce post decryption?  My
> recollection is that the RME firmware cleans the caches to the PoPA,
> making the data immediately visible to the hypervisor. Obviously, this
> isn't the case anymore, since the zeroing comes after that, and I
> don't see any CMO enforcing this.

The firmware should be ensuring that things are cleaned sufficiently
that the original data is inaccessible - that's required as part of the
wiping when converting from private. However the wipe doesn't have to be
writing zeros, indeed the RMM spec suggests that two "possible
implementations" are:

* The RMM (or other platform firmware) writing either random data or
  zeroes to the memory location
* The MEC of the memory location being changed

My assumption (I have to admit I haven't checked) is that the GIC code
is doing sufficient CMO to ensure that the zeros that are being written
after the conversion are visible to the hypervisor - but that's no
different to the non-CCA case.

> I'm concerned that this relies on undocumented behaviours that may
> hold today on some undisclosed combinations of HW and hypervisors, but
> that are not guaranteed at all. set_memory_decrypted() doesn't really
> say anything, and I have the feeling that we may want some hypervisor
> specific hook to perform the correct CMO magic. I don't think this is
> required right now, but I'm not excluding anything!

This is reducing how much Linux relies on undocumented behaviour - at
the moment Linux is relying on either the zeros it has written still
being visible or the firmware/hypervisor writing zeros after any
private->shared transition. This patch makes the guest do it rather than
relying on anything else.

You have a point that a spec clarification about CMOs might be worth
having - the RMM spec doesn't make clear what is required of the
firmware. Clearly for security it should be doing something to ensure
that the old data from the realm doesn't become visible.

>>
>> Mask out __GFP_ZERO from the allocation request, and do the zeroing as a
>> separate step after decryption.
>>
>> Fixes: b08e2f42e86b ("irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor")
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>>  drivers/irqchip/irq-gic-v3-its.c | 7 ++++++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
>> index 6f5811aae59c..c954bbe9f4db 100644
>> --- a/drivers/irqchip/irq-gic-v3-its.c
>> +++ b/drivers/irqchip/irq-gic-v3-its.c
>> @@ -213,10 +213,12 @@ static gfp_t gfp_flags_quirk;
>>  static struct page *its_alloc_pages_node(int node, gfp_t gfp,
>>  					 unsigned int order)
>>  {
>> +	bool want_zero = gfp & __GFP_ZERO;
>>  	struct page *page;
>>  	int ret = 0;
>>  
>> -	page = alloc_pages_node(node, gfp | gfp_flags_quirk, order);
>> +	page = alloc_pages_node(node, (gfp & ~__GFP_ZERO) | gfp_flags_quirk,
>> +				order);
>>  
>>  	if (!page)
>>  		return NULL;
>> @@ -231,6 +233,9 @@ static struct page *its_alloc_pages_node(int node, gfp_t gfp,
>>  	if (ret)
>>  		return NULL;
>>  
>> +	if (want_zero)
>> +		clear_pages(page_address(page), 1 << order);
>> +
> 
> nit: please use BIT(order), which matches the type required for
> clear_pages().

Sure, this was matching the use in set_memory_decrypted(), but I can
update that too.

> But I'd really like some discussion about the CMO side of things.

I'm not sure what more to say about CMO - if you want changes in the
commit message(s) then please suggest something. AFAICT this patch
doesn't change anything about cache maintenance. You're welcome to raise
spec clarifications if you want to.

Thanks,
Steve

PS. I'll take a look at the Sashiko comments - but they are both
pre-existing issues not issues with this series.

Re: [PATCH 1/2] irqchip/gic-v3-its: Zero shared pages after conversion

[Jason Gunthorpe]

On Thu, Aug 20, 2026 at 01:32:05PM +0100, Marc Zyngier wrote:
> On Thu, 20 Aug 2026 11:50:24 +0100,
> Steven Price <steven.price@arm.com> wrote:
> > 
> > its_alloc_pages_node() passes __GFP_ZERO to the page allocator before
> > calling set_memory_decrypted(). This assumes that converting a page from
> > private to shared preserves its contents.
> > 
> > For Arm CCA with MEC (Memory Encryption Contexts) the key used to access
> > the page will change, and so by default the visible data will change.
> > The host could ensure that it zeros the page, but rather than relying on
> > the host's behaviour it's best if the guest simply zeros after the
> > decryption rather than before. Specifically in this case the ITS tables
> > are required to be zeroed.
> 
> What are the guarantees that we want to enforce post decryption?  My
> recollection is that the RME firmware cleans the caches to the PoPA,
> making the data immediately visible to the hypervisor. Obviously, this
> isn't the case anymore, since the zeroing comes after that, and I
> don't see any CMO enforcing this.

I thought any CMO stuff was principally about cleaning things as part
of the MEC change? Coherency after the memory is made shared should
follow the normal cachable memory model rules, just like in a non-CC
VM? We don't need further explicit CMOs for that.

Post decryption I would expect from all architectures:

 1) Neither the guest or host take a fault/error when accessing the
    memory. ie the host may immediately pass this memory to an
    O_DIRECT system call and have its kernel read from it.
    It must not crash the kernel.

 2) So long as the memory is mapped cachable it should follow the
    normal memory model visibility rules. ie it works the same as
    VM CPU memory prior to CC

 3) Rules for actual DMA are the same as prior to CC, the VM is
    expected to issue its own flushes prior to DMA if the platform
    requires it.

Given the requirements for #1, is there actually any case on any
platform where the host doesn't *have* to fill the memory? Is there a
platform with MEC that doesn't generate an error on reading with the
wrong MEC? Without MEC it surely has to be zero'd in the RMM world,
right?

I've argued before that set_memory_decrypted() should be defined to
return 0'd memory. I think there are real systems that *have* to zero
the memory as part of the state change and this API is now
forcing an extra zeroing.

> I'm concerned that this relies on undocumented behaviours that may
> hold today on some undisclosed combinations of HW and hypervisors, but
> that are not guaranteed at all. set_memory_decrypted() doesn't really
> say anything, and I have the feeling that we may want some hypervisor
> specific hook to perform the correct CMO magic. I don't think this is
> required right now, but I'm not excluding anything!

I would expect any required CMOs to be part of the arch's
implementation of set_memory_decrypted()?

Jason