[RFC PATCH] arm64: mm: support set_memory_encrypted/decrypted for vmalloc addresses

[RFC PATCH] arm64: mm: support set_memory_encrypted/decrypted for vmalloc addresses

[Kameron Carr]

Currently __set_memory_enc_dec() only handles linear map (lm) addresses
and returns -EINVAL for anything else. This means callers using
vmalloc'd buffers cannot mark memory as shared/protected with the RMM
via set_memory_decrypted()/set_memory_encrypted().

Extend the implementation to handle vmalloc (non-linear-map) addresses
by introducing __set_va_addr_enc_dec(). For vmalloc addresses, the page
table entries are not contiguous in the physical address space, so the
function walks the vm_area's pages array and issues per-page RSI calls
to transition each page between shared and protected states.

The original linear-map path is factored out into __set_lm_addr_enc_dec(),
and __set_memory_enc_dec() now dispatches to the appropriate helper based
on whether the address is a linear map address.

Signed-off-by: Kameron Carr <kameroncarr@linux.microsoft.com>
---
 arch/arm64/mm/pageattr.c | 74 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 65 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c
index ce035e1b4eaf..45058f61b957 100644
--- a/arch/arm64/mm/pageattr.c
+++ b/arch/arm64/mm/pageattr.c
@@ -275,20 +275,12 @@ int set_direct_map_default_noflush(struct page *page)
 				 PAGE_SIZE, set_mask, clear_mask);
 }
 
-static int __set_memory_enc_dec(unsigned long addr,
-				int numpages,
-				bool encrypt)
+static int __set_lm_addr_enc_dec(unsigned long addr, int numpages, bool encrypt)
 {
 	unsigned long set_prot = 0, clear_prot = 0;
 	phys_addr_t start, end;
 	int ret;
 
-	if (!is_realm_world())
-		return 0;
-
-	if (!__is_lm_address(addr))
-		return -EINVAL;
-
 	start = __virt_to_phys(addr);
 	end = start + numpages * PAGE_SIZE;
 
@@ -321,6 +313,70 @@ static int __set_memory_enc_dec(unsigned long addr,
 				      __pgprot(PTE_PRESENT_INVALID));
 }
 
+static int __set_va_addr_enc_dec(unsigned long addr, int numpages, bool encrypt)
+{
+	unsigned long set_prot = 0, clear_prot = 0, start_idx;
+	struct vm_struct *area;
+	int i, ret;
+
+	if (encrypt)
+		clear_prot = PROT_NS_SHARED;
+	else
+		set_prot = PROT_NS_SHARED;
+
+	area = find_vm_area((void *)addr);
+	if (!area)
+		return -EINVAL;
+
+	start_idx = ((unsigned long)kasan_reset_tag((void *)addr) -
+		     (unsigned long)kasan_reset_tag(area->addr)) >>
+		    PAGE_SHIFT;
+
+	if (start_idx + numpages > area->nr_pages)
+		return -EINVAL;
+
+	/*
+	 * Break the mapping before we make any changes to avoid stale TLB
+	 * entries or Synchronous External Aborts caused by RIPAS_EMPTY
+	 */
+	ret = change_memory_common(addr, numpages,
+		__pgprot(set_prot | PTE_PRESENT_INVALID),
+		__pgprot(clear_prot | PTE_PRESENT_VALID_KERNEL));
+
+	if (ret)
+		return ret;
+
+	for (i = 0; i < numpages; i++) {
+		struct page *page = area->pages[start_idx + i];
+		phys_addr_t phys = page_to_phys(page);
+
+		if (encrypt) {
+			ret = rsi_set_memory_range_protected(phys,
+							     phys + PAGE_SIZE);
+		} else {
+			ret = rsi_set_memory_range_shared(phys,
+							  phys + PAGE_SIZE);
+		}
+		if (ret)
+			return ret;
+	}
+
+	return change_memory_common(addr, numpages,
+				    __pgprot(PTE_PRESENT_VALID_KERNEL),
+				    __pgprot(PTE_PRESENT_INVALID));
+}
+
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool encrypt)
+{
+	if (!is_realm_world())
+		return 0;
+
+	if (!__is_lm_address(addr))
+		return __set_va_addr_enc_dec(addr, numpages, encrypt);
+
+	return __set_lm_addr_enc_dec(addr, numpages, encrypt);
+}
+
 static int realm_set_memory_encrypted(unsigned long addr, int numpages)
 {
 	int ret = __set_memory_enc_dec(addr, numpages, true);

base-commit: 25f6040ca43a78048466b949994b8d637fe2fe07
-- 
2.45.4

Re: [RFC PATCH] arm64: mm: support set_memory_encrypted/decrypted for vmalloc addresses

[Catalin Marinas]

On Mon, Apr 06, 2026 at 02:33:17PM -0700, Kameron Carr wrote:
> Currently __set_memory_enc_dec() only handles linear map (lm) addresses
> and returns -EINVAL for anything else. This means callers using
> vmalloc'd buffers cannot mark memory as shared/protected with the RMM
> via set_memory_decrypted()/set_memory_encrypted().
> 
> Extend the implementation to handle vmalloc (non-linear-map) addresses
> by introducing __set_va_addr_enc_dec(). For vmalloc addresses, the page
> table entries are not contiguous in the physical address space, so the
> function walks the vm_area's pages array and issues per-page RSI calls
> to transition each page between shared and protected states.
> 
> The original linear-map path is factored out into __set_lm_addr_enc_dec(),
> and __set_memory_enc_dec() now dispatches to the appropriate helper based
> on whether the address is a linear map address.

Could you give more details about the user of set_memory_decrypted() on
vmalloc()'ed addresses? I think this came up in the past and I wondered
whether something like GFP_DECRYPTED would be simpler to implement (even
posted a hack but without vmalloc() support). If it is known upfront
that the memory will be decrypted, it's easier/cheaper to do this on the
page allocation time to change the linear map and just use
pgprot_decrypted() for vmap(). No need to rewrite the page table after
mapping the pages.

-- 
Catalin

RE: [RFC PATCH] arm64: mm: support set_memory_encrypted/decrypted for vmalloc addresses

[Kameron Carr]

On Friday, April 10, 2026 4:06 AM, Catalin Marinas wrote:
> Could you give more details about the user of set_memory_decrypted() on
> vmalloc()'ed addresses? I think this came up in the past and I wondered
> whether something like GFP_DECRYPTED would be simpler to implement (even
> posted a hack but without vmalloc() support). If it is known upfront
> that the memory will be decrypted, it's easier/cheaper to do this on the
> page allocation time to change the linear map and just use
> pgprot_decrypted() for vmap(). No need to rewrite the page table after
> mapping the pages.

Thank you for the review. I understand that my approach is not ideal in
terms of speed / cost. For my use case it was not an issue since the
memory is typically only initialized once during device initialization,
but I understand there could be a larger performance impact in other
uses.

The use case I am interested in is Hyper-V netvsc
(drivers/net/hyperv/netvsc.c). The network driver allocates large send
and receive buffers (typically 16MB+) with vzalloc(), then registers them
as Guest Physical Address Descriptor Lists (GPADLs) via
vmbus_establish_gpadl(). Inside __vmbus_establish_gpadl()
(drivers/hv/channel.c), set_memory_decrypted() is called on the buffer so
the hypervisor can access the shared memory.

In this use case, whether to decrypt the memory can always be known at
time of allocation, so a solution like GFP_DECRYPTED is an option.

I think I found the hack you mentioned
(https://lore.kernel.org/linux-arm-kernel/ZmNJdSxSz-sYpVgI@arm.com/). The
feedback in Michael Kelley's reply covers the key considerations well. He
likely had netvsc's use of vmalloc in mind when he made the point
"GFP_DECRYPTED should work for the three memory allocation interfaces and
their variants: alloc_pages(), kmalloc(), and vmalloc()." His other
points already cover the concerns I had in mind around handling errors
from set_memory_decrypted()/encrypted(), etc.

What is the current status of your proposed GFP_DECRYPTED implementation?
Is this something you are actively working on?

I'd be happy to work on an RFC following the GFP_DECRYPTED approach if
you think that's the right direction.

Regards,
Kameron Carr

Re: [RFC PATCH] arm64: mm: support set_memory_encrypted/decrypted for vmalloc addresses

[Catalin Marinas]

On Fri, Apr 10, 2026 at 02:36:42PM -0700, Kameron Carr wrote:
> On Friday, April 10, 2026 4:06 AM, Catalin Marinas wrote:
> > Could you give more details about the user of set_memory_decrypted() on
> > vmalloc()'ed addresses? I think this came up in the past and I wondered
> > whether something like GFP_DECRYPTED would be simpler to implement (even
> > posted a hack but without vmalloc() support). If it is known upfront
> > that the memory will be decrypted, it's easier/cheaper to do this on the
> > page allocation time to change the linear map and just use
> > pgprot_decrypted() for vmap(). No need to rewrite the page table after
> > mapping the pages.
[...]
> In this use case, whether to decrypt the memory can always be known at
> time of allocation, so a solution like GFP_DECRYPTED is an option.
> 
> I think I found the hack you mentioned
> (https://lore.kernel.org/linux-arm-kernel/ZmNJdSxSz-sYpVgI@arm.com/). The
> feedback in Michael Kelley's reply covers the key considerations well.

Yes, that's the thread. It started originally as a GICv3 need
(eventually we went for genpool).

> He likely had netvsc's use of vmalloc in mind when he made the point
> "GFP_DECRYPTED should work for the three memory allocation interfaces and
> their variants: alloc_pages(), kmalloc(), and vmalloc()." His other
> points already cover the concerns I had in mind around handling errors
> from set_memory_decrypted()/encrypted(), etc.
> 
> What is the current status of your proposed GFP_DECRYPTED implementation?
> Is this something you are actively working on?

Not really. But I've been looking at it again and I think it adds more
problems than it solves. A GFP flag would be passed down to
kmem_cache_alloc() and confuse the slab management if some pages are
encrypted, others not for the same kmem_cache (SLAB_NO_MERGE wouldn't
help). I wonder whether something like SLAB_DECRYPTED would work better
for this if we really need it (not aware of any user though).

Anyway, let's ignore slab for now and look at vmalloc(). I can see
hv_ringbuffer_init() using an explicit vmap(pgprot_decrypted()). While
you could do this, it might be better to just add a VM_DECRYPTED flag
and a few wrappers like vmalloc_decrypted(). It would call
set_memory_decrypted() for the allocated pages and use
pgprot_decrypted() for vmap. On vfree(), it will have to set the pages
back to encrypted. It should be fairly mechanical to do (or a 5 min job
for an LLM ;)).

-- 
Catalin

Re: [RFC PATCH] arm64: mm: support set_memory_encrypted/decrypted for vmalloc addresses

[Mike Rapoport]

Hi Catalin,

On Tue, Apr 14, 2026 at 05:46:06PM +0100, Catalin Marinas wrote:
> On Fri, Apr 10, 2026 at 02:36:42PM -0700, Kameron Carr wrote:
> > On Friday, April 10, 2026 4:06 AM, Catalin Marinas wrote:
> > > Could you give more details about the user of set_memory_decrypted() on
> > > vmalloc()'ed addresses? I think this came up in the past and I wondered
> > > whether something like GFP_DECRYPTED would be simpler to implement (even
> > > posted a hack but without vmalloc() support). If it is known upfront
> > > that the memory will be decrypted, it's easier/cheaper to do this on the
> > > page allocation time to change the linear map and just use
> > > pgprot_decrypted() for vmap(). No need to rewrite the page table after
> > > mapping the pages.
> [...]
> > In this use case, whether to decrypt the memory can always be known at
> > time of allocation, so a solution like GFP_DECRYPTED is an option.
> > 
> > I think I found the hack you mentioned
> > (https://lore.kernel.org/linux-arm-kernel/ZmNJdSxSz-sYpVgI@arm.com/). The
> > feedback in Michael Kelley's reply covers the key considerations well.
> 
> Yes, that's the thread. It started originally as a GICv3 need
> (eventually we went for genpool).
> 
> > He likely had netvsc's use of vmalloc in mind when he made the point
> > "GFP_DECRYPTED should work for the three memory allocation interfaces and
> > their variants: alloc_pages(), kmalloc(), and vmalloc()." His other
> > points already cover the concerns I had in mind around handling errors
> > from set_memory_decrypted()/encrypted(), etc.
> > 
> > What is the current status of your proposed GFP_DECRYPTED implementation?
> > Is this something you are actively working on?
> 
> Not really. But I've been looking at it again and I think it adds more
> problems than it solves. A GFP flag would be passed down to
> kmem_cache_alloc() and confuse the slab management if some pages are
> encrypted, others not for the same kmem_cache (SLAB_NO_MERGE wouldn't
> help). I wonder whether something like SLAB_DECRYPTED would work better
> for this if we really need it (not aware of any user though).
> 
> Anyway, let's ignore slab for now and look at vmalloc(). I can see
> hv_ringbuffer_init() using an explicit vmap(pgprot_decrypted()). While
> you could do this, it might be better to just add a VM_DECRYPTED flag
> and a few wrappers like vmalloc_decrypted(). It would call
> set_memory_decrypted() for the allocated pages and use
> pgprot_decrypted() for vmap. On vfree(), it will have to set the pages
> back to encrypted. It should be fairly mechanical to do (or a 5 min job
> for an LLM ;)).
 
A GFP + SLAB flag is semantically better than only implementing decrypted
allocations only in vmalloc, but it's surely way more complex and intrusive
and we are running low on gfp flags on 32 bits :)

But I think we can push the _decrypted to kvmalloc() that would do either
kmalloc() + set_memory_decrypted() or vmalloc_decrypted() and it seems that
x86 and drivers could use that instead of alloc_pages() +
set_memory_decrypted().

We also would want also to make x86::set_memory_decrypted() to only accept
linear^w direct map addresses to ensure consistency between architectures.

> -- 
> Catalin
> 

-- 
Sincerely yours,
Mike.