[PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Lorenzo Stoakes]

x86 implements page attribute modification using its Change Page
Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page
attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
fragmentation") ranges of kernel page table entries can be collapsed into
huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed
entries previously pointed to, and it does so without any relevant locks
being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and
this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable
mappings.

However, this is problematic for ptdump, as it walks ranges it does not own
and thus runs the risk of a use-after-free on page tables freed underneath
it.

This patch resolves the issue by acquiring the mmap read lock on init_mm to
provide mutual exclusion against ptdump, which acquires the init_mm write
lock.

It is safe to acquire a sleeping lock as all the callers invoke
set_memory_rox() from process context and in any case,
change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
mutex, disallowing atomic context here.

We also include cleanup.h in order to use a scoped_guard() to implement
this cleanly.

Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Cc: stable@vger.kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
---
 arch/x86/mm/pat/set_memory.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index d023a40a1e03..4c4b8244502f 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -436,9 +437,16 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	flush_tlb_all();
 
-	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
-		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+	/*
+	 * ptdump might read these page tables, so avoid a use-after-free by
+	 * acquiring the mmap read lock on init_mm (ptdump acquires the mmap
+	 * write lock).
+	 */
+	scoped_guard(mmap_read_lock, &init_mm) {
+		list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
+			list_del(&ptdesc->pt_list);
+			pagetable_free(ptdesc);
+		}
 	}
 }
 

---
base-commit: a635d6748234582ea287c5ffeae28b9b23f91c7e
change-id: 20260710-fix-cpa-ptdump-race-0aa3b65d878d

Cheers,
-- 
Lorenzo Stoakes <ljs@kernel.org>

Re: [PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Kiryl Shutsemau]

On Fri, Jul 10, 2026 at 12:56:40PM +0100, Lorenzo Stoakes wrote:
> x86 implements page attribute modification using its Change Page
> Attributes (CPA) mechanism.
> 
> This tracks properties of ranges such as cache mode through x86 page
> attributes, and as part of that logic manipulates kernel page tables.
> 
> Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
> fragmentation") ranges of kernel page table entries can be collapsed into
> huge page table entries as part of this logic.
> 
> As part of this collapse, it frees the page tables which the collapsed
> entries previously pointed to, and it does so without any relevant locks
> being held to preclude concurrent kernel page table walkers.
> 
> The only way this code can be reached is if CPA_COLLAPSE is specified, and
> this is only set in set_memory_rox() via:
> 
> set_memory_rox()
> -> change_page_attr_set_clr()
> -> cpa_flush()
> -> cpa_collapse_large_pages()
> 
> Notable users of this are execmem and bpf when manipulating executable
> mappings.
> 
> However, this is problematic for ptdump, as it walks ranges it does not own
> and thus runs the risk of a use-after-free on page tables freed underneath
> it.
> 
> This patch resolves the issue by acquiring the mmap read lock on init_mm to
> provide mutual exclusion against ptdump, which acquires the init_mm write
> lock.
> 
> It is safe to acquire a sleeping lock as all the callers invoke
> set_memory_rox() from process context and in any case,
> change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
> mutex, disallowing atomic context here.
> 
> We also include cleanup.h in order to use a scoped_guard() to implement
> this cleanly.
> 
> Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
> Cc: stable@vger.kernel.org
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
 
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Dave Hansen]

On 7/10/26 04:56, Lorenzo Stoakes wrote:
> This patch resolves the issue by acquiring the mmap read lock on init_mm to
> provide mutual exclusion against ptdump, which acquires the init_mm write
> lock.

Hey Lorenzo,

Thanks for looking at this!

This isn't wrong per-se. ptdump does _sometimes_ acquire the init_mm
write lock.

But the fun comes when ptdump_curknl_show() passes current->mm in to the
ptdump code. In that case, there's no init_mm locking. I think the
'efi_mm' code has the same issue since it shares some of the kernel page
tables.

Is that your read on it too?

In the end, I think the issue is that there's not even *a* correct mmap
lock to take. The userspace half of the address space needs the
current->mm mmap lock and the kernel half needs the init_mm mmap lock.

The naming here doesn't help because the "current_kernel" file doesn't
dump the current kernel page tables, it dumps the whole kernel *copy*
(the only copy with PTI off) which includes userspace.

(Note: maybe we should hide "current_user" when PTI is off at runtime)

So what do we do?

1. We could just bite the bullet and have separate ptdump files for the
   top and bottom of the address space:
	current_kernel_top
	current_kernel_bottom
	current_user_top
	current_user_bottom
	etc..
   Then the lock you take is dictated by the file.
2. We could always take both init_mm and current->mm locks. That seems
   icky.
3. We could have ptdump_walk_pgd() take a different lock for each
   'range'. Logically:

	if (range->start < PAGE_OFFSET)
		mmap_write_lock(mm);
	else
		mmap_write_lock(&init_mm);

   That's icky too and it means a range can't cross PAGE_OFFSET, but
   that doesn't seem too bad (it could also WARN() if it sees bad
   ranges).
4. We do something fancier with the free like RCU (I think this may have
   been discussed already).

I'm kinda leaning toward #3.

Re: [PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Lorenzo Stoakes]

On Fri, Jul 10, 2026 at 09:26:48AM -0700, Dave Hansen wrote:
> On 7/10/26 04:56, Lorenzo Stoakes wrote:
> > This patch resolves the issue by acquiring the mmap read lock on init_mm to
> > provide mutual exclusion against ptdump, which acquires the init_mm write
> > lock.
>
> Hey Lorenzo,
>
> Thanks for looking at this!
>
> This isn't wrong per-se. ptdump does _sometimes_ acquire the init_mm
> write lock.
>
> But the fun comes when ptdump_curknl_show() passes current->mm in to the
> ptdump code. In that case, there's no init_mm locking. I think the
> 'efi_mm' code has the same issue since it shares some of the kernel page
> tables.
>
> Is that your read on it too?

Yeah sashiko reminded me of this, I had glossed over it :)

I already came up with a fix at [0], I think it's fine to just take the init_mm
lock in this case.

x86 is the only case in which an arbitrary process mm is read (fun!) arm64 does
the efi_mm thing too.

>
> In the end, I think the issue is that there's not even *a* correct mmap
> lock to take. The userspace half of the address space needs the
> current->mm mmap lock and the kernel half needs the init_mm mmap lock.

Well there's 2 that are correct because you are actually traversing both
ranges...

>
> The naming here doesn't help because the "current_kernel" file doesn't
> dump the current kernel page tables, it dumps the whole kernel *copy*
> (the only copy with PTI off) which includes userspace.

Yeah this is misleading, and I misunderstood on assumption x86 wouldn't and
 be so silly to... and didn't read the code too closely assuming you did
 this PAGE_OFFSET split and... yeah :)) guys :)

>
> (Note: maybe we should hide "current_user" when PTI is off at runtime)
>
> So what do we do?
>
> 1. We could just bite the bullet and have separate ptdump files for the
>    top and bottom of the address space:
> 	current_kernel_top
> 	current_kernel_bottom
> 	current_user_top
> 	current_user_bottom
> 	etc..
>    Then the lock you take is dictated by the file.

I mean that'd break userspace though wouldn't it?

> 2. We could always take both init_mm and current->mm locks. That seems
>    icky.

It's actually the least awful of all of these I think :) and the one I
implemented ([0]).

It logically makes sense because there's nothing that relies on an
arbitrary mm in order to acquire an mmap lock around the init_mm. None of
the mitigation relies on that.

Also the user mappings (modulo PTI obv) share kernel mappings, so it's
actually logical and reasonable to hold both.

> 3. We could have ptdump_walk_pgd() take a different lock for each
>    'range'. Logically:
>
> 	if (range->start < PAGE_OFFSET)
> 		mmap_write_lock(mm);
> 	else
> 		mmap_write_lock(&init_mm);

I don't love this. It feels a hack for x86 that's put in the wrong place,
i.e. core code.

And can you can make this assumption for efi_mm for all arches? Could other
arches might be weird about this?

>
>    That's icky too and it means a range can't cross PAGE_OFFSET, but
>    that doesn't seem too bad (it could also WARN() if it sees bad
>    ranges).
> 4. We do something fancier with the free like RCU (I think this may have
>    been discussed already).

No :) please no.

 Implementing a crazy RCU scheme just for ptdump was already mad enough,
but this is begging for an RCU stall. It's just not the right tool for
this.

Literally the issue is vmap (and CPA it turns out) not properly locking
when freeing page tables.

The convention already exists that the mmap lock on init_mm is how we
handle this kind of thing and everything fits except vmap, CPA which I've
now fixed.

And with [0] I fix this too :)

It's icky but the least bad IMO.

>
> I'm kinda leaning toward #3.

Another way forwards might be simply have the caller _call
ptdump_walk_pgd() twice_ once with the range set to [0, PAGE_OFFSET) passing whatever mm
!= init_mm, and again for [PAGE_OFFSET, ~0) passing init_mm?

Are there cases where you expect to see a delta in the kernel range in x86
for an arbitrary mm?

And would this work for efi_mm and also for arm64?

So far I think the 2 locks thing, as horrid as it is is the least fraught
way. And we have a live UAF here so good to get a fix in ASAP.

Thanks, Lorenzo

[0]:https://lore.kernel.org/linux-mm/20260710-b4-fix-non-init_mm-ptdump-v1-1-2d40982c98ec@kernel.org/

Re: [PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Dave Hansen]

On 7/10/26 11:53, Lorenzo Stoakes wrote:
> On Fri, Jul 10, 2026 at 09:26:48AM -0700, Dave Hansen wrote:
>> 1. We could just bite the bullet and have separate ptdump files for the
>>    top and bottom of the address space:
>> 	current_kernel_top
>> 	current_kernel_bottom
>> 	current_user_top
>> 	current_user_bottom
>> 	etc..
>>    Then the lock you take is dictated by the file.
> 
> I mean that'd break userspace though wouldn't it?

It's debugfs. So, yeah, I can see it breaking things, but it's also way
less of a concern. Nobody ever complained about the new PTI file getting
added in there.

>> 2. We could always take both init_mm and current->mm locks. That seems
>>    icky.
> 
> It's actually the least awful of all of these I think :) and the one I
> implemented ([0]).

Oh, cool, I missed that. That's a good pairing with this one!

>> 3. We could have ptdump_walk_pgd() take a different lock for each
>>    'range'. Logically:
>>
>> 	if (range->start < PAGE_OFFSET)
>> 		mmap_write_lock(mm);
>> 	else
>> 		mmap_write_lock(&init_mm);
> 
> I don't love this. It feels a hack for x86 that's put in the wrong place,
> i.e. core code.
> 
> And can you can make this assumption for efi_mm for all arches? Could other
> arches might be weird about this?

Yeah, it's possible they're weird. But I thought the whole idea of
efi_mm was to reuse the non-kernel part of the address space. So oddly
enough it kinda makes sense.

But, yeah, I totally get the reluctance to do this.

>> I'm kinda leaning toward #3.
> 
> Another way forwards might be simply have the caller _call
> ptdump_walk_pgd() twice_ once with the range set to [0, PAGE_OFFSET) passing whatever mm
> != init_mm, and again for [PAGE_OFFSET, ~0) passing init_mm?

Ahh, yeah, that's a good point. It could be done a layer up too.

> Are there cases where you expect to see a delta in the kernel range in x86
> for an arbitrary mm?

Are you asking if current->mm->pgd[255->511] is always the same as
init_mm->pgd[255->511]?

I think so, except for the LDT PGD when PTI is on. That can be different
between mms, and it's a single pgd_t entry. I think Brendan had some
grand plans to use this PGD for other things for ASI as well.

So, yeah, the upper half of the address space is *normally* identical.
But PTI plus set_ldt() is abnormal and we have to deal with it. The only
times that code frees page tables is at exit time and in an error path.

So, how does this interact with mmap_lock? Surely, someone looked at
this recently because the comment says:

 * Lock order:
 *      context.ldt_usr_sem
 *        mmap_lock
 *          context.lock

and not mmap_sem. But, alas, I don't see any mmap_lock anywhere. Someone
changed the comment and didn't look at the code.

Is there some mmap_lock interaction that I'm missing? I don't see it
_anywhere_ in the ldt code.

Re: [PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Lorenzo Stoakes]

+cc Brendan for fun's sake ;)

On Fri, Jul 10, 2026 at 12:50:26PM -0700, Dave Hansen wrote:
> On 7/10/26 11:53, Lorenzo Stoakes wrote:
> > On Fri, Jul 10, 2026 at 09:26:48AM -0700, Dave Hansen wrote:
> >> 1. We could just bite the bullet and have separate ptdump files for the
> >>    top and bottom of the address space:
> >> 	current_kernel_top
> >> 	current_kernel_bottom
> >> 	current_user_top
> >> 	current_user_bottom
> >> 	etc..
> >>    Then the lock you take is dictated by the file.
> >
> > I mean that'd break userspace though wouldn't it?
>
> It's debugfs. So, yeah, I can see it breaking things, but it's also way
> less of a concern. Nobody ever complained about the new PTI file getting
> added in there.

I wonder how many people use this :P but I expect some do and would moan about a
_removal_.

I guess we could have a warning if somebody used it.

>
> >> 2. We could always take both init_mm and current->mm locks. That seems
> >>    icky.
> >
> > It's actually the least awful of all of these I think :) and the one I
> > implemented ([0]).
>
> Oh, cool, I missed that. That's a good pairing with this one!

Yeah, sent it separately as they stand separately and this already addresses the
init_mm side of it anyway.

>
> >> 3. We could have ptdump_walk_pgd() take a different lock for each
> >>    'range'. Logically:
> >>
> >> 	if (range->start < PAGE_OFFSET)
> >> 		mmap_write_lock(mm);
> >> 	else
> >> 		mmap_write_lock(&init_mm);
> >
> > I don't love this. It feels a hack for x86 that's put in the wrong place,
> > i.e. core code.
> >
> > And can you can make this assumption for efi_mm for all arches? Could other
> > arches might be weird about this?
>
> Yeah, it's possible they're weird. But I thought the whole idea of
> efi_mm was to reuse the non-kernel part of the address space. So oddly
> enough it kinda makes sense.
>
> But, yeah, I totally get the reluctance to do this.

Yeah it's all a bit sucky this :(

>
> >> I'm kinda leaning toward #3.
> >
> > Another way forwards might be simply have the caller _call
> > ptdump_walk_pgd() twice_ once with the range set to [0, PAGE_OFFSET) passing whatever mm
> > != init_mm, and again for [PAGE_OFFSET, ~0) passing init_mm?
>
> Ahh, yeah, that's a good point. It could be done a layer up too.
>
> > Are there cases where you expect to see a delta in the kernel range in x86
> > for an arbitrary mm?
>
> Are you asking if current->mm->pgd[255->511] is always the same as
> init_mm->pgd[255->511]?

Yes :)

>
> I think so, except for the LDT PGD when PTI is on. That can be different
> between mms, and it's a single pgd_t entry. I think Brendan had some
> grand plans to use this PGD for other things for ASI as well.

Yeah I wondered if PTI or something else funky (Brendan's mermap?) might have
changed this.

But whether that'd actually be accessible from the debugfs "this mm's "?

>
> So, yeah, the upper half of the address space is *normally* identical.
> But PTI plus set_ldt() is abnormal and we have to deal with it. The only
> times that code frees page tables is at exit time and in an error path.
>
> So, how does this interact with mmap_lock? Surely, someone looked at
> this recently because the comment says:
>
>  * Lock order:
>  *      context.ldt_usr_sem
>  *        mmap_lock
>  *          context.lock
>
> and not mmap_sem. But, alas, I don't see any mmap_lock anywhere. Someone
> changed the comment and didn't look at the code.

Was a simple mmap_sem -> mmap_lock change since we renamed the lock right?

>
> Is there some mmap_lock interaction that I'm missing? I don't see it
> _anywhere_ in the ldt code.

Presumably dup_mmap() -> arch_dup_mmap() -> ldt_dup_context()?

But I don't think that interacts with the init_mm nested lock stuff we're
looking at here, as it's not touching init_mm and in any case ptdump is
unconditionally taking the mm's mmap write lock as-is without issue.

Cheers, Lorenzo

Re: [PATCH] x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF

[Mike Rapoport]

On Fri, Jul 10, 2026 at 12:56:40PM +0100, Lorenzo Stoakes wrote:
> x86 implements page attribute modification using its Change Page
> Attributes (CPA) mechanism.
> 
> This tracks properties of ranges such as cache mode through x86 page
> attributes, and as part of that logic manipulates kernel page tables.
> 
> Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
> fragmentation") ranges of kernel page table entries can be collapsed into
> huge page table entries as part of this logic.
> 
> As part of this collapse, it frees the page tables which the collapsed
> entries previously pointed to, and it does so without any relevant locks
> being held to preclude concurrent kernel page table walkers.
> 
> The only way this code can be reached is if CPA_COLLAPSE is specified, and
> this is only set in set_memory_rox() via:
> 
> set_memory_rox()
> -> change_page_attr_set_clr()
> -> cpa_flush()
> -> cpa_collapse_large_pages()
> 
> Notable users of this are execmem and bpf when manipulating executable
> mappings.
> 
> However, this is problematic for ptdump, as it walks ranges it does not own
> and thus runs the risk of a use-after-free on page tables freed underneath
> it.
> 
> This patch resolves the issue by acquiring the mmap read lock on init_mm to
> provide mutual exclusion against ptdump, which acquires the init_mm write
> lock.
> 
> It is safe to acquire a sleeping lock as all the callers invoke
> set_memory_rox() from process context and in any case,
> change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
> mutex, disallowing atomic context here.
> 
> We also include cleanup.h in order to use a scoped_guard() to implement
> this cleanly.

This is a part of another patch, isn't it?
 
> Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
> Cc: stable@vger.kernel.org
> Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>

Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>

> ---
>  arch/x86/mm/pat/set_memory.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)

-- 
Sincerely yours,
Mike.