[PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Lorenzo Stoakes (ARM)]

When splitting a large page in CPA in __split_large_page() we allocate a
PTE directly without going through the standard page table allocation
routines such as pte_alloc_one_kernel().

This means the page table constructor is never called nor is the page table
marked as a kernel page table.

The former results in the folio associated with the page table not being
marked as a page table (__pagetable_ctor() is never called thus neither is
__folio_set_pgtable()) nor are statistics updated to reflect
it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page
table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
tables") kernel page table freeing has been batched and since the
subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
for kernel address space") IOTLB cache entries for kernel page tables have
been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can
contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor,
and thus requires a page table destructor.

Since we cannot assume one is always present (early allocated direct map
page tables are not marked as such), we conditionally call
pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables,
which now includes split page tables, take the correct route through
pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear
in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit
5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
choose this as the Fixes target.

Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
Cc: stable@vger.kernel.org
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
---
 arch/x86/mm/pat/set_memory.c | 21 ++++++++++++---------
 1 file changed, 12 insertions(+), 9 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 301fb9e77d91..a67ca33b9dd1 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -439,7 +439,11 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+
+		if (folio_test_pgtable(ptdesc_folio(ptdesc)))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1138,11 +1142,10 @@ static void split_set_pte(struct cpa_data *cpa, pte_t *pte, unsigned long pfn,
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1246,18 +1249,18 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	if (!debug_pagealloc_enabled())
 		spin_unlock(&cpa_lock);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (!debug_pagealloc_enabled())
 		spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }

---
base-commit: 890f8c4e827c918dac668a12eaf63180ba8a9e6d
change-id: 20260720-fix-cpa-kernel-pagetables-e641bd41c281

Cheers,
-- 
Lorenzo Stoakes (ARM) <ljs@kernel.org>

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Vishal Moola]

On Mon, Jul 20, 2026 at 10:27:29AM +0100, Lorenzo Stoakes (ARM) wrote:
> When splitting a large page in CPA in __split_large_page() we allocate a
> PTE directly without going through the standard page table allocation
> routines such as pte_alloc_one_kernel().
> 
> This means the page table constructor is never called nor is the page table
> marked as a kernel page table.
> 
> The former results in the folio associated with the page table not being
> marked as a page table (__pagetable_ctor() is never called thus neither is
> __folio_set_pgtable()) nor are statistics updated to reflect
> it (lruvec_stat_add_folio() is never called).
> 
> The latter issue of failing to mark the page table as a kernel page
> table (ptdesc_set_kernel() is never called) is far more problematic.
> 
> Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
> tables") kernel page table freeing has been batched and since the
> subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
> for kernel address space") IOTLB cache entries for kernel page tables have
> been invalidated upon being freed.
> 
> Since split page tables are freed without this invalidation, the IOTLB can
> contain stale entries for them.
> 
> Resolve the issue by using the ordinary PTE allocation API at split time.
> 
> This results in these kernel page tables invoking a page table constructor,
> and thus requires a page table destructor.
> 
> Since we cannot assume one is always present (early allocated direct map
> page tables are not marked as such), we conditionally call
> pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
> otherwise we free the page table via pagetable_free().
> 
> Regardless of which path is taken page tables marked as kernel page tables,
> which now includes split page tables, take the correct route through
> pagetable_free_kernel().
> 
> There is a user-visible side effect in that split page tables will appear
> in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
> allocated after early boot), however this is a positive change.
> 
> This issue started being markedly problematic after commit
> 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
> choose this as the Fixes target.
> 
> Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
> Cc: stable@vger.kernel.org
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> ---
>  arch/x86/mm/pat/set_memory.c | 21 ++++++++++++---------
>  1 file changed, 12 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index 301fb9e77d91..a67ca33b9dd1 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -439,7 +439,11 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
>  
>  	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
>  		list_del(&ptdesc->pt_list);
> -		pagetable_free(ptdesc);
> +
> +		if (folio_test_pgtable(ptdesc_folio(ptdesc)))
> +			pagetable_dtor_free(ptdesc);
> +		else
> +			pagetable_free(ptdesc);

Lets not introduce more folio-ptdesc crossovers, we're trying to get
rid of them :)

I believe pagetable_dtor_free() should do what you're looking for on its
own anyway.

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Vishal Moola]

On Mon, Jul 20, 2026 at 01:01:00PM -0700, Vishal Moola wrote:
> On Mon, Jul 20, 2026 at 10:27:29AM +0100, Lorenzo Stoakes (ARM) wrote:
> > When splitting a large page in CPA in __split_large_page() we allocate a
> > PTE directly without going through the standard page table allocation
> > routines such as pte_alloc_one_kernel().
> > 
> > This means the page table constructor is never called nor is the page table
> > marked as a kernel page table.
> > 
> > The former results in the folio associated with the page table not being
> > marked as a page table (__pagetable_ctor() is never called thus neither is
> > __folio_set_pgtable()) nor are statistics updated to reflect
> > it (lruvec_stat_add_folio() is never called).
> > 
> > The latter issue of failing to mark the page table as a kernel page
> > table (ptdesc_set_kernel() is never called) is far more problematic.
> > 
> > Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
> > tables") kernel page table freeing has been batched and since the
> > subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
> > for kernel address space") IOTLB cache entries for kernel page tables have
> > been invalidated upon being freed.
> > 
> > Since split page tables are freed without this invalidation, the IOTLB can
> > contain stale entries for them.
> > 
> > Resolve the issue by using the ordinary PTE allocation API at split time.
> > 
> > This results in these kernel page tables invoking a page table constructor,
> > and thus requires a page table destructor.
> > 
> > Since we cannot assume one is always present (early allocated direct map
> > page tables are not marked as such), we conditionally call
> > pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
> > otherwise we free the page table via pagetable_free().
> > 
> > Regardless of which path is taken page tables marked as kernel page tables,
> > which now includes split page tables, take the correct route through
> > pagetable_free_kernel().
> > 
> > There is a user-visible side effect in that split page tables will appear
> > in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
> > allocated after early boot), however this is a positive change.
> > 
> > This issue started being markedly problematic after commit
> > 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
> > choose this as the Fixes target.
> > 
> > Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > ---
> >  arch/x86/mm/pat/set_memory.c | 21 ++++++++++++---------
> >  1 file changed, 12 insertions(+), 9 deletions(-)
> > 
> > diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> > index 301fb9e77d91..a67ca33b9dd1 100644
> > --- a/arch/x86/mm/pat/set_memory.c
> > +++ b/arch/x86/mm/pat/set_memory.c
> > @@ -439,7 +439,11 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
> >  
> >  	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
> >  		list_del(&ptdesc->pt_list);
> > -		pagetable_free(ptdesc);
> > +
> > +		if (folio_test_pgtable(ptdesc_folio(ptdesc)))
> > +			pagetable_dtor_free(ptdesc);
> > +		else
> > +			pagetable_free(ptdesc);
> 
> Lets not introduce more folio-ptdesc crossovers, we're trying to get
> rid of them :)
> 
> I believe pagetable_dtor_free() should do what you're looking for on its
> own anyway.

Actually, looking at it closer, maybe not because of the conditional
portion? But that makes me think it might be better to just replace the
ptdesc_clear_kernel() with pagetable_dtor() in the free function...

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Andrew Morton]

On Mon, 20 Jul 2026 10:27:29 +0100 "Lorenzo Stoakes (ARM)" <ljs@kernel.org> wrote:

> When splitting a large page in CPA in __split_large_page() we allocate a
> PTE directly without going through the standard page table allocation
> routines such as pte_alloc_one_kernel().
> 
> This means the page table constructor is never called nor is the page table
> marked as a kernel page table.

Thanks, I'll await x86 review on this (Dave?).  Also, Sashiko might
have found a pre-existing thing:

	https://sashiko.dev/#/patchset/20260720-fix-cpa-kernel-pagetables-v1-1-0766e782cefe@kernel.org

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Lorenzo Stoakes (ARM)]

On Mon, Jul 20, 2026 at 01:03:57PM -0700, Vishal Moola wrote:
> On Mon, Jul 20, 2026 at 01:01:00PM -0700, Vishal Moola wrote:
> > On Mon, Jul 20, 2026 at 10:27:29AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > When splitting a large page in CPA in __split_large_page() we allocate a
> > > PTE directly without going through the standard page table allocation
> > > routines such as pte_alloc_one_kernel().
> > >
> > > This means the page table constructor is never called nor is the page table
> > > marked as a kernel page table.
> > >
> > > The former results in the folio associated with the page table not being
> > > marked as a page table (__pagetable_ctor() is never called thus neither is
> > > __folio_set_pgtable()) nor are statistics updated to reflect
> > > it (lruvec_stat_add_folio() is never called).
> > >
> > > The latter issue of failing to mark the page table as a kernel page
> > > table (ptdesc_set_kernel() is never called) is far more problematic.
> > >
> > > Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
> > > tables") kernel page table freeing has been batched and since the
> > > subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
> > > for kernel address space") IOTLB cache entries for kernel page tables have
> > > been invalidated upon being freed.
> > >
> > > Since split page tables are freed without this invalidation, the IOTLB can
> > > contain stale entries for them.
> > >
> > > Resolve the issue by using the ordinary PTE allocation API at split time.
> > >
> > > This results in these kernel page tables invoking a page table constructor,
> > > and thus requires a page table destructor.
> > >
> > > Since we cannot assume one is always present (early allocated direct map
> > > page tables are not marked as such), we conditionally call
> > > pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
> > > otherwise we free the page table via pagetable_free().
> > >
> > > Regardless of which path is taken page tables marked as kernel page tables,
> > > which now includes split page tables, take the correct route through
> > > pagetable_free_kernel().
> > >
> > > There is a user-visible side effect in that split page tables will appear
> > > in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
> > > allocated after early boot), however this is a positive change.
> > >
> > > This issue started being markedly problematic after commit
> > > 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
> > > choose this as the Fixes target.
> > >
> > > Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
> > > Cc: stable@vger.kernel.org
> > > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > > ---
> > >  arch/x86/mm/pat/set_memory.c | 21 ++++++++++++---------
> > >  1 file changed, 12 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> > > index 301fb9e77d91..a67ca33b9dd1 100644
> > > --- a/arch/x86/mm/pat/set_memory.c
> > > +++ b/arch/x86/mm/pat/set_memory.c
> > > @@ -439,7 +439,11 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
> > >
> > >  	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
> > >  		list_del(&ptdesc->pt_list);
> > > -		pagetable_free(ptdesc);
> > > +
> > > +		if (folio_test_pgtable(ptdesc_folio(ptdesc)))
> > > +			pagetable_dtor_free(ptdesc);
> > > +		else
> > > +			pagetable_free(ptdesc);
> >
> > Lets not introduce more folio-ptdesc crossovers, we're trying to get
> > rid of them :)
> >
> > I believe pagetable_dtor_free() should do what you're looking for on its
> > own anyway.
>
> Actually, looking at it closer, maybe not because of the conditional
> portion? But that makes me think it might be better to just replace the
> ptdesc_clear_kernel() with pagetable_dtor() in the free function...

Well some kernel page tables are still allocated without ctor (early allocated
direct map for isntance), and if you did pagetable_dtor_free() it
unconditionally calls pagetable_dtor().

The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
assigned for kernel page table, and if PG_table never set clearing it is a noop)
but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
nr_page_table_pages.

It sucks, but until everything is updated to call the ctor we have to do it this
way :>)

Cheers, Lorenzo

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Vishal Moola]

On Tue, Jul 21, 2026 at 08:43:51AM +0100, Lorenzo Stoakes (ARM) wrote:
> On Mon, Jul 20, 2026 at 01:03:57PM -0700, Vishal Moola wrote:
> > On Mon, Jul 20, 2026 at 01:01:00PM -0700, Vishal Moola wrote:
> > > On Mon, Jul 20, 2026 at 10:27:29AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > > When splitting a large page in CPA in __split_large_page() we allocate a
> > > > PTE directly without going through the standard page table allocation
> > > > routines such as pte_alloc_one_kernel().
> > > >
> > > > This means the page table constructor is never called nor is the page table
> > > > marked as a kernel page table.
> > > >
> > > > The former results in the folio associated with the page table not being
> > > > marked as a page table (__pagetable_ctor() is never called thus neither is
> > > > __folio_set_pgtable()) nor are statistics updated to reflect
> > > > it (lruvec_stat_add_folio() is never called).
> > > >
> > > > The latter issue of failing to mark the page table as a kernel page
> > > > table (ptdesc_set_kernel() is never called) is far more problematic.
> > > >
> > > > Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
> > > > tables") kernel page table freeing has been batched and since the
> > > > subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
> > > > for kernel address space") IOTLB cache entries for kernel page tables have
> > > > been invalidated upon being freed.
> > > >
> > > > Since split page tables are freed without this invalidation, the IOTLB can
> > > > contain stale entries for them.
> > > >
> > > > Resolve the issue by using the ordinary PTE allocation API at split time.
> > > >
> > > > This results in these kernel page tables invoking a page table constructor,
> > > > and thus requires a page table destructor.
> > > >
> > > > Since we cannot assume one is always present (early allocated direct map
> > > > page tables are not marked as such), we conditionally call
> > > > pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
> > > > otherwise we free the page table via pagetable_free().
> > > >
> > > > Regardless of which path is taken page tables marked as kernel page tables,
> > > > which now includes split page tables, take the correct route through
> > > > pagetable_free_kernel().
> > > >
> > > > There is a user-visible side effect in that split page tables will appear
> > > > in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
> > > > allocated after early boot), however this is a positive change.
> > > >
> > > > This issue started being markedly problematic after commit
> > > > 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
> > > > choose this as the Fixes target.
> > > >
> > > > Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
> > > > Cc: stable@vger.kernel.org
> > > > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > > > ---
> > > >  arch/x86/mm/pat/set_memory.c | 21 ++++++++++++---------
> > > >  1 file changed, 12 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> > > > index 301fb9e77d91..a67ca33b9dd1 100644
> > > > --- a/arch/x86/mm/pat/set_memory.c
> > > > +++ b/arch/x86/mm/pat/set_memory.c
> > > > @@ -439,7 +439,11 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
> > > >
> > > >  	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
> > > >  		list_del(&ptdesc->pt_list);
> > > > -		pagetable_free(ptdesc);
> > > > +
> > > > +		if (folio_test_pgtable(ptdesc_folio(ptdesc)))
> > > > +			pagetable_dtor_free(ptdesc);
> > > > +		else
> > > > +			pagetable_free(ptdesc);
> > >
> > > Lets not introduce more folio-ptdesc crossovers, we're trying to get
> > > rid of them :)
> > >
> > > I believe pagetable_dtor_free() should do what you're looking for on its
> > > own anyway.
> >
> > Actually, looking at it closer, maybe not because of the conditional
> > portion? But that makes me think it might be better to just replace the
> > ptdesc_clear_kernel() with pagetable_dtor() in the free function...
> 
> Well some kernel page tables are still allocated without ctor (early allocated
> direct map for isntance), and if you did pagetable_dtor_free() it
> unconditionally calls pagetable_dtor().
> 
> The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> assigned for kernel page table, and if PG_table never set clearing it is a noop)
> but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> nr_page_table_pages.

Gotcha, thanks for the explanation :)

> It sucks, but until everything is updated to call the ctor we have to do it this
> way :>)

Yeah that makes sense. Although I'd rather see the condition as:
if(PageTable(ptdesc_page(...)))

We really shouldn't be calling ptdesc_folio() anywhere anymore.

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Lorenzo Stoakes (ARM)]

On Tue, Jul 21, 2026 at 02:45:43AM -0700, Vishal Moola wrote:
> On Tue, Jul 21, 2026 at 08:43:51AM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Mon, Jul 20, 2026 at 01:03:57PM -0700, Vishal Moola wrote:
> > > On Mon, Jul 20, 2026 at 01:01:00PM -0700, Vishal Moola wrote:
> > > > On Mon, Jul 20, 2026 at 10:27:29AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > > > When splitting a large page in CPA in __split_large_page() we allocate a
> > > > > PTE directly without going through the standard page table allocation
> > > > > routines such as pte_alloc_one_kernel().
> > > > >
> > > > > This means the page table constructor is never called nor is the page table
> > > > > marked as a kernel page table.
> > > > >
> > > > > The former results in the folio associated with the page table not being
> > > > > marked as a page table (__pagetable_ctor() is never called thus neither is
> > > > > __folio_set_pgtable()) nor are statistics updated to reflect
> > > > > it (lruvec_stat_add_folio() is never called).
> > > > >
> > > > > The latter issue of failing to mark the page table as a kernel page
> > > > > table (ptdesc_set_kernel() is never called) is far more problematic.
> > > > >
> > > > > Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
> > > > > tables") kernel page table freeing has been batched and since the
> > > > > subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
> > > > > for kernel address space") IOTLB cache entries for kernel page tables have
> > > > > been invalidated upon being freed.
> > > > >
> > > > > Since split page tables are freed without this invalidation, the IOTLB can
> > > > > contain stale entries for them.
> > > > >
> > > > > Resolve the issue by using the ordinary PTE allocation API at split time.
> > > > >
> > > > > This results in these kernel page tables invoking a page table constructor,
> > > > > and thus requires a page table destructor.
> > > > >
> > > > > Since we cannot assume one is always present (early allocated direct map
> > > > > page tables are not marked as such), we conditionally call
> > > > > pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
> > > > > otherwise we free the page table via pagetable_free().
> > > > >
> > > > > Regardless of which path is taken page tables marked as kernel page tables,
> > > > > which now includes split page tables, take the correct route through
> > > > > pagetable_free_kernel().
> > > > >
> > > > > There is a user-visible side effect in that split page tables will appear
> > > > > in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
> > > > > allocated after early boot), however this is a positive change.
> > > > >
> > > > > This issue started being markedly problematic after commit
> > > > > 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
> > > > > choose this as the Fixes target.
> > > > >
> > > > > Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
> > > > > Cc: stable@vger.kernel.org
> > > > > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > > > > ---
> > > > >  arch/x86/mm/pat/set_memory.c | 21 ++++++++++++---------
> > > > >  1 file changed, 12 insertions(+), 9 deletions(-)
> > > > >
> > > > > diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> > > > > index 301fb9e77d91..a67ca33b9dd1 100644
> > > > > --- a/arch/x86/mm/pat/set_memory.c
> > > > > +++ b/arch/x86/mm/pat/set_memory.c
> > > > > @@ -439,7 +439,11 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
> > > > >
> > > > >  	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
> > > > >  		list_del(&ptdesc->pt_list);
> > > > > -		pagetable_free(ptdesc);
> > > > > +
> > > > > +		if (folio_test_pgtable(ptdesc_folio(ptdesc)))
> > > > > +			pagetable_dtor_free(ptdesc);
> > > > > +		else
> > > > > +			pagetable_free(ptdesc);
> > > >
> > > > Lets not introduce more folio-ptdesc crossovers, we're trying to get
> > > > rid of them :)
> > > >
> > > > I believe pagetable_dtor_free() should do what you're looking for on its
> > > > own anyway.
> > >
> > > Actually, looking at it closer, maybe not because of the conditional
> > > portion? But that makes me think it might be better to just replace the
> > > ptdesc_clear_kernel() with pagetable_dtor() in the free function...
> >
> > Well some kernel page tables are still allocated without ctor (early allocated
> > direct map for isntance), and if you did pagetable_dtor_free() it
> > unconditionally calls pagetable_dtor().
> >
> > The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> > assigned for kernel page table, and if PG_table never set clearing it is a noop)
> > but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> > nr_page_table_pages.
>
> Gotcha, thanks for the explanation :)

No worries, this is subtle stuff with lots of weird gotchas and stuff we need to
improve... I seem to have fallen down an unexpected rabbit hole with these fixes
:)

>
> > It sucks, but until everything is updated to call the ctor we have to do it this
> > way :>)
>
> Yeah that makes sense. Although I'd rather see the condition as:
> if(PageTable(ptdesc_page(...)))
>
> We really shouldn't be calling ptdesc_folio() anywhere anymore.

I think better for a follow up since the code already uses ptdesc all over the
place (fundamental to the approach really, keeping a list of page tables etc.)
and this is a fix that needs backporting.

The follow up be something that you could look at if you have time? :>)

Cheers, Lorenzo

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Mike Rapoport]

On Tue, Jul 21, 2026 at 10:58:50AM +0100, Lorenzo Stoakes (ARM) wrote:
> On Tue, Jul 21, 2026 at 02:45:43AM -0700, Vishal Moola wrote:
> > >
> > > Well some kernel page tables are still allocated without ctor (early allocated
> > > direct map for isntance), and if you did pagetable_dtor_free() it
> > > unconditionally calls pagetable_dtor().

TBH, I cannot think of a scenario when page tables allocated at boot would
be collapsed. But surely, checking the page type is safer just in case.

> > > The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> > > assigned for kernel page table, and if PG_table never set clearing it is a noop)
> > > but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> > > nr_page_table_pages.
> >
> > Gotcha, thanks for the explanation :)
> 
> No worries, this is subtle stuff with lots of weird gotchas and stuff we need to
> improve... I seem to have fallen down an unexpected rabbit hole with these fixes
> :)
> 
> >
> > > It sucks, but until everything is updated to call the ctor we have to do it this
> > > way :>)
> >
> > Yeah that makes sense. Although I'd rather see the condition as:
> > if(PageTable(ptdesc_page(...)))
> >
> > We really shouldn't be calling ptdesc_folio() anywhere anymore.
> 
> I think better for a follow up since the code already uses ptdesc all over the
> place (fundamental to the approach really, keeping a list of page tables etc.)
> and this is a fix that needs backporting.

I agree with Vishal that it's better to use page type rather than folio
type. And it's the same for backporting ;-) 
 
> Cheers, Lorenzo

-- 
Sincerely yours,
Mike.

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Lorenzo Stoakes (ARM)]

On Tue, Jul 21, 2026 at 01:32:44PM +0300, Mike Rapoport wrote:
> On Tue, Jul 21, 2026 at 10:58:50AM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Tue, Jul 21, 2026 at 02:45:43AM -0700, Vishal Moola wrote:
> > > >
> > > > Well some kernel page tables are still allocated without ctor (early allocated
> > > > direct map for isntance), and if you did pagetable_dtor_free() it
> > > > unconditionally calls pagetable_dtor().
>
> TBH, I cannot think of a scenario when page tables allocated at boot would
> be collapsed. But surely, checking the page type is safer just in case.

Yeah nor can to be honest, anything that could be made large in the direct map
would already be large right?

But it's 'just in case' somebody did something dumb :) Later can maybe make it a
WARN_ON(). But just to fix the proximate issue for now.

>
> > > > The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> > > > assigned for kernel page table, and if PG_table never set clearing it is a noop)
> > > > but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> > > > nr_page_table_pages.
> > >
> > > Gotcha, thanks for the explanation :)
> >
> > No worries, this is subtle stuff with lots of weird gotchas and stuff we need to
> > improve... I seem to have fallen down an unexpected rabbit hole with these fixes
> > :)
> >
> > >
> > > > It sucks, but until everything is updated to call the ctor we have to do it this
> > > > way :>)
> > >
> > > Yeah that makes sense. Although I'd rather see the condition as:
> > > if(PageTable(ptdesc_page(...)))
> > >
> > > We really shouldn't be calling ptdesc_folio() anywhere anymore.
> >
> > I think better for a follow up since the code already uses ptdesc all over the
> > place (fundamental to the approach really, keeping a list of page tables etc.)
> > and this is a fix that needs backporting.
>
> I agree with Vishal that it's better to use page type rather than folio
> type. And it's the same for backporting ;-)

Ah sorry misunderstood, you mean straight up PageTable(ptdesc_page()), I thought
Vishal was saying we shouldn't be directly referencing ptdesc's at all (which
would be the rework).

I guess definitionally page tables are never folios. I lazily went with what I
saw elsewhere, my bad :)

Will respin!


>
> > Cheers, Lorenzo
>
> --
> Sincerely yours,
> Mike.

Cheers, Lorenzo

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Lorenzo Stoakes (ARM)]

On Tue, Jul 21, 2026 at 01:00:53PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Tue, Jul 21, 2026 at 01:32:44PM +0300, Mike Rapoport wrote:
> > On Tue, Jul 21, 2026 at 10:58:50AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > On Tue, Jul 21, 2026 at 02:45:43AM -0700, Vishal Moola wrote:
> > > > >
> > > > > Well some kernel page tables are still allocated without ctor (early allocated
> > > > > direct map for isntance), and if you did pagetable_dtor_free() it
> > > > > unconditionally calls pagetable_dtor().
> >
> > TBH, I cannot think of a scenario when page tables allocated at boot would
> > be collapsed. But surely, checking the page type is safer just in case.
>
> Yeah nor can to be honest, anything that could be made large in the direct map
> would already be large right?
>
> But it's 'just in case' somebody did something dumb :) Later can maybe make it a
> WARN_ON(). But just to fix the proximate issue for now.
>
> >
> > > > > The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> > > > > assigned for kernel page table, and if PG_table never set clearing it is a noop)
> > > > > but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> > > > > nr_page_table_pages.
> > > >
> > > > Gotcha, thanks for the explanation :)
> > >
> > > No worries, this is subtle stuff with lots of weird gotchas and stuff we need to
> > > improve... I seem to have fallen down an unexpected rabbit hole with these fixes
> > > :)
> > >
> > > >
> > > > > It sucks, but until everything is updated to call the ctor we have to do it this
> > > > > way :>)
> > > >
> > > > Yeah that makes sense. Although I'd rather see the condition as:
> > > > if(PageTable(ptdesc_page(...)))
> > > >
> > > > We really shouldn't be calling ptdesc_folio() anywhere anymore.
> > >
> > > I think better for a follow up since the code already uses ptdesc all over the
> > > place (fundamental to the approach really, keeping a list of page tables etc.)
> > > and this is a fix that needs backporting.
> >
> > I agree with Vishal that it's better to use page type rather than folio
> > type. And it's the same for backporting ;-)
>
> Ah sorry misunderstood, you mean straight up PageTable(ptdesc_page()), I thought
> Vishal was saying we shouldn't be directly referencing ptdesc's at all (which
> would be the rework).
>
> I guess definitionally page tables are never folios. I lazily went with what I
> saw elsewhere, my bad :)

Ah yeah I remember now, i saw __pagetable_ctor() dealt with folios:

static inline void __pagetable_ctor(struct ptdesc *ptdesc)
{
	struct folio *folio = ptdesc_folio(ptdesc);

	__folio_set_pgtable(folio);
	lruvec_stat_add_folio(folio, NR_PAGETABLE);
}


And was like 'huh?' (surely definitionally they're _not_ folios) but went with
that on that basis.

Another place to clean up I guess? (that one _definitely_ is a follow up though
;)

>
> Will respin!
>
>
> >
> > > Cheers, Lorenzo
> >
> > --
> > Sincerely yours,
> > Mike.
>
> Cheers, Lorenzo

Cheers, Lorenzo

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Mike Rapoport]

On Tue, Jul 21, 2026 at 01:09:27PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Tue, Jul 21, 2026 at 01:00:53PM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Tue, Jul 21, 2026 at 01:32:44PM +0300, Mike Rapoport wrote:
> > > On Tue, Jul 21, 2026 at 10:58:50AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > > On Tue, Jul 21, 2026 at 02:45:43AM -0700, Vishal Moola wrote:
> > > > > >
> > > > > > Well some kernel page tables are still allocated without ctor (early allocated
> > > > > > direct map for isntance), and if you did pagetable_dtor_free() it
> > > > > > unconditionally calls pagetable_dtor().
> > >
> > > TBH, I cannot think of a scenario when page tables allocated at boot would
> > > be collapsed. But surely, checking the page type is safer just in case.
> >
> > Yeah nor can to be honest, anything that could be made large in the direct map
> > would already be large right?
> >
> > But it's 'just in case' somebody did something dumb :) Later can maybe make it a
> > WARN_ON(). But just to fix the proximate issue for now.
> >
> > >
> > > > > > The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> > > > > > assigned for kernel page table, and if PG_table never set clearing it is a noop)
> > > > > > but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> > > > > > nr_page_table_pages.
> > > > >
> > > > > Gotcha, thanks for the explanation :)
> > > >
> > > > No worries, this is subtle stuff with lots of weird gotchas and stuff we need to
> > > > improve... I seem to have fallen down an unexpected rabbit hole with these fixes
> > > > :)
> > > >
> > > > >
> > > > > > It sucks, but until everything is updated to call the ctor we have to do it this
> > > > > > way :>)
> > > > >
> > > > > Yeah that makes sense. Although I'd rather see the condition as:
> > > > > if(PageTable(ptdesc_page(...)))
> > > > >
> > > > > We really shouldn't be calling ptdesc_folio() anywhere anymore.
> > > >
> > > > I think better for a follow up since the code already uses ptdesc all over the
> > > > place (fundamental to the approach really, keeping a list of page tables etc.)
> > > > and this is a fix that needs backporting.
> > >
> > > I agree with Vishal that it's better to use page type rather than folio
> > > type. And it's the same for backporting ;-)
> >
> > Ah sorry misunderstood, you mean straight up PageTable(ptdesc_page()), I thought
> > Vishal was saying we shouldn't be directly referencing ptdesc's at all (which
> > would be the rework).
> >
> > I guess definitionally page tables are never folios. I lazily went with what I
> > saw elsewhere, my bad :)
> 
> Ah yeah I remember now, i saw __pagetable_ctor() dealt with folios:
> 
> static inline void __pagetable_ctor(struct ptdesc *ptdesc)
> {
> 	struct folio *folio = ptdesc_folio(ptdesc);
> 
> 	__folio_set_pgtable(folio);
> 	lruvec_stat_add_folio(folio, NR_PAGETABLE);
> }
> 
> 
> And was like 'huh?' (surely definitionally they're _not_ folios) but went with
> that on that basis.

I wonder why setting the type is even in ctor rather than in allocation.
 
> Another place to clean up I guess? (that one _definitely_ is a follow up though
> ;)

Yep :)
 
> Cheers, Lorenzo

-- 
Sincerely yours,
Mike.

Re: [PATCH] x86/mm/pat: allocate split page tables as kernel page tables

[Vishal Moola]

On Tue, Jul 21, 2026 at 04:51:31PM +0300, Mike Rapoport wrote:
> On Tue, Jul 21, 2026 at 01:09:27PM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Tue, Jul 21, 2026 at 01:00:53PM +0100, Lorenzo Stoakes (ARM) wrote:
> > > On Tue, Jul 21, 2026 at 01:32:44PM +0300, Mike Rapoport wrote:
> > > > On Tue, Jul 21, 2026 at 10:58:50AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > > > On Tue, Jul 21, 2026 at 02:45:43AM -0700, Vishal Moola wrote:
> > > > > > >
> > > > > > > Well some kernel page tables are still allocated without ctor (early allocated
> > > > > > > direct map for isntance), and if you did pagetable_dtor_free() it
> > > > > > > unconditionally calls pagetable_dtor().
> > > >
> > > > TBH, I cannot think of a scenario when page tables allocated at boot would
> > > > be collapsed. But surely, checking the page type is safer just in case.
> > >
> > > Yeah nor can to be honest, anything that could be made large in the direct map
> > > would already be large right?
> > >
> > > But it's 'just in case' somebody did something dumb :) Later can maybe make it a
> > > WARN_ON(). But just to fix the proximate issue for now.
> > >
> > > >
> > > > > > > The ptlock_free() and __folio_clear_pgtable() there would be harmelss (no locks
> > > > > > > assigned for kernel page table, and if PG_table never set clearing it is a noop)
> > > > > > > but the lruvec_stat_sub_folio() would cause an unbalanced decrement of
> > > > > > > nr_page_table_pages.
> > > > > >
> > > > > > Gotcha, thanks for the explanation :)
> > > > >
> > > > > No worries, this is subtle stuff with lots of weird gotchas and stuff we need to
> > > > > improve... I seem to have fallen down an unexpected rabbit hole with these fixes
> > > > > :)
> > > > >
> > > > > >
> > > > > > > It sucks, but until everything is updated to call the ctor we have to do it this
> > > > > > > way :>)
> > > > > >
> > > > > > Yeah that makes sense. Although I'd rather see the condition as:
> > > > > > if(PageTable(ptdesc_page(...)))
> > > > > >
> > > > > > We really shouldn't be calling ptdesc_folio() anywhere anymore.
> > > > >
> > > > > I think better for a follow up since the code already uses ptdesc all over the
> > > > > place (fundamental to the approach really, keeping a list of page tables etc.)
> > > > > and this is a fix that needs backporting.
> > > >
> > > > I agree with Vishal that it's better to use page type rather than folio
> > > > type. And it's the same for backporting ;-)
> > >
> > > Ah sorry misunderstood, you mean straight up PageTable(ptdesc_page()), I thought
> > > Vishal was saying we shouldn't be directly referencing ptdesc's at all (which
> > > would be the rework).
> > >
> > > I guess definitionally page tables are never folios. I lazily went with what I
> > > saw elsewhere, my bad :)
> > 
> > Ah yeah I remember now, i saw __pagetable_ctor() dealt with folios:
> > 
> > static inline void __pagetable_ctor(struct ptdesc *ptdesc)
> > {
> > 	struct folio *folio = ptdesc_folio(ptdesc);
> > 
> > 	__folio_set_pgtable(folio);
> > 	lruvec_stat_add_folio(folio, NR_PAGETABLE);
> > }
> > 
> > 
> > And was like 'huh?' (surely definitionally they're _not_ folios) but went with
> > that on that basis.
> 
> I wonder why setting the type is even in ctor rather than in allocation.

Yeah, you're not the only one wondering that ;)

Kevin is actively looking at moving those to the allocation/free site instead[1]!

> > Another place to clean up I guess? (that one _definitely_ is a follow up though
> > ;)
> 
> Yep :)

Yup. It makes most sense to clean those up when we add a per-memdesc
api (i.e. for memcg in this case). I haven't really had the time to work
on those though :/

> > Cheers, Lorenzo
> 
> -- 
> Sincerely yours,
> Mike.

[1] https://lore.kernel.org/linux-mm/20260714-remove_pgtable_cdtor-v1-14-44be8a7685d7@arm.com/