[PATCH 00/22] Simplify special kernel page table handling

[Kevin Brodsky <kevin.brodsky@arm.com>]

Page table constructors and destructors were originally introduced
specifically for user PTE pages. Over time, their use became more
widespread, and starting from v6.16 they are called at all levels
and for both user and kernel page tables [1]. Or so I thought. Turns
out, we still have various special kernel page tables where the
ctor/dtor are not called; vmemmap page tables is one such case.

Looking closer, we do not actually need per-level ctors for kernel
page tables. In fact we do not need ctors for them at all, because
nothing is specific to kernel page tables in the ctor/dtor and
nowadays they should be allocated with pagetable_alloc().

Here's the plan: all the generic page table handling moves to
pagetable_{alloc,free}(). Whatever is left is specific to user page
tables; the ctor/dtor are only called for user page tables at the
required levels (PTE/PMD), and renamed to make that clear.

Doing this will allow us to remove a good amount of special casing,
but there's quite a bit of churn to get there. Here's what this
series does:

1. Introduce the concept of "kernel mm" and replace most comparisons
   with &init_mm with a flag check (patch 1-12); no functional change
   so far.

2. Move the page table accounting/type setting to pagetable_{alloc,free}
   (patch 13-15). Kernel mm's do not need to call the ctor/dtor any
   more.

3. Mark special mm's (efi_mm, tboot_mm on x86) as kernel mm's (patch
   16-19). They now abide by the same rules as init_mm, which means
   in particular that split page table locks are no longer used.

4. Remove the ctor/dtor calls (now no-ops) for special kernel page
   tables on arm/arm64/riscv (patch 20-22).

[1] https://lore.kernel.org/all/20250408095222.860601-1-kevin.brodsky@arm.com/

---
I stopped there for part 1 to keep it digestible. Part 2 will then
ensure that the ctor/dtor are only called for user PTE/PMD pages and
rename them accordingly. This is comparatively simpler but also
involves a lot of churn.

This series was suggested in reaction to [2]; removing dtor calls
entirely for kernel page tables systematically solves such issues.

Worth noting that some comparisons against &init_mm have been left
untouched, as they are truly specific to init_mm. Reviewers are
encouraged to take a look at the remaining cases:

  $ git grep -nE '([=!]= *&init_mm|&init_mm *[=!]=)'
  Documentation/mm/active_mm.rst:78:      if (current->mm == &init_mm)
  Documentation/translations/zh_CN/mm/active_mm.rst:76:   if (current->mm == &init_mm)
  arch/arm/include/asm/mmu_context.h:144: if (mm != &init_mm)
  arch/arm64/include/asm/mmu_context.h:100:       if (mm != &init_mm && !system_uses_ttbr0_pan())
  arch/arm64/include/asm/mmu_context.h:210:       if (mm == &init_mm)
  arch/arm64/include/asm/mmu_context.h:242:       if (next == &init_mm) {
  arch/arm64/mm/fault.c:121:      if (mm == &init_mm)
  arch/arm64/mm/fault.c:139:              if (mm == &init_mm) {
  arch/arm64/mm/fault.c:154:               mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K,
  arch/arm64/mm/mmu.c:997:        BUG_ON(mm == &init_mm);
  arch/powerpc/xmon/xmon.c:3314:  if (mm == &init_mm)
  arch/riscv/mm/fault.c:42:                mm == &init_mm ? (u64)__pa_symbol(mm->pgd) : virt_to_phys(mm->pgd));
  kernel/cpu.c:917:       WARN_ON(mm != &init_mm);
  kernel/fork.c:725:      BUG_ON(mm == &init_mm);
  kernel/sched/core.c:8390:       if (mm != &init_mm) {
  mm/ptdump.c:182:        if (mm != &init_mm)
  mm/ptdump.c:191:        if (mm != &init_mm)

Testing:

* Built all modified architectures
* arm64
    - mm kselftests (FVP)
    - Hot adding/removing memory + memory-hotplug kselftests (QEMU)
    - Reading /sys/kernel/debug/{efi,kernel}_page_tables
    - Reading from /sys/firmware/efi/efivars/
*riscv
    - memory-hotplug kselftests (QEMU)
* x86
    - mm kselftests (lkvm)
    - tboot page table setup using a fake tboot shared page (QEMU)
    - CONFIG_EFI_PGT_DUMP=y (QEMU)

Patches based on mm-unstable.

Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>

[2] https://lore.kernel.org/all/20260521032730.2104017-1-apopple@nvidia.com/

---

To: linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>

Cc: Russell King <linux@armlinux.org.uk>

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: linux-arm-kernel@lists.infradead.org

Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: loongarch@lists.linux.dev

Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: linux-parisc@vger.kernel.org

Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: linuxppc-dev@lists.ozlabs.org

Cc: Paul Walmsley <pjw@kernel.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: linux-riscv@lists.infradead.org

Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: linux-s390@vger.kernel.org

Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andreas Larsson <andreas@gaisler.com>
Cc: sparclinux@vger.kernel.org

Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: Johannes Berg <johannes@sipsolutions.net>
Cc: linux-um@lists.infradead.org

Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ning Sun <ning.sun@intel.com>
Cc: x86@kernel.org
Cc: tboot-devel@lists.sourceforge.net

Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Cc: linux-efi@vger.kernel.org

Cc: Vishal Moola <vishal.moola@gmail.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>

Cc: linux-kernel@vger.kernel.org
Cc: linux-arch@vger.kernel.org

---
Kevin Brodsky (22):
      mm: drop unused __mm_flags_set_mask_bits_word()
      mm: move mm_flags helpers to mm_types.h
      mm: introduce MMF_KERNEL flag and set it for init_mm
      mm: use mm_is_kernel() in generic page table code
      arm64: mm: use mm_is_kernel() for kernel mm checks
      loongarch: mm: use mm_is_kernel() in switch_mm_irqs_off()
      parisc: mm: use mm_is_kernel() for kernel mm checks
      powerpc: mm: use mm_is_kernel() for kernel mm checks
      s390: mm: use mm_is_kernel() for kernel mm checks
      sparc: mm: use mm_is_kernel() for kernel mm checks
      um: mm: use mm_is_kernel() in TLB sync
      x86/mm: use mm_is_kernel() for kernel mm checks
      mm: account page table pages when allocated
      mm: set page table page type when allocated
      mm: only initialise pt_share_count for user pgtables
      efi: mark efi_mm as a kernel mm
      mm: pagewalk: drop redundant address check for kernel mm walks
      arm64: mm: drop explicit mm_is_efi() check in contpte
      x86/tboot: mark tboot_mm as a kernel mm
      arm64: mm: drop ctor/dtor calls for kernel page tables
      arm: mm: drop ctor call for kernel page tables
      riscv: mm: drop ctor/dtor calls for kernel page tables

 arch/arm/mm/mmu.c                        |  3 +-
 arch/arm64/include/asm/pgalloc.h         | 10 ++--
 arch/arm64/mm/contpte.c                  | 26 ++++------
 arch/arm64/mm/hugetlbpage.c              |  2 +-
 arch/arm64/mm/mmu.c                      | 82 +++++++++-----------------------
 arch/loongarch/include/asm/mmu_context.h |  2 +-
 arch/parisc/include/asm/mmu_context.h    |  2 +-
 arch/parisc/include/asm/pgalloc.h        |  2 +-
 arch/parisc/include/asm/tlbflush.h       |  2 +-
 arch/powerpc/include/asm/pgalloc.h       |  2 +-
 arch/powerpc/mm/book3s64/pgtable.c       |  2 +-
 arch/powerpc/mm/mmu_context.c            |  2 +-
 arch/powerpc/mm/pgtable.c                |  2 +-
 arch/riscv/mm/init.c                     | 63 ++++++------------------
 arch/s390/include/asm/mmu_context.h      |  2 +-
 arch/s390/include/asm/pgtable.h          |  6 +--
 arch/s390/mm/pgalloc.c                   |  4 +-
 arch/sparc/include/asm/mmu_context_64.h  |  2 +-
 arch/sparc/include/asm/pgtable_64.h      |  6 +--
 arch/sparc/mm/init_64.c                  |  2 +-
 arch/sparc/mm/tlb.c                      |  2 +-
 arch/um/kernel/tlb.c                     |  2 +-
 arch/x86/include/asm/tlbflush.h          |  8 ++--
 arch/x86/kernel/tboot.c                  |  4 +-
 arch/x86/mm/pgtable.c                    |  2 +-
 arch/x86/mm/tlb.c                        |  9 ++--
 drivers/firmware/efi/efi.c               |  1 +
 include/asm-generic/pgalloc.h            | 16 +++----
 include/linux/mm.h                       | 69 +++++++--------------------
 include/linux/mm_types.h                 | 52 +++++++++++++++-----
 mm/init-mm.c                             |  1 +
 mm/memory.c                              |  6 +--
 mm/page_table_check.c                    | 14 +++---
 mm/pagewalk.c                            |  7 +--
 34 files changed, 163 insertions(+), 254 deletions(-)
---
base-commit: a531f0cf705af3374388eafe4115f651c625d8db
change-id: 20260611-remove_pgtable_cdtor-3b123938f8cd