Re: [PATCH v6 00/14] Nesting support for lazy MMU mode

Yeoreum Yun
2025-12-15T16:53:55Z

> When the lazy MMU mode was introduced eons ago, it wasn't made clear
> whether such a sequence was legal:
>
> arch_enter_lazy_mmu_mode()
> ...
> arch_enter_lazy_mmu_mode()
> ...
> arch_leave_lazy_mmu_mode()
> ...
> arch_leave_lazy_mmu_mode()
>
> It seems fair to say that nested calls to
> arch_{enter,leave}_lazy_mmu_mode() were not expected, and most
> architectures never explicitly supported it.
>
> Nesting does in fact occur in certain configurations, and avoiding it
> has proved difficult. This series therefore enables lazy_mmu sections to
> nest, on all architectures.
>
> Nesting is handled using a counter in task_struct (patch 8), like other
> stateless APIs such as pagefault_{disable,enable}(). This is fully
> handled in a new generic layer in <linux/pgtable.h>; the arch_* API
> remains unchanged. A new pair of calls, lazy_mmu_mode_{pause,resume}(),
> is also introduced to allow functions that are called with the lazy MMU
> mode enabled to temporarily pause it, regardless of nesting.
>
> An arch now opts in to using the lazy MMU mode by selecting
> CONFIG_ARCH_LAZY_MMU; this is more appropriate now that we have a
> generic API, especially with state conditionally added to task_struct.
>
> ---
>
> Background: Ryan Roberts' series from March [1] attempted to prevent
> nesting from ever occurring, and mostly succeeded. Unfortunately, a
> corner case (DEBUG_PAGEALLOC) may still cause nesting to occur on arm64.
> Ryan proposed [2] to address that corner case at the generic level but
> this approach received pushback; [3] then attempted to solve the issue
> on arm64 only, but it was deemed too fragile.
>
> It feels generally difficult to guarantee that lazy_mmu sections don't
> nest, because callers of various standard mm functions do not know if
> the function uses lazy_mmu itself.
>
> The overall approach in v3/v4 is very close to what David Hildenbrand
> proposed on v2 [4].
>
> Unlike in v1/v2, no special provision is made for architectures to
> save/restore extra state when entering/leaving the mode. Based on the
> discussions so far, this does not seem to be required - an arch can
> store any relevant state in thread_struct during arch_enter() and
> restore it in arch_leave(). Nesting is not a concern as these functions
> are only called at the top level, not in nested sections.
>
> The introduction of a generic layer, and tracking of the lazy MMU state
> in task_struct, also allows to streamline the arch callbacks - this
> series removes 67 lines from arch/.
>
> Patch overview:
>
> * Patch 1: cleanup - avoids having to deal with the powerpc
> context-switching code
>
> * Patch 2-4: prepare arch_flush_lazy_mmu_mode() to be called from the
> generic layer (patch 9)
>
> * Patch 5: documentation clarification (not directly related to the
> changes in this series)
>
> * Patch 6-7: new API + CONFIG_ARCH_LAZY_MMU
>
> * Patch 8: ensure correctness in interrupt context
>
> * Patch 9: nesting support
>
> * Patch 10-13: replace arch-specific tracking of lazy MMU mode with
> generic API
>
> * Patch 14: basic tests to ensure that the state added in patch 9 is
> tracked correctly
>
> This series has been tested by running the mm kselftests on arm64 with
> DEBUG_VM, DEBUG_PAGEALLOC, KFENCE and KASAN. Extensive testing on
> powerpc was also kindly provided by Venkat Rao Bagalkote [5]. It was
> build-tested on other architectures (with and without XEN_PV on x86).
>
> - Kevin
>
> [1] https://lore.kernel.org/all/20250303141542.3371656-1-ryan.roberts@arm.com/
> [2] https://lore.kernel.org/all/20250530140446.2387131-1-ryan.roberts@arm.com/
> [3] https://lore.kernel.org/all/20250606135654.178300-1-ryan.roberts@arm.com/
> [4] https://lore.kernel.org/all/ef343405-c394-4763-a79f-21381f217b6c@redhat.com/
> [5] https://lore.kernel.org/all/94889730-1AEF-458F-B623-04092C0D6819@linux.ibm.com/
> ---
> Changelog
>
> v5..v6:
>
> - Rebased on v6.19-rc1
> - Overall: no functional change
> - Patch 5: new patch clarifying that generic code may not sleep while in lazy
> MMU mode [Alexander Gordeev]
> - Patch 6: added description for the ARCH_HAS_LAZY_MMU_MODE option
> [Anshuman Khandual]
> - Patch 9: rename in_lazy_mmu_mode() to is_lazy_mmu_mode_active() [Alexander]
> - Patch 14: new patch with basic KUnit tests [Anshuman]
> - Collected R-b/A-b/T-b tags
>
> v5: https://lore.kernel.org/all/20251124132228.622678-1-kevin.brodsky@arm.com/
>
> v4..v5:
>
> - Rebased on mm-unstable
> - Patch 3: added missing radix_enabled() check in arch_flush()
> [Ritesh Harjani]
> - Patch 6: declare arch_flush_lazy_mmu_mode() as static inline on x86
> [Ryan Roberts]
> - Patch 7 (formerly 12): moved before patch 8 to ensure correctness in
> interrupt context [Ryan]. The diffs in in_lazy_mmu_mode() and
> queue_pte_barriers() are moved to patch 8 and 9 resp.
> - Patch 8:
> * Removed all restrictions regarding lazy_mmu_mode_{pause,resume}().
> They may now be called even when lazy MMU isn't enabled, and
> any call to lazy_mmu_mode_* may be made while paused (such calls
> will be ignored). [David, Ryan]
> * lazy_mmu_state.{nesting_level,active} are replaced with
> {enable_count,pause_count} to track arbitrary nesting of both
> enable/disable and pause/resume [Ryan]
> * Added __task_lazy_mmu_mode_active() for use in patch 12 [David]
> * Added documentation for all the functions [Ryan]
> - Patch 9: keep existing test + set TIF_LAZY_MMU_PENDING instead of
> atomic RMW [David, Ryan]
> - Patch 12: use __task_lazy_mmu_mode_active() instead of accessing
> lazy_mmu_state directly [David]
> - Collected R-b/A-b tags
>
> v4: https://lore.kernel.org/all/20251029100909.3381140-1-kevin.brodsky@arm.com/
>
> v3..v4:
>
> - Patch 2: restored ordering of preempt_{disable,enable}() [Dave Hansen]
> - Patch 5 onwards: s/ARCH_LAZY_MMU/ARCH_HAS_LAZY_MMU_MODE/ [Mike Rapoport]
> - Patch 7: renamed lazy_mmu_state members, removed VM_BUG_ON(),
> reordered writes to lazy_mmu_state members [David Hildenbrand]
> - Dropped patch 13 as it doesn't seem justified [David H]
> - Various improvements to commit messages [David H]
>
> v3: https://lore.kernel.org/all/20251015082727.2395128-1-kevin.brodsky@arm.com/
>
> v2..v3:
>
> - Full rewrite; dropped all Acked-by/Reviewed-by.
> - Rebased on v6.18-rc1.
>
> v2: https://lore.kernel.org/all/20250908073931.4159362-1-kevin.brodsky@arm.com/
>
> v1..v2:
> - Rebased on mm-unstable.
> - Patch 2: handled new calls to enter()/leave(), clarified how the "flush"
> pattern (leave() followed by enter()) is handled.
> - Patch 5,6: removed unnecessary local variable [Alexander Gordeev's
> suggestion].
> - Added Mike Rapoport's Acked-by.
>
> v1: https://lore.kernel.org/all/20250904125736.3918646-1-kevin.brodsky@arm.com/
> ---
> Cc: Alexander Gordeev <agordeev@linux.ibm.com>
> Cc: Andreas Larsson <andreas@gaisler.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Anshuman Khandual <anshuman.khandual@arm.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: David Woodhouse <dwmw2@infradead.org>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Jann Horn <jannh@google.com>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Mike Rapoport <rppt@kernel.org>
> Cc: Nicholas Piggin <npiggin@gmail.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> Cc: Ryan Roberts <ryan.roberts@arm.com>
> Cc: Suren Baghdasaryan <surenb@google.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: Will Deacon <will@kernel.org>
> Cc: Yeoreum Yun <yeoreum.yun@arm.com>
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-kernel@vger.kernel.org
> Cc: linuxppc-dev@lists.ozlabs.org
> Cc: sparclinux@vger.kernel.org
> Cc: xen-devel@lists.xenproject.org
> Cc: x86@kernel.org
> ---
> Alexander Gordeev (1):
> powerpc/64s: Do not re-activate batched TLB flush
>
> Kevin Brodsky (13):
> x86/xen: simplify flush_lazy_mmu()
> powerpc/mm: implement arch_flush_lazy_mmu_mode()
> sparc/mm: implement arch_flush_lazy_mmu_mode()
> mm: clarify lazy_mmu sleeping constraints
> mm: introduce CONFIG_ARCH_HAS_LAZY_MMU_MODE
> mm: introduce generic lazy_mmu helpers
> mm: bail out of lazy_mmu_mode_* in interrupt context
> mm: enable lazy_mmu sections to nest
> arm64: mm: replace TIF_LAZY_MMU with is_lazy_mmu_mode_active()
> powerpc/mm: replace batch->active with is_lazy_mmu_mode_active()
> sparc/mm: replace batch->active with is_lazy_mmu_mode_active()
> x86/xen: use lazy_mmu_state when context-switching
> mm: Add basic tests for lazy_mmu
>
> arch/arm64/Kconfig | 1 +
> arch/arm64/include/asm/pgtable.h | 41 +----
> arch/arm64/include/asm/thread_info.h | 3 +-
> arch/arm64/mm/mmu.c | 8 +-
> arch/arm64/mm/pageattr.c | 4 +-
> .../include/asm/book3s/64/tlbflush-hash.h | 20 +--
> arch/powerpc/include/asm/thread_info.h | 2 -
> arch/powerpc/kernel/process.c | 25 ---
> arch/powerpc/mm/book3s64/hash_tlb.c | 10 +-
> arch/powerpc/mm/book3s64/subpage_prot.c | 4 +-
> arch/powerpc/platforms/Kconfig.cputype | 1 +
> arch/sparc/Kconfig | 1 +
> arch/sparc/include/asm/tlbflush_64.h | 5 +-
> arch/sparc/mm/tlb.c | 14 +-
> arch/x86/Kconfig | 1 +
> arch/x86/boot/compressed/misc.h | 1 +
> arch/x86/boot/startup/sme.c | 1 +
> arch/x86/include/asm/paravirt.h | 1 -
> arch/x86/include/asm/pgtable.h | 1 +
> arch/x86/include/asm/thread_info.h | 4 +-
> arch/x86/xen/enlighten_pv.c | 3 +-
> arch/x86/xen/mmu_pv.c | 6 +-
> fs/proc/task_mmu.c | 4 +-
> include/linux/mm_types_task.h | 5 +
> include/linux/pgtable.h | 158 +++++++++++++++++-
> include/linux/sched.h | 45 +++++
> mm/Kconfig | 19 +++
> mm/Makefile | 1 +
> mm/kasan/shadow.c | 8 +-
> mm/madvise.c | 18 +-
> mm/memory.c | 16 +-
> mm/migrate_device.c | 8 +-
> mm/mprotect.c | 4 +-
> mm/mremap.c | 4 +-
> mm/tests/lazy_mmu_mode_kunit.c | 71 ++++++++
> mm/userfaultfd.c | 4 +-
> mm/vmalloc.c | 12 +-
> mm/vmscan.c | 12 +-
> 38 files changed, 380 insertions(+), 166 deletions(-)
> create mode 100644 mm/tests/lazy_mmu_mode_kunit.c
>
>
> base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
> --
> 2.51.2

All of these look good to me.

Reviewed-by: Yeoreum Yun <yeoreum.yun@arm.com>

--
Sincerely,
Yeoreum Yun