From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3FED46EF92 for ; Tue, 8 Sep 2026 21:16:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788902213; cv=none; b=nBV1cbPBEWI3UmpxGPJPZkrIkMo47gMxzNVDNffhd0Ss7lYEb3xpo6e65J/XHk7c4SEk3ZNA9gV0jW2/U90rwxmIuJ32Xkv/xM6QA1WFPpqZDrCtvc2ffiqTsc1pjWmGdw41P1gi1r/8HWMCszoKixJvOh72QAUxOgDjrbN58/8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788902213; c=relaxed/simple; bh=/3cx4MGpWrfiLdaiNw2RYAl0B2kZ+XaWHOcThfrNOnk=; h=Date:To:From:Subject:Message-Id; b=EolYPFQPDM7WadJ4IwOX/egl8FZU63qzuYUHayqnMJVZdeC76+XiFTY6OYv0yXd8QCBJ5E7LuH763+rMcMI+ajINVeb7KJfxq+61jnBGhQ2GUsWGMJXBPT/CmvbXQsji+oi2w2FbjCvNutyQSSiPM9omuS6PtEjbkdAwU44UAMQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=I4whqF8o; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="I4whqF8o" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B5011F00A3A; Tue, 8 Sep 2026 21:16:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1788902210; bh=ywxucpbnKDM12WWB8GjzeHfT4iL+17RU9Wg5OEie63s=; h=Date:To:From:Subject; b=I4whqF8oyk8NiTnnXeN/LIK/a/V9LM7i6ERYGHzgWmL2QS01rTYaQsv/zJ0eQrZS5 GH5MJKrGJdm1CpkdwOBt0ss9iFdF0yacJ8Uomb88WB7kgUoJuRVU5g+MQStQUvS88i LJtyUuaw2V9c7XZ9pw/29hL/KDpFYOwHnNYJ0+nA= Date: Tue, 08 Sep 2026 14:16:50 -0700 To: mm-commits@vger.kernel.org,ljs@kernel.org,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-make-userland-page-table-freeing-rcu-safe.patch added to mm-new branch Message-Id: <20260908211650.6B5011F00A3A@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: mm: make userland page table freeing RCU-safe has been added to the -mm mm-new branch. Its filename is mm-make-userland-page-table-freeing-rcu-safe.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-make-userland-page-table-freeing-rcu-safe.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: "Lorenzo Stoakes (ARM)" Subject: mm: make userland page table freeing RCU-safe Date: Tue, 08 Sep 2026 13:32:20 +0100 Now every architecture has been converted to support CONFIG_MMU_GATHER_RCU_TABLE_FREE, this configuration option no longer makes any sense to keep around. Therefore remove it, and remove all the dead code that existed for !CONFIG_MMU_GATHER_RCU_TABLE_FREE architectures previously. Additionally, CONFIG_MMU_GATHER_TABLE_FREE is no longer necessary, as all architectures instead use CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE when a custom __tlb_remove_table() is required, so remove this too. A number of architectures only enabled CONFIG_MMU_GATHER_RCU_TABLE_FREE if CONFIG_MMU was set, however the mmu_gather logic only actually does something meaningful if CONFIG_MMU is set (mmu_gather.c is only compiled in this case, for instance). As a result, there's no need to gate any of this logic on CONFIG_MMU explicitly. CONFIG_PT_RECLAIM however does have a strict dependency on CONFIG_MMU, so make this dependency explicit. Additionally, correct comments to remove references to non-RCU page table gathering and make it clear that this is not 'semi-RCU', nor has it been since commit 1fb3d8c20bfa ("mm/mmu_gather: replace IPI with synchronize_rcu() when batch allocation fails"). With this change in place the kernel policy is now that all page tables are freed after an RCU grace period, and thus it is now safe to unconditionally perform page table walks under RCU, safe in the knowledge that page tables will not be freed underneath the walker. This is all that is guaranteed, however, so naturally it is still incumbent upon page table walkers to ensure that the page table entries are as expected. Link: https://lore.kernel.org/20260908-rcu-pagetable-freeing-v2-11-1f60b64e878e@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Cc: Albert Ou Cc: Alexander Gordeev Cc: Alexandre Ghiti Cc: Andreas Larsson Cc: "Aneesh Kumar K.V" Cc: Anton Ivanov Cc: Arnd Bergmann Cc: Baolin Wang Cc: Barry Song Cc: "Borislav Petkov (AMD)" Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Christian Zankel Cc: Dave Hansen Cc: David Hildenbrand Cc: David S. Miller Cc: Dev Jain Cc: Dinh Nguyen Cc: Geert Uytterhoeven Cc: Guo Ren Cc: Heiko Carstens Cc: Helge Deller Cc: "H. Peter Anvin" Cc: Huacai Chen Cc: Hugh Dickins Cc: Ingo Molnar Cc: James Bottomley Cc: Jason Gunthorpe Cc: Johannes Berg Cc: John Hubbard Cc: John Paul Adrian Glaubitz Cc: Jonas Bonn Cc: Kiryl Shutsemau Cc: Lance Yang Cc: Liam R. Howlett Cc: Madhavan Srinivasan Cc: Magnus Lindholm Cc: Marc Rutland Cc: Matt Turner Cc: Max Filippov Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Simek Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Palmer Dabbelt Cc: Peter Xu Cc: Peter Zijlstra Cc: Richard Henderson Cc: Richard Weinberger Cc: Rich Felker Cc: Russell King Cc: Ryan Roberts Cc: Stafford Horne Cc: Stefan Kristiansson Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Bogendoerfer Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Vlastimil Babka Cc: WANG Xuerui Cc: Will Deacon Cc: Yoshinori Sato Cc: Zi Yan Signed-off-by: Andrew Morton --- arch/Kconfig | 8 --- arch/alpha/Kconfig | 1 arch/arc/Kconfig | 1 arch/arm/Kconfig | 1 arch/arm64/Kconfig | 1 arch/csky/Kconfig | 1 arch/hexagon/Kconfig | 1 arch/loongarch/Kconfig | 1 arch/m68k/Kconfig | 1 arch/microblaze/Kconfig | 1 arch/mips/Kconfig | 1 arch/nios2/Kconfig | 1 arch/openrisc/Kconfig | 1 arch/parisc/Kconfig | 1 arch/powerpc/Kconfig | 1 arch/riscv/Kconfig | 1 arch/s390/Kconfig | 1 arch/sh/Kconfig | 1 arch/sparc/Kconfig | 2 arch/sparc/include/asm/tlb_64.h | 2 arch/um/Kconfig | 1 arch/x86/Kconfig | 1 arch/xtensa/Kconfig | 1 include/asm-generic/tlb.h | 66 ++++-------------------------- mm/Kconfig | 2 mm/gup.c | 5 +- mm/mmu_gather.c | 30 ++----------- 27 files changed, 18 insertions(+), 117 deletions(-) --- a/arch/alpha/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/alpha/Kconfig @@ -42,7 +42,6 @@ config ALPHA select ARCH_STACKWALK select CPU_NO_EFFICIENT_FFS if !ALPHA_EV67 select MMU_GATHER_NO_RANGE - select MMU_GATHER_RCU_TABLE_FREE select SPARSEMEM_EXTREME if SPARSEMEM select ZONE_DMA select TRACE_IRQFLAGS_SUPPORT --- a/arch/arc/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/arc/Kconfig @@ -47,7 +47,6 @@ config ARC select HAVE_SYSCALL_TRACEPOINTS select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select OF select OF_EARLY_FLATTREE --- a/arch/arm64/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/arm64/Kconfig @@ -221,7 +221,6 @@ config ARM64 select HAVE_RELIABLE_STACKTRACE select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_FUNCTION_ARG_ACCESS_API - select MMU_GATHER_RCU_TABLE_FREE select HAVE_RSEQ select HAVE_RUST if RUSTC_SUPPORTS_ARM64 select HAVE_STACKPROTECTOR --- a/arch/arm/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/arm/Kconfig @@ -134,7 +134,6 @@ config ARM select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select HAVE_POSIX_CPU_TIMERS_TASK_WORK - select MMU_GATHER_RCU_TABLE_FREE if MMU select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RSEQ select HAVE_RUST if CPU_LITTLE_ENDIAN && CPU_32v7 && !KASAN --- a/arch/csky/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/csky/Kconfig @@ -96,7 +96,6 @@ config CSKY select HAVE_SYSCALL_TRACEPOINTS select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MAY_HAVE_SPARSE_IRQ select MODULES_USE_ELF_RELA if MODULES select OF --- a/arch/hexagon/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/hexagon/Kconfig @@ -23,7 +23,6 @@ config HEXAGON # select HAVE_CLK select GENERIC_ATOMIC64 select HAVE_PERF_EVENTS - select MMU_GATHER_RCU_TABLE_FREE # GENERIC_ALLOCATOR is used by dma_alloc_coherent() select GENERIC_ALLOCATOR select GENERIC_IRQ_PROBE --- a/arch/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/Kconfig @@ -526,13 +526,6 @@ config HAVE_ARCH_JUMP_LABEL config HAVE_ARCH_JUMP_LABEL_RELATIVE bool -config MMU_GATHER_TABLE_FREE - bool - -config MMU_GATHER_RCU_TABLE_FREE - bool - select MMU_GATHER_TABLE_FREE - config MMU_GATHER_PAGE_SIZE bool @@ -548,7 +541,6 @@ config MMU_GATHER_MERGE_VMAS config MMU_GATHER_NO_GATHER bool - depends on MMU_GATHER_TABLE_FREE config ARCH_WANT_IRQS_OFF_ACTIVATE_MM bool --- a/arch/loongarch/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/loongarch/Kconfig @@ -188,7 +188,6 @@ config LOONGARCH select IRQ_LOONGARCH_CPU select LOCK_MM_AND_FIND_VMA select MMU_GATHER_MERGE_VMAS if MMU - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA if MODULES select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK --- a/arch/m68k/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/m68k/Kconfig @@ -37,7 +37,6 @@ config M68K select HAVE_MOD_ARCH_SPECIFIC select HAVE_UID16 select MMU_GATHER_NO_RANGE if MMU - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_REL select MODULES_USE_ELF_RELA select NO_DMA if !MMU && !COLDFIRE --- a/arch/microblaze/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/microblaze/Kconfig @@ -41,7 +41,6 @@ config MICROBLAZE select PCI_SYSCALL if PCI select CPU_NO_EFFICIENT_FFS select MMU_GATHER_NO_RANGE - select MMU_GATHER_RCU_TABLE_FREE select SPARSE_IRQ select ZONE_DMA select TRACE_IRQFLAGS_SUPPORT --- a/arch/mips/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/mips/Kconfig @@ -97,7 +97,6 @@ config MIPS select IRQ_FORCED_THREADING select ISA if EISA select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_REL if MODULES select MODULES_USE_ELF_RELA if MODULES && 64BIT select PERF_USE_VMALLOC --- a/arch/nios2/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/nios2/Kconfig @@ -19,7 +19,6 @@ config NIOS2 select HAVE_PAGE_SIZE_4KB select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select OF select OF_EARLY_FLATTREE --- a/arch/openrisc/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/openrisc/Kconfig @@ -35,7 +35,6 @@ config OPENRISC select GENERIC_ATOMIC64 select GENERIC_CLOCKEVENTS_BROADCAST select GENERIC_SMP_IDLE_THREAD - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC --- a/arch/parisc/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/parisc/Kconfig @@ -80,7 +80,6 @@ config PARISC select GENERIC_CLOCKEVENTS select CPU_NO_EFFICIENT_FFS select THREAD_INFO_IN_TASK - select MMU_GATHER_RCU_TABLE_FREE select NEED_DMA_MAP_STATE select NEED_SG_DMA_LENGTH select HAVE_ARCH_KGDB --- a/arch/powerpc/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/powerpc/Kconfig @@ -307,7 +307,6 @@ config PPC select KASAN_VMALLOC if KASAN && EXECMEM select LOCK_MM_AND_FIND_VMA select MMU_GATHER_PAGE_SIZE - select MMU_GATHER_RCU_TABLE_FREE select HAVE_ARCH_TLB_REMOVE_TABLE select MMU_GATHER_MERGE_VMAS select MMU_LAZY_TLB_SHOOTDOWN if PPC_BOOK3S_64 --- a/arch/riscv/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/riscv/Kconfig @@ -208,7 +208,6 @@ config RISCV select IRQ_FORCED_THREADING select KASAN_VMALLOC if KASAN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA if MODULES select OF select OF_EARLY_FLATTREE --- a/arch/s390/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/s390/Kconfig @@ -267,7 +267,6 @@ config S390 select LOCK_MM_AND_FIND_VMA select MMU_GATHER_MERGE_VMAS select MMU_GATHER_NO_GATHER - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select NEED_DMA_MAP_STATE if PCI select NEED_PER_CPU_EMBED_FIRST_CHUNK --- a/arch/sh/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/sh/Kconfig @@ -62,7 +62,6 @@ config SUPERH select HAVE_SYSCALL_TRACEPOINTS select IRQ_FORCED_THREADING select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select NEED_SG_DMA_LENGTH select NO_DMA if !MMU && !DMA_COHERENT --- a/arch/sparc/include/asm/tlb_64.h~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/sparc/include/asm/tlb_64.h @@ -29,9 +29,7 @@ void flush_tlb_pending(void); * and therefore we don't need a TLBI when freeing page-table pages. */ -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE #define tlb_needs_table_invalidate() (false) -#endif #include --- a/arch/sparc/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/sparc/Kconfig @@ -64,7 +64,6 @@ config SPARC32 select HAVE_UID16 select HAVE_PAGE_SIZE_4KB select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select HAVE_ARCH_TLB_REMOVE_TABLE select OLD_SIGACTION select ZONE_DMA @@ -77,7 +76,6 @@ config SPARC64 select HAVE_FUNCTION_GRAPH_TRACER select HAVE_KRETPROBES select HAVE_KPROBES - select MMU_GATHER_RCU_TABLE_FREE select HAVE_ARCH_TLB_REMOVE_TABLE select MMU_GATHER_MERGE_VMAS select MMU_GATHER_NO_FLUSH_CACHE --- a/arch/um/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/um/Kconfig @@ -44,7 +44,6 @@ config UML select HAVE_SYSCALL_TRACEPOINTS select THREAD_INFO_IN_TASK select SPARSE_IRQ - select MMU_GATHER_RCU_TABLE_FREE config MMU bool --- a/arch/x86/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/x86/Kconfig @@ -283,7 +283,6 @@ config X86 select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select ASYNC_KERNEL_PGTABLE_FREE if IOMMU_SVA - select MMU_GATHER_RCU_TABLE_FREE select MMU_GATHER_MERGE_VMAS select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_REGS_AND_STACK_ACCESS_API --- a/arch/xtensa/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/arch/xtensa/Kconfig @@ -55,7 +55,6 @@ config XTENSA select HAVE_VIRT_CPU_ACCOUNTING_GEN select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select PERF_USE_VMALLOC select TRACE_IRQFLAGS_SUPPORT --- a/include/asm-generic/tlb.h~mm-make-userland-page-table-freeing-rcu-safe +++ a/include/asm-generic/tlb.h @@ -67,11 +67,8 @@ * - tlb_remove_table() * * tlb_remove_table() is the basic primitive to free page-table directories - * (__p*_free_tlb()). In it's most primitive form it is an alias for - * tlb_remove_page() below, for when page directories are pages and have no - * additional constraints. - * - * See also MMU_GATHER_TABLE_FREE and MMU_GATHER_RCU_TABLE_FREE. + * (__p*_free_tlb()). Page directories are freed after an RCU grace + * period - see the comment in mm/mmu_gather.c. * * - tlb_remove_page() / tlb_remove_page_size() * - __tlb_remove_folio_pages() / __tlb_remove_page_size() @@ -151,24 +148,15 @@ * This might be useful if your architecture has size specific TLB * invalidation instructions. * - * MMU_GATHER_TABLE_FREE - * - * This provides tlb_remove_table(), to be used instead of tlb_remove_page() - * for page directores (__p*_free_tlb()). - * - * Useful if your architecture has non-page page directories. + * Page directories (__p*_free_tlb()) are always freed via tlb_remove_table(), + * after an RCU grace period (see mm/mmu_gather.c). * - * When used, an architecture is expected to provide __tlb_remove_table() or - * use the generic __tlb_remove_table(), which does the actual freeing of these - * pages. + * This serialises against software page-table walkers, including architectures + * which do not use IPIs for remote TLB invalidates. * - * MMU_GATHER_RCU_TABLE_FREE - * - * Like MMU_GATHER_TABLE_FREE, and adds semi-RCU semantics to the free (see - * comment below). - * - * Useful if your architecture doesn't use IPIs for remote TLB invalidates - * and therefore doesn't naturally serialize with software page-table walkers. + * An architecture is expected to provide __tlb_remove_table() (see + * HAVE_ARCH_TLB_REMOVE_TABLE) or use the generic __tlb_remove_table(), which + * does the actual freeing of these pages. * * MMU_GATHER_NO_FLUSH_CACHE * @@ -200,12 +188,8 @@ * various ptep_get_and_clear() functions. */ -#ifdef CONFIG_MMU_GATHER_TABLE_FREE - struct mmu_table_batch { -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE struct rcu_head rcu; -#endif unsigned int nr; void *tables[]; }; @@ -224,23 +208,6 @@ static inline void __tlb_remove_table(vo extern void tlb_remove_table(struct mmu_gather *tlb, void *table); -#else /* !CONFIG_MMU_GATHER_TABLE_FREE */ - -static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page); -/* - * Without MMU_GATHER_TABLE_FREE the architecture is assumed to have page based - * page directories and we can use the normal page batching to free them. - */ -static inline void tlb_remove_table(struct mmu_gather *tlb, void *table) -{ - struct ptdesc *ptdesc = (struct ptdesc *)table; - - pagetable_dtor(ptdesc); - tlb_remove_page(tlb, ptdesc_page(ptdesc)); -} -#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ - -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* * This allows an architecture that does not use the linux page-tables for * hardware to skip the TLBI when freeing page tables. @@ -253,19 +220,6 @@ void tlb_remove_table_sync_one(void); void tlb_remove_table_sync_rcu(void); -#else - -#ifdef tlb_needs_table_invalidate -#error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE -#endif - -static inline void tlb_remove_table_sync_one(void) { } - -static inline void tlb_remove_table_sync_rcu(void) { } - -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - - #ifndef CONFIG_MMU_GATHER_NO_GATHER /* * If we can't allocate a page to make a big batch of page pointers @@ -325,9 +279,7 @@ static inline void tlb_flush_rmaps(struc struct mmu_gather { struct mm_struct *mm; -#ifdef CONFIG_MMU_GATHER_TABLE_FREE struct mmu_table_batch *batch; -#endif unsigned long start; unsigned long end; --- a/mm/gup.c~mm-make-userland-page-table-freeing-rcu-safe +++ a/mm/gup.c @@ -2700,8 +2700,9 @@ EXPORT_SYMBOL(get_user_pages_unlocked); * Before activating this code, please be aware that the following assumptions * are currently made: * - * *) Either MMU_GATHER_RCU_TABLE_FREE is enabled, and tlb_remove_table() is used to - * free pages containing page tables or TLB flushing requires IPI broadcast. + * *) tlb_remove_table() is used to free pages containing page tables, with + * the free deferred until an RCU grace period has elapsed (see + * mm/mmu_gather.c). * * *) ptes can be read atomically by the architecture. * --- a/mm/Kconfig~mm-make-userland-page-table-freeing-rcu-safe +++ a/mm/Kconfig @@ -1464,7 +1464,7 @@ config HAVE_ARCH_TLB_REMOVE_TABLE config PT_RECLAIM def_bool y - depends on MMU_GATHER_RCU_TABLE_FREE && !HAVE_ARCH_TLB_REMOVE_TABLE + depends on MMU && !HAVE_ARCH_TLB_REMOVE_TABLE help Try to reclaim empty user page table pages in paths other than munmap and exit_mmap path. --- a/mm/mmu_gather.c~mm-make-userland-page-table-freeing-rcu-safe +++ a/mm/mmu_gather.c @@ -218,8 +218,6 @@ bool __tlb_remove_page_size(struct mmu_g #endif /* MMU_GATHER_NO_GATHER */ -#ifdef CONFIG_MMU_GATHER_TABLE_FREE - static void __tlb_remove_table_free(struct mmu_table_batch *batch) { int i; @@ -230,10 +228,8 @@ static void __tlb_remove_table_free(stru free_page((unsigned long)batch); } -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE - /* - * Semi RCU freeing of the page directories. + * RCU freeing of the page directories. * * This is needed by some architectures to implement software pagetable walkers. * @@ -259,13 +255,13 @@ static void __tlb_remove_table_free(stru * means. * * What we do is batch the freed directory pages (tables) and RCU free them. - * We use the sched RCU variant, as that guarantees that IRQ/preempt disabling - * holds off grace periods. + * Disabling IRQs or preemption holds off RCU grace periods, so this protects + * both rcu_read_lock() and IRQ-disabling walkers. * * However, in order to batch these pages we need to allocate storage, this * allocation is deep inside the MM code and can thus easily fail on memory - * pressure. To guarantee progress we fall back to single table freeing, see - * the implementation of tlb_remove_table_one(). + * pressure. To guarantee progress we fall back to single table freeing, which + * is also RCU-deferred - see the implementation of tlb_remove_table_one(). * */ @@ -315,15 +311,6 @@ void tlb_remove_table_sync_rcu(void) synchronize_rcu(); } -#else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - -static void tlb_remove_table_free(struct mmu_table_batch *batch) -{ - __tlb_remove_table_free(batch); -} - -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - /* * If we want tlb_remove_table() to imply TLB invalidates. */ @@ -403,13 +390,6 @@ static inline void tlb_table_init(struct tlb->batch = NULL; } -#else /* !CONFIG_MMU_GATHER_TABLE_FREE */ - -static inline void tlb_table_flush(struct mmu_gather *tlb) { } -static inline void tlb_table_init(struct mmu_gather *tlb) { } - -#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ - static void tlb_flush_mmu_free(struct mmu_gather *tlb) { tlb_table_flush(tlb); _ Patches currently in -mm which might be from ljs@kernel.org are mm-vma-correctly-unaccount-on-mmap_prepare-failure.patch mm-vmpressure-remove-window-size-todo.patch tools-testing-selftests-mm-add-missing-gitignore-entries.patch mm-move-drivers-char-memc-to-mm-char-memc.patch mm-implement-file_is_dev_zero-to-uniquely-identify-dev-zero.patch mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous.patch mm-vma-make-map_private-mapped-dev-zero-mappings-truly-anonymous.patch tools-testing-vma-add-test-to-assert-map_private-dev-zero-is-anon.patch tools-testing-selftests-mm-add-map_private-dev-zero-merge-tests.patch mm-madvise-swap-in-cowd-map_private-file-mappings-on-madv_willneed.patch mm-huge_memory-zap-deposited-page-tables-after-an-rcu-grace-period.patch mm-enable-mmu_gather_rcu_table_free-for-most-2-level-architectures.patch mm-enable-mmu_gather_rcu_table_free-for-mmu-riscv.patch mm-enable-mmu_gather_rcu_table_free-for-mmu-arm.patch mm-enable-mmu_gather_rcu_table_free-for-arc-microblaze-xtensa.patch mm-enable-mmu_gather_rcu_table_free-for-sparc64.patch mm-enable-mmu_gather_rcu_table_free-for-m68k-coldfire.patch mm-enable-mmu_gather_rcu_table_free-for-sh-x2.patch mm-enable-mmu_gather_rcu_table_free-for-m68k-motorola.patch mm-enable-mmu_gather_rcu_table_free-for-sparc32.patch mm-make-userland-page-table-freeing-rcu-safe.patch mm-change-the-contract-for-free_pgtables-update-docs.patch