From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 535C253CA97; Tue, 8 Sep 2026 12:36:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788870980; cv=none; b=KSAeVj38kVx+n3CbOWT4xF99pOGPqeLJlMgnEaYAxYaoFzXyUB89K93tJR6IGuFrIVcbU9FAWy+RpEy+JXGzxf0+5hWjSLPBOJi7CSvLR9tOhF6WNWQuYIdJKqgkdKlJAlGuc2M8dVRUPh5SrYl/8k++yT5BOnSv03hHQVM2ntA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788870980; c=relaxed/simple; bh=xRAtx33ggaHgqCgBnjU11lMi7S9z53z/52qjI/kL87I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CGPVaL571l656x/jFAqCaf9gFfaYhn9VAJVy3dLFGcO7m3zwhStWiS4C/JwV8HhXqTQ364CrHFrosxPodua540VvRrZg2xkOlC2UwK3ZZ6N5l5iSy7mMnZcf3CnkqhciSo6Q7vkyEz1UB7VNQmINtdA1OJlNlZkdd6vQceI1rls= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fSEE3eLH; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fSEE3eLH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06A831F00A3A; Tue, 8 Sep 2026 12:35:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788870976; bh=2Yn+fN4XsVpOM7QbS7dPPxzgR+QgYqe1aKcSqRsRi08=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=fSEE3eLHGOreADMqTIpRBXDPW8kHs+J5YWpjb6hYguRtB+Vd0jKYl0g0I5pPsdQrY 3/HWsmMYm/CB+4uGXpSQSU+ROg2cc9c4t5tZNEwLgSJ0Cx+a2psMDNcOg4TGmksvbW TEjcmghJU2LrRZp70WdDGeYzHR0mXMSXMONZcUFKwehCZYymCQj+6CMvgSFaZyL0TX rEr3SVTUNHVPc2eTr6ZTt3fP9i/TyD7A58XUTLw+wXHCjySSe7Xebwm0EozW0fnH7y ZAyQUtxhHzhGO/OhVJprTWfbm+H90tMFBOGQT6TVA5xsfA7hqL6lH4LlKPzVXhxcyB Hj91DOv3HE/Pw== From: "Lorenzo Stoakes (ARM)" Date: Tue, 08 Sep 2026 13:32:19 +0100 Subject: [PATCH v2 10/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for sparc32 Precedence: bulk X-Mailing-List: linux-arch@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260908-rcu-pagetable-freeing-v2-10-1f60b64e878e@kernel.org> References: <20260908-rcu-pagetable-freeing-v2-0-1f60b64e878e@kernel.org> In-Reply-To: <20260908-rcu-pagetable-freeing-v2-0-1f60b64e878e@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Yoshinori Sato , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=7406; i=ljs@kernel.org; h=from:subject:message-id; bh=xRAtx33ggaHgqCgBnjU11lMi7S9z53z/52qjI/kL87I=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLIWMMQ8YVGr+bRni3h5117Vn3fUZGZdeVP2o/mZuv/FY LPFnLzJHaUsDGJcDLJiiizPv4jvDxIJm9d5wd8NZg4rE8gQBi5OAZhI1TSGf6rp0/bZXL1/vmWB hZP4hpAVHGZnClzv94Q3/J3lcT5x1QVGhuaIsAazqbzbCv05chaYtP7SLNZ6ePJMtOOc+zJ/tt6 8xQYA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Careful handling is required for sparc32 which implements page tables as part of a shared backing page. To support this, a custom __tlb_remove_table() function is required, as specified by CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE. This allows __pte_free_tlb() and __pmd_free_tlb() to specify which page table level is being freed, which is transmitted to __tlb_remove_table() through setting the lowest bit of the page table to 1 for a PMD and 0 for a PTE (the page tables are 256-byte aligned so this is safe to do). Next, since the page table freeing is done via RCU callback, and thus might be executed in softirq context, update the spin locks to IRQ save/restore. Then, in __tlb_remove_table(), figure out whether to free a PMD page table via free_pmd_fast() or a PTE via the newly introduced __pte_free() function, using the lower bit encoded in __pte_free_tlb() or __pmd_free_tlb() to determine which to call. As part of this change use this spin lock rather than mm->page_table_lock for all shared page table exclusion, as RCU freeing means that page tables can be freed from soft IRQ context so both don't have an mm and also mm->page_table_lock is not IRQ-safe. Note that the specification of CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE disables CONFIG_PT_RECLAIM for sparc32, which mirrors sparc64. This forms part of an overall effort to switch every architecture to this mode, and with it complete, means every architecture now supports CONFIG_MMU_GATHER_RCU_TABLE_FREE. Signed-off-by: Lorenzo Stoakes (ARM) --- arch/sparc/Kconfig | 2 ++ arch/sparc/include/asm/pgalloc_32.h | 7 +++++-- arch/sparc/lib/bitext.c | 14 +++++++------- arch/sparc/mm/srmmu.c | 32 +++++++++++++++++++++++++++----- 4 files changed, 41 insertions(+), 14 deletions(-) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 8d42ebc6d302..79c09d6ee466 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -64,6 +64,8 @@ config SPARC32 select HAVE_UID16 select HAVE_PAGE_SIZE_4KB select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE + select HAVE_ARCH_TLB_REMOVE_TABLE select OLD_SIGACTION select ZONE_DMA diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h index 4f73e87b22a3..36010852ba0c 100644 --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -48,7 +48,9 @@ static inline void free_pmd_fast(pmd_t * pmd) } #define pmd_free(mm, pmd) free_pmd_fast(pmd) -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) + +#define __pmd_free_tlb(tlb, pmd, addr) \ + tlb_remove_table((tlb), (void *)((unsigned long)(pmd) | 1UL)) #define pmd_populate(mm, pmd, pte) pmd_set(pmd, pte) @@ -72,6 +74,7 @@ static inline void free_pte_fast(pte_t *pte) #define pte_free_kernel(mm, pte) free_pte_fast(pte) void pte_free(struct mm_struct * mm, pgtable_t pte); -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +void __tlb_remove_table(void *table); +#define __pte_free_tlb(tlb, pte, addr) tlb_remove_table((tlb), (void *)(pte)) #endif /* _SPARC_PGALLOC_H */ diff --git a/arch/sparc/lib/bitext.c b/arch/sparc/lib/bitext.c index 32a5c1d9459c..c309e27973ce 100644 --- a/arch/sparc/lib/bitext.c +++ b/arch/sparc/lib/bitext.c @@ -22,8 +22,6 @@ * @align: requested alignment * * Returns offset in the map or -1 if out of space. - * - * Not safe to call from an interrupt (uses spin_lock). */ int bit_map_string_get(struct bit_map *t, int len, int align) { @@ -31,6 +29,7 @@ int bit_map_string_get(struct bit_map *t, int len, int align) int off_new; int align1; int i, color; + unsigned long flags; if (t->num_colors) { /* align is overloaded to be the page color */ @@ -50,7 +49,7 @@ int bit_map_string_get(struct bit_map *t, int len, int align) BUG(); color &= align1; - spin_lock(&t->lock); + spin_lock_irqsave(&t->lock, flags); if (len < t->last_size) offset = t->first_free; else @@ -64,7 +63,7 @@ int bit_map_string_get(struct bit_map *t, int len, int align) if (offset >= t->size) offset = 0; if (count + len > t->size) { - spin_unlock(&t->lock); + spin_unlock_irqrestore(&t->lock, flags); /* P3 */ printk(KERN_ERR "bitmap out: size %d used %d off %d len %d align %d count %d\n", t->size, t->used, offset, len, align, count); @@ -90,7 +89,7 @@ int bit_map_string_get(struct bit_map *t, int len, int align) t->last_off = 0; t->used += len; t->last_size = len; - spin_unlock(&t->lock); + spin_unlock_irqrestore(&t->lock, flags); return offset; } } @@ -103,10 +102,11 @@ int bit_map_string_get(struct bit_map *t, int len, int align) void bit_map_clear(struct bit_map *t, int offset, int len) { int i; + unsigned long flags; if (t->used < len) BUG(); /* Much too late to do any good, but alas... */ - spin_lock(&t->lock); + spin_lock_irqsave(&t->lock, flags); for (i = 0; i < len; i++) { if (test_bit(offset + i, t->map) == 0) BUG(); @@ -115,7 +115,7 @@ void bit_map_clear(struct bit_map *t, int offset, int len) if (offset < t->first_free) t->first_free = offset; t->used -= len; - spin_unlock(&t->lock); + spin_unlock_irqrestore(&t->lock, flags); } void bit_map_init(struct bit_map *t, unsigned long *map, int size) diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index 9a74902ad181..1c277ab3cdb8 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -340,38 +340,60 @@ pgd_t *get_pgd_fast(void) * Alignments up to the page size are the same for physical and virtual * addresses of the nocache area. */ + +static DEFINE_SPINLOCK(pte_page_lock); + pgtable_t pte_alloc_one(struct mm_struct *mm) { + unsigned long flags; pte_t *ptep; struct page *page; if (!(ptep = pte_alloc_one_kernel(mm))) return NULL; page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); - spin_lock(&mm->page_table_lock); + spin_lock_irqsave(&pte_page_lock, flags); if (page_ref_inc_return(page) == 2 && !pagetable_pte_ctor(mm, page_ptdesc(page))) { page_ref_dec(page); ptep = NULL; } - spin_unlock(&mm->page_table_lock); + spin_unlock_irqrestore(&pte_page_lock, flags); return ptep; } -void pte_free(struct mm_struct *mm, pgtable_t ptep) +static void __pte_free(pgtable_t ptep) { struct page *page; + unsigned long flags; page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); - spin_lock(&mm->page_table_lock); + spin_lock_irqsave(&pte_page_lock, flags); if (page_ref_dec_return(page) == 1) pagetable_dtor(page_ptdesc(page)); - spin_unlock(&mm->page_table_lock); + spin_unlock_irqrestore(&pte_page_lock, flags); srmmu_free_nocache(ptep, SRMMU_PTE_TABLE_SIZE); } +void pte_free(struct mm_struct *mm, pgtable_t ptep) +{ + __pte_free(ptep); +} + +void __tlb_remove_table(void *table) +{ + const unsigned long encoded = (unsigned long)table; + const unsigned long addr = encoded & ~1UL; + const bool is_pmd = encoded & 1; + + if (is_pmd) + free_pmd_fast((pmd_t *)addr); + else + __pte_free((pgtable_t)addr); +} + /* context handling - a dynamically sized pool is used */ #define NO_CONTEXT -1 -- 2.55.0