From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B763AC4706F for ; Tue, 2 Jan 2024 03:36:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bivMwKHVUxR056P6LVYycrimS9bb9oVnL339S+46W2Y=; b=AgZGpT3jZTSi+/ ql02YbzaWglUHYgWRA7gdV35WBCeJns8xOSE8VPeSUuD/ik3J47Dj829tuVHE0+uaJXzD72UXjUvb v3etSUhbmfTYtA9wOZ6euu+o1iZoSK5Dnphu/zDqLgal5UWIcf6HQS0B6hMFWm/+L4uNnw63hFdm7 2x38OswMk+v/G0jR0fz4CyOwkAPLkT5Q66vKzrj+wlzT7U27fMERDvepMr4EDW33xk85IFmV3JohE tUf66ChCdDc0ik4IQkASqSb095lO1+13c1S24gzVfLQ0EWCAnDDuW6AAXNbPSKVtX1fZVHA78KnOV Lb8XwNLbxaPj3yaOsxrg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rKVa0-0071bl-1o; Tue, 02 Jan 2024 03:36:36 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rKVZv-0071Zb-2S for linux-riscv@lists.infradead.org; Tue, 02 Jan 2024 03:36:34 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 6074BCE0B79; Tue, 2 Jan 2024 03:36:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BD48C433CB; Tue, 2 Jan 2024 03:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704166588; bh=WeKR+pf7kCbWxOsXcSMjqhGRylxECnkrQmi2F37hamY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=uhoENczfe/ZlasEP9n18xnig3cHIoUjmDo0y+kaeTW2aQPf4Q7JPGDu13cK0KzFB+ lczgZu64UvAltdriaMUQxV4aStNyNpdrA2029uchXbxhK4VjLRLD7GYMJFE1Hlc4gG 5PjSWM68hTZjL1hIx8Js9isYEozJnVEnoJQe+BcbB6iIzpGrce13sMf6CXBs7T0M9L zdIcyIlhpVcQrW284GZtCj+bhrG1P79B1sbg97Sr+Fp9HlCTfQ3aVHKqrBbyPU+t6a X/TqjsJ7LoZCACDBGSCbRo7YVE3d8Akt7boHerKUngY/ERnnlcyR5uqWp2jiVRe9xP Jkw5sfob8hM9A== Date: Tue, 2 Jan 2024 11:23:46 +0800 From: Jisheng Zhang To: Alexandre Ghiti Cc: Paul Walmsley , Palmer Dabbelt , Albert Ou , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 3/4] riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU Message-ID: References: <20231219175046.2496-1-jszhang@kernel.org> <20231219175046.2496-4-jszhang@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240101_193632_339685_D26E3287 X-CRM114-Status: GOOD ( 31.13 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On Sun, Dec 31, 2023 at 07:32:47AM +0100, Alexandre Ghiti wrote: > On 19/12/2023 18:50, Jisheng Zhang wrote: > > In order to implement fast gup we need to ensure that the page > > table walker is protected from page table pages being freed from > > under it. > > > > riscv situation is more complicated than other architectures: some > > riscv platforms may use IPI to perform TLB shootdown, for example, > > those platforms which support AIA, usually the riscv_ipi_for_rfence is > > true on these platforms; some riscv platforms may rely on the SBI to > > perform TLB shootdown, usually the riscv_ipi_for_rfence is false on > > these platforms. To keep software pagetable walkers safe in this case > > we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the > > comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in > > include/asm-generic/tlb.h for more details. > > > > This patch enables MMU_GATHER_RCU_TABLE_FREE, then use > > > > *tlb_remove_page_ptdesc() for those platforms which use IPI to perform > > TLB shootdown; > > > > *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB > > shootdown; > > > Can you elaborate a bit more on what those functions do differently and why > we need to differentiate IPI vs SBI TLB shootdown? I don't understand this. Hi Alex, If IPI, the local_irq_save in lockless_pages_from_mm() of fast gup code path will block page table pages from being freed, I think the comments there is execellent. If SBI, the local_irq_save in lockless_pages_from_mm() can't acchieve the goal however. Because local_irq_save() only disable S-privilege IPI irq, it can't disable M-privilege's, which the SBI implementation use to shootdown TLB entry. So we need MMU_GATHER_RCU_TABLE_FREE helper for SBI case. Thanks > > > Both case mean that disabling interrupts will block the free and > > protect the fast gup page walker. > > > > Signed-off-by: Jisheng Zhang > > --- > > arch/riscv/Kconfig | 1 + > > arch/riscv/include/asm/pgalloc.h | 23 ++++++++++++++++++----- > > arch/riscv/include/asm/tlb.h | 18 ++++++++++++++++++ > > 3 files changed, 37 insertions(+), 5 deletions(-) > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > > index 24c1799e2ec4..d3555173d9f4 100644 > > --- a/arch/riscv/Kconfig > > +++ b/arch/riscv/Kconfig > > @@ -147,6 +147,7 @@ config RISCV > > select IRQ_FORCED_THREADING > > select KASAN_VMALLOC if KASAN > > select LOCK_MM_AND_FIND_VMA > > + select MMU_GATHER_RCU_TABLE_FREE if SMP && MMU > > select MODULES_USE_ELF_RELA if MODULES > > select MODULE_SECTIONS if MODULES > > select OF > > diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h > > index 3c5e3bd15f46..deaf971253a2 100644 > > --- a/arch/riscv/include/asm/pgalloc.h > > +++ b/arch/riscv/include/asm/pgalloc.h > > @@ -102,7 +102,10 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, > > struct ptdesc *ptdesc = virt_to_ptdesc(pud); > > pagetable_pud_dtor(ptdesc); > > - tlb_remove_page_ptdesc(tlb, ptdesc); > > + if (riscv_use_ipi_for_rfence()) > > + tlb_remove_page_ptdesc(tlb, ptdesc); > > + else > > + tlb_remove_ptdesc(tlb, ptdesc); > > } > > } > > @@ -136,8 +139,12 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) > > static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, > > unsigned long addr) > > { > > - if (pgtable_l5_enabled) > > - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); > > + if (pgtable_l5_enabled) { > > + if (riscv_use_ipi_for_rfence()) > > + tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); > > + else > > + tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); > > + } > > } > > #endif /* __PAGETABLE_PMD_FOLDED */ > > @@ -169,7 +176,10 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, > > struct ptdesc *ptdesc = virt_to_ptdesc(pmd); > > pagetable_pmd_dtor(ptdesc); > > - tlb_remove_page_ptdesc(tlb, ptdesc); > > + if (riscv_use_ipi_for_rfence()) > > + tlb_remove_page_ptdesc(tlb, ptdesc); > > + else > > + tlb_remove_ptdesc(tlb, ptdesc); > > } > > #endif /* __PAGETABLE_PMD_FOLDED */ > > @@ -180,7 +190,10 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, > > struct ptdesc *ptdesc = page_ptdesc(pte); > > pagetable_pte_dtor(ptdesc); > > - tlb_remove_page_ptdesc(tlb, ptdesc); > > + if (riscv_use_ipi_for_rfence()) > > + tlb_remove_page_ptdesc(tlb, ptdesc); > > + else > > + tlb_remove_ptdesc(tlb, ptdesc); > > } > > #endif /* CONFIG_MMU */ > > diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h > > index 1eb5682b2af6..a0b8b853503f 100644 > > --- a/arch/riscv/include/asm/tlb.h > > +++ b/arch/riscv/include/asm/tlb.h > > @@ -10,6 +10,24 @@ struct mmu_gather; > > static void tlb_flush(struct mmu_gather *tlb); > > +#ifdef CONFIG_MMU > > +#include > > + > > +/* > > + * While riscv platforms with riscv_ipi_for_rfence as true require an IPI to > > + * perform TLB shootdown, some platforms with riscv_ipi_for_rfence as false use > > + * SBI to perform TLB shootdown. To keep software pagetable walkers safe in this > > + * case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the > > + * comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h > > + * for more details. > > + */ > > +static inline void __tlb_remove_table(void *table) > > +{ > > + free_page_and_swap_cache(table); > > +} > > + > > +#endif /* CONFIG_MMU */ > > + > > #define tlb_flush tlb_flush > > #include _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv