From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 01064CD3441 for ; Tue, 19 Sep 2023 06:49:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ryxQcdWYsvJrzx1dMKgZK2tMIwqp/ieVf1RxBdqLuUU=; b=Bhk8b+iTXQDC/C Fx5SoawBdcWyoY/nZGFJzp3S3DwyAhv8C2vpUT0APH0eK2iTRh2LcIDpN8bd1zEQ3BMeIjRSHEZUv ia2h1BzpXfQBQmbNDbECDoDP5PolEdglOPtdhewuVyuuH4kBQqKaWmHKf0fcAlYH14XzIw798YilM dqsnVLHhIJfLY9PfxFD4OpsuOg/13jud575BKJOPzmRZE62A8G8WolJTEEBMfeeF6l9fSZ86Pgjho CqQXoS+W4DiSimQYS4sZgezLD1R5HyhdSeV9xxEh8xmkDI5Nh76cjdEayGzGAiM6d7LqTSyP2ZOFk RpRyLs/jPro8a3dg0iTA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qiUXG-00HA0A-1v; Tue, 19 Sep 2023 06:48:38 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qiUXD-00H9zN-39 for linux-arm-kernel@lists.infradead.org; Tue, 19 Sep 2023 06:48:37 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 08B54CE11A1; Tue, 19 Sep 2023 06:48:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45C45C433C8; Tue, 19 Sep 2023 06:48:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695106111; bh=v1PBG6Gu/3eqRITlKaenIy1Iaq3gKBzyqYlv0ai4zO8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=g81NpwhjgWC5UNpbUGov4u227vlRjEbZixKM1tjaZpaM/VY6T9DXSa/NEFVhg41PN vnrRJL/9n7SjreVQLTCw1zfG/HO8EMMJtOSa/z1D/CFbU114nEbsr06aQ520n/3u1j g1ERXarMOaeSRRWVrRVjkj3QcghAZT2RhMDgEx8AQqjIbY6048kFxlHKikjCWSyuLM iuv2rfpWoG3+m+AHxfjjzMFu11ynzIQlPTHtjh2xfDgQPFPLFv48dM8n6h4j5U4CaH Pokk/9WjdSFC6CHoDZa/xnP+QGd+sc3B23/ldGMkARS2g+aCBL6v5tzoijY2vKM65J rxr74ifBr+LMw== Date: Tue, 19 Sep 2023 09:47:44 +0300 From: Mike Rapoport To: Baolin Wang Cc: akpm@linux-foundation.org, will@kernel.org, aneesh.kumar@linux.ibm.com, npiggin@gmail.com, peterz@infradead.org, catalin.marinas@arm.com, chenhuacai@kernel.org, tsbogend@alpha.franken.de, dave.hansen@linux.intel.com, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, arnd@arndb.de, willy@infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org Subject: Re: [PATCH] mm: add statistics for PUD level pagetable Message-ID: <20230919064744.GE3303@kernel.org> References: <876c71c03a7e69c17722a690e3225a4f7b172fb2.1695017383.git.baolin.wang@linux.alibaba.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <876c71c03a7e69c17722a690e3225a4f7b172fb2.1695017383.git.baolin.wang@linux.alibaba.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230918_234836_487291_52AA2631 X-CRM114-Status: GOOD ( 26.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Sep 18, 2023 at 02:31:42PM +0800, Baolin Wang wrote: > Recently, we found that cross-die access to pagetable pages on ARM64 > machines can cause performance fluctuations in our business. Currently, > there are no PMU events available to track this situation on our ARM64 > machines, so an accurate pagetable accounting can help to analyze this > issue, but now the PUD level pagetable accounting is missed. > > So introducing pagetable_pud_ctor/dtor() to help to get an accurate > PUD pagetable accounting, as well as converting the architectures with > using generic PUD pagatable allocation to add corresponding PUD pagetable > accounting. Moreover this patch will also mark the PUD level pagetable > with PG_table flag, which will help to do sanity validation in unpoison_memory(). > > On my testing machine, I can see more pagetables statistics after the patch > with page-types tool: > > Before patch: > flags page-count MB symbolic-flags long-symbolic-flags > 0x0000000004000000 27326 106 __________________________g_________________ pgtable > After patch: > 0x0000000004000000 27541 107 __________________________g_________________ pgtable > > Signed-off-by: Baolin Wang Acked-by: Mike Rapoport (IBM) > --- > arch/arm64/include/asm/tlb.h | 5 ++++- > arch/loongarch/include/asm/pgalloc.h | 1 + > arch/mips/include/asm/pgalloc.h | 1 + > arch/x86/mm/pgtable.c | 3 +++ > include/asm-generic/pgalloc.h | 7 ++++++- > include/linux/mm.h | 16 ++++++++++++++++ > 6 files changed, 31 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h > index 2c29239d05c3..846c563689a8 100644 > --- a/arch/arm64/include/asm/tlb.h > +++ b/arch/arm64/include/asm/tlb.h > @@ -96,7 +96,10 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp, > static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pudp, > unsigned long addr) > { > - tlb_remove_ptdesc(tlb, virt_to_ptdesc(pudp)); > + struct ptdesc *ptdesc = virt_to_ptdesc(pudp); > + > + pagetable_pud_dtor(ptdesc); > + tlb_remove_ptdesc(tlb, ptdesc); > } > #endif > > diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h > index 79470f0b4f1d..4e2d6b7ca2ee 100644 > --- a/arch/loongarch/include/asm/pgalloc.h > +++ b/arch/loongarch/include/asm/pgalloc.h > @@ -84,6 +84,7 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address) > > if (!ptdesc) > return NULL; > + pagetable_pud_ctor(ptdesc); > pud = ptdesc_address(ptdesc); > > pud_init(pud); > diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h > index 40e40a7eb94a..f4440edcd8fe 100644 > --- a/arch/mips/include/asm/pgalloc.h > +++ b/arch/mips/include/asm/pgalloc.h > @@ -95,6 +95,7 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address) > > if (!ptdesc) > return NULL; > + pagetable_pud_ctor(ptdesc); > pud = ptdesc_address(ptdesc); > > pud_init(pud); > diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c > index 9deadf517f14..0cbc1b8e8e3d 100644 > --- a/arch/x86/mm/pgtable.c > +++ b/arch/x86/mm/pgtable.c > @@ -76,6 +76,9 @@ void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd) > #if CONFIG_PGTABLE_LEVELS > 3 > void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud) > { > + struct ptdesc *ptdesc = virt_to_ptdesc(pud); > + > + pagetable_pud_dtor(ptdesc); > paravirt_release_pud(__pa(pud) >> PAGE_SHIFT); > paravirt_tlb_remove_table(tlb, virt_to_page(pud)); > } > diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h > index c75d4a753849..879e5f8aa5e9 100644 > --- a/include/asm-generic/pgalloc.h > +++ b/include/asm-generic/pgalloc.h > @@ -169,6 +169,8 @@ static inline pud_t *__pud_alloc_one(struct mm_struct *mm, unsigned long addr) > ptdesc = pagetable_alloc(gfp, 0); > if (!ptdesc) > return NULL; > + > + pagetable_pud_ctor(ptdesc); > return ptdesc_address(ptdesc); > } > > @@ -190,8 +192,11 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) > > static inline void __pud_free(struct mm_struct *mm, pud_t *pud) > { > + struct ptdesc *ptdesc = virt_to_ptdesc(pud); > + > BUG_ON((unsigned long)pud & (PAGE_SIZE-1)); > - pagetable_free(virt_to_ptdesc(pud)); > + pagetable_pud_dtor(ptdesc); > + pagetable_free(ptdesc); > } > > #ifndef __HAVE_ARCH_PUD_FREE > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 12335de50140..2232bfebb88a 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -3049,6 +3049,22 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud) > return ptl; > } > > +static inline void pagetable_pud_ctor(struct ptdesc *ptdesc) > +{ > + struct folio *folio = ptdesc_folio(ptdesc); > + > + __folio_set_pgtable(folio); > + lruvec_stat_add_folio(folio, NR_PAGETABLE); > +} > + > +static inline void pagetable_pud_dtor(struct ptdesc *ptdesc) > +{ > + struct folio *folio = ptdesc_folio(ptdesc); > + > + __folio_clear_pgtable(folio); > + lruvec_stat_sub_folio(folio, NR_PAGETABLE); > +} > + > extern void __init pagecache_init(void); > extern void free_initmem(void); > > -- > 2.39.3 > > -- Sincerely yours, Mike. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel