From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C5B2C77B73 for ; Thu, 27 Apr 2023 07:31:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ELJvI5d9WGEeRx4Yoy8c0BcZ2WOPbkc/B2VT5+f+xzs=; b=nKhEfyOkJOvXNF 3FlpruP720CIp3+IiSOs6Ig3Wuh0ePc64ZWOyOeXOTJU4S2IIAHLsYdZeQSVPN6vC6vPTYTmr5A5p jcpOw1eXNyO8c9111Jhuk05yyGNIE05UHQstD8EU7h3r4d3G62pDBmtoSZp+nWMjizGNl7lV/QBvW HvTrJX2r2C1jmszAaQBgaXhuacvEpBxMWMhL4MU151YrMwvZ+npt61FS7MB4diYOiCyE2IU5NJ2/G mnkk1uuyT/xI1WuhD5/f4+9Z7nsYqg2fmpDeqfE7rXDVtr1HdIBK9JTqNO7HIFAs49PuSJVuOps2w VRF7UjK+49ib20aFd+mQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1prw5a-005gFr-2J; Thu, 27 Apr 2023 07:30:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1prw5W-005gEp-2l for linux-arm-kernel@lists.infradead.org; Thu, 27 Apr 2023 07:30:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EC8972F4; Thu, 27 Apr 2023 00:31:26 -0700 (PDT) Received: from FVFF77S0Q05N (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 840A63F64C; Thu, 27 Apr 2023 00:30:41 -0700 (PDT) Date: Thu, 27 Apr 2023 08:30:32 +0100 From: Mark Rutland To: Gang Li Cc: Will Deacon , Tomasz Nowicki , Laura Abbott , Catalin Marinas , Ard Biesheuvel , Anshuman Khandual , Kefeng Wang , Feiyang Chen , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [QUESTION FOR ARM64 TLB] performance issue and implementation difference of TLB flush Message-ID: References: <2eb026b8-9e13-2b60-9e14-06417b142ac9@bytedance.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <2eb026b8-9e13-2b60-9e14-06417b142ac9@bytedance.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230427_003046_989319_2399BE7F X-CRM114-Status: GOOD ( 25.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, On Thu, Apr 27, 2023 at 11:26:50AM +0800, Gang Li wrote: > Hi all, > > I have encountered a performance issue on our ARM64 machine, which seems > to be caused by the flush_tlb_kernel_range. Can you please provide a few more details on what you're seeing? What does your performance issue look like? Are you sure that the performance issue is caused by flush_tlb_kernel_range() specifically? > Here is the stack on the ARM64 machine: > > # ARM64: > ``` > ghes_unmap > clear_fixmap > __set_fixmap > flush_tlb_kernel_range > ``` > > As we can see, the ARM64 implementation eventually calls > flush_tlb_kernel_range, which flushes the TLB on all cores. However, on > AMD64, the implementation calls flush_tlb_one_kernel instead. > > # AMD64: > ``` > ghes_unmap > clear_fixmap > __set_fixmap > mmu.set_fixmap > native_set_fixmap > __native_set_fixmap > set_pte_vaddr > set_pte_vaddr_p4d > __set_pte_vaddr > flush_tlb_one_kernel > ``` > > On our ARM64 machine, flush_tlb_kernel_range is causing a noticeable > performance degradation. As above, could you please provide more details on this? > This arm64 patch said: > https://lore.kernel.org/all/20161201135112.15396-1-fu.wei@linaro.org/ > (commit 9f9a35a7b654e006250530425eb1fb527f0d32e9) > > ``` > /* > * Despite its name, this function must still broadcast the TLB > * invalidation in order to ensure other CPUs don't end up with junk > * entries as a result of speculation. Unusually, its also called in > * IRQ context (ghes_iounmap_irq) so if we ever need to use IPIs for > * TLB broadcasting, then we're in trouble here. > */ > static inline void arch_apei_flush_tlb_one(unsigned long addr) > { > flush_tlb_kernel_range(addr, addr + PAGE_SIZE); > } > ``` > > 1. I am curious to know the reason behind the design choice of flushing > the TLB on all cores for ARM64's clear_fixmap, while AMD64 only flushes > the TLB on a single core. Are there any TLB design details that make a > difference here? I don't know why arm64 only clears this on a single CPU. On arm64 we *must* invalidate the TLB on all CPUs as the kernel page tables are shared by all CPUs, and the architectural Break-Before-Make rules in require the TLB to be invalidated between two valid (but distinct) entries. > 2. Is it possible to let the ARM64 to flush the TLB on just one core, > similar to the AMD64? No. If we omitted the broadcast TLB invalidation, then a different CPU may fetch the old value into a TLB, then fetch the new value. When this happens, the architecture permits "amalgamation", with UNPREDICTABLE results, which could result in memory corruption, taking SErrors, etc. > 3. If so, would there be any potential drawbacks or limitations to > making such a change? As above, we must use broadcast TLB invalidation here. Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel