From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 249D9C77B75 for ; Mon, 15 May 2023 21:31:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h8vy7sxIx03mY3FQXWN8AlgYaUBN4WO+mCzycKKGe7E=; b=H3FZmoUvV8sFRi Xxlp3XK7mXvSwGfO1ffRZOiu99xZAlcnHV8OexcSI7en7taDfIcfEV5G6tcZtOXLqrc4ztY2zFnaa N+Y5fAq5tOogM3uuehGyCjd1oMvSBDsrU53oMFYE8+gPQs51/fjJQI7ReRFXtRZCn3Sho6l9zkepn upLMEzeho2x+OnONkmZMAH/zLNRYlE6il1lrEzcHvaudaROLDye8zDLDjJktQk3yQ2cdmTXlPHAA6 93z0gxgFnzHOwgjtrMe8kOC4N1kxdDJZBi4MmKJ6K89NpEW/iPNl9WcIozqqLYb82j4P6SnVDKZpN hwm0z21VoL42sPE5e3iw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pyfmx-003Ym0-2t; Mon, 15 May 2023 21:31:27 +0000 Received: from pandora.armlinux.org.uk ([2001:4d48:ad52:32c8:5054:ff:fe00:142]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pyfmv-003Yjj-0T for linux-arm-kernel@lists.infradead.org; Mon, 15 May 2023 21:31:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=DIvmSIRVOIKMZeHdo7YQVs1U/DxOWcq2bRz535cFiIE=; b=sNknB2LyuDLlgkAznc+sZRXj8l AEQdyDs1a4fSzuDBJzh0ja4J5wEiT6UoEV2mpgmxb9TdwrxMrioIjuzalXM0AL/UPLZefSuKgRbU0 wZ4Ss61DdRyrtBV0YupcjAbzHYnHbvldoEov2Z17uobuSKPt7weAkKibcCvVpNVnKZWWZUxVYHZrQ 3VSflSVQLldIX7agLpbaiAKSBTspDsUyO2Di1rDAKncFF/SomYD8ZvhmW0XBYBehMyd2n+3M1vVJV EnmO1gphX3WR54jc7+Ft/sSIwHRTG2ohf02pDs/fAR9MCrQW5sV8jfmCHEjmPejFOnT1INMjQtUEf Y1iSanLA==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:38002) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pyfme-0004eG-Tc; Mon, 15 May 2023 22:31:08 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.94.2) (envelope-from ) id 1pyfmb-00005k-OJ; Mon, 15 May 2023 22:31:05 +0100 Date: Mon, 15 May 2023 22:31:05 +0100 From: "Russell King (Oracle)" To: Thomas Gleixner Cc: Andrew Morton , linux-mm@kvack.org, Christoph Hellwig , Uladzislau Rezki , Lorenzo Stoakes , Peter Zijlstra , Baoquan He , John Ogness , linux-arm-kernel@lists.infradead.org, Mark Rutland , Marc Zyngier , x86@kernel.org Subject: Re: Excessive TLB flush ranges Message-ID: References: <87a5y5a6kj.ffs@tglx> <87353x9y3l.ffs@tglx> <87zg658fla.ffs@tglx> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87zg658fla.ffs@tglx> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230515_143125_184920_C8FBE182 X-CRM114-Status: GOOD ( 32.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, May 15, 2023 at 11:11:45PM +0200, Thomas Gleixner wrote: > On Mon, May 15 2023 at 21:46, Thomas Gleixner wrote: > > On Mon, May 15 2023 at 17:59, Russell King wrote: > >> On Mon, May 15, 2023 at 06:43:40PM +0200, Thomas Gleixner wrote: > > That reproduces in a VM easily and has exactly the same behaviour: > > > > Extra page[s] via The actual allocation > > _vm_unmap_aliases() Pages Pages Flush start Pages > > alloc: ffffc9000058e000 2 > > free : ffff888144751000 1 ffffc9000058e000 2 ffff888144751000 17312759359 > > > > alloc: ffffc90000595000 2 > > free : ffff8881424f0000 1 ffffc90000595000 2 ffff8881424f0000 17312768167 > > > > ..... > > > > seccomp seems to install 29 BPF programs for that process. So on exit() > > this results in 29 full TLB flushes on x86, where each of them is used > > to flush exactly three TLB entries. > > > > The actual two page allocation (ffffc9...) is in the vmalloc space, the > > extra page (ffff88...) is in the direct mapping. > > I tried to flush them one by one, which is actually slightly slower. > That's not surprising as there are 3 * 29 instead of 29 IPIs and the > IPIs dominate the picture. > > But that's not necessarily true for ARM32 as there are no IPIs involved > on the machine we are using, which is a dual-core Cortex-A9. > > So I came up with the hack below, which is equally fast as the full > flush variant while the performance impact on the other CPUs is minimally > lower according to perf. > > That probably should have another argument which tells how many TLBs > this flush affects, i.e. 3 in this example, so an architecture can > sensibly decide whether it wants to use flush all or not. > > Thanks, > > tglx > --- > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -1728,6 +1728,7 @@ static bool __purge_vmap_area_lazy(unsig > unsigned int num_purged_areas = 0; > struct list_head local_purge_list; > struct vmap_area *va, *n_va; > + struct vmap_area tmp = { .va_start = start, .va_end = end }; > > lockdep_assert_held(&vmap_purge_lock); > > @@ -1747,7 +1748,12 @@ static bool __purge_vmap_area_lazy(unsig > list_last_entry(&local_purge_list, > struct vmap_area, list)->va_end); > > - flush_tlb_kernel_range(start, end); > + if (tmp.va_end > tmp.va_start) > + list_add(&tmp.list, &local_purge_list); > + flush_tlb_kernel_vas(&local_purge_list); > + if (tmp.va_end > tmp.va_start) > + list_del(&tmp.list); So basically we end up iterating over each VA range, which seems sensible if the range is large and we have to iterate over it page by page. In the case you have, are "start" and "end" set on function entry to a range, or are they set to ULONG_MAX,0 ? What I'm wondering is whether we could get away with just having flush_tlb_kernel_vas(). Whether that's acceptable to others is a different question :) > + > resched_threshold = lazy_max_pages() << 1; > > spin_lock(&free_vmap_area_lock); > --- a/arch/x86/mm/tlb.c > +++ b/arch/x86/mm/tlb.c > @@ -10,6 +10,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -1081,6 +1082,24 @@ void flush_tlb_kernel_range(unsigned lon > } > } > > +static void do_flush_vas(void *arg) > +{ > + struct list_head *list = arg; > + struct vmap_area *va; > + unsigned long addr; > + > + list_for_each_entry(va, list, list) { > + /* flush range by one by one 'invlpg' */ > + for (addr = va->va_start; addr < va->va_end; addr += PAGE_SIZE) > + flush_tlb_one_kernel(addr); Isn't this just the same as: flush_tlb_kernel_range(va->va_start, va->va_end); at least on ARM32, it should be - the range will be iterated over in assembly instead of C, although it'll be out of line but should be slightly faster. Thanks. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel