From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0079BC7EE24 for ; Mon, 15 May 2023 19:47:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oLmaB1VYswZHRdvONkgP6EsZk9EH5eps+xyWXl/agGg=; b=E/xNuvG5GTCNAB aC/rV2bzketL6DpYFlRU3tcv5KPPsUs5kuZU4FC9pqFmTAq3jnM0znUiEVOdCqUmCXny/WtVjb4SQ jrM7CBtoruOBZmc/8RfMV6CxXcA94tJOaJpDSKMK7946GXMIA0+d3i61tPShm1fWNV0W6jaS45446 KE1P7tdkP+JqsoqGcOGfTGbcsJTrX816jbi6hpBoLSJye4RPxvJ8dvG1uhKMn41XD3sdsV5HENb12 ZRzHfMPXqiaZWn3U8rfY7CqxYj+64Rf/AZSu2TdKuvYY8cLkrlZUYB8hEiMsFYtkH7jY4zFjQIwgR B9q6RBteHQaGwFRpv1aQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pye9f-003Juv-1g; Mon, 15 May 2023 19:46:47 +0000 Received: from galois.linutronix.de ([193.142.43.55]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pye9c-003Jt1-2g for linux-arm-kernel@lists.infradead.org; Mon, 15 May 2023 19:46:46 +0000 From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1684179999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6fgURv4U0P1Gi8D14D0+ZT9POL819NJFtWCbYalWclc=; b=iq3CnA9HTWbDFQtJTbehbmhSY8ZftE9nfCmY/WFZCEz8V27QNWp3G1xoXe42RyBvbyhCVn hXMzwHb3PdDXNjhr7B+Mcf8IvHeiIe9INa/uPIffPYLs5yi3EVRsyBwfCJyBE6y37AQAWU hBpT+ZPflVSKUHwVSwPYMHptfJMr9HPkkrxJ/JySiI/Ba6+AKLgUPeWtobwqQBsDL94AJy fuDglD1OZ7FSk6xL+iUuVCeM71iSD8J0MMRTj5eQQK+CCzghVhRrtfA8U58bU5L3ZETYsF 70scZIvhfWpZapnEuWM8uvMUrv4PMM4RFMVWDOFwsEiAVUHZ3OrvIMV9pqQC1w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1684179999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6fgURv4U0P1Gi8D14D0+ZT9POL819NJFtWCbYalWclc=; b=Kec+mAjxrAaOyun3rDIky9jiMyw7/PHAsYhHrBqJkQ7ne3vXmj/ISiYJRV6jRCInxTUTOq GBQDVK3f/n1RhUCQ== To: "Russell King (Oracle)" Cc: Andrew Morton , linux-mm@kvack.org, Christoph Hellwig , Uladzislau Rezki , Lorenzo Stoakes , Peter Zijlstra , Baoquan He , John Ogness , linux-arm-kernel@lists.infradead.org, Mark Rutland , Marc Zyngier , x86@kernel.org Subject: Re: Excessive TLB flush ranges In-Reply-To: References: <87a5y5a6kj.ffs@tglx> Date: Mon, 15 May 2023 21:46:38 +0200 Message-ID: <87353x9y3l.ffs@tglx> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230515_124645_034466_D5D0AC85 X-CRM114-Status: GOOD ( 18.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, May 15 2023 at 17:59, Russell King wrote: > On Mon, May 15, 2023 at 06:43:40PM +0200, Thomas Gleixner wrote: >> bpf_prog_free_deferred() >> vfree() >> _vm_unmap_aliases() >> collect_per_cpu_vmap_blocks: start:0x95c8d000 end:0x95c8e000 size:0x1000 >> __purge_vmap_area_lazy(start:0x95c8d000, end:0x95c8e000) >> >> va_start:0xf08a1000 va_end:0xf08a5000 size:0x00004000 gap:0x5ac13000 (371731 pages) >> va_start:0xf08a5000 va_end:0xf08a9000 size:0x00004000 gap:0x00000000 ( 0 pages) >> va_start:0xf08a9000 va_end:0xf08ad000 size:0x00004000 gap:0x00000000 ( 0 pages) >> va_start:0xf08ad000 va_end:0xf08b1000 size:0x00004000 gap:0x00000000 ( 0 pages) >> va_start:0xf08b3000 va_end:0xf08b7000 size:0x00004000 gap:0x00002000 ( 2 pages) >> va_start:0xf08b7000 va_end:0xf08bb000 size:0x00004000 gap:0x00000000 ( 0 pages) >> va_start:0xf08bb000 va_end:0xf08bf000 size:0x00004000 gap:0x00000000 ( 0 pages) >> va_start:0xf0a15000 va_end:0xf0a17000 size:0x00002000 gap:0x00156000 ( 342 pages) >> >> flush_tlb_kernel_range(start:0x95c8d000, end:0xf0a17000) >> >> Does 372106 flush operations where only 31 are useful > > So, you asked the architecture to flush a large range, and are then > surprised if it takes a long time. There is no way to know how many > of those are useful. I did not ask for that. That's the merge ranges logic in __purge_vmap_area_lazy() which decides that the one page at 0x95c8d000 should build a flush range with the rest. I'm just the messenger :) > Now, while using the sledge hammer of flushing all TLB entries may > sound like a good answer, if we're only evicting 31 entries, the > other entries are probably useful to have, no? That's what I was asking already in the part you removed from the reply, no? > I think that you'd only run into this if you had a huge BPF > program and you tore it down, no? There was no huge BPF program. Some default seccomp muck. I have another trace which shows that seccomp creates 10 BPF programs for one process where each allocates 8K vmalloc memory in the 0xf0a.... address range. On teardown this is even more horrible than the above. Every allocation is deleted separately, i.e. 8k at a time and the pattern is always the same. One extra page in the 0xca6..... address range is handed in via _vm_unmap_aliases(), which expands the range insanely. So this means ~1.5M flush operations to flush a total of 30 TLB entries. That reproduces in a VM easily and has exactly the same behaviour: Extra page[s] via The actual allocation _vm_unmap_aliases() Pages Pages Flush start Pages alloc: ffffc9000058e000 2 free : ffff888144751000 1 ffffc9000058e000 2 ffff888144751000 17312759359 alloc: ffffc90000595000 2 free : ffff8881424f0000 1 ffffc90000595000 2 ffff8881424f0000 17312768167 ..... seccomp seems to install 29 BPF programs for that process. So on exit() this results in 29 full TLB flushes on x86, where each of them is used to flush exactly three TLB entries. The actual two page allocation (ffffc9...) is in the vmalloc space, the extra page (ffff88...) is in the direct mapping. This is a plain debian install with the a 6.4-rc1 kernel. The reproducer is: # systemctl start logrotate Thanks, tglx _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel