From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7DCEC433FE for ; Fri, 4 Nov 2022 03:29:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4E396B0071; Thu, 3 Nov 2022 23:29:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFDB66B0073; Thu, 3 Nov 2022 23:29:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEC986B0074; Thu, 3 Nov 2022 23:29:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BB8F16B0071 for ; Thu, 3 Nov 2022 23:29:38 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9486D1C67AD for ; Fri, 4 Nov 2022 03:29:38 +0000 (UTC) X-FDA: 80094330036.23.D4385D2 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf25.hostedemail.com (Postfix) with ESMTP id BAEEAA0002 for ; Fri, 4 Nov 2022 03:29:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=bcSYSNvDHZdAJ46wXRu1CIpoQa17xIwvlEBsfNRpqtI=; b=xKgDuZO7/QGr1PXuSZbxoDjeKm JkZeptvQzbKI4Cni3sPxrOq0F8MglBd3bqb5Pb1V83fIHeL4agiMg7ZbMdPhfyiZscL7mx+d21+IN hyKIbEj9u/bKJPcDwIAT1/C5t8I0kDay23yJL/4XwdkOGpI109g4Fjtzp+6KWKQW/80yhUeo+4JCM H0HEhR5+NP2cBfQR273vw2iZ+cFvyPrVRpDc9NDwij9Bermy+Fva4hh8YM4Qc6ti+T2CifPf7cW2z eWifPTMguRvS/4k/zuUGsDBljVJwCXw6Zd/zgkbNQL8vxHw1WY5jLU6N6d4OALzwP+7S5uEOoblM2 M1nHs/+w==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oqnOT-002Ixy-Rt; Fri, 04 Nov 2022 03:29:21 +0000 Date: Thu, 3 Nov 2022 20:29:21 -0700 From: Luis Chamberlain To: "Edgecombe, Rick P" Cc: "rppt@kernel.org" , "p.raghav@samsung.com" , "peterz@infradead.org" , "bpf@vger.kernel.org" , "dave@stgolabs.net" , "willy@infradead.org" , "linux-mm@kvack.org" , "song@kernel.org" , "hch@lst.de" , "vbabka@suse.cz" , "zhengjun.xing@linux.intel.com" , "x86@kernel.org" , "akpm@linux-foundation.org" , "Torvalds, Linus" , "Hansen, Dave" , "kbusch@kernel.org" , "mgorman@suse.de" , "a.manzanares@samsung.com" Subject: Re: [PATCH bpf-next v1 RESEND 1/5] vmalloc: introduce vmalloc_exec, vfree_exec, and vcopy_exec Message-ID: References: <20221031222541.1773452-1-song@kernel.org> <20221031222541.1773452-2-song@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667532577; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bcSYSNvDHZdAJ46wXRu1CIpoQa17xIwvlEBsfNRpqtI=; b=n94w5wcleYhSxwc+/zyjig+oxWtFKgj7RDJXCyIXDlsYqb5stCBwv1cehNnpGyRqpDt7eX iRxR+2pbKIdXdA6kgk2J7bOiOaN4RdGd45Vd2TVL/Uy5ZQlY5ytojsyLq1qTDv7S6iVjON mgEPLb3YYOL4Wn7ulmY+00VL80IcmgM= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=xKgDuZO7; spf=none (imf25.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667532577; a=rsa-sha256; cv=none; b=DRtCGahrua2Dg8vHbs1JrYdREbg6YIsEg8NRXfwHFZV4Mmc+jc357W++fS01NItK4tmuGv loHBZcTJSkIRTbXQj4W5+00Ov03ETj+ZTY2jTrReMZ+AVXhE99wIEx+0cjId8x4pyvQVdO hrTWPXR7jbipZ4OPGsFbRKOHiIeANSg= X-Stat-Signature: niqhe5gmtyxykpgi3momh4y8q9oxg97m X-Rspamd-Queue-Id: BAEEAA0002 X-Rspamd-Server: rspam06 X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=xKgDuZO7; spf=none (imf25.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-HE-Tag: 1667532577-450183 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Nov 03, 2022 at 05:18:51PM -0700, Luis Chamberlain wrote: > On Thu, Nov 03, 2022 at 09:19:25PM +0000, Edgecombe, Rick P wrote: > > On Thu, 2022-11-03 at 11:59 -0700, Luis Chamberlain wrote: > > > > > Mike Rapoport had presented about the Direct map fragmentation > > > > > problem > > > > > at Plumbers 2021 [0], and clearly mentioned modules / BPF / > > > > > ftrace / > > > > > kprobes as possible sources for this. Then Xing Zhengjun's 2021 > > > > > performance > > > > > evaluation on whether using 2M/1G pages aggressively for the > > > > > kernel direct map > > > > > help performance [1] ends up generally recommending huge pages. > > > > > The work by Xing > > > > > though was about using huge pages *alone*, not using a strategy > > > > > such as in the > > > > > "bpf prog pack" to share one 2 MiB huge page for *all* small eBPF > > > > > programs, > > > > > and that I think is the real golden nugget here. > > > > > > > > > > I contend therefore that the theoretical reduction of iTLB misses > > > > > by using > > > > > huge pages for "bpf prog pack" is not what gets your systems to > > > > > perform > > > > > somehow better. It should be simply that it reduces fragmentation > > > > > and > > > > > *this* generally can help with performance long term. If this is > > > > > accurate > > > > > then let's please separate the two aspects to this. > > > > > > > > The direct map fragmentation is the reason for higher TLB miss > > > > rate, both > > > > for iTLB and dTLB. > > > > > > OK so then whatever benchmark is running in tandem as eBPF JIT is > > > hammered > > > should *also* be measured with perf for iTLB and dTLB. ie, the patch > > > can > > > provide such results as a justifications. > > > > Song had done some tests on the old prog pack version that to me seemed > > to indicate most (or possibly all) of the benefit was direct map > > fragmentation reduction. > > Matches my observations but I also provided quite a bit of hints as to > *why* I think that is. I suggested lib/test_kmod.c as an example beefy > multithreaded selftests which really kicks the hell out of the kernel > with whatever crap you want to run. That is precicely how I uncovered > some odd kmod bug lingering for years. *and*, *perhaps*... it may be that you need another memory intensive benchmark to run in tandem, one which mimics the behaviour of the internal "shadow production benchmark", whatever that is. Luis