From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 359BFC4332F for ; Mon, 7 Nov 2022 22:55:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5B916B0071; Mon, 7 Nov 2022 17:55:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B0B378E0001; Mon, 7 Nov 2022 17:55:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FACB6B0074; Mon, 7 Nov 2022 17:55:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9125F6B0071 for ; Mon, 7 Nov 2022 17:55:35 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6E66D1C6249 for ; Mon, 7 Nov 2022 22:55:35 +0000 (UTC) X-FDA: 80108154630.25.7D6916D Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf30.hostedemail.com (Postfix) with ESMTP id 0845880006 for ; Mon, 7 Nov 2022 22:55:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=fJcfgxXmL+GKcXUyKfPyWxmmj0zZYYjlCkHwiQsbtvQ=; b=aYlFGfCyqs4VEApNYDQshl9gdT 1G6JUKI3EY+kbMzXNOlW5V1oHr1l6glDcDR5myrSfpNmPuKCrt+vpRWvURttBUURjAvZQTyYKCE8k wGTn6oDx7m1NUz2/39wP5UP46k18DHYUiJk6++0fQrJL+zbZGNgVB7zRP+60JIbF27TfDiF+phUXH eWE5kjn8wkfeO5183iSAJAIz/Q7RUpH+WbJU8kPsBwmXvCNiEiQ6tt593VQeaJ9V/1l2jrkFWuSSS DBN/ZC5QLoCtRmfzNmOefGKKR3s/1MRLGMcmuJQI6lkNqD19QHFcxh9/4XED1Kl2HDtS0AvvyVyir /C3vBIfQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1osB1Z-001A3Y-Ay; Mon, 07 Nov 2022 22:55:25 +0000 Date: Mon, 7 Nov 2022 14:55:25 -0800 From: Luis Chamberlain To: Song Liu Cc: bpf@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org, peterz@infradead.org, hch@lst.de, rick.p.edgecombe@intel.com, aaron.lu@intel.com, rppt@kernel.org, dave@stgolabs.net, torvalds@linux-foundation.org Subject: Re: [PATCH bpf-next v2 0/5] execmem_alloc for BPF programs Message-ID: References: <20221107223921.3451913-1-song@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221107223921.3451913-1-song@kernel.org> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667861734; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fJcfgxXmL+GKcXUyKfPyWxmmj0zZYYjlCkHwiQsbtvQ=; b=J/zhVz7bndiGGu8QcUuQz45hzSEx/I2kqILBV/f4aWPCYAB1zLou+hJb8mYgb8GDMIA0hK N+ol5a1VSLCYWhzRazcG/GXEg/b0pufBkbLFAGD98k9ljSXmQyr3j3ZJYgYuhbqIJuj3ao vb+tFyLfpbXyTKfd3HAXGvAR+KgW6SU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=aYlFGfCy; spf=none (imf30.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667861734; a=rsa-sha256; cv=none; b=WMiZaKakM97H9tzueglPiQ1iXNiE70dkpQWEwdPtikHjzrjCMSpU+BFJG+kU4HQOFOfnka Xc+NImAi+1G62xVY71DVi+yeG4bJWx6/HgzaS4FKd/cDcbbQSWXmCF51XWv+LsuhV6Fbnw U0D7b16X2JKtkDmrTLm6bNTIs5Yhk1M= X-Rspamd-Queue-Id: 0845880006 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=aYlFGfCy; spf=none (imf30.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-Rspamd-Server: rspam10 X-Rspam-User: X-Stat-Signature: omhmani1yudh5mwycf7ojenngcjyy1b8 X-HE-Tag: 1667861731-748102 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000021, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 07, 2022 at 02:39:16PM -0800, Song Liu wrote: > This patchset tries to address the following issues: > > 1. Direct map fragmentation > > On x86, STRICT_*_RWX requires the direct map of any RO+X memory to be also > RO+X. These set_memory_* calls cause 1GB page table entries to be split > into 2MB and 4kB ones. This fragmentation in direct map results in bigger > and slower page table, and pressure for both instruction and data TLB. > > Our previous work in bpf_prog_pack tries to address this issue from BPF > program side. Based on the experiments by Aaron Lu [4], bpf_prog_pack has > greatly reduced direct map fragmentation from BPF programs. You should be able to past the results there into the respecite commit from non-bpf-prog-pack to the new generalized solution here. > 2. iTLB pressure from BPF program > > Dynamic kernel text such as modules and BPF programs (even with current > bpf_prog_pack) use 4kB pages on x86, when the total size of modules and > BPF program is big, we can see visible performance drop caused by high > iTLB miss rate. This is arbitrary, please provide some real stat and in the commit with some reproducible benchmark. > 3. TLB shootdown for short-living BPF programs > > Before bpf_prog_pack loading and unloading BPF programs requires global > TLB shootdown. This patchset (and bpf_prog_pack) replaces it with a local > TLB flush. > > 4. Reduce memory usage by BPF programs (in some cases) > > Most BPF programs and various trampolines are small, and they often > occupies a whole page. From a random server in our fleet, 50% of the > loaded BPF programs are less than 500 byte in size, and 75% of them are > less than 2kB in size. Allowing these BPF programs to share 2MB pages > would yield some memory saving for systems with many BPF programs. For > systems with only small number of BPF programs, this patch may waste a > little memory by allocating one 2MB page, but using only part of it. Should be easy to provide some real numbers with at least selftests and onto the commit as well. > Based on our experiments [5], we measured 0.5% performance improvement > from bpf_prog_pack. This patchset further boosts the improvement to 0.7%. > The difference is because bpf_prog_pack uses 512x 4kB pages instead of > 1x 2MB page, bpf_prog_pack as-is doesn't resolve #2 above. > > This patchset replaces bpf_prog_pack with a better API and makes it > available for other dynamic kernel text, such as modules, ftrace, kprobe. And likewise here, please no arbitrary internal benchmark, real numbers. Luis