From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34DF7C43334 for ; Tue, 12 Jul 2022 19:04:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 928459400CA; Tue, 12 Jul 2022 15:04:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D7B9940063; Tue, 12 Jul 2022 15:04:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79DFE9400CA; Tue, 12 Jul 2022 15:04:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6A5E8940063 for ; Tue, 12 Jul 2022 15:04:54 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 37EF920519 for ; Tue, 12 Jul 2022 19:04:54 +0000 (UTC) X-FDA: 79679374908.31.793D9F7 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf24.hostedemail.com (Postfix) with ESMTP id 47FE8180075 for ; Tue, 12 Jul 2022 19:04:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=5zn6hvHj4eKtfzqRScUEHMiIgZur5MyCCs29W1zBYWo=; b=xxs5oqmrZEfO2E2kxTPGNrkRA7 vBxlIZp9V+hqHw7UsxneA0WB6uEnzq62HTiQMSJlzBKphdX/f13tQpG50mucKGSRLyum5pR1eqxDj RFiCrTx/E6tlShgUgl6TZtHLYONAUGmmDpljk3YyL2wNRddKtfsrQFBX31dnnl4UmgIcDQn6Bhiy2 R1AYFBFhR1bQRM6WmKGxKr2aLGpVzm5FLCWYcpg23IdcOzHEdsHa14+A8AmKHKpyw95W/1onf4lBW HyF0WFSv56iDl0v13QjsccAx2nNtGNXoTClDAz2uatbs1mqzeOgB8GGcmUtfCbvtv9ss8P+BrIeKb gVzKuaVA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oBLBN-00E3qI-Ub; Tue, 12 Jul 2022 19:04:29 +0000 Date: Tue, 12 Jul 2022 12:04:29 -0700 From: Luis Chamberlain To: Song Liu Cc: Peter Zijlstra , Steven Rostedt , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Masami Hiramatsu , "Naveen N. Rao" , "David S. Miller" , Anil S Keshavamurthy , Kees Cook , Song Liu , bpf , Christoph Hellwig , Davidlohr Bueso , lkml , Linux-MM , Daniel Borkmann , Kernel Team , "x86@kernel.org" , "dave.hansen@linux.intel.com" , "rick.p.edgecombe@intel.com" , "linux-modules@vger.kernel.org" Subject: Re: [PATCH v6 bpf-next 0/5] bpf_prog_pack followup Message-ID: References: <863A2D5B-976D-4724-AEB1-B2A494AD2BDB@fb.com> <6214B9C9-557B-4DC0-BFDE-77EAC425E577@fb.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657652692; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5zn6hvHj4eKtfzqRScUEHMiIgZur5MyCCs29W1zBYWo=; b=1Qs4e7C/vpfh7eVYmH5yEM8P0Ju+9BwXJOr4qf4enWPYWU3C9qjDs71Greu6wtmqbRsqXN yJTG9nnh0JLSytp/yiwZwwz55O4z4TmSPHUTIQFiyBwKIEbcJRkK/HY5RLbKiXB8vtekng f9snBxBV5iKtfOtHSB/1e6rDvKLCT3w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657652692; a=rsa-sha256; cv=none; b=2eowaeCSI94Ll2eHxiTEpWhvnsmq7liRlE07+QzfJ3mk1Qb6jm4PzmlAw1Vw8GfIsrrplz CAxHkvd09Y2ybDT5MMC8MGIp/HN5+gKxZNdaGN4/keUQkX0VPGHqQOBHwrs+7Q+W+Aghp2 +gIJCahWBVMAN5XtKxbiNxQCdYS+zXs= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=xxs5oqmr; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf24.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org X-Stat-Signature: r6hfwrfp19qxxs1jddc6oufw5f9titxg X-Rspamd-Queue-Id: 47FE8180075 X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=xxs5oqmr; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf24.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org X-Rspamd-Server: rspam05 X-HE-Tag: 1657652692-330090 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 12, 2022 at 05:49:32AM +0000, Song Liu wrote: > > On Jul 11, 2022, at 9:18 PM, Luis Chamberlain wrote: > > > I believe you are mentioning requiring text_poke() because the way > > eBPF code uses the module_alloc() is different. Correct me if I'm > > wrong, but from what I gather is you use the text_poke_copy() as the data > > is already RO+X, contrary module_alloc() use cases. You do this since your > > bpf_prog_pack_alloc() calls set_memory_ro() and set_memory_x() after > > module_alloc() and before you can use this memory. This is a different type > > of allocator. And, again please correct me if I'm wrong but now you want to > > share *one* 2 MiB huge-page for multiple BPF programs to help with the > > impact of TLB misses. > > Yes, sharing 1x 2MiB huge page is the main reason to require text_poke. > OTOH, 2MiB huge pages without sharing is not really useful. Both kprobe > and ftrace only uses a fraction of a 4kB page. Most BPF programs and > modules cannot use 2MiB either. Therefore, vmalloc_rw_exec() doesn't add > much value on top of current module_alloc(). Thanks for the clarification. > > A vmalloc_ro_exec() by definition would imply a text_poke(). > > > > Can kprobes, ftrace and modules use it too? It would be nice > > so to not have to deal with the loose semantics on the user to > > have to use set_vm_flush_reset_perms() on ro+x later, but > > I think this can be addressed separately on a case by case basis. > > I am pretty confident that kprobe and ftrace can share huge pages with > BPF programs. Then wonderful, we know where to go in terms of a new API then as it can be shared in the future for sure and there are gains. > I haven't looked into all the details with modules, but > given CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC, I think it is also > possible. Sure. > Once this is done, a regular system (without huge BPF program or huge > modules) will just use 1x 2MB page for text from module, ftrace, kprobe, > and bpf programs. That would be nice, if possible, however modules will require likely its own thing, on my system I see about 57 MiB used on coresize alone. lsmod | grep -v Module | cut -f1 -d ' ' | \ xargs sudo modinfo | grep filename | \ grep -o '/.*' | xargs stat -c "%s - %n" | \ awk 'BEGIN {sum=0} {sum+=$1} END {print sum}' 60001272 And so perhaps we need such a pool size to be configurable. > > But a vmalloc_ro_exec() with a respective free can remove the > > requirement to do set_vm_flush_reset_perms(). > > Removing the requirement to set_vm_flush_reset_perms() is the other > reason to go directly to vmalloc_ro_exec(). Yes fantastic. > My current version looks like this: > > void *vmalloc_exec(unsigned long size); > void vfree_exec(void *ptr, unsigned int size); > > ro is eliminated as there is no rw version of the API. Alright. I am not sure if 2 MiB will suffice given what I mentioned above, and what to do to ensure this grows at a reasonable pace. Then, at least for usage for all architectures since not all will support text_poke() we will want to consider a way to make it easy to users to use non huge page fallbacks, but that would be up to those users, so we can wait for that. > The ugly part is @size for vfree_exec(). We need it to share huge > pages. I suppose this will become evident during patch review. > Under the hood, it looks similar to current bpf_prog_pack_alloc > and bpf_prog_pack_free. Groovy. Luis