From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCC1DC4332F for ; Mon, 7 Nov 2022 18:30:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 67DAD6B0071; Mon, 7 Nov 2022 13:30:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 62D316B0072; Mon, 7 Nov 2022 13:30:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F4C36B0073; Mon, 7 Nov 2022 13:30:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3EE1C6B0071 for ; Mon, 7 Nov 2022 13:30:19 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id EA726160D2C for ; Mon, 7 Nov 2022 18:30:18 +0000 (UTC) X-FDA: 80107486116.01.252DB39 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf11.hostedemail.com (Postfix) with ESMTP id 1F1DA40002 for ; Mon, 7 Nov 2022 18:30:16 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id EB3CDB81619 for ; Mon, 7 Nov 2022 18:30:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 186B8C4347C for ; Mon, 7 Nov 2022 18:30:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1667845814; bh=o/YCtXWPnRRvkfEvLC2RGV2wHGxZZXcfgU1DXVdalWI=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=KmvozpLhtG5nUW3KxXmhv7tluTCkwfWoTLpkHaH7+J7xeTZSp9Nf6Y6cjQtRrJGxZ b8sM/q//O+/yVdqr9mzbh1bh2c35ik/dRIjCRvw3LLhBT5YUOtwClt7hEB7GXuoLT9 jtcy/O1FJsI1Uw19T61/n8W8gFPic402kjjnr+BeK4DKpD7Q9glUlRD6+v2jH7nFFV UvHxUskRoghYeNT6yzw5AGItKcwtYU0ciBHjgKfVVYrpB2MpjRP8TDGyhj4XM+J0j6 mRQ900WDNCq3XlA1kY3zeqTXD/57x+T/A5c+jHq49OeGO0HOyJaW219sEaVnWtgzOX Ha1HkLe2VjYmQ== Received: by mail-ed1-f53.google.com with SMTP id i21so18955260edj.10 for ; Mon, 07 Nov 2022 10:30:13 -0800 (PST) X-Gm-Message-State: ACrzQf2l1lIQo7es5r7adlDvZ9DMDmlINcGPUYbCiVN1vU8Uwk4NjfIx IkUOS+YzBMp5NgnejZSPvmYOGFg3/02ic8vx78M= X-Google-Smtp-Source: AMsMyM6yvL8EBv5HsTSCgTpShLlvGOhe/tgzzJvjhG/rTtLgMeXubj2JsRZW3Ni6upqc/ZiFV+/K7g2Gh43dnSYJbss= X-Received: by 2002:aa7:d710:0:b0:463:bd7b:2b44 with SMTP id t16-20020aa7d710000000b00463bd7b2b44mr35604880edq.385.1667845812235; Mon, 07 Nov 2022 10:30:12 -0800 (PST) MIME-Version: 1.0 References: <20221031222541.1773452-1-song@kernel.org> <20221031222541.1773452-2-song@kernel.org> In-Reply-To: From: Song Liu Date: Mon, 7 Nov 2022 10:30:00 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH bpf-next v1 RESEND 1/5] vmalloc: introduce vmalloc_exec, vfree_exec, and vcopy_exec To: Aaron Lu Cc: Luis Chamberlain , bpf@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org, peterz@infradead.org, hch@lst.de, rick.p.edgecombe@intel.com, dave.hansen@intel.com, rppt@kernel.org, zhengjun.xing@linux.intel.com, kbusch@kernel.org, p.raghav@samsung.com, dave@stgolabs.net, vbabka@suse.cz, mgorman@suse.de, willy@infradead.org, torvalds@linux-foundation.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Content-Type: text/plain; charset="UTF-8" ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KmvozpLh; spf=pass (imf11.hostedemail.com: domain of song@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=song@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667845817; a=rsa-sha256; cv=none; b=5JS0SVPIna7NWebRFYmLyrBG+xDFCgOvTo+rEUHvi1IqGqD2C8EEkL6p/ck374rS0AaZ6h OnTI5E18poG9RlANMo8kAsjU+lifD4137p5CdGhYVul+a6ZN1K2tSoS+biuie6u0pqliup PCVBzF1wrsBjhAf78MEpRBMUWYDDYS8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667845817; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o/YCtXWPnRRvkfEvLC2RGV2wHGxZZXcfgU1DXVdalWI=; b=fMPMUzUOUlhL7or/VnAiRrZ5x6rGntUV4C728aPwp4tM2hcGPlVqDDVjInOLe0RsDdXp1I Hna/0IiOcvAZ23jnEmcljDalko2CcgrCtB+B0+5oPUL8Uu1Ohw2tIVidfcSaeh6YIa6yJq XarkswkfUR+/iXN9e6GB1HtnXV6OxX0= X-Rspam-User: Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KmvozpLh; spf=pass (imf11.hostedemail.com: domain of song@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=song@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1F1DA40002 X-Stat-Signature: 8yo46akuyj388p1k44niojoccyi8t9z7 X-HE-Tag: 1667845816-458654 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Aaron, On Sun, Nov 6, 2022 at 10:40 PM Aaron Lu wrote: > [...] > > and that I think is the real golden nugget here. > > I'm interested in how this patchset (further) improves direct map > fragmentation so would like to evaluate it to see if my previous work to > merge small mappings back in architecture layer[1] is still necessary. > > I tried to apply this patchset on v6.1-rc3/2/1 and v6.0 but all failed, > so I took one step back and evaluated the existing bpf_prog_pack. I'm > aware of this patchset would make things even better by using order-9 > page to backup the vmalloced range. The patchset was based on bpf-next tree: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/ > > I used the sample bpf prog: sample/bpf/sockex1 because it looks easy to > run, feel free to let me know a better way to evaluate this. > > - In kernels before bpf_prog_pack(v5.17 and earlier), this prog would > cause 3 pages to change protection to RO+X from RW+NX; And if the three > pages are far apart, each would cause a level 3 split then a level 2 > split; Reality is, allocated pages tend to stay close physically so > actual result will not be this bad. [...] > > - v6.1-rc3 > There is no difference because I can't trigger another pack alloc before > system is OOMed. > > Conclusion: I think bpf_prog_pack is very good at reducing direct map > fragmentation and this patchset can further improve this situation on > large machines(with huge amount of memory) or with more large bpf progs > loaded etc. Thanks a lot for these experiments! I will include the data in the next version of the set. > > Some imperfect things I can think of are(not related to this patchset): > 1 Once a split happened, it remains happened. This may not be a big deal > now with bpf_prog_pack and this patchset because the need to allocate a > new order-9 page and thus cause a potential split should happen much much > less; I think we will need to regroup the direct map for some scenarios. But I am not aware of such workloads. > 2 When a new order-9 page has to be allocated, there is no way to tell > the allocator to allocate this order-9 page from an already splitted PUD > range to avoid another PUD mapping split; This would be a good improvement. > 3 As Mike and others have mentioned, there are other users that can also > cause direct map split. Thanks, Song