From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47446C8303C for ; Wed, 9 Jul 2025 01:53:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC1D76B00A6; Tue, 8 Jul 2025 21:53:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A72876B00A7; Tue, 8 Jul 2025 21:53:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 919306B00A9; Tue, 8 Jul 2025 21:53:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7D1C96B00A6 for ; Tue, 8 Jul 2025 21:53:20 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2B3E9C02CF for ; Wed, 9 Jul 2025 01:53:20 +0000 (UTC) X-FDA: 83643053760.09.5398CC6 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf07.hostedemail.com (Postfix) with ESMTP id 4818940004 for ; Wed, 9 Jul 2025 01:53:18 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ib6b8O88; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752025998; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oGJ5phVBe26AHffUjtQ0C5chQhrjMeWzPyV2gXCSEws=; b=eaFcTEwpQqCNOtY9s7GLkQxSo12DyeF8J7OIgGhiCLhCkshsj1ice3mQ1ydPzskUBZKRK+ SY+YzalC4y3CE1BwFdcTPvng3jwIJ8vv1DPlUHhU3F2XaGPb7dMuMDI3+Vaqv4udjNO7m+ sJ2ajyETSeA2YtLPyDsvGftnC7zxGeI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752025998; a=rsa-sha256; cv=none; b=5nTSKDTy2pTFC7glj5jK64KwQJ+7VjQBJdpq2zeeDfN68rBuUNWI1Ztve7L10PXxX9X56N I9utSHZrRi7Zv+dah292Ew1EogUlNf8JoZzv0NmDBaZi3qjKTyxvLlfevPQFa25BX6FpMw HVSYoureyz3QXYvZJk7Lc0BhdBBRu2Q= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ib6b8O88; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-879d2e419b9so3942710a12.2 for ; Tue, 08 Jul 2025 18:53:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752025997; x=1752630797; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oGJ5phVBe26AHffUjtQ0C5chQhrjMeWzPyV2gXCSEws=; b=ib6b8O88mTEvLHgV5bVaFFDqTyfn4C0A8cK5libvDqq8Y5lmrS1HqOS4sNQh/WIOjv Rq8fAC6HiajFU3Jmmy0Y6T0b4hAHPv4utj0qJVzMR9++oefNg7N8oNqQV1WETYPsmjbl DDIBKuYObSQPepzXV034Psj4vNwGd+5e0c7AJ7moXKxjbbi3qu4raZ9eCxKUdV/tsL4Q T/6jm95iI+r25J3h1DGveN7M8wCYtlxMjbiH4YruymbiS0gJe9Pcf9MaIKWWWo7o4L0w LHW5Iv8I3rDE8HmPuuuBh+ta6MatY6B5RtmmsAXSRkWujsWv8KQ42MYQKvPZbXaGR/wx 6Mvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752025997; x=1752630797; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oGJ5phVBe26AHffUjtQ0C5chQhrjMeWzPyV2gXCSEws=; b=QIednQs5Z4pQ2lDLBzth79GQ/vdT8oHh7HHSROk4DJjRMvqKbe2N3PcHmEvyOZ8NgE o0rvk2543kYLIbIm0p9QNSmrKPvAPaeGJ/KwVhr7ADC+pmeyKqK928Z+72ckBbtq7sdW hejCNdv2ZgEd+l1BOJNNQ6kYD/iVtxLThf0MFYQRD6TnUX5ukWXoKiJSeMiqdd6Q1sJc 87GsiboBIZIUSBnE7OraEv5lUJWf8xW0JilZkkSb86GGOd4V4gk/+ku0wLNDStG2EnUY OhLd0aY5Svj79b7a4+nae/9EuOhh4HH/8TXcka8dpavgwfCT/XdU670RzfZiSarPZd/f nVqQ== X-Forwarded-Encrypted: i=1; AJvYcCVYgyeM3DW255SZP5l9YqxG978HosTim6IfBhwQ1XoUuis0/h8teED/1jk10cMSu+2JK1+kdqyOcg==@kvack.org X-Gm-Message-State: AOJu0YyenQTNFqJn1tF++6vRWrICE1GHne+mbJ45kAfSJuKrmL4mzOcV Y+jVtZGhNiEErq9B/8k5Q+fMVqw+eEfBl8n20cEBjJV9eje8oHfToMeB X-Gm-Gg: ASbGncta3bd7vBNmP6R5Qt43WlFQxbjBvcpmKx3xfMHwyGBX82BsZDuhBXUJefQ9eci s+pyuQMJjpi8B1aZzbJYZjkxtN4p4qnLu+AD0ol3yTJUCcIVdAoIJSoK+6gHSf7abpOVIN1jTw8 SKr/8cIr3Uou/g1DGF53t6C0vignAx7e0Q94w8jqL2/Q8eqTUZUQIWnNXgAGQIsbDjaxd3Be6Bo /uaH6C66H4jq6T+MHBGByZwPnqiMWZePVr/yrh/ai4bHtEcGUL5k6HS2pilqMnKhavtQhyPi2YU qyow6KR58m3BoqRbJamFUheM051uBs0dW/xuii8Lr/Vh9S+LasP2W/MDtOGw9rn+MyLTOF82pfK DLlqJ9CXvoW0Ac4FNKlQHMhA7zDw= X-Google-Smtp-Source: AGHT+IHGHvw2R/ZLy4YM57AnOJL7xjem1ShfJH6FuzBAF2fTOGwIMvfNS2UPCGKW+uTTJb4HLFOW6w== X-Received: by 2002:a17:90b:1cc3:b0:311:b0d3:865 with SMTP id 98e67ed59e1d1-31c2fe04666mr916648a91.32.1752025996974; Tue, 08 Jul 2025 18:53:16 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-31c300f7177sm453103a91.45.2025.07.08.18.53.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 08 Jul 2025 18:53:16 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v2 4/6] mm: Allow GFP_ACCOUNT to be used in alloc_pages_nolock(). Date: Tue, 8 Jul 2025 18:53:01 -0700 Message-Id: <20250709015303.8107-5-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250709015303.8107-1-alexei.starovoitov@gmail.com> References: <20250709015303.8107-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4818940004 X-Stat-Signature: xckj5otmkirb4gaqsqjwf3c5z8y55qtm X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1752025998-52147 X-HE-Meta: U2FsdGVkX19gkaYAOmt6qUGExE7lEYOLWoQRGKqodVYZcQl+0aO+wv39XyyOperXeiAHmD91JCC/fePAzV5lLhFDiBLL2eh/IQXxaDKe/3u1UqJ2qIww9pGDLVIIBGsu+Hr3Ok4erxa9SA9EFSpHsyeE5jKusErwgu+QtEqAGGxMHvPkBkvqAKNlg+WENEPuqaHw0mXM6/M3NTXSp4VcEyeUcNEcOLyPBNG74Wdgk/ImBjfV6rikNsYbk34EcfUCmX9FHDZmyOzXNi4iWTQ1JVE87KTN25kmDk0JKU6YGEci045wod9db4BF5pnpyfUyqb04K3RDfgDjt3I7AWogj20OVE9M5uLP630/fYb8z9MStStXOCahAVT7B0cnr9t2CnIHUzzNtL1HN5TbG9zSZ6/iwMa0SWTdJ4dMSFObUrozghJ7iiumQOz1YHEkZgFq8bIdyq+VMuuCAkOcyecHBjXiRGxw2j9Zptv6IYpIGUYgMYtpdy2PJQhUlAZt47p953b4AEx8Cza9Up3S2vU6Xyj96JsnV/UMINQBsAyQBbEBnWQ+is1/0t1IrVCtAsSW467wq1TplMQBy/N5GA2xIOaFvBlamAZ+Fivlq5w8r7BJ0VCZe2OnZV2kDkxaMs4WPdJp9v8AwS8DG8S1FZVrVN8w70ds9glr4DvJ9+ahvWtlaZDTY7EWziz5Ciw/ymw93zPIwewcURW7NtIvaNM9YBkPxkNu+GShRyxj6TYy++VG319MHs6PilRw6fomV2ScOnvJNLsm9qouBFgqucy33s2LPnk1ygENDnldqyinyA8m/+5jtj4zXBy2JLXVFsWtnY3Z6sGY2ee6GAnI01xzbxWss8nm84Umg/cxL0CD+eqx0eu/3Zrh2now7f7WbI4rPr3XsiXtZ5l++gPeThSJgFXuH2+qIY6vJgDstks3gquqVN6rSuGhOj0zoNaJp8rSOUVsD7/Z1/jVfA11q4q Mq03gdKs PARHMLkkYFJJcGYnEs62kLHcpIzy/hDSycRbSiXlF6nT0IEnGv+TltMdFNKHEEfyXk5CXVgndN6flg+PtZbr4XASdvHzPQlMJ6IEYp+7lGv2HRb++9tQyFzVR6iJ5tkRL2CO1uekyRUS6Wbg0eItL2Mn3HFT0DX5gS2s5CMVNyLzI5nxNrDq/cqfdZdf1Bhz8YCX+ivwHpMHUjcGQIb5NG2+NZnqI6oJL6piCqwRlATuD5B9n9n0Ont22STTUG0FrQZOBRN5CF3Ul9DYbEQdJneEcinZbqxpeQvmoWVNuZvospJmuYHQbZZIRx6zsE/4ayxGs3baJjRA9bEQuKnYj3wOb64quIuUJYc5gjJmLka+Bnkl5bCqFcMOZF3eQQaPmVdh9nfovDpchDeNc2b3Jh/dgggvRh2CWbUSl73x+Z3dlm5NiDy846piS01l0C4U/utIQtO1KH33/wdT6aoeiFZ8/rjvrxNJfPOTaCWseR+x9mf9Z5V1LZgxTAHkw0jXl0XdX1xNKAepjwLzN6b+E/HDW0D/GzMJT4g+1lwMtb132bvGjbOXZJ23fENEZP/uaPKue X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov Change alloc_pages_nolock() to default to __GFP_COMP when allocating pages, since upcoming reentrant alloc_slab_page() needs __GFP_COMP. Also allow __GFP_ACCOUNT flag to be specified, since BPF infra needs __GFP_ACCOUNT. Signed-off-by: Alexei Starovoitov --- include/linux/gfp.h | 2 +- kernel/bpf/syscall.c | 2 +- mm/page_alloc.c | 10 ++++++---- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 5ebf26fcdcfa..0ceb4e09306c 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -354,7 +354,7 @@ static inline struct page *alloc_page_vma_noprof(gfp_t gfp, } #define alloc_page_vma(...) alloc_hooks(alloc_page_vma_noprof(__VA_ARGS__)) -struct page *alloc_pages_nolock_noprof(int nid, unsigned int order); +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order); #define alloc_pages_nolock(...) alloc_hooks(alloc_pages_nolock_noprof(__VA_ARGS__)) extern unsigned long get_free_pages_noprof(gfp_t gfp_mask, unsigned int order); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index dd5304c6ac3c..eb9b6c4c10e9 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -579,7 +579,7 @@ static bool can_alloc_pages(void) static struct page *__bpf_alloc_page(int nid) { if (!can_alloc_pages()) - return alloc_pages_nolock(nid, 0); + return alloc_pages_nolock(__GFP_ACCOUNT, nid, 0); return alloc_pages_node(nid, GFP_KERNEL | __GFP_ZERO | __GFP_ACCOUNT diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 78ddf1d43c6c..148945f0b667 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7506,6 +7506,7 @@ static bool __free_unaccepted(struct page *page) /** * alloc_pages_nolock - opportunistic reentrant allocation from any context + * @gfp_flags: GFP flags. Only __GFP_ACCOUNT allowed. * @nid: node to allocate from * @order: allocation order size * @@ -7519,7 +7520,7 @@ static bool __free_unaccepted(struct page *page) * Return: allocated page or NULL on failure. NULL does not mean EBUSY or EAGAIN. * It means ENOMEM. There is no reason to call it again and expect !NULL. */ -struct page *alloc_pages_nolock_noprof(int nid, unsigned int order) +struct page *alloc_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order) { /* * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. @@ -7541,12 +7542,13 @@ struct page *alloc_pages_nolock_noprof(int nid, unsigned int order) * specify it here to highlight that alloc_pages_nolock() * doesn't want to deplete reserves. */ - gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC - | __GFP_ACCOUNT; + gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_COMP + | gfp_flags; unsigned int alloc_flags = ALLOC_TRYLOCK; struct alloc_context ac = { }; struct page *page; + VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); /* * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is * unsafe in NMI. If spin_trylock() is called from hard IRQ the current @@ -7584,7 +7586,7 @@ struct page *alloc_pages_nolock_noprof(int nid, unsigned int order) if (page) set_page_refcounted(page); - if (memcg_kmem_online() && page && + if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) && unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) { free_pages_nolock(page, order); page = NULL; -- 2.47.1