From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1793797344; b=a92FdZjxe5dhyzkklKkQP9CQCI0bGe6JGMh5JEnwJuYgf/JmnKQGkQHwD/VoBjz6XNGE 9FWXGftGwAP6TsrtNPNgTG738FK8w9CaaHWcvZzyCISbJINCB9VVFXkZT7a8Gy9cg1EyQ fF4T0TrRyBphYJcA3KCO+wAK2xlHeuJR/oaB5eHquyeUDjFTD6ecYESot60DcU1x1DxIk SLVPJfxSmg3wq7UIZRHCdgD1GoJVdBKucNmXLp37l0pu7SEE3aCIiv4NiE/PmF7xp5JZj 0wXQgWtTTG2Abo5CQvrieH5RL6nhS83xlqN948fIcFbp1++vVxASN9PVhBhC1NDTUpg== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1793797344; h=Received:DKIM-Signature:Received:Received:X-ME-Sender:X-ME-Received: X-ME-Proxy-Cause:X-ME-Proxy:Feedback-ID:Received:From:To:Cc:Subject: Date:Message-ID:X-Mailer:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding; bh=R5z5YvwsAVG0TIHGlZc17HJ/f1VSVEW4yJDbx1J4Pa8=; b=jntfIFiIb8STISOFHbytImDu8mdTC3/AU+g85T3ubnWlZJbsvaAcRV+ngAz/iO2xh9Id EN8y7liZVFOeLUF5vMPjaGdbHnH+JL9JJxMJCeGlPnMXGdnW9OyoWOqWeSAGHuZBN/alv LoclTiikaPTXWZRThanPfDUV2fZ4JbYXyWUryRdpKrqDfuZjksiiw9/s9PgbFO9o2QI7Q x4DhsjZIIjITa7j1UeQ/5AjQ2GgV5/bnXuz6JLzZzBVy0mf6YnG3nLi5ZjkWY9JTAGHyf 0qyoLBlswss1NPqqUZarPraVIs+961SKCYthHIvteyEqwXhs3ymXZYZUVKOBiFJG3Ag== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=G6jgAQ4z; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DC80606A31B for ; Wed, 4 Nov 2026 13:02:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=G6jgAQ4z Received: by smtp.kernel.org (Postfix) with ESMTPSA id 980C21F00A16; Wed, 4 Nov 2026 13:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1793797344; bh=R5z5YvwsAVG0TIHGlZc17HJ/f1VSVEW4yJDbx1J4Pa8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=G6jgAQ4zSafmaTS9UlNrY1jGOZcAW2BsNX+vutxBD94x8bUtpWz3suneBiDOPNTOS 4rXToTundsSKfThrE+WgrX7Khlx29EXlwqibyIXVKNSQyq+fGu461EOzYNW86Hr1dk U2Vd/Ik03cnaPgMqLHjNmbV7Fd3r7++JuftYHWMLKsW9iN4uJ6BU94wajXEDfjNw/N PhGlZimsZfUcpergXZ8DEfZxeWTuEaw6VGSBUTvskq5sgdTc578Qq+ra98an1qdGS8 IJr9MCoR/KIlcude5BsdxjSAtjjyH0MJuH9OwVkgouBIhEcmydUER2HbsdsalGd8xP 9C6eZS80vdtcA== Received: from phl-compute-03.internal (phl-compute-03.internal [10.202.2.43]) by mailfauth.ams.internal (Postfix) with ESMTP id 71C021980054; Wed, 4 Nov 2026 08:02:17 -0500 (EST) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-03.internal (MEProxy); Wed, 04 Nov 2026 08:02:21 -0500 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGcnWxWNdwzviArTqwO8SleswujEAoF3G5cAnzgWuj9VIXYzVn8L/YtSSoCVpDQCx AFAjkaMBKJDECegKseyQPi3BoJ2ULdzgEWHQh0kOOdV3qcZ96T5qYw8zvnYBduU6y+7gOl ju1hRqdfaUd029Zpq0M1QKVfvNYpVJXA03mIXM/FVYF2n1TVZdGxcwDBil6kVRafbiJtv0 QuPNDwYksKVrYzT4QGS7HIxmz8ayVgiPQk5G9zoQji1nSBMTr/zV+0QH6w0a9BT4L54N+d cRhB7KhKgiZgmWNJc1G8l9d+gvjQ75B75nEIsy2loqzEXGvtqjUcmfJ5UhZu2fDzNX4RZ+ 6oR/S6KZE3B+Oj59PydT7wq4+cy2/+Do57EyU/spfCtxf/P36vf5deC7qOxvjMAtoOJNe8 NXXd1ZuH7iDV5jJ0bBxXLG1LqVa7P9BOmaxRW5JTT1/jF3adOCntQdSPvAU4twjqIyOJnD il3SYhGq0rYp1kMyeBc99l97LZnhmPCerSKBxilv0VoIbfSZ/r0kRgqeaaq8vWQFyTKMq9 JqWfqAb8kO8+Eaq32Gnd9ghE13syUX2tJAxCvmgtCsp/xZfVlMEAx0mCJon0C/Z+o5ex05 Jd9V0ubEBlvCJwgW+XBjmEPpFPlMDPilID7ueQ2EeSfc1TgzPHQ9hBJktdzQ X-ME-Proxy: Feedback-ID: i10464835:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 4 Nov 2026 08:02:16 -0500 (EST) From: "Kiryl Shutsemau (Meta)" To: Andrew Morton , Linus Torvalds , David Hildenbrand , Matthew Wilcox , Vlastimil Babka , Lorenzo Stoakes , Jan Kara , Hugh Dickins , Christian Brauner , Alexander Viro , Harry Yoo Cc: "Kiryl Shutsemau (Meta)" , Miklos Szeredi , Miaohe Lin , Naoya Horiguchi , Oscar Salvador , "Liam R. Howlett" , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kairui Song , Qi Zheng , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Baoquan He , "Paul E. McKenney" , Pedro Falcato , Luis Chamberlain , Ilya Gladyshev , Jaegeuk Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, fuse-devel@lists.linux.dev, kernel-team@meta.com Subject: [RFC PATCH v2 09/18] mm/memory-failure: handle folios with PG_rcu_free Date: Wed, 4 Nov 2026 13:01:05 +0000 Message-ID: <20261104130118.2507513-10-kas@kernel.org> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20261104130118.2507513-1-kas@kernel.org> References: <20261104130118.2507513-1-kas@kernel.org> X-Mailing-List: fuse-devel@lists.linux.dev List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit A page whose free waits for an RCU grace period has refcount 0 and is neither on the LRU nor in the buddy allocator. memory_failure() and soft_offline_page() cannot handle such a page: get_any_page() retries a few times and gives up. Soft offline of the page fails, and a hard error is ignored. If the page is part of a large folio, the folio is later freed with the poisoned page in it, and that page can be handed out again. Free the waiting pages first when the page looks like one of them, and again on each retry in get_any_page(): the page can be freed while it retries. memory_failure() has to do it before it sets PG_hwpoison. The free of a poisoned order-0 page drops the page instead of returning it to the buddy allocator, so a later drain would still leave the error ignored. A folio with PG_rcu_free that is still in use may be copied by a reader that holds no reference. When memory_failure() succeeds, its caller can unmap the page from the direct map under such a reader. Wait for a grace period before returning success. By then the page has left the page cache, or is an order-0 folio with PG_hwpoison, which a reader that starts after the grace period has to check. Wait at the end, not right after setting PG_hwpoison. A large folio is split later, and until the new folios are in the page cache, a reader can still copy the poisoned page as part of the large one. Both callers hold mf_mutex and can wait for the grace period. Signed-off-by: Kiryl Shutsemau (Meta) --- mm/memory-failure.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 7dde7c3c9a0c..398463b2e82d 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1419,6 +1419,19 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags) return 0; } +/* A page waiting for a grace period is not in the buddy yet: free it */ +static void mf_free_rcu_pages(struct page *p) +{ + if (!zone_rcu_free_waiting(page_zone(p))) + return; + if (page_count(p)) + return; + if (is_free_buddy_page(p)) + return; + + rcu_free_sync(); +} + #define GET_PAGE_MAX_RETRY_NUM 3 static int get_any_page(struct page *p, unsigned long flags) @@ -1443,6 +1456,7 @@ static int get_any_page(struct page *p, unsigned long flags) } try_again: + mf_free_rcu_pages(p); if (!count_increased) { ret = __get_hwpoison_page(p, flags); if (!ret) { @@ -2440,6 +2454,7 @@ int memory_failure(unsigned long pfn, int flags) if (res != -ENOENT) goto unlock_mutex; + mf_free_rcu_pages(p); if (TestSetPageHWPoison(p)) { res = -EHWPOISON; if (flags & MF_ACTION_REQUIRED) @@ -2447,6 +2462,13 @@ int memory_failure(unsigned long pfn, int flags) if (flags & MF_COUNT_INCREASED) put_page(p); action_result(pfn, MF_MSG_ALREADY_POISONED, MF_FAILED); + /* + * The memory_failure() that set PG_hwpoison may have failed + * without waiting for lockless readers, and the caller may + * unmap the page once this returns 0. + */ + if (!res && folio_test_rcu_free(page_folio(p))) + synchronize_rcu(); goto unlock_mutex; } @@ -2612,6 +2634,13 @@ int memory_failure(unsigned long pfn, int flags) identify_page_state: res = identify_page_state(pfn, p, page_flags); + /* + * The caller may unmap the page once this returns 0. Lockless + * readers that started before the page got PG_hwpoison, or before a + * split left it in a folio of its own, may still be copying it. + */ + if (!res && folio_test_rcu_free(folio)) + synchronize_rcu(); mutex_unlock(&mf_mutex); return res; unlock_page: -- 2.54.0