From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794323740; b=eksS8kVadpOQ/MJZcmvw15QnveupNBXNUzKU5PeOOGSKzMWYgxDkuR16+Fh6DRzqFyfI 8KTkytlGskdY+m3vXlkrvh06rEwBBi2cb6qsxbqoNo3ft9UsymI2pf6O3ikgKnhgKsIR3 qlaG/xX9+OOe9jPW8bZVvB+uXbsHCIIeayn8LFA9RcR0cD3xFTGcwywRwydSW+fYDpunz JcctiN3Fl5nblMaCe77hBCgjv8OES/uJMp7VfjaFPl/9ftTj36RBqfd73MTbUdbQADIc7 Tma2IaaKjkiUTGEjgkg3T+xXF9DCb5O40y1T7hIAkHM+BTSXg5p13oWM73/xvyTvV/A== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794323740; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Transfer-Encoding; bh=fQdjF6iS/55tbDZxN0BWCpAand9QrIht+ZfplLSgF5E=; b=PmVRmwxOxM4yEfUNN+naj+7jUrfq5zDlEl3KWYL9KjtG/L93eOJfybulgj5MfaSfNC8t 4K2P04bOQZU7+C+QpNReHoLCOczn+s9aThAdQpTRPrQkh9UwNzpCj1hdQqB+sg+PqKSOT v6DHK2wdO8d8j2Nnm27P0Vdk7iXK9lAZr0EYhf3eW+7lGfZAVNubfeFIBgiw8yp4fsXX1 rjACmSamZ3FFPdU7ny2epOUQQdtU9J2biPGZDYPsilaU/7c1yb+x5ro/fAdfWAHAW+lge mlXxg+f2owuEFql9Yr5kNPRfpNusadBVoBFnsUGv7QuW7TDpbhpjP9TycGM030XzsFQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b="a/LpPgoE"; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8ED7E606A326 for ; Tue, 10 Nov 2026 15:15:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=a/LpPgoE Received: by smtp.kernel.org (Postfix) with ESMTPSA id 390EF1F008A1; Tue, 10 Nov 2026 15:15:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1794323740; bh=fQdjF6iS/55tbDZxN0BWCpAand9QrIht+ZfplLSgF5E=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=a/LpPgoEqBJLVbANM6VfB4Gm/7ZxLv8UTkrB4d3rDjX2+cIKUwZUKgEUKSFymwUuo ymNXsiR65U7tB04bblVPldFat/a5hcbVf5FPM0seT0Q758etrb+JWv7TRwGnZkVJ+C +JQS7VuvtHAuTcnMyfaoe7rzdl7j5mQpDw2PD4T/thXld91mdN748zh0q/ZWYfmfNc HGre6MSANNJfj8mGKjyTLZ7/JqNPuRWdc0X/gfIj5k3JmaudkSQ/y6H2i8jeh9Kzhe Ar601BRUcTfJFntElS89DsNZbNs9c/3sHvzeMbvuwKK7nG/tybqjxYPLKM9GGcmPA3 MZm9HtRemuoFQ== From: Sasha Levin To: stable@vger.kernel.org Cc: Ridong Chen , Andrew Morton , Barry Song , Johannes Weiner , Axel Rasmussen , Chris Down , David Hildenbrand , Kairui Song , Lorenzo Stoakes , Michal Hocko , Muchun Song , Roman Gushchin , Shakeel Butt , Tejun Heo , Wei Xu , Yuanchu Xie , Yu Zhao , Sasha Levin Subject: [PATCH 6.12.y] mm/mglru: fix ineffective memory protection for non-kswapd reclaim Date: Tue, 10 Nov 2026 10:15:36 -0500 Message-ID: <20261110151459.2041392-1-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261107052035-failed-6.12-15da1e4978dd@kernel.org> References: <20261107052035-failed-6.12-15da1e4978dd@kernel.org> X-Mailing-List: stable@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Ridong Chen [ Upstream commit 15da1e4978dd4ea8f5faa6e15ee967ba4ad9a30a ] For MGLRU, memory.min/low is not honored during global proactive reclaim (writing to the root memory.reclaim) and global direct reclaim, because these paths shrink memcgs using stale protection (emin/elow). It can be reproduced as follows: # echo 7 > /sys/kernel/mm/lru_gen/enabled # cd /sys/fs/cgroup # mkdir -p a/b # echo 100M > a/memory.min # echo +memory > a/cgroup.subtree_control # echo 100M > a/b/memory.min # echo $$ > a/b/cgroup.procs # dd if=/dev/zero of=/tmp/testfile bs=1M count=200 # cat a/b/memory.current 222650368 # echo 500M > memory.reclaim -bash: echo: write error: Resource temporarily unavailable # cat a/b/memory.current 6070272 memory.min is 100M, yet reclaim drops a/b down to 6M, breaking the protection. The traditional LRU path is not affected because shrink_node() calls mem_cgroup_calculate_protection() for each memcg it visits during a top-down tree walk. Commit 30d77b7eef01 ("mm/mglru: fix ineffective protection calculation") moved the protection computation into lru_gen_age_node(), which only runs for kswapd. Non-kswapd global reclaim reaches shrink_one() through lru_gen_shrink_node() -> shrink_many() without any protection computation, so emin/elow are whatever a previous kswapd run left behind - or zero if kswapd never ran on this node. Relying on a prior kswapd pass is not correct either: a memcg's emin/elow are derived from its ancestors' memory.min/low settings and from children_min_usage, both of which change over time, so emin/elow go stale even after kswapd has run and must be recomputed at the point of reclaim. Introduce mem_cgroup_calculate_protection_path() which computes emin/elow along the root-to-target path only, by iterating through the cgroup ancestors array top-down. This avoids the full tree traversal that would be needed with mem_cgroup_calculate_protection(), limiting the cost to O(depth) per memcg - typically 3-5 levels. Call it from shrink_one() for the non-kswapd path so that each memcg about to be shrunk has correct protection values. Link: https://lore.kernel.org/20260907025445.1836238-3-ridong.chen@linux.dev Fixes: e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists") Signed-off-by: Ridong Chen Signed-off-by: Andrew Morton Reviewed-by: Barry Song Reviewed-by: Johannes Weiner Assisted-by: Claude:claude-opus-4-8 Cc: Axel Rasmussen Cc: Chris Down Cc: David Hildenbrand Cc: Kairui Song Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Wei Xu Cc: Yuanchu Xie Cc: Yu Zhao Cc: [ sashal: Replaced unavailable cgroup_css() with an explicit rcu_dereference() lookup of the memory controller pointer. ] Signed-off-by: Sasha Levin --- include/linux/memcontrol.h | 10 +++++++++ mm/memcontrol.c | 45 ++++++++++++++++++++++++++++++++++++++ mm/vmscan.c | 8 ++++++- 3 files changed, 62 insertions(+), 1 deletion(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 8dab6c755e107..2aba72d1de3ab 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1884,6 +1884,16 @@ static inline void count_objcg_events(struct obj_cgroup *objcg, #endif /* CONFIG_MEMCG */ +#if defined(CONFIG_MEMCG) && defined(CONFIG_LRU_GEN) +void mem_cgroup_calculate_protection_path(struct mem_cgroup *root, + struct mem_cgroup *memcg); +#else +static inline void mem_cgroup_calculate_protection_path(struct mem_cgroup *root, + struct mem_cgroup *memcg) +{ +} +#endif + #if defined(CONFIG_MEMCG) && defined(CONFIG_ZSWAP) bool obj_cgroup_may_zswap(struct obj_cgroup *objcg); void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4d1caa4faf69f..b713cda2628a8 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4503,6 +4503,51 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root, page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection); } +#ifdef CONFIG_LRU_GEN +/** + * mem_cgroup_calculate_protection_path - compute protection along a path + * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup) + * @memcg: the target memory cgroup + * + * Walk the ancestor path from @root down to @memcg and compute the effective + * protection at each level. This is safe for isolated queries because it + * ensures parents are computed before children. + */ +void mem_cgroup_calculate_protection_path(struct mem_cgroup *root, + struct mem_cgroup *memcg) +{ + bool recursive_protection = + cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT; + struct cgroup *cg; + int root_level, i; + + if (mem_cgroup_disabled()) + return; + + if (!root) + root = root_mem_cgroup; + + if (memcg == root) + return; + + root_level = root->css.cgroup->level; + cg = memcg->css.cgroup; + + rcu_read_lock(); + for (i = root_level + 1; i <= cg->level; i++) { + struct mem_cgroup *cur; + + cur = mem_cgroup_from_css(rcu_dereference( + cg->ancestors[i]->subsys[memory_cgrp_id])); + if (cur) + page_counter_calculate_protection(&root->memory, + &cur->memory, + recursive_protection); + } + rcu_read_unlock(); +} +#endif /* CONFIG_LRU_GEN */ + static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg, gfp_t gfp) { diff --git a/mm/vmscan.c b/mm/vmscan.c index ae6325a11d42f..b248dcb883697 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4835,7 +4835,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct pglist_data *pgdat = lruvec_pgdat(lruvec); - /* lru_gen_age_node() called mem_cgroup_calculate_protection() */ + /* + * For kswapd, mem_cgroup_calculate_protection() has already + * been called during the top-down cgroup traversal. + */ + if (!current_is_kswapd()) + mem_cgroup_calculate_protection_path(NULL, memcg); + if (mem_cgroup_below_min(NULL, memcg)) return MEMCG_LRU_YOUNG; -- 2.53.0