Re: [PATCH] mm/mglru: skip oom killing for high order allocs in kswapd

[sashiko-bot@kernel.org]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] mm/mglru: lru_gen_age_node() still OOM-kills for order 1..3 reclaim
--

commit cb44e931601f0546134f19e7185758d84dbc23fb
Author: Bingfang Guo <bingfangguo@tencent.com>

mm/mglru: skip oom killing for high order allocs in kswapd

This patch skips invoking the OOM killer in lru_gen_age_node() when
sc->order > PAGE_ALLOC_COSTLY_ORDER so that high-order allocations in
fragmented systems with MGLRU min_ttl_ms enabled do not repeatedly trigger OOM
kills from kswapd.

> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 9959ca158ce83..b32777468ead8 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -4407,6 +4407,12 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc)
>  			reclaimable = lruvec_is_reclaimable(lruvec, sc, min_ttl);
>  	} while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)));
>  
> +	/*
> +	 * OOM killing won't help high order allocations. Skip and let them fail.
> +	 */
> +	if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
> +		return;

[Severity: High]
Can kswapd still prematurely OOM-kill user processes during compaction-induced
reclaim for orders 1 through PAGE_ALLOC_COSTLY_ORDER?

Prior to commit 30d77b7eef019 ("mm/mglru: fix ineffective protection
calculation"), lru_gen_age_node() skipped OOM kills for any non-zero
sc->order, matching should_abort_scan():

mm/vmscan.c:lru_gen_age_node() {
	/* check the order to exclude compaction-induced reclaim */
	if (!min_ttl || sc->order || sc->priority == DEF_PRIORITY)
		return;
}

mm/vmscan.c:should_abort_scan() {
	/* check the order to exclude compaction-induced reclaim */
	if (!current_is_kswapd() || sc->order)
		return false;
}

In __alloc_pages_slowpath(), any order 1..3 allocation with
__GFP_KSWAPD_RECLAIM calls wake_all_kswapds(), even when the caller specifies
GFP_NOWAIT, __GFP_NORETRY, or __GFP_RETRY_MAYFAIL (for example, multi-page
slab allocations in allocate_slab(), network fragment refills in
skb_page_frag_refill(), or large-folio readahead).

When kswapd runs balance_pgdat(), it sets sc.gfp_mask to GFP_KERNEL (losing
the original caller's GFP flags) and does not run compaction inside the
reclaim loop. In kswapd_shrink_node(), sc->order is only reset to 0 after
reclaiming compact_gap(sc->order) pages:

mm/vmscan.c:kswapd_shrink_node() {
	if (sc->order && sc->nr_reclaimed >= compact_gap(sc->order))
		sc->order = 0;
}

If physical memory is fragmented for orders 1..3 while order-0 watermarks are
healthy, and aging pushes all memcg generations younger than min_ttl_ms before
compact_gap(sc->order) pages are reclaimed, sc->order remains in 1..3 and
lru_gen_age_node() will still invoke out_of_memory() on each priority step
until all processes are killed.

Additionally, checking sc->order after the mem_cgroup_iter() loop means
lruvec_is_reclaimable() is evaluated across every memcg even when sc->order
causes an early return.

Would initializing reclaimable as:

	bool reclaimable = !min_ttl || sc->order;

at the top of lru_gen_age_node() restore the original exclusion for all
compaction-induced non-zero orders and avoid the unnecessary
lruvec_is_reclaimable() checks (while still allowing OOM kills if kswapd falls
back to order-0 reclaim in kswapd() when order-0 watermarks are unbalanced)?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261027-fix-mglru-kswapd-age-v1-1-b1ffdec4ae52@tencent.com?part=1