Re: [PATCH 6.18.y 3/3] mm/page_alloc: avoid direct reclaim and compaction for costly __GFP_NORETRY allocations

[sashiko-bot@kernel.org]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] mm/page_alloc: clearing __GFP_DIRECT_RECLAIM breaks THP
  defrag=always
It also noticed 1 related pre-existing issue this patch does not
introduce:
- [High] mm/page_alloc: premature OOM kill due to concurrent defrag_mode
  update [1]
--

commit c68a1136d4935b5c8880839164e923ec2040b9fb
Author: Salvatore Dipietro <dipiets@amazon.it>

mm/page_alloc: avoid direct reclaim and compaction for costly __GFP_NORETRY allocations

This commit clears __GFP_DIRECT_RECLAIM at the start of
__alloc_pages_slowpath() for costly-order __GFP_NORETRY allocations that
do not also specify __GFP_THISNODE, avoiding direct reclaim and direct
compaction.

Link: https://lore.kernel.org/20261001082152.2879289-1-dipiets@amazon.it
Link: https://lore.kernel.org/all/20260403193535.9970-1-dipiets@amazon.it/T/#t [v1]
Link: https://lore.kernel.org/linux-mm/20260420161404.642-1-dipiets@amazon.it/T/#u [v2]
Link: https://lore.kernel.org/all/20260710143437.12379-1-dipiets@amazon.it/T/#u [v3]
Link: https://lore.kernel.org/all/20260904115629.3993331-1-dipiets@amazon.it/T/#u [v4]
Link: https://lore.kernel.org/all/20260911142102.2294202-1-dipiets@amazon.it/T/#u [v5]

> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index c1ba49ba51d75..a7aab8dde3ad6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4743,7 +4743,7 @@ static inline struct page *
>  __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  						struct alloc_context *ac)
>  {
> -	bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;
> +	bool can_direct_reclaim;
>  	bool can_compact = gfp_compaction_allowed(gfp_mask);
>  	bool nofail = gfp_mask & __GFP_NOFAIL;
>  	const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER;
> @@ -4758,6 +4758,16 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	unsigned int zonelist_iter_cookie;
>  	int reserve_flags;
>  
> +	/*
> +	 * Costly __GFP_NORETRY callers have a cheap fallback, so don't stall
> +	 * them in reclaim or compaction. __GFP_THISNODE callers are exempt.
> +	 */
> +	if (costly_order && (gfp_mask & __GFP_NORETRY) &&
> +	    !(gfp_mask & __GFP_THISNODE))
> +		gfp_mask &= ~__GFP_DIRECT_RECLAIM;
> +
> +	can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;

[Severity: Medium]
Does clearing __GFP_DIRECT_RECLAIM here cause a regression for transparent
hugepage allocations under transparent_hugepage/defrag=always?

When TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG is enabled,
vma_thp_gfp_mask() returns GFP_TRANSHUGE | __GFP_NORETRY for non-madvised
VMAs, which lacks both __GFP_THISNODE and __GFP_KSWAPD_RECLAIM.
alloc_pages_mpol() only adds __GFP_THISNODE on CONFIG_NUMA=y for the
initial local-node pass of order == HPAGE_PMD_ORDER.  As a result, costly
mTHP allocations (orders 4..8), !CONFIG_NUMA THP allocations, and the
all-node retry in alloc_pages_mpol() have __GFP_DIRECT_RECLAIM stripped
here and bail out without direct compaction, direct reclaim, or waking
kswapd.

In addition, because reaching the costly_order && (gfp_mask &
__GFP_NORETRY) check inside the initial direct-compaction block now
requires __GFP_THISNODE to be set, the if (gfp_mask & __GFP_THISNODE)
goto nopage branch there is always taken, leaving compact_priority =
INIT_COMPACT_PRIORITY unreachable.

[Severity: High]
This problem wasn't introduced by this patch, but can a concurrent sysctl
write to vm.defrag_mode [1] cause __alloc_pages_slowpath() to trigger
a premature OOM kill?

When defrag_mode is enabled at the start of the slowpath,
gfp_to_alloc_flags() sets ALLOC_NOFRAGMENT in alloc_flags.  If reclaim and
compaction fail, the fallback check before __alloc_pages_may_oom() re-reads
defrag_mode:

mm/page_alloc.c:__alloc_pages_slowpath() {
	...
	/* Reclaim/compaction failed to prevent the fallback */
	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
		alloc_flags &= ~ALLOC_NOFRAGMENT;
		goto retry;
	}
	...
	/* Reclaim has failed us, start killing things */
	page = __alloc_pages_may_oom(gfp_mask, order, ac, &did_some_progress);
}

If vm.defrag_mode is cleared to 0 while the allocation is in the slowpath,
this check evaluates to false, so __alloc_pages_slowpath() never drops
ALLOC_NOFRAGMENT and proceeds to __alloc_pages_may_oom() even though
unfragmented fallback pageblocks may be available.  Checking only
alloc_flags & ALLOC_NOFRAGMENT avoids depending on the live sysctl value.

[1] https://sashiko.dev/#/bug/linux-47853a87-6054-4869-a159-48ddf577eff4

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261113110259.2901456-1-dipiets@amazon.it?part=3