Re: + mm-page_alloc-avoid-direct-compaction-for-costly-__gfp_noretry-allocations.patch added to mm-hotfixes-unstable branch

[Johannes Weiner <hannes@cmpxchg.org>]

On Sun, Sep 13, 2026 at 03:30:09AM +0100, Matthew Wilcox wrote:
> On Fri, Sep 11, 2026 at 09:32:43AM -0700, Andrew Morton wrote:
> > From: Salvatore Dipietro <dipiets@amazon.it>
> > Subject: mm/page_alloc: avoid direct compaction for costly __GFP_NORETRY allocations
> > Date: Fri, 11 Sep 2026 14:21:02 +0000
> > 
> > Commit 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
> > introduced high-order folio allocations in the iomap buffered write path. 
> > When memory is fragmented, each failed costly-order allocation enters
> > __alloc_pages_slowpath() which runs direct compaction and
> > drain_all_pages(), causing a 0.38x throughput drop on PostgreSQL pgbench
> > (simple-update) with 1024 clients on a 96-vCPU arm64 system.
> > 
> > The root issue is that direct compaction is too expensive for hot
> > allocation paths that have fallbacks to smaller allocations. 
> > __filemap_get_folio_mpol() already marks higher-order allocations with
> > __GFP_NORETRY | __GFP_NOWARN, signalling that the caller can handle
> > failure.  However, the page allocator still attempts full direct
> > compaction for costly orders with __GFP_NORETRY, which is unnecessarily
> > aggressive when the caller will simply retry at a lower order.
> > 
> > For costly-order allocations with __GFP_NORETRY, suppress direct reclaim
> > for the whole slowpath by computing can_direct_reclaim (and hence
> > can_compact) as false.  The !can_direct_reclaim check near the top of the
> > slowpath then short-circuits to nopage: no direct reclaim, no direct
> > compaction and no drain_all_pages() IPI across every CPU.  kswapd (and in
> > turn kcompactd) is still woken further down for background
> > defragmentation, so compaction keeps working for long-term system health
> > while being removed from the latency-critical direct allocation path.
> > 
> > Allocations that also request __GFP_THISNODE are exempted.  That flag
> > pairing identifies the local-node-first THP attempt issued by
> > alloc_pages_mpol() (mempolicy.c), which relies on direct compaction to
> > form transparent huge pages.  __GFP_NOFAIL is exempted as well, so a
> > must-not-fail allocation is never made to fail.
> > 
> > Test environment:
> >   Hardware:  AWS EC2 m8g.24xlarge (96 vCPU, arm64)
> >              12x 1TB IO2 32000 IOPS RAID0 XFS
> >   OS:        AL2023
> >   Kernel:    v7.3-rc1
> >   Database:  PostgreSQL 18.4
> >   Workload:  pgbench simple-update, 1024 clients, 96 threads, 1200s
> > 
> > Results (average of 3 runs, TPS):
> >   Config                                       Avg TPS   % vs baseline
> >   AL2023 stock 6.1 kernel (pre-5d8edfb900d5)   136,942           n/a
> >   v7.3-rc1 baseline (no patch)                  59,408             -
> >   v7.3-rc1 + this patch                        161,994       +172.7%
> > 
> > The patch fully recovers the pre-5d8edfb900d5 performance, bringing
> > throughput back above the pre-regression level and well clear of the ~59k
> > baseline.  The AL2023 6.1 row runs a kernel that predates commit
> > 5d8edfb900d5 ("iomap: Copy larger chunks from userspace"), so it is not
> > directly comparable, but it shows the pre-regression level and confirms
> > that the ~59k baseline is the anomaly and not the norm.
> > 
> > Link: https://lore.kernel.org/all/20260403193535.9970-1-dipiets@amazon.it/T/#t [v1]
> > Link: https://lore.kernel.org/linux-mm/20260420161404.642-1-dipiets@amazon.it/T/#u [v2]
> > Link: https://lore.kernel.org/all/20260710143437.12379-1-dipiets@amazon.it/T/#u [v3]
> > Link: https://lore.kernel.org/all/20260904115629.3993331-1-dipiets@amazon.it/T/#u [v4]
> > Link: https://lore.kernel.org/20260911142102.2294202-1-dipiets@amazon.it
> > Fixes: 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
> > Signed-off-by: Salvatore Dipietro <dipiets@amazon.it>
> > Acked-by: Zi Yan <ziy@nvidia.com>
> > Cc: Vlastimil Babka <vbabka@suse.cz>
> > Cc: David Hildenbrand <david@redhat.com>
> > Cc: Michal Hocko <mhocko@suse.com>
> > Cc: Johannes Weiner <hannes@cmpxchg.org>
> > Cc: Matthew Wilcox <willy@infradead.org>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: Dave Chinner <dgc@kernel.org>
> > Cc: Ritesh Harjani <ritesh.list@gmail.com>
> > Cc: <stable@vger.kernel.org>
> > Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> 
> i'm just back from vacation, but i think the patch i posted was the
> better way to fic this.  at this point i'm confused why this one is
> being considered, but i'll review all the email from the last two weeks
> and see what's happened to cause this one to be included.

Welcome back, Willy.

The alternate proposal didn't seem to solve Salvatore's problem:
https://lore.kernel.org/all/20260714120204.542300-1-dipiets@amazon.it/

Apologize if you were referring to something more recent, this one was
from mid-July.