[PATCH v2 0/3] mm: make watermark checks honour ALLOC_NOFRAGMENT

["Kiryl Shutsemau (Meta)" <kas@kernel.org>]

With defrag_mode, 64K mTHP faults under memory pressure stall in
compaction 18 times as often as without it, and get about a third of the
folios.  Dev Jain reported it [1].

The high-order watermark check counts free pages of every migratetype.
With defrag_mode, every slowpath allocation is ALLOC_NOFRAGMENT, and a
request below pageblock_order cannot take free pages from another
migratetype's pageblock.  Under pressure, that is where the large free
pages end up.  Two callers act on the wrong answer:

  - direct compaction's up-front check returns COMPACT_SUCCESS without
    compacting, so compaction is never deferred;
  - direct reclaim for a costly order stops early, having freed little.

Patches 1-2 give the check the request's migratetype and make it skip
the free lists the request cannot take.  Patch 3 tells direct reclaim
that the allocation is ALLOC_NOFRAGMENT.

Dev's benchmark in a 7G VM with only 64K mTHP enabled, medians of 15
runs:

                              defrag_mode=0  defrag_mode=1  defrag_mode=1
                                                            + series
    compact_stall                      1690          30669          7366
    64K faults allocated              20750           7391         20129
    64K faults falling back to 4K       725          13832          1339

defrag_mode=1 still compacts more than defrag_mode=0: by design, it
keeps 64K faults out of unmovable and reclaimable pageblocks [2].
defrag_mode=0 is unchanged: 20698 64K faults allocated with the series.

v2:
 - Patch 2 changelog: describe what the fast path change does to pages
   on the per-cpu lists (Andrew); drop the should_reclaim_retry()
   paragraph
 - Patch 3 changelog: say that the benchmark baseline already has the
   high-order check honouring ALLOC_NOFRAGMENT (Andrew)
 - Rebased onto v7.4-rc1

v1: https://lore.kernel.org/all/20261028132638.2560361-1-kirill@shutemov.name/

[1] https://lore.kernel.org/all/4aa14b41-9b89-494c-8825-55fffd5a541c@arm.com/
[2] https://lore.kernel.org/all/auCDnqJc7fsBpUW2@thinkstation/


Kiryl Shutsemau (Meta) (3):
  mm: page_alloc: add a watermark check that knows the migratetype
  mm: page_alloc: make the high-order watermark check honour
    ALLOC_NOFRAGMENT
  mm: vmscan: tell direct reclaim the allocation is ALLOC_NOFRAGMENT

 include/linux/mmzone.h |  3 --
 mm/compaction.c        | 16 +++++++----
 mm/internal.h          | 10 ++++++-
 mm/page_alloc.c        | 64 +++++++++++++++++++++++++++++++++---------
 mm/vmscan.c            | 24 ++++++++++++----
 5 files changed, 89 insertions(+), 28 deletions(-)

Range-diff against v1:
1:  c630170105ac = 1:  ed3ef9f9fd13 mm: page_alloc: add a watermark check that knows the migratetype
2:  918bed282fed ! 2:  3347b0a24b0e mm: page_alloc: make the high-order watermark check honour ALLOC_NOFRAGMENT
    @@ Commit message
         __compact_finished() the same rule for its fallback test, so that
         compaction, once it runs, stops on a page the request can take.
     
    -    The allocator fast path gets the same answer.  A zone whose large free
    -    pages all belong to other migratetypes now fails the watermark check,
    -    where before rmqueue failed, and the allocator moves on to the next zone.
    -
    -    should_reclaim_retry() keeps counting free pages of every migratetype.
    -    It does not ask whether the allocation can be served now, but whether
    -    reclaim and compaction can still make progress, and they work across
    -    migratetypes to free a whole pageblock the allocation can claim.
    +    zone_watermark_fast() calls __zone_watermark_ok_mt() too.  With
    +    defrag_mode, an allocation now wakes kswapd and starts reclaim or
    +    compaction as soon as the free lists hold nothing it can take.  For the
    +    orders the per-cpu lists serve, that is earlier than before: the check
    +    does not count their pages, so the allocation no longer uses up the local
    +    list first.  A request that may not wake kswapd or reclaim, such as an
    +    mTHP fault with THP defrag=never, falls back to a smaller folio instead.
     
         Fixes: e3aa7df331bc ("mm: page_alloc: defrag_mode")
         Reported-by: Dev Jain <dev.jain@arm.com>
3:  a7ebde665647 ! 3:  0ecbcc5f5fb0 mm: vmscan: tell direct reclaim the allocation is ALLOC_NOFRAGMENT
    @@ Commit message
         and cannot take the free pages left in unmovable and reclaimable
         pageblocks.  Reclaim stops on them anyway, having freed little.
     
    -    In Dev Jain's 64K mTHP benchmark, an order-4 direct reclaim frees 16
    +    In Dev Jain's 64K mTHP benchmark, with the high-order watermark check
    +    already honouring ALLOC_NOFRAGMENT, an order-4 direct reclaim frees 16
         pages on average with defrag_mode and 45 without.  64K faults get a bit
         over half as many folios as without defrag_mode (medians of 15 runs):
     

base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.54.0