From: Zi Yan <ziy@nvidia.com>
To: Usama Arif <usama.arif@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>,
Nimrod Oren <noren@nvidia.com>,
Andrew Morton <akpm@linux-foundation.org>,
David Hildenbrand <david@kernel.org>,
Lorenzo Stoakes <ljs@kernel.org>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
"Liam R. Howlett" <liam@infradead.org>,
Nico Pache <nico.pache@linux.dev>,
Ryan Roberts <ryan.roberts@arm.com>, Dev Jain <dev.jain@arm.com>,
Barry Song <baohua@kernel.org>, Lance Yang <lance.yang@linux.dev>,
Kiryl Shutsemau <kas@kernel.org>,
Vlastimil Babka <vbabka@kernel.org>,
Mike Rapoport <rppt@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
Michal Hocko <mhocko@suse.com>,
Brendan Jackman <brendan.jackman@linux.dev>,
Hugh Dickins <hughd@google.com>, Nirmoy Das <nirmoyd@nvidia.com>,
Dragos Tatulea <dtatulea@nvidia.com>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3] mm: remove min_free_kbytes adjustment for THP
Date: Wed, 02 Sep 2026 13:02:29 -0400 [thread overview]
Message-ID: <A36CC394-0954-437A-B408-359C548DA3E1@nvidia.com> (raw)
In-Reply-To: <583c4718-f763-41f0-b757-02b46655c0b5@linux.dev>
On 2 Sep 2026, at 12:47, Usama Arif wrote:
> On 02/09/2026 17:04, Johannes Weiner wrote:
>> On Tue, Sep 01, 2026 at 09:49:58PM -0400, Zi Yan wrote:
>>> On 1 Sep 2026, at 18:09, Johannes Weiner wrote:
>>>
>>>> On Tue, Sep 01, 2026 at 05:09:52PM -0400, Zi Yan wrote:
>>>>> On 1 Sep 2026, at 16:44, Johannes Weiner wrote:
>>>>>
>>>>>> On Tue, Sep 01, 2026 at 10:01:23PM +0300, Nimrod Oren wrote:
>>>>>>> When THP is enabled, set_recommended_min_free_kbytes() may raise
>>>>>>> min_free_kbytes using a heuristic that scales with pageblock_nr_pages.
>>>>>>> Commit f000565adb77 ("thp: set recommended min free kbytes") added this
>>>>>>> heuristic to help keep pageblocks free and reduce fragmentation for THP
>>>>>>> allocations.
>>>>>>
>>>>>> We've had problems with compaction before when min_free_kbytes was too
>>>>>> small on large machines. Competing free space scanners do a lot of
>>>>>> work only to fight over a very small set of possible target pages.
>>>>>>
>>>>>> So I'm a bit uneasy that you didn't include any benchmark numbers with
>>>>>> this that prove basic functionality on larger hosts isn't regressed.
>>>>>>
>>>>>>> The recommendation scales poorly with larger base page sizes. With the
>>>>>>> default arm64 pageblock sizes, the contribution per eligible zone
>>>>>>> before applying the existing cap of 5% of low memory is:
>>>>>>>
>>>>>>> 4 KiB pages: 2 MiB pageblock, 22 MiB per zone
>>>>>>> 16 KiB pages: 32 MiB pageblock, 352 MiB per zone
>>>>>>> 64 KiB pages: 512 MiB pageblock, 5.5 GiB per zone
>>>>>>
>>>>>> I question whether pageblocks need to be 512M on those machines to
>>>>>> begin with. After this patch, you're still asking the page allocator
>>>>>> to optimize grouping such that 512M pages can be allocated at
>>>>>> runtime. Only now you took away part of the mechanism to do so.
>>>>>>
>>>>>> If you're using 512M THPs, I would kind of assume it's on machines
>>>>>> with a memory size where 5.5G for defrag purposes isn't devastating.
>>>>>>
>>>>>> And if you're not, it would make more sense to lower the pageblock
>>>>>> size to the mTHP size you're actually using. And that would fix the
>>>>>> "excessive" min_free_kbytes issue as well.
>>>>>
>>>>> But lowering pageblock size requires a kernel compilation. That means
>>>>> maintaining two sets of kernels for different needs.
>>>>
>>>> That depends on whether anyone actually wants 512M pageblocks...
>>>>
>>>>> The ultimate solution is to enable better compaction to generate
>>>>> THPs bigger than a pageblock size, like Rik's super-pageblock
>>>>> proposal.
>>>>
>>>> ...or whether we can say, at that point, use gigablocks/cma+hugetlb.
>>>>
>>>> And then the static pageblock size for the fallback logic etc. can be
>>>> a smaller, saner default for everybody.
>>>>
>>>> Because the point you didn't address: it doesn't make really sense to
>>>> have 512M pageblocks on smaller machines, beyond the min_free_kbytes
>>>> issue: Fragmentation events will poison half a gig at once,
>>>> should_try_claim_block() becomes harder which results in less
>>>> conversions and more allocations falling through to stealing, page
>>>> isolation is more likely to fail, compaction locks and operates on
>>>> oversized chunks which is bad for latency and concurrency...
>>>
>>> I actually wonder why such a big pageblock would still result in a lot
>>> of fallbacks.
>>
>> If you look at try_to_claim_block(), you need half of a block to be
>> free or compatible with the requested migratetype in order to convert
>> it. When memory is full, LRU pages are scattered all over, and
>> compaction is not involved (order-0), this gets more difficult the
>> bigger the block is. You can get into a situation where LRU reclaim
>> will not clear sufficient room for conversion in any given pageblock
>> anymore and you're stuck with the type distribution. A large share of
>> buddy requests then go permanently through the slower fallback path.
>>
>> Usama knows more about this, but we have seen this on GB300 hosts, and
>> have JUST started to deploy kernels with smaller pageblocks (2M).
>
> Yes, just to confirm above, unfortunately, there were too many problems
> we were seeing in Meta production with 512M pageblock size for arm 64K
> base page size and just had to switch to 2M.
Do you mind elaborating on the problems? Johannes has given good arguments.
I would like to hear more from you, since he said you know more about the
issues.
Thanks.
Best Regards,
Yan, Zi
next prev parent reply other threads:[~2026-09-02 17:02 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-01 19:01 [PATCH v3] mm: remove min_free_kbytes adjustment for THP Nimrod Oren
2026-09-01 19:12 ` Michal Hocko
2026-09-01 20:01 ` Zi Yan
2026-09-01 20:44 ` Johannes Weiner
2026-09-01 21:09 ` Zi Yan
2026-09-01 22:09 ` Johannes Weiner
2026-09-02 1:49 ` Zi Yan
2026-09-02 13:02 ` Lorenzo Stoakes (ARM)
2026-09-02 16:04 ` Johannes Weiner
2026-09-02 16:47 ` Usama Arif
2026-09-02 17:02 ` Zi Yan [this message]
2026-09-02 17:24 ` Usama Arif
2026-09-02 17:13 ` Lorenzo Stoakes (ARM)
2026-09-03 18:58 ` Zi Yan
2026-09-04 15:20 ` Johannes Weiner
2026-09-02 12:07 ` Nirmoy Das
2026-09-02 13:46 ` Lorenzo Stoakes (ARM)
2026-09-02 16:23 ` Johannes Weiner
2026-09-02 17:00 ` Lorenzo Stoakes (ARM)
2026-09-02 18:37 ` Johannes Weiner
2026-09-03 15:46 ` Lorenzo Stoakes (ARM)
2026-09-04 15:43 ` Johannes Weiner
2026-09-04 16:46 ` Lorenzo Stoakes (ARM)
2026-09-03 12:52 ` Michal Hocko
2026-09-02 16:41 ` Usama Arif
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=A36CC394-0954-437A-B408-359C548DA3E1@nvidia.com \
--to=ziy@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=baohua@kernel.org \
--cc=baolin.wang@linux.alibaba.com \
--cc=brendan.jackman@linux.dev \
--cc=david@kernel.org \
--cc=dev.jain@arm.com \
--cc=dtatulea@nvidia.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=kas@kernel.org \
--cc=lance.yang@linux.dev \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mhocko@suse.com \
--cc=nico.pache@linux.dev \
--cc=nirmoyd@nvidia.com \
--cc=noren@nvidia.com \
--cc=rppt@kernel.org \
--cc=ryan.roberts@arm.com \
--cc=surenb@google.com \
--cc=usama.arif@linux.dev \
--cc=vbabka@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox