Re: [RFC PATCH 0/8] Introducte Reserved THP

[Qi Zheng <qi.zheng@linux.dev>]

On 7/1/26 7:45 AM, Zi Yan wrote:
> On Mon Jun 29, 2026 at 8:20 AM EDT, David Hildenbrand (Arm) wrote:
>> On 6/27/26 09:21, Qi Zheng wrote:
>>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>>
>>> Hi all,
>>>
>>
>> Hi,
>>
>>> This RFC patchset introduces a new feature called "Reserved THP", and I'd like
>>> to open up a discussion on how to use this as a stepping stone toward unifying
>>> HugeTLB and THP (Transparent Huge Page).
>>>
>>> 1. Background
>>> =============
>>>
>>> Currently, two huge page solutions co-exist in the kernel:
>>>
>>> 1. HugeTLB: Supports reservation, guaranteeing successful allocation within the
>>>              reserved pool. However, it does not support features like swap. And
>>>              it is a relatively independent subsystem.
>>> 2. THP: Does not support reservation and may fail to allocate and fallback to
>>>          small pages when system memory is fragmented, but it is more tightly
>>>          integrated with mm core and supports features like swap.
>>>
>>> Both have their pros and cons. However, in one of our internal scenarios, it
>>> seems we need to combine the features of both to meet the requirements.
>>>
>>> In our internal scenario, a user process needs to reserve double the amount
>>> of Hugetlb memory due to hot-upgrade requirements. For example, if the
>>> process needs 16GB of Hugetlb, an additional 16GB is required during the
>>> hot-upgrade to satisfy memory allocations. After the upgrade, the old
>>> process exits and releases the 16GB of HugeTLB. Therefore, in most cases,
>>> the extra 16GB of HugeTLB is wasted.
>>>
>>> A straightforward idea is to use the Hugetlb CMA feature, reserving a total
>>> of 32GB of hugetlb_cma. During normal operation, 16GB is consumed, and the
>>> remaining 16GB can be used by other processes. During hot-upgrade, we could
>>> try to migrate the memory used by other processes to allocate the required
>>> extra 16GB of Hugetlb. This might work, but it still requires reserving 32GB
>>> of memory.
>>>
>>> We also found that during the hot upgrade, about 10GB of the old process's
>>> hugetlb is actually cold memory, which could theoretically be reclaimed. In
>>> extreme cases, we could reserve only 22GB of memory and reclaim the
>>> remaining 10GB during the hot upgrade. But unfortunately, hugetlb currently
>>> does not support swap, and supporting it seems quite difficult.
>>>
>>> Therefore, we are wondering if we can introduce "reserved THP", which is THP
>>> that can be reserved. It can be consumed through methods like madvise(), while
>>> normal memory allocation cannot consume it.
>>
>> madvise(). Gah. No :)
>>
>>> This can achieve an effect similar
>>> to hugetlb. And because it is THP, it can relatively easily support swap
>>> features, which perfectly solves the above problem.
>>
>> No, this is the wrong approach. We really shouldn't be making the same mistake
>> hugetlb did and support reserving of non-filebacked memory (IOW anonymous memory).
>>
>> And even for files, the hugetlb mechanism is an absolute trainwreck, because
>> it's not NUMA aware.
>>
>> This really needs some proper thought.
> 
> You mean the reservation should be done via some file handle, like
> memfd, so that it is easy to apply memory policies to determine where
> reserved memory locates?
> 
> For existing hugetlb reservation, there is no fine control, like NUMA,
> or cgroup, of the reserved free memory.
> 
> Is that what you mean above?
> 
>>
>>>
>>> Additionally, in 2024 (or possibly earlier), there have been discussions about
>>> the possibility of unifying Hugetlb and THP:
>>>
>>> Link: https://lwn.net/Articles/974491/
>>>
>>> After all, hugetlb's management is relatively independent and requires too
>>> much special handling in mm core. The introduction of reserved THP might be
>>> an opportunity. In the future, reserved THP could be enhanced to support
>>> various hugetlb features, such as acting as a backend for hugetlbfs. When
>>> reserved THP can completely replace HugeTLB, HugeTLB could be entirely
>>> removed, and reserved THP would just become a feature of THP.
>>>
>>> 2. Implementation
>>> =================
>>>
>>> In 2024, Yu Zhao proposed a similar idea:
>>>
>>> Link: https://lore.kernel.org/all/20240229183436.4110845-2-yuzhao@google.com/
>>>
>>> The idea was to introduce two virt zones: ZONE_NOSPLIT and ZONE_NOMERGE to
>>> guarantee the allocation success rate of THP, achieving an effect similar to
>>> reservation. However, it seems there was no further progress, perhaps because of
>>> reluctance to introduce more virt zones like ZONE_MOVABLE.
>>>
>>> This RFC wants to discuss another implementation:
>>>
>>> 1. Introduce a new migratetype: MIGRATE_RESERVED_THP.
>>> 2. Introduce two new hugetlb-like kernel boot parameters: `thp_reserved_size`
>>>     and `thp_reserved_nr`. When set, the required memory is marked as
>>>     MIGRATE_RESERVED_THP and put back into the buddy allocator.
>>
>> I'm all for some mechanism to make runtime allocation of large chunks of memory
>> easier, by adding a pool from where multiple consumers (THP, guest_memfd,
>> hugetlb, whatever) can allocate memory.
> 
> I agree with this one. We do not want to invent different free memory
> reservation mechanisms for each possible consumer. A shared reservation
> mechanism with different reservation and allocation policies is better.

Hi all, thanks a lot for the feedback! It seems that introducing a new
reservation approach isn't the best way to go. Is the consensus to
address/optimize the problem by doing the following?

1. make THP allocation more reliable.
    (pointed by Gregory. And I think this correspondingly requires
     swap-in to support bringing in the THP folio as a whole. This is
     also the issue Matthew mentioned that the swap subsystem needs
     to address.)

2. design a shared memory reservation mechanism.
    (suggested by David and Zi)

3. Minimize memory fragmentation as much as possible.
    (Like Barry suggested, we could introduce something at pageblock
     level to record memory order preferences.)

Thanks,
Qi

>