[PATCH] mm/hugetlb: fix resv_huge_pages double decrement in memfd error path

[PATCH] mm/hugetlb: fix resv_huge_pages double decrement in memfd error path

[Hongfu Li]

From: Hongfu Li <lihongfu@kylinos.cn>

alloc_hugetlb_folio_reserve() decrements h->resv_huge_pages when
dequeuing a folio, but unlike the use_global_reservation handling in
hugetlb_alloc_folio(), it does not set HPageRestoreReserve on the folio.

Its sole caller memfd_alloc_folio() pre-allocates a reservation via
hugetlb_reserve_pages() before allocating. When hugetlb_add_to_page_cache()
fails, folio_put() drops the folio without HPageRestoreReserve set, so
free_huge_folio() does not restore the reservation. The subsequent
hugetlb_unreserve_pages() on the err_unresv path decrements the counter
a second time, leaving resv_huge_pages off by one for every failed
allocation.

Set HPageRestoreReserve when consuming the reservation in
alloc_hugetlb_folio_reserve(). On the error path, free_huge_folio() then
restores the reservation before hugetlb_unreserve_pages() releases it.
The success path is unaffected, as hugetlb_add_to_page_cache() clears
the flag once the folio is added to the page cache.

Fixes: 26a8ea80929c ("mm/hugetlb: fix memfd_pin_folios resv_huge_pages leak")
Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
---
 mm/hugetlb.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index d8849196ad9b..fadfe1dbc047 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2178,8 +2178,10 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
 
 	folio = dequeue_hugetlb_folio_nodemask(h, gfp_mask, preferred_nid,
 					       nmask);
-	if (folio)
+	if (folio) {
+		folio_set_hugetlb_restore_reserve(folio);
 		h->resv_huge_pages--;
+	}
 
 	spin_unlock_irq(&hugetlb_lock);
 	return folio;
-- 
2.54.0

Re: [PATCH] mm/hugetlb: fix resv_huge_pages double decrement in memfd error path

[Muchun Song]

> On Aug 25, 2026, at 10:10, Hongfu Li <hongfu.li@linux.dev> wrote:
> 
> From: Hongfu Li <lihongfu@kylinos.cn>
> 
> alloc_hugetlb_folio_reserve() decrements h->resv_huge_pages when
> dequeuing a folio, but unlike the use_global_reservation handling in
> hugetlb_alloc_folio(), it does not set HPageRestoreReserve on the folio.
> 
> Its sole caller memfd_alloc_folio() pre-allocates a reservation via
> hugetlb_reserve_pages() before allocating. When hugetlb_add_to_page_cache()
> fails, folio_put() drops the folio without HPageRestoreReserve set, so
> free_huge_folio() does not restore the reservation. The subsequent
> hugetlb_unreserve_pages() on the err_unresv path decrements the counter
> a second time, leaving resv_huge_pages off by one for every failed
> allocation.
> 
> Set HPageRestoreReserve when consuming the reservation in
> alloc_hugetlb_folio_reserve(). On the error path, free_huge_folio() then
> restores the reservation before hugetlb_unreserve_pages() releases it.
> The success path is unaffected, as hugetlb_add_to_page_cache() clears
> the flag once the folio is added to the page cache.
> 
> Fixes: 26a8ea80929c ("mm/hugetlb: fix memfd_pin_folios resv_huge_pages leak")
> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>

Reviewed-by: Muchun Song <muchun.song@linux.dev>

Thanks.

Re: [PATCH] mm/hugetlb: fix resv_huge_pages double decrement in memfd error path

[Andrew Morton]

On Tue, 25 Aug 2026 10:10:13 +0800 Hongfu Li <hongfu.li@linux.dev> wrote:

> From: Hongfu Li <lihongfu@kylinos.cn>
> 
> alloc_hugetlb_folio_reserve() decrements h->resv_huge_pages when
> dequeuing a folio, but unlike the use_global_reservation handling in
> hugetlb_alloc_folio(), it does not set HPageRestoreReserve on the folio.
> 
> Its sole caller memfd_alloc_folio() pre-allocates a reservation via
> hugetlb_reserve_pages() before allocating. When hugetlb_add_to_page_cache()
> fails, folio_put() drops the folio without HPageRestoreReserve set, so
> free_huge_folio() does not restore the reservation. The subsequent
> hugetlb_unreserve_pages() on the err_unresv path decrements the counter
> a second time, leaving resv_huge_pages off by one for every failed
> allocation.
> 
> Set HPageRestoreReserve when consuming the reservation in
> alloc_hugetlb_folio_reserve(). On the error path, free_huge_folio() then
> restores the reservation before hugetlb_unreserve_pages() releases it.
> The success path is unaffected, as hugetlb_add_to_page_cache() clears
> the flag once the folio is added to the page cache.
>
> ...
>
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2178,8 +2178,10 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
>  
>  	folio = dequeue_hugetlb_folio_nodemask(h, gfp_mask, preferred_nid,
>  					       nmask);
> -	if (folio)
> +	if (folio) {
> +		folio_set_hugetlb_restore_reserve(folio);
>  		h->resv_huge_pages--;
> +	}
>  
>  	spin_unlock_irq(&hugetlb_lock);
>  	return folio;

Thanks.  I pasted an AI-generated test case which might demonstrate
this bug.  Requires fault-injection so I won't add cc:stable.


Also, Sashiko might have found an accounting issue in the nearby code
(Sashiko doesn't like hugetlb.c):

	https://sashiko.dev/#/patchset/20260825021013.25672-1-hongfu.li@linux.dev




#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <linux/memfd.h>

/* Read resv_hugepages from sysfs */
static long get_resv_hugepages(void) {
    FILE *f = fopen("/sys/kernel/mm/hugepages/hugepages-2048kB/resv_hugepages", "r");
    if (!f) return -1;
    long val = -1;
    fscanf(f, "%ld", &val);
    fclose(f);
    return val;
}

int main(void) {
    long orig_resv = get_resv_hugepages();
    printf("[1] Initial resv_hugepages: %ld\n", orig_resv);

    /* Enable fail_function for hugetlb_add_to_page_cache via debugfs */
    system("echo hugetlb_add_to_page_cache > /sys/kernel/debug/fail_function/inject");
    system("echo 100 > /sys/kernel/debug/fail_function/probability");

    /* Create memfd and attempt write to allocate hugetlb folio */
    int fd = memfd_create("test_memfd", MFD_HUGETLB);
    if (fd >= 0) {
        /* Force allocation path which triggers memfd_alloc_folio() */
        ftruncate(fd, 2 * 1024 * 1024);
        write(fd, "a", 1); 
        close(fd);
    }

    /* Disable error injection */
    system("echo > /sys/kernel/debug/fail_function/inject");

    long post_resv = get_resv_hugepages();
    printf("[2] Post-failure resv_hugepages: %ld\n", post_resv);

    if (post_resv < orig_resv) {
        printf("[!] BUG DEMONSTRATED: resv_hugepages double-decremented by %ld!\n", 
               orig_resv - post_resv);
    }

    return 0;
}

Re: [PATCH] mm/hugetlb: fix resv_huge_pages double decrement in memfd error path

[Hongfu Li]

On 8/27/26 10:47 AM, Andrew Morton wrote:
> On Tue, 25 Aug 2026 10:10:13 +0800 Hongfu Li <hongfu.li@linux.dev> wrote:
>
>> From: Hongfu Li <lihongfu@kylinos.cn>
>>
>> alloc_hugetlb_folio_reserve() decrements h->resv_huge_pages when
>> dequeuing a folio, but unlike the use_global_reservation handling in
>> hugetlb_alloc_folio(), it does not set HPageRestoreReserve on the folio.
>>
>> Its sole caller memfd_alloc_folio() pre-allocates a reservation via
>> hugetlb_reserve_pages() before allocating. When hugetlb_add_to_page_cache()
>> fails, folio_put() drops the folio without HPageRestoreReserve set, so
>> free_huge_folio() does not restore the reservation. The subsequent
>> hugetlb_unreserve_pages() on the err_unresv path decrements the counter
>> a second time, leaving resv_huge_pages off by one for every failed
>> allocation.
>>
>> Set HPageRestoreReserve when consuming the reservation in
>> alloc_hugetlb_folio_reserve(). On the error path, free_huge_folio() then
>> restores the reservation before hugetlb_unreserve_pages() releases it.
>> The success path is unaffected, as hugetlb_add_to_page_cache() clears
>> the flag once the folio is added to the page cache.
>>
>> ...
>>
>> --- a/mm/hugetlb.c
>> +++ b/mm/hugetlb.c
>> @@ -2178,8 +2178,10 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
>>   
>>   	folio = dequeue_hugetlb_folio_nodemask(h, gfp_mask, preferred_nid,
>>   					       nmask);
>> -	if (folio)
>> +	if (folio) {
>> +		folio_set_hugetlb_restore_reserve(folio);
>>   		h->resv_huge_pages--;
>> +	}
>>   
>>   	spin_unlock_irq(&hugetlb_lock);
>>   	return folio;
> Thanks.  I pasted an AI-generated test case which might demonstrate
> this bug.  Requires fault-injection so I won't add cc:stable.

Thanks for sharing the test case.

I have run your provided test program and successfully triggered
the double‑decrement bug of resv_hugepages with fault‑injection.
Testing confirms that my patch fixes this bug.

> Also, Sashiko might have found an accounting issue in the nearby code
> (Sashiko doesn't like hugetlb.c):
>
> 	https://sashiko.dev/#/patchset/20260825021013.25672-1-hongfu.li@linux.dev

I have taken a close look at the nearby accounting issue reported
by Sashiko in hugetlb.c. It appears to be a real issue, and I intend
to submit a separate patch for review to fix this.

> #define _GNU_SOURCE
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
> #include <fcntl.h>
> #include <sys/mman.h>
> #include <linux/memfd.h>
>
> /* Read resv_hugepages from sysfs */
> static long get_resv_hugepages(void) {
>      FILE *f = fopen("/sys/kernel/mm/hugepages/hugepages-2048kB/resv_hugepages", "r");
>      if (!f) return -1;
>      long val = -1;
>      fscanf(f, "%ld", &val);
>      fclose(f);
>      return val;
> }
>
> int main(void) {
>      long orig_resv = get_resv_hugepages();
>      printf("[1] Initial resv_hugepages: %ld\n", orig_resv);
>
>      /* Enable fail_function for hugetlb_add_to_page_cache via debugfs */
>      system("echo hugetlb_add_to_page_cache > /sys/kernel/debug/fail_function/inject");
>      system("echo 100 > /sys/kernel/debug/fail_function/probability");
>
>      /* Create memfd and attempt write to allocate hugetlb folio */
>      int fd = memfd_create("test_memfd", MFD_HUGETLB);
>      if (fd >= 0) {
>          /* Force allocation path which triggers memfd_alloc_folio() */
>          ftruncate(fd, 2 * 1024 * 1024);
>          write(fd, "a", 1);
>          close(fd);
>      }
>
>      /* Disable error injection */
>      system("echo > /sys/kernel/debug/fail_function/inject");
>
>      long post_resv = get_resv_hugepages();
>      printf("[2] Post-failure resv_hugepages: %ld\n", post_resv);
>
>      if (post_resv < orig_resv) {
>          printf("[!] BUG DEMONSTRATED: resv_hugepages double-decremented by %ld!\n",
>                 orig_resv - post_resv);
>      }
>
>      return 0;
> }

-- 
Best regards,
Hongfu