Linux Documentation
 help / color / mirror / Atom feed
From: "David Hildenbrand (Arm)" <david@kernel.org>
To: Muchun Song <muchun.song@linux.dev>,
	Oscar Salvador <osalvador@suse.de>,
	Naoya Horiguchi <nao.horiguchi@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Mel Gorman <mgorman@suse.de>, Andy Whitcroft <apw@canonical.com>,
	Miaohe Lin <linmiaohe@huawei.com>, Michal Hocko <mhocko@suse.com>,
	Rik van Riel <riel@surriel.com>, Hugh Dickins <hughd@google.com>,
	Baolin Wang <baolin.wang@linux.alibaba.com>,
	Vivek Kasireddy <vivek.kasireddy@intel.com>,
	Dave Airlie <airlied@redhat.com>, Jason Gunthorpe <jgg@ziepe.ca>,
	Lorenzo Stoakes <ljs@kernel.org>,
	"Liam R. Howlett" <liam@infradead.org>,
	Vlastimil Babka <vbabka@kernel.org>,
	Mike Rapoport <rppt@kernel.org>,
	Suren Baghdasaryan <surenb@google.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Peter Xu <peterx@redhat.com>, Breno Leitao <leitao@debian.org>,
	Will Deacon <will@kernel.org>,
	"Aneesh Kumar K.V" <aneesh.kumar@kernel.org>,
	Nick Piggin <npiggin@gmail.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Heiko Carstens <hca@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Alexander Gordeev <agordeev@linux.ibm.com>,
	Christian Borntraeger <borntraeger@linux.ibm.com>,
	Sven Schnelle <svens@linux.ibm.com>,
	Arnd Bergmann <arnd@arndb.de>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	Gerd Hoffmann <kraxel@redhat.com>,
	linux-doc@vger.kernel.org, linux-arch@vger.kernel.org,
	linux-s390@vger.kernel.org, stable@kernel.org,
	Josef Bacik <josef@toxicpanda.com>
Subject: Re: [PATCH v2 04/15] mm/hugetlb: account temporary folios separately
Date: Sun, 22 Nov 2026 19:44:35 +0100	[thread overview]
Message-ID: <5f477c4f-9874-4f51-ab84-5b044a3ea568@kernel.org> (raw)
In-Reply-To: <45997a78-95bd-4e50-b433-f64237f9ec1a@kernel.org>

On 11/22/26 19:21, David Hildenbrand (Arm) wrote:
> On 11/16/26 19:29, David Hildenbrand (Arm) wrote:
>> Hugetlb migration might have to allocate a temporary folio through
>> alloc_migrate_hugetlb_folio(). Such folios are currently accounted in
>> "nr_huge_pages" but not in "surplus_huge_pages". persistent_huge_pages()
>> can mistake them as persistent huge pages, as it merely calculates the
>> difference.
>>
>> If set_max_huge_pages() shrinks the pool at the wrong time, it will turn
>> hugetlb folios into surplus folios until persistent_huge_pages() returns
>> the desired number.
>>
>> After the migration (successful or unsuccessful) either the new (temporary)
>> folio or the old folio gets freed: this will reduce "nr_huge_pages" but
>> not "surplus_huge_pages", to undo what was done when allocating a
>> temporary folio.
>>
>> This can result in having more surplus folios than actually allocated
>> folios:
>>
>>   HugePages_Total:       0
>>   HugePages_Surp:        1
>>
>> From then on persistent_huge_pages() returns garbage.
>>
>> To fix it, add internal temporary counters and subtract them when
>> calculating persistent pool sizes. Update persistent_huge_pages() and add
>> persistent_huge_pages_node() to make the code easier to read.
>>
>> Count migration targets and rejected surplus allocations as temporary.
>> Transfer the per-node temporary count along with the temporary flag on
>> migration, such that folio freeing will always do the right thing.
>>
>> Take care of calling account_new_hugetlb_folio() only after setting the
>> folios to be temporary.
>>
>> For now, keep temporary folios in nr_huge_pages, so HugePages_Total and
>> Hugetlb continue to include them and overcommit accounting remains
>> unchanged. The new temporary counters are internal for now and are not
>> exported to userspace.
>>
>> Fixes: ab5ac90aecf5 ("mm, hugetlb: do not rely on overcommit limit during migration")
>> Cc: stable@kernel.org
>> Reported-by: Josef Bacik <josef@toxicpanda.com>
>> Acked-by: Muchun Song <muchun.song@linux.dev>
>> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
>> ---
> 
> In combination with
> 
> 	mm/hugetlb: preserve persistent flags when returning folios to the pool
> 
> This results in an issue during boot.
> 
> I think the following is required:
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index bfc88d42ef0d..aa6ad8d93266 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1804,8 +1804,8 @@ void free_huge_folio(struct folio *folio)
>                 h->resv_huge_pages++;
> 
>         if (folio_test_hugetlb_temporary(folio)) {
> -               folio_clear_hugetlb_temporary(folio);
>                 remove_hugetlb_folio(h, folio, false);
> +               folio_clear_hugetlb_temporary(folio);
>                 spin_unlock_irqrestore(&hugetlb_lock, flags);
>                 update_and_free_hugetlb_folio(h, folio, true);
>         } else if (h->surplus_huge_pages_node[nid] &&
> 
> I'll test real quick whether that resolves CI issues ...
> 

Yes, that makes it work. @Andrew, can you fix that up?

-- 
Cheers,

David

  reply	other threads:[~2026-11-22 18:44 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 18:29 [PATCH v2 00/15] mm/hugetlb: various fixes David Hildenbrand (Arm)
2026-11-16 18:29 ` [PATCH v2 01/15] hugetlbfs: fix integer overflow in fallocate() David Hildenbrand (Arm)
2026-11-16 18:29 ` [PATCH v2 02/15] mm/hugetlb: compare against persistent count when freeing lowmem folios David Hildenbrand (Arm)
2026-11-16 22:24   ` Rik van Riel
2026-11-17 10:11   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 03/15] mm/hugetlb: preserve reservations when shrinking on HIGHMEM David Hildenbrand (Arm)
2026-11-16 22:43   ` Rik van Riel
2026-11-17 10:13   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 04/15] mm/hugetlb: account temporary folios separately David Hildenbrand (Arm)
2026-11-17  8:14   ` Gregory Price (Meta)
2026-11-17 10:16   ` Oscar Salvador (SUSE)
2026-11-22 18:21   ` David Hildenbrand (Arm)
2026-11-22 18:44     ` David Hildenbrand (Arm) [this message]
2026-11-22 20:10       ` Andrew Morton
2026-11-16 18:29 ` [PATCH v2 05/15] mm/hugetlb: keep surplus folios when the pool shrinks David Hildenbrand (Arm)
2026-11-18  9:01   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 06/15] mm/hugetlb: fix mremap() racing with page migration David Hildenbrand (Arm)
2026-11-17  8:42   ` Gregory Price (Meta)
2026-11-18  9:02   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 07/15] hugetlbfs: fix unmapping folios from private VMAs David Hildenbrand (Arm)
2026-11-17  8:31   ` Gregory Price (Meta)
2026-11-18  9:03   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 08/15] mm/hugetlb: serialize page-cache allocations with truncation David Hildenbrand (Arm)
2026-11-19  9:09   ` Oscar Salvador (SUSE)
2026-11-19 16:58     ` David Hildenbrand (Arm)
2026-11-16 18:29 ` [PATCH v2 09/15] mm/memfd: recheck the page cache before allocating a hugetlb folio David Hildenbrand (Arm)
2026-11-17  8:34   ` Gregory Price (Meta)
2026-11-19  9:11   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 10/15] mm/hugetlb: count again if the reserve map shrank David Hildenbrand (Arm)
2026-11-16 18:29 ` [PATCH v2 11/15] hugetlbfs: do not grow a sealed file in mmap() David Hildenbrand (Arm)
2026-11-19  9:19   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 12/15] mm/hugetlb: keep surplus folios that are reserved David Hildenbrand (Arm)
2026-11-19  9:21   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 13/15] mm/hugetlb: fix reservation leak on min_size= mounts David Hildenbrand (Arm)
2026-11-19  9:25   ` Oscar Salvador (SUSE)
2026-11-16 18:29 ` [PATCH v2 14/15] mm/hugetlb: account restored reservations before freeing folios David Hildenbrand (Arm)
2026-11-16 18:29 ` [PATCH v2 15/15] mm/hugetlb: delay anon folio rmap removal until after the TLB flush David Hildenbrand (Arm)
2026-11-17  9:12 ` [PATCH v2 00/15] mm/hugetlb: various fixes David Hildenbrand (Arm)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5f477c4f-9874-4f51-ab84-5b044a3ea568@kernel.org \
    --to=david@kernel.org \
    --cc=agordeev@linux.ibm.com \
    --cc=airlied@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=aneesh.kumar@kernel.org \
    --cc=apw@canonical.com \
    --cc=arnd@arndb.de \
    --cc=baolin.wang@linux.alibaba.com \
    --cc=borntraeger@linux.ibm.com \
    --cc=corbet@lwn.net \
    --cc=gor@linux.ibm.com \
    --cc=hca@linux.ibm.com \
    --cc=hughd@google.com \
    --cc=jgg@ziepe.ca \
    --cc=josef@toxicpanda.com \
    --cc=kraxel@redhat.com \
    --cc=leitao@debian.org \
    --cc=liam@infradead.org \
    --cc=linmiaohe@huawei.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=ljs@kernel.org \
    --cc=mgorman@suse.de \
    --cc=mhocko@suse.com \
    --cc=muchun.song@linux.dev \
    --cc=nao.horiguchi@gmail.com \
    --cc=npiggin@gmail.com \
    --cc=osalvador@suse.de \
    --cc=peterx@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=riel@surriel.com \
    --cc=rppt@kernel.org \
    --cc=skhan@linuxfoundation.org \
    --cc=stable@kernel.org \
    --cc=surenb@google.com \
    --cc=svens@linux.ibm.com \
    --cc=vbabka@kernel.org \
    --cc=vivek.kasireddy@intel.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox