From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68E8B35977 for ; Fri, 4 Jul 2025 00:59:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751590761; cv=none; b=Ct9s2dRdpHPMmkD83//4WzPoxIA4+aYIrbXPnFr9A3LKxwt+CKcPUlX93aY0CZcbkvfcMEFrfaKTBlxaVYS7G8Jx+j5Ac1FUojQGA4/CbuvzZZh+Rb7M8E8S6ec5hloqzE5Qw/fo6u0LzXZp24CR+97pL9K+icL44ks+txv0+zo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751590761; c=relaxed/simple; bh=TVwb1mvcxZGkSQV5LeI+pNydpqcxNSkDeW/KH1JuWfI=; h=Date:To:From:Subject:Message-Id; b=tjgtI45NNswKxBplxkWjJV9lt7twbx2svd4/+aiK5R8LEmLPLDjdiEHLWPXMpDp92LHf4i2jEISj8mHPM/q+NIGwQP4WgKrROnC/R5CQRwlg20+GsNKX9OkyTxd36yaezHt4TUoaCMltEMwvIW3bizWDb+dMD7wU4yjHxenpBEA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=g4HgPKda; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="g4HgPKda" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB315C4CEE3; Fri, 4 Jul 2025 00:59:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1751590758; bh=TVwb1mvcxZGkSQV5LeI+pNydpqcxNSkDeW/KH1JuWfI=; h=Date:To:From:Subject:From; b=g4HgPKdaL8CweqBaRV2SHzYFRd0rwVmG2sKa6r6qsK78G2cNtznRd6E2ZUrqYAGh9 VtquEPzOJYkeQoFJPkbg6JNKhFHAomio3Zss4i12ssoE2pLSSAnm9X6y8GrlxzOHDw Rtpe5eGzJd3oXEcyVl/vo2yzxkEcQokaU4fL7Mz4= Date: Thu, 03 Jul 2025 17:59:17 -0700 To: mm-commits@vger.kernel.org,zokeefe@google.com,ziy@nvidia.com,willy@infradead.org,will@kernel.org,wangkefeng.wang@huawei.com,vishal.moola@gmail.com,usamaarif642@gmail.com,tiwai@suse.de,thomas.hellstrom@linux.intel.com,surenb@google.com,sunnanyong@huawei.com,ryan.roberts@arm.com,rostedt@goodmis.org,rientjes@google.com,rdunlap@infradead.org,raquini@redhat.com,peterx@redhat.com,mhocko@suse.com,mhiramat@kernel.org,mathieu.desnoyers@efficios.com,lorenzo.stoakes@oracle.com,liam.howlett@oracle.com,kirill.shutemov@linux.intel.com,jack@suse.cz,hannes@cmpxchg.org,dev.jain@arm.com,david@redhat.com,corbet@lwn.net,cl@gentwo.org,catalin.marinas@arm.com,baolin.wang@linux.alibaba.com,baohua@kernel.org,bagasdotme@gmail.com,anshuman.khandual@arm.com,aarcange@redhat.com,npache@redhat.com,akpm@linux-foundation.org From: Andrew Morton Subject: + khugepaged-introduce-khugepaged_scan_bitmap-for-mthp-support.patch added to mm-new branch Message-Id: <20250704005917.DB315C4CEE3@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: khugepaged: introduce khugepaged_scan_bitmap for mTHP support has been added to the -mm mm-new branch. Its filename is khugepaged-introduce-khugepaged_scan_bitmap-for-mthp-support.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/khugepaged-introduce-khugepaged_scan_bitmap-for-mthp-support.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via the mm-everything branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there every 2-3 working days ------------------------------------------------------ From: Nico Pache Subject: khugepaged: introduce khugepaged_scan_bitmap for mTHP support Date: Tue, 1 Jul 2025 23:57:33 -0600 khugepaged scans anons PMD ranges for potential collapse to a hugepage. To add mTHP support we use this scan to instead record chunks of utilized sections of the PMD. khugepaged_scan_bitmap uses a stack struct to recursively scan a bitmap that represents chunks of utilized regions. We can then determine what mTHP size fits best and in the following patch, we set this bitmap while scanning the anon PMD. A minimum collapse order of 2 is used as this is the lowest order supported by anon memory. max_ptes_none is used as a scale to determine how "full" an order must be before being considered for collapse. When attempting to collapse an order that has its order set to "always" lets always collapse to that order in a greedy manner without considering the number of bits set. Link: https://lkml.kernel.org/r/20250702055742.102808-7-npache@redhat.com Signed-off-by: Nico Pache Cc: Andrea Arcangeli Cc: Anshuman Khandual Cc: Bagas Sanjaya Cc: Baolin Wang Cc: Barry Song Cc: Catalin Marinas Cc: Christoph Lameter (Ampere) Cc: David Hildenbrand Cc: David Rientjes Cc: Dev Jain Cc: Jan Kara Cc: Johannes Weiner Cc: Jonathan Corbet Cc: Kefeng Wang Cc: Kirill A. Shuemov Cc: Liam Howlett Cc: Lorenzo Stoakes Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Nanyong Sun Cc: Peter Xu Cc: Rafael Aquini Cc: Randy Dunlap Cc: Reported-by:Takashi Iwai Cc: Ryan Roberts Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Thomas Hellstrom Cc: Usama Arif Cc: Vishal Moola (Oracle) Cc: Will Deacon Cc: Zach O'Keefe Cc: Zi Yan Signed-off-by: Andrew Morton --- include/linux/khugepaged.h | 4 + mm/khugepaged.c | 96 +++++++++++++++++++++++++++++++---- 2 files changed, 90 insertions(+), 10 deletions(-) --- a/include/linux/khugepaged.h~khugepaged-introduce-khugepaged_scan_bitmap-for-mthp-support +++ a/include/linux/khugepaged.h @@ -1,6 +1,10 @@ /* SPDX-License-Identifier: GPL-2.0 */ #ifndef _LINUX_KHUGEPAGED_H #define _LINUX_KHUGEPAGED_H +#define KHUGEPAGED_MIN_MTHP_ORDER 2 +#define KHUGEPAGED_MIN_MTHP_NR (1<mthp_bitmap_stack[++top] = (struct scan_bit_state) + { HPAGE_PMD_ORDER - KHUGEPAGED_MIN_MTHP_ORDER, 0 }; + + while (top >= 0) { + state = cc->mthp_bitmap_stack[top--]; + order = state.order + KHUGEPAGED_MIN_MTHP_ORDER; + offset = state.offset; + num_chunks = 1 << (state.order); + // Skip mTHP orders that are not enabled + if (!test_bit(order, &enabled_orders)) + goto next; + + // copy the relavant section to a new bitmap + bitmap_shift_right(cc->mthp_bitmap_temp, cc->mthp_bitmap, offset, + MTHP_BITMAP_SIZE); + + bits_set = bitmap_weight(cc->mthp_bitmap_temp, num_chunks); + threshold_bits = (HPAGE_PMD_NR - khugepaged_max_ptes_none - 1) + >> (HPAGE_PMD_ORDER - state.order); + + //Check if the region is "almost full" based on the threshold + if (bits_set > threshold_bits || is_pmd_only + || test_bit(order, &huge_anon_orders_always)) { + ret = collapse_huge_page(mm, address, referenced, unmapped, cc, + mmap_locked, order, offset * KHUGEPAGED_MIN_MTHP_NR); + if (ret == SCAN_SUCCEED) { + collapsed += (1 << order); + continue; + } + } + +next: + if (state.order > 0) { + next_order = state.order - 1; + mid_offset = offset + (num_chunks / 2); + cc->mthp_bitmap_stack[++top] = (struct scan_bit_state) + { next_order, mid_offset }; + cc->mthp_bitmap_stack[++top] = (struct scan_bit_state) + { next_order, offset }; + } + } + return collapsed; +} + static int khugepaged_scan_pmd(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long address, bool *mmap_locked, @@ -1435,9 +1513,7 @@ out_unmap: pte_unmap_unlock(pte, ptl); if (result == SCAN_SUCCEED) { result = collapse_huge_page(mm, address, referenced, - unmapped, cc); - /* collapse_huge_page will return with the mmap_lock released */ - *mmap_locked = false; + unmapped, cc, mmap_locked, HPAGE_PMD_ORDER, 0); } out: trace_mm_khugepaged_scan_pmd(mm, folio, writable, referenced, @@ -2385,7 +2461,7 @@ static int khugepaged_collapse_single_pm int result = SCAN_FAIL; struct mm_struct *mm = vma->vm_mm; - if (IS_ENABLED(CONFIG_SHMEM) && !vma_is_anonymous(vma)) { + if (!vma_is_anonymous(vma)) { struct file *file = get_file(vma->vm_file); pgoff_t pgoff = linear_page_index(vma, addr); _ Patches currently in -mm which might be from npache@redhat.com are khugepaged-rename-hpage_collapse_-to-khugepaged_.patch introduce-khugepaged_collapse_single_pmd-to-unify-khugepaged-and-madvise_collapse.patch khugepaged-generalize-hugepage_vma_revalidate-for-mthp-support.patch khugepaged-generalize-__collapse_huge_page_-for-mthp-support.patch khugepaged-introduce-khugepaged_scan_bitmap-for-mthp-support.patch khugepaged-add-mthp-support.patch khugepaged-skip-collapsing-mthp-to-smaller-orders.patch khugepaged-avoid-unnecessary-mthp-collapse-attempts.patch khugepaged-allow-madvise_collapse-to-check-all-anonymous-mthp-orders.patch khugepaged-improve-tracepoints-for-mthp-orders.patch khugepaged-add-per-order-mthp-khugepaged-stats.patch documentation-mm-update-the-admin-guide-for-mthp-collapse.patch