Re: [RFC PATCH v2 06/11] mm: khugepaged: allow khugepaged to check all shmem mTHP-sized orders

[Lance Yang <lance.yang@linux.dev>]

On Wed, Jun 10, 2026 at 06:29:14PM +0800, Baolin Wang wrote:
>We are now ready to enable shmem mTHP collapse, allowing
>thp_vma_allowable_orders() to check all permissible shmem large orders.
>
>Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>---
> mm/khugepaged.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
>diff --git a/mm/khugepaged.c b/mm/khugepaged.c
>index 75b18ec4a6c3..a87918b7e18c 100644
>--- a/mm/khugepaged.c
>+++ b/mm/khugepaged.c
>@@ -578,9 +578,14 @@ static unsigned long collapse_possible_orders(struct vm_area_struct *vma,
> {
> 	unsigned long orders;
> 
>-	/* If khugepaged is scanning an anonymous vma, allow mTHP collapse */
>+	/*
>+	 * If khugepaged is scanning an anonymous or shmem vma,
>+	 * allow mTHP collapse.
>+	 */
> 	if ((tva_flags == TVA_KHUGEPAGED) && vma_is_anonymous(vma))
> 		orders = THP_ORDERS_ALL_ANON;
>+	else if ((tva_flags == TVA_KHUGEPAGED) && vma_is_shmem(vma))
>+		orders = THP_ORDERS_ALL_FILE_DEFAULT;

Hmm... for shmem, is the lower bound in mthp_collapse() expected?

#define KHUGEPAGED_MIN_MTHP_ORDER	2

#define THP_ORDERS_ALL_FILE_DEFAULT	\
	((BIT(MAX_PAGECACHE_ORDER + 1) - 1) & ~BIT(0))

KHUGEPAGED_MIN_MTHP_ORDER is 2, which was introduced for anon collapse.
For shmem, though, we're feeding mthp_collapse() orders from
THP_ORDERS_ALL_FILE_DEFAULT, and that only filter out order-0 ...

		if (order > KHUGEPAGED_MIN_MTHP_ORDER &&
			(enabled_orders & GENMASK(order - 1, 0))) {
			order--;
			continue;
		}

So order-1 never gets a chance. The walker stops at order-2 and never
tries it, right?

> 	else
> 		orders = BIT(HPAGE_PMD_ORDER);
> 
>-- 
>2.47.3
>
>