Re: [PATCH v3] mm/khugepaged: fix inconsistent MMF_VM_HUGEPAGE flag due to allocation failure order

[Lorenzo Stoakes <ljs@kernel.org>]

On Mon, May 11, 2026 at 10:54:07AM +0800, Ye Liu wrote:
> From: Ye Liu <liuye@kylinos.cn>
>
> __khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the
> corresponding mm_slot.  If mm_slot_alloc() fails, the function
> returns with the flag set but without inserting the mm into the
> khugepaged tracking structures, leaving the mm in an inconsistent
> state where future registration attempts are skipped.
>
> Fix this by reordering: allocate the mm_slot first, then check and
> set the flag.  If the flag is already set, free the allocated slot
> and return.  This ensures the flag is only set when the mm is
> successfully registered in the khugepaged tracking structures.
>
> Fixes: 16618670276a ("mm: khugepaged: avoid pointless allocation for "struct mm_slot"")
> Suggested-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Ye Liu <liuye@kylinos.cn>

LGTM, so:

Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>

> ---
> Changes since v2:
> - Reorder to allocate mm_slot first, free it when flag already set,
>   as suggested by David, Dev Jain and Lance Yang
> - Update the subject line to better match the patch.
> - Link: https://lore.kernel.org/all/20260506012130.9306-1-ye.liu@linux.dev/
>
> Changes since v1:
> - Add Fixes tag as suggested by Dev Jain and Lance Yang
> - Link: https://lore.kernel.org/all/20260501075708.327217-1-ye.liu@linux.dev/
>
>  mm/khugepaged.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index 5f4e009593e0..78735f34250a 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -437,13 +437,16 @@ void __khugepaged_enter(struct mm_struct *mm)
>
>  	/* __khugepaged_exit() must not run from under us */
>  	VM_BUG_ON_MM(collapse_test_exit(mm), mm);
> -	if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm)))
> -		return;
>
>  	slot = mm_slot_alloc(mm_slot_cache);
>  	if (!slot)
>  		return;
>
> +	if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
> +		mm_slot_free(mm_slot_cache, slot);
> +		return;
> +	}
> +
>  	spin_lock(&khugepaged_mm_lock);
>  	mm_slot_insert(mm_slots_hash, mm, slot);
>  	/*
> --
> 2.43.0
>