From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-172.mta1.migadu.com (out-172.mta1.migadu.com [95.215.58.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30D763624C3 for ; Fri, 1 May 2026 07:58:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777622295; cv=none; b=OuvV17HDHm8nwwmgAFVz6eBmfD2A181bscpcrZjtrnO9/ILQ9uH3Ns04Q/q0XTaQzHfpHVFhb/9yalXwF1kdg5MK15wBQektAXGAvktDFd8+u1niXhMdS8wuNzfRK/+jVJleoWU8QlKQs+7C5HmuI5P/yIwoV/+ZhHAFK6pZaeg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777622295; c=relaxed/simple; bh=f5+Kuz8q6eaSK1aAfTjgcBtMXOjTA/B+4XoivhLxSy0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=FXcRzGdShRLACgtNgnUbSx6JFqjTuyGsD+kqTTX07Nrw2aHiWMle5S8MX88OHCFVvyLKCT6tIwvR3QlJw1ExkbHt+hFDHAA/O35ImiMccr4M5DnqvgxD+MmAi0K6khiJNxT6lEIDANcgTH9/UqeNglzcKYUdbCoShIMDWCLkzs0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=lPQlJNof; arc=none smtp.client-ip=95.215.58.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="lPQlJNof" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1777622251; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=kyAk8NA2J8YQgw/gGqBaVOWUHWOA6WWmvSiQgKxkb6M=; b=lPQlJNofp7eLTUX3eNkcXz9TPqr+ahTmftjh5QsREqS0S2al9UrBTHMVb/Z0shGSMIRX40 4KGc2Fz54IwH7pt0+UvqdG6n31XxJkVdF27ly2VC0Em2A1d8opzTh/GXu+mT6pOP0EHFat xC7Gxe3IU10Uf5VxmAcoqg21T9MdDQg= From: Ye Liu To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Ye Liu , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH] mm/khugepaged: clear MMF_VM_HUGEPAGE on mm_slot_alloc() failure Date: Fri, 1 May 2026 15:57:07 +0800 Message-ID: <20260501075708.327217-1-ye.liu@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT From: Ye Liu __khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures. This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early. Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later. Signed-off-by: Ye Liu --- mm/khugepaged.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 7d48d4fbd5f3..60ab7c1b61dd 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm) return; slot = mm_slot_alloc(mm_slot_cache); - if (!slot) + if (!slot) { + mm_flags_clear(MMF_VM_HUGEPAGE, mm); return; + } spin_lock(&khugepaged_mm_lock); mm_slot_insert(mm_slots_hash, mm, slot); -- 2.43.0