From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DA6D3E5EDA for ; Mon, 20 Jul 2026 14:29:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557773; cv=none; b=g68eXt8fLYhMzNQO6IDKoumwm5c1bS9z1vgjwPXvDL0Snq8ZW0Bi9BW5HpUbPYCCl2i/WlI+ikOmowe10u584Udf9RJ5jOtiq/rOVf7wsMLXCT+pyyDtYPAQUrQ/LLEGOBCi2cgD8l/WxlmYcRXR4RCflVfHCvW8LMtjzp6DvZs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557773; c=relaxed/simple; bh=pAlioVGcAe/e3f7WyV06lzak5Q7e6yUDJt0T3QvBv2A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iwfXQ/3k6VXwJNiovAH7MCUYhDxQpOX2Bh1k+R4HMK9NltYboxbmbsdHpp/nFleUHBcy3Itzn+N1cS+lq/a0bdF4N8rt3M2toDTUC6sKOjnUiiM96ucUwgaYCdYDgZdVEJUARUmt3sGSqdmGxkLXkLWp0kPg9yI4Rc4/tyVoqO8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B95653E18; Mon, 20 Jul 2026 14:29:23 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BA09F779AA; Mon, 20 Jul 2026 14:29:22 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cLYiKsIwXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:22 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] mm/khugepaged: hoist isolation into collapse_isolate_folio() Date: Mon, 20 Jul 2026 15:29:11 +0100 Message-ID: <20260720142913.846902-7-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: B95653E18 X-Rspamd-Action: no action Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 118 +++++++++++++++++++++++++++--------------------- 1 file changed, 66 insertions(+), 52 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index db1f765b6ead..429e2c5833d0 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2332,6 +2332,69 @@ static enum scan_result prepare_collapse_file_folio(pgoff_t index, struct collap return ret; } +static enum scan_result collapse_isolate_folio(struct collapse_file_state *state) +{ + struct folio *folio = state->folio; + enum scan_result result; + + /* + * The folio must be locked, so we can drop the i_pages lock + * without racing with truncate. + */ + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + + /* make sure the folio is up to date */ + if (unlikely(!folio_test_uptodate(folio))) { + result = SCAN_FAIL; + goto out_unlock; + } + + /* + * If file was truncated then extended, or hole-punched, before + * we locked the first folio, then a THP might be there already. + * This will be discovered on the first iteration. + */ + if (is_pmd_order(folio_order(folio))) { + result = SCAN_PTE_MAPPED_HUGEPAGE; + goto out_unlock; + } + + if (folio_mapping(folio) != state->mapping) { + result = SCAN_TRUNCATED; + goto out_unlock; + } + + if (!state->is_shmem && (folio_test_dirty(folio) || + folio_test_writeback(folio))) { + /* + * khugepaged only works on clean file-backed folios, + * so this folio is dirty because it hasn't been flushed + * since first write. + */ + result = SCAN_PAGE_DIRTY_OR_WRITEBACK; + goto out_unlock; + } + + if (!folio_isolate_lru(folio)) { + result = SCAN_DEL_PAGE_LRU; + goto out_unlock; + } + + if (!filemap_release_folio(folio, GFP_KERNEL)) { + result = SCAN_PAGE_HAS_PRIVATE; + folio_putback_lru(folio); + goto out_unlock; + } + + if (folio_mapped(folio)) + try_to_unmap(folio, TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH); + return SCAN_SUCCEED; +out_unlock: + folio_unlock(folio); + folio_put(folio); + return result; +} + /** * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * @@ -2442,61 +2505,12 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, folio = state.folio; if (result != SCAN_SUCCEED) goto xa_unlocked; - /* - * The folio must be locked, so we can drop the i_pages lock - * without racing with truncate. - */ - VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); - - /* make sure the folio is up to date */ - if (unlikely(!folio_test_uptodate(folio))) { - result = SCAN_FAIL; - goto out_unlock; - } - - /* - * If file was truncated then extended, or hole-punched, before - * we locked the first folio, then a THP might be there already. - * This will be discovered on the first iteration. - */ - if (is_pmd_order(folio_order(folio))) { - result = SCAN_PTE_MAPPED_HUGEPAGE; - goto out_unlock; - } - - if (folio_mapping(folio) != mapping) { - result = SCAN_TRUNCATED; - goto out_unlock; - } - - if (!is_shmem && (folio_test_dirty(folio) || - folio_test_writeback(folio))) { - /* - * khugepaged only works on clean file-backed folios, - * so this folio is dirty because it hasn't been flushed - * since first write. - */ - result = SCAN_PAGE_DIRTY_OR_WRITEBACK; - goto out_unlock; - } - if (!folio_isolate_lru(folio)) { - result = SCAN_DEL_PAGE_LRU; - goto out_unlock; - } - - if (!filemap_release_folio(folio, GFP_KERNEL)) { - result = SCAN_PAGE_HAS_PRIVATE; - folio_putback_lru(folio); - goto out_unlock; - } - - if (folio_mapped(folio)) - try_to_unmap(folio, - TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH); + result = collapse_isolate_folio(&state); + if (result != SCAN_SUCCEED) + goto xa_unlocked; xas_lock_irq(&xas); - VM_BUG_ON_FOLIO(folio != xa_load(xas.xa, index), folio); /* -- 2.55.0