From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80BE1217F55; Tue, 30 Sep 2025 15:17:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759245449; cv=none; b=qc74X76d+zYMxsELxieaemYspY/H76Qnm6g64AAKAiQN/irarqbBKCf7lUmfgm1BN2yH7NoU/1yEySNf/CEUOxeL1waTS20mb5FHcl+q8WaVUynXBa8nb6t6gM0hRvOj9NjTPcZFnzMiBEGvFeWrIxn+m7c/+Rp50Pl+eYFTEwg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759245449; c=relaxed/simple; bh=Btc4rSCe/HJ/vzlHF8etXZYnpDnTtqYeukhuff9YtGs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DEgKN/waeZNY/0wyfWPRkJ7IRNegQ3tvWXMrVah0bL1Ch/k9RNjAVyHGsGRx1hZnp95sRm7L4r4viidHKUdt+JEnOpLjkSp6LI9uNKAH4EPyRfp2Dc3IxHEVEI1oMaewXgG/5BXEyBWSpaIHHaoXNQc60TQj8T+d5IfAcRB9RjY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=QsOW+uVI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="QsOW+uVI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AC982C4CEF0; Tue, 30 Sep 2025 15:17:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1759245449; bh=Btc4rSCe/HJ/vzlHF8etXZYnpDnTtqYeukhuff9YtGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QsOW+uVI43xLLnCfx7yuFUfOaimif2tpBD3Ira/jX5gO1puc3YT/6HXqNpErsYGly nz/HMUP102dPbGbWfNAGq+MV4D0HSi/rPiOqX2tXPQ4sZU8igUIgPK/Ww1ZqfBJWjB Lsbq2nB+aSYgJ6XDtdA/chO2UEuXsnj1HneebM+w= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Hugh Dickins , David Hildenbrand , "Aneesh Kumar K.V" , Axel Rasmussen , Chris Li , Christoph Hellwig , Jason Gunthorpe , Johannes Weiner , John Hubbard , Keir Fraser , Konstantin Khlebnikov , Li Zhe , "Matthew Wilcox (Oracle)" , Peter Xu , Rik van Riel , Shivank Garg , Vlastimil Babka , Wei Xu , Will Deacon , yangge , Yuanchu Xie , Yu Zhao , Andrew Morton , Sasha Levin Subject: [PATCH 6.1 21/73] mm/gup: local lru_add_drain() to avoid lru_add_drain_all() Date: Tue, 30 Sep 2025 16:47:25 +0200 Message-ID: <20250930143821.446321742@linuxfoundation.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250930143820.537407601@linuxfoundation.org> References: <20250930143820.537407601@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.1-stable review patch. If anyone has any objections, please let me know. ------------------ From: Hugh Dickins [ Upstream commit a09a8a1fbb374e0053b97306da9dbc05bd384685 ] In many cases, if collect_longterm_unpinnable_folios() does need to drain the LRU cache to release a reference, the cache in question is on this same CPU, and much more efficiently drained by a preliminary local lru_add_drain(), than the later cross-CPU lru_add_drain_all(). Marked for stable, to counter the increase in lru_add_drain_all()s from "mm/gup: check ref_count instead of lru before migration". Note for clean backports: can take 6.16 commit a03db236aebf ("gup: optimize longterm pin_user_pages() for large folio") first. Link: https://lkml.kernel.org/r/66f2751f-283e-816d-9530-765db7edc465@google.com Signed-off-by: Hugh Dickins Acked-by: David Hildenbrand Cc: "Aneesh Kumar K.V" Cc: Axel Rasmussen Cc: Chris Li Cc: Christoph Hellwig Cc: Jason Gunthorpe Cc: Johannes Weiner Cc: John Hubbard Cc: Keir Fraser Cc: Konstantin Khlebnikov Cc: Li Zhe Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Rik van Riel Cc: Shivank Garg Cc: Vlastimil Babka Cc: Wei Xu Cc: Will Deacon Cc: yangge Cc: Yuanchu Xie Cc: Yu Zhao Cc: Signed-off-by: Andrew Morton [ Resolved minor conflicts ] Signed-off-by: Hugh Dickins Signed-off-by: Sasha Levin --- mm/gup.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 44e5fe2535d0e..e1f125af9c844 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1968,7 +1968,7 @@ static unsigned long collect_longterm_unpinnable_pages( { unsigned long i, collected = 0; struct folio *prev_folio = NULL; - bool drain_allow = true; + int drained = 0; for (i = 0; i < nr_pages; i++) { struct folio *folio = page_folio(pages[i]); @@ -1990,10 +1990,17 @@ static unsigned long collect_longterm_unpinnable_pages( continue; } - if (drain_allow && folio_ref_count(folio) != - folio_expected_ref_count(folio) + 1) { + if (drained == 0 && + folio_ref_count(folio) != + folio_expected_ref_count(folio) + 1) { + lru_add_drain(); + drained = 1; + } + if (drained == 1 && + folio_ref_count(folio) != + folio_expected_ref_count(folio) + 1) { lru_add_drain_all(); - drain_allow = false; + drained = 2; } if (folio_isolate_lru(folio)) -- 2.51.0