From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 219F82F0C52; Mon, 22 Sep 2025 19:44:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758570241; cv=none; b=kFT2Lmj6ST06Fvb4czEdw3UUfovi68eeZr1Wr1BwxOspof49NEJyIe1mt8jYrPaS0WHNPY8boTzPP6lmcoNi/H45sHyN3LPoM3eFsKmXWmQYnbyEyDTm0pfEK6BL6r3FiFGqREKmSCAABRYEZWXbdQE0tOjvmVlNeTcOTVJ1kzU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758570241; c=relaxed/simple; bh=4/FlbNyk14T/84dH6be2H1bk1/Hm0/bOebmlRLRqIfk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h2C1fWrtUWWwIjcK8ooW16LKFmAtjcKChHciOuDvwS+g82tLf028vxXOB0YTcVxu5kb4U5LLurodHG9QoZHzv2LR2f9QP3NZBV4FZ9XxgGL5OTthsnz0/M/RkP5BztlwdL4ZB5K0GDk+c937+ABzzim/4aRuRR8cVhnobwbFNcY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=S/R907Zn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="S/R907Zn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8A218C4CEF0; Mon, 22 Sep 2025 19:44:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1758570241; bh=4/FlbNyk14T/84dH6be2H1bk1/Hm0/bOebmlRLRqIfk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S/R907ZnDiAkf1jHvkUwGW9ZkPuCJ9pEwuLNQ1vVmPdDZ07xWICdI3qa9YUKk+1uX c/+hFMvIZ4G42LQ8ay7ExRFLdjnARXsVS6LRc5Vi7DWqOBzfXZ9GYjHdVtaF2AhG8B Hmd+vTsEcbar4LygP+kKbBj+MubGuB45EDweQi0k= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Hugh Dickins , David Hildenbrand , "Aneesh Kumar K.V" , Axel Rasmussen , Chris Li , Christoph Hellwig , Jason Gunthorpe , Johannes Weiner , John Hubbard , Keir Fraser , Konstantin Khlebnikov , Li Zhe , "Matthew Wilcox (Oracle)" , Peter Xu , Rik van Riel , Shivank Garg , Vlastimil Babka , Wei Xu , Will Deacon , yangge , Yuanchu Xie , Yu Zhao , Andrew Morton Subject: [PATCH 6.16 061/149] mm/gup: local lru_add_drain() to avoid lru_add_drain_all() Date: Mon, 22 Sep 2025 21:29:21 +0200 Message-ID: <20250922192414.422164822@linuxfoundation.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250922192412.885919229@linuxfoundation.org> References: <20250922192412.885919229@linuxfoundation.org> User-Agent: quilt/0.68 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.16-stable review patch. If anyone has any objections, please let me know. ------------------ From: Hugh Dickins commit a09a8a1fbb374e0053b97306da9dbc05bd384685 upstream. In many cases, if collect_longterm_unpinnable_folios() does need to drain the LRU cache to release a reference, the cache in question is on this same CPU, and much more efficiently drained by a preliminary local lru_add_drain(), than the later cross-CPU lru_add_drain_all(). Marked for stable, to counter the increase in lru_add_drain_all()s from "mm/gup: check ref_count instead of lru before migration". Note for clean backports: can take 6.16 commit a03db236aebf ("gup: optimize longterm pin_user_pages() for large folio") first. Link: https://lkml.kernel.org/r/66f2751f-283e-816d-9530-765db7edc465@google.com Signed-off-by: Hugh Dickins Acked-by: David Hildenbrand Cc: "Aneesh Kumar K.V" Cc: Axel Rasmussen Cc: Chris Li Cc: Christoph Hellwig Cc: Jason Gunthorpe Cc: Johannes Weiner Cc: John Hubbard Cc: Keir Fraser Cc: Konstantin Khlebnikov Cc: Li Zhe Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Rik van Riel Cc: Shivank Garg Cc: Vlastimil Babka Cc: Wei Xu Cc: Will Deacon Cc: yangge Cc: Yuanchu Xie Cc: Yu Zhao Cc: Signed-off-by: Andrew Morton Signed-off-by: Greg Kroah-Hartman --- mm/gup.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) --- a/mm/gup.c +++ b/mm/gup.c @@ -2333,8 +2333,8 @@ static unsigned long collect_longterm_un struct pages_or_folios *pofs) { unsigned long collected = 0; - bool drain_allow = true; struct folio *folio; + int drained = 0; long i = 0; for (folio = pofs_get_folio(pofs, i); folio; @@ -2353,10 +2353,17 @@ static unsigned long collect_longterm_un continue; } - if (drain_allow && folio_ref_count(folio) != - folio_expected_ref_count(folio) + 1) { + if (drained == 0 && + folio_ref_count(folio) != + folio_expected_ref_count(folio) + 1) { + lru_add_drain(); + drained = 1; + } + if (drained == 1 && + folio_ref_count(folio) != + folio_expected_ref_count(folio) + 1) { lru_add_drain_all(); - drain_allow = false; + drained = 2; } if (!folio_isolate_lru(folio))