From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E044C83F1B for ; Fri, 11 Jul 2025 18:18:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 29A596B00A5; Fri, 11 Jul 2025 14:18:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 24B186B00A7; Fri, 11 Jul 2025 14:18:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1885A6B00A8; Fri, 11 Jul 2025 14:18:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0A5036B00A5 for ; Fri, 11 Jul 2025 14:18:18 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id AA3F280286 for ; Fri, 11 Jul 2025 18:18:17 +0000 (UTC) X-FDA: 83652793434.29.60BB954 Received: from out-173.mta0.migadu.com (out-173.mta0.migadu.com [91.218.175.173]) by imf20.hostedemail.com (Postfix) with ESMTP id DC0DD1C000C for ; Fri, 11 Jul 2025 18:18:15 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=m0n2Rey9; spf=pass (imf20.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752257896; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z6PERit5UuXRcx14tZIOd7Jc1I+i6g4+8Ry4BzEZdK0=; b=qnctVuFk3X8xDSegrbvcF+jS81YP/mecrioFWwsfwhZkokRCyMW3Ys6RYXnOrTI8Gl9ZrS qI2f24sBeCKe5TdvDErD4T1FviCRM0EzToM6DQxutkRs1AAJooGbehN08tM5qXnUa4+NuQ /Tf2W20b51rt8ztDFbq1vZT89GMhnlA= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=m0n2Rey9; spf=pass (imf20.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752257896; a=rsa-sha256; cv=none; b=jfkUKBfD/+fJZ+VNsfeIAJgraBCu1gT0ijLLCL1+gNUUaaZeNjizXd8O65NTja08SSmNEG WUyu5Vpz1CZ7HiqGtdBz2MrRhqgtKkQHskPrvHcsGCCa0JZvkPEPT2gssOnIFTbgQ0Qdfd 62KMPZb0udfVtk+FQAJjw+e/h2Ds25U= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1752257894; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Z6PERit5UuXRcx14tZIOd7Jc1I+i6g4+8Ry4BzEZdK0=; b=m0n2Rey97rQreuqAifFigbhzwg2fr8jP/EiBz1+mpQDdmBYnQ2CzM5O5mYPvXttig3a3A7 ua0+UHdyHZvoxg6WKp7a3PSrfYsoTSec8V3Ie8qBOMnkRee5wcrrlVku8FoBnc8jEcMeir Cyz4TTLJzI6IOpdn8yNiUy4xkjcCIpw= From: Roman Gushchin To: Andrew Morton Cc: Johannes Weiner , Shakeel Butt , Lorenzo Stoakes , Michal Hocko , David Hildenbrand , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm: skip lru_note_cost() when scanning only file or anon In-Reply-To: <20250711155044.137652-1-roman.gushchin@linux.dev> (Roman Gushchin's message of "Fri, 11 Jul 2025 08:50:44 -0700") References: <20250711155044.137652-1-roman.gushchin@linux.dev> Date: Fri, 11 Jul 2025 11:18:01 -0700 Message-ID: <8734b21tie.fsf@linux.dev> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Flow: FLOW_OUT X-Stat-Signature: xzk3781nnif5mcuuai55wpnyptu6sckp X-Rspamd-Queue-Id: DC0DD1C000C X-Rspamd-Server: rspam01 X-Rspam-User: X-HE-Tag: 1752257895-149573 X-HE-Meta: U2FsdGVkX1/Ep1Jtxux4BC+0AzDish4l5dFuXT1SDi3fprwAUNJh3FKVUodgwdBjCkt5ts8Qq/UGG8n2/hsTTyIE0eGIIop9AM2eLwf03tbRJNJnLWFJDM0efJrg75Gk04Kei9i9k1R525cYYFsfpnfhOSAIdXOdlLyGonrN9auqUgSIxoyUl04n/RMJj9mA72eKV1SQ4QrOIywmlBztpNudyIp9/LWCCHDPQvLjYnBHReCRrsmQSu6X7AsSA7Q9lVdEPCbuXipt54+60DfQ2OXdyNQKsDCsh54bTZUhcIXWRwG9UIEQJSzDSMHfjgU7bNHqKYvzcMboBDpFKnpHiXn2QWmLI18OchZocfFxi1SVfpomicAl4llwXbqQBQlt6xI2Umjoq4RhOHnuko7jNzSMHPazjJWnnwsbyggu4oBv/83qINnOaSFlZLV8zai2psRiqLcXCSuHXHWR+jvJFsBa5GhiwCvCDkwiOgzWwMuUzzHWRCr6pQJUbxU6FvojmQLqvXiadriqAZ8hmIk/rWHQYWEBKJ3DDac8kHJn2BcE6HGHh6bq+/9SjfiarDXL4LcgwofM9oMtRWViGam8e0KXsnacfUgx378sQCOsRnGR24AI3fU94eHaO72Hp45ZgHU7MhE6/YXN2EZa40nsD82NccwucBbI2uu70+tIqH7P/fP5w7BEZ8NYRfzc2RqxDW4II/U4hKN5A/EpBbYasgodtcx+kFXkANJOkPQscI0giqZUAWnB3X/9PK4rig4n1YwIA0BV0htnJZPeMs6S67yidobwOYTAGgHwL3YjxZlZ/eG4I9MEyMbiiVbnXKXeMB+npiIhPeg1QW1hPHfVdIkdKWfH3FSHE9Uc6lS4EYLsaOH63A0ridoDer8gy8g/70PyswfKqnUOa006bhQdawPUjBLiJHakLvQoXRw8tHEFTVq5mtjpoI+B08Y5Eg183lO5q61CEKFZFvOxdnd Nn+PYSBU Es/IML7LmHIDHlde9l/MYqr+owGXCj+s2ky+SQfQOsOfSFZgWodPKv53Qw0VOPBFzvn09tH7aGIG14J7Qzhk2EhoRzHDIjlOIO/N4sJucChi8rzB8L8u/Rd8ggxl3ZRAlSDM7lZ3lX78mKemo40cj2oGE3e97MzRu3inkNaz7mCuH+kRPjmPzhFcrLZHKmOPjyrO96re07HfKGFTgeGZzh2/9zdzkdqEPKJQEeAucA/fxTHVKXYn7UgC1msOZ8/staAstbVEJEn9evW+CnVI0vaxsV/l2z3Pw2QTJkD0VaPpf7bc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Sorry, sent a wrong version with a trivial bug. Below is the correct one. The only difference is s/&sc/sc when calling scan_balance_biased(). -- >From c06530edfb8a11139f2d7878ce3956b9238cc702 Mon Sep 17 00:00:00 2001 From: Roman Gushchin Subject: [PATCH] mm: skip lru_note_cost() when scanning only file or anon lru_note_cost() records relative cost of incurring io and cpu spent on lru rotations, which is used to balance the pressure on file and anon memory. The applied pressure is inversely proportional to the recorded cost of reclaiming, but only within 2/3 of the range (swappiness aside). This is useful when both anon and file memory is reclaimable, however in many cases it's not the case: e.g. there might be no swap, proactive reclaim can target anon memory specifically, the memory pressure can come from cgroup v1's memsw limit, etc. In all these cases recording the cost will only bias all following reclaim, also potentially outside of the scope of the original memcg. So it's better to not record the cost if it comes from the initially biased reclaim. lru_note_cost() is a relatively expensive function, which traverses the memcg tree up to the root and takes the lruvec lock on each level. Overall it's responsible for about 50% of cycles spent on lruvec lock, which might be a non-trivial number overall under heavy memory pressure. So optimizing out a large number of lru_note_cost() calls is also beneficial from the performance perspective. Signed-off-by: Roman Gushchin --- mm/vmscan.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index c86a2495138a..7d08606b08ea 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -71,6 +71,13 @@ #define CREATE_TRACE_POINTS #include +enum scan_balance { + SCAN_EQUAL, + SCAN_FRACT, + SCAN_ANON, + SCAN_FILE, +}; + struct scan_control { /* How many pages shrink_list() should reclaim */ unsigned long nr_to_reclaim; @@ -90,6 +97,7 @@ struct scan_control { /* * Scan pressure balancing between anon and file LRUs */ + enum scan_balance scan_balance; unsigned long anon_cost; unsigned long file_cost; @@ -1988,6 +1996,17 @@ static int current_may_throttle(void) return !(current->flags & PF_LOCAL_THROTTLE); } +static bool scan_balance_biased(struct scan_control *sc) +{ + switch (sc->scan_balance) { + case SCAN_EQUAL: + case SCAN_FRACT: + return false; + default: + return true; + } +} + /* * shrink_inactive_list() is a helper for shrink_node(). It returns the number * of reclaimed pages @@ -2054,7 +2073,9 @@ static unsigned long shrink_inactive_list(unsigned long nr_to_scan, __count_vm_events(PGSTEAL_ANON + file, nr_reclaimed); spin_unlock_irq(&lruvec->lru_lock); - lru_note_cost(lruvec, file, stat.nr_pageout, nr_scanned - nr_reclaimed); + if (!scan_balance_biased(sc)) + lru_note_cost(lruvec, file, stat.nr_pageout, + nr_scanned - nr_reclaimed); /* * If dirty folios are scanned that are not queued for IO, it @@ -2202,7 +2223,7 @@ static void shrink_active_list(unsigned long nr_to_scan, __mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, -nr_taken); spin_unlock_irq(&lruvec->lru_lock); - if (nr_rotated) + if (nr_rotated && !scan_balance_biased(sc)) lru_note_cost(lruvec, file, 0, nr_rotated); trace_mm_vmscan_lru_shrink_active(pgdat->node_id, nr_taken, nr_activate, nr_deactivate, nr_rotated, sc->priority, file); @@ -2327,13 +2348,6 @@ static bool inactive_is_low(struct lruvec *lruvec, enum lru_list inactive_lru) return inactive * inactive_ratio < active; } -enum scan_balance { - SCAN_EQUAL, - SCAN_FRACT, - SCAN_ANON, - SCAN_FILE, -}; - static void prepare_scan_control(pg_data_t *pgdat, struct scan_control *sc) { unsigned long file; @@ -2613,6 +2627,8 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, calculate_pressure_balance(sc, swappiness, fraction, &denominator); out: + sc->scan_balance = scan_balance; + for_each_evictable_lru(lru) { bool file = is_file_lru(lru); unsigned long lruvec_size; -- 2.50.0