From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC90141B340 for ; Wed, 29 Apr 2026 18:55:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777488903; cv=none; b=WNTN9q4cRD7CwdYPSTakRI+Mso1NShpJrlu0df/54KJuGkQqsXhljECgeYEGVMKHmBGy7ksr+gq3Z3NuEc5IO5veFN1T9gj2wsO+eBkJGoibD+BNSWJ0VoXX81ZNxYDhrttPzkkBxsE1Rf4q0baRnxaabwNJXrVGH9s6oo/43hI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777488903; c=relaxed/simple; bh=+tcJwPe3ENgOMr1AVld+0sFjb7UaMOe396YP4cP3REs=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Gz5IHY4sI7MCkMVIwXnr5JPQ1T7ALI/hQPoSSfi1DF7iPup1UbTbjhKI7eph+X1NWKCGtnpvOSPspL5e3Ve9CIi8ZWphpDuIosGkItZ34Gb8HF0V6RkUBp0/uvjj2PeAHC0ma+RNx+fxunHqFnHumaNrkFOC30mim5kjTs2grAg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EGfExnTj; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EGfExnTj" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-82f1f6103afso81859b3a.1 for ; Wed, 29 Apr 2026 11:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1777488901; x=1778093701; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=31n0og1/dxcJF7KY10Vv8Lw8nirerVCiMfZrG3XZkhs=; b=EGfExnTjlydXHfWq6tGRwHOHzGC7iCa84K0liMHULrWse1rFiJ/yGZyOXA687NtZSt 1sh9b2TGYmmDnVkw3i743TFT5qAlCXJiQ767hgdIOlO58Nuav48w05FGRpwCHldRVqU+ zmIqhWpXk2AiZ2gt5XAo7RGsJqrLwh2Y7sSIIUDMm1KJ4SK8DfTFE7msLirNV46nmb8p Dz4WYEXT+IzjrS7fqHfDnxxCWw6Zi9DS2TfZrypdbTWKDX5jPDDagZZrn4hW6A223SW4 ceQHKwYIGjMZ1FyNkygIWAyqCa9PgJ1SdtA7o78XFbLGxX9qvMxrswq1rTI8G3Bceo+C 8HBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777488901; x=1778093701; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=31n0og1/dxcJF7KY10Vv8Lw8nirerVCiMfZrG3XZkhs=; b=oicQoxMgYi6p/WSU3ui3QRynSyXh1NaKxaaqt/MIZcdVtJgJ7i0H8RI5rBLuDU+hPO bNndy+3lLk5v4SOgh9JJBnqbjnynzguZOQztBcljRA5OCPdXK0kbHa8h5ujMkVkHSP6W TClXoT4J8WzHysNpEIzQMn9YpsSxcEiheTxqLRIeDMiMREbeVVWxeK2T6gpjATuZHtEv bW/DKz1vkEGl8DE//ibMrUBTOOZ7XCqzZW/LGm5caFHVjMnrbketS5neVUnuDMarZtsW NucyZ68Hho5386ePs0w3GamhzQNrPrtY0qqHJkrKDe1UQti5s2On/XW9umgi42q7WbwF gHMQ== X-Forwarded-Encrypted: i=1; AFNElJ/lsbxPtSUQSp1vH9xbS2nQjD4dz8N7m76L47v0FpTVfnXj/d9fk6mPpEWB5nff2n3GxI4R/wl0NKH+yBo=@vger.kernel.org X-Gm-Message-State: AOJu0YzS+gjD4/H4yb0B4VbxaB9iEAwyWLqY7kZHbjizDIu/OFGQFDjf Ur5aTVRqLjWCKlhUTqXF00AX+OILEUwbyMrhkIA9O5b2ILb6vAe1H+/w X-Gm-Gg: AeBDies+qsOpQrdFKNRwaHvo12qETLOOCLYSMTjVULLIqL54KHaCeQniJRDKR49iISC zSo29CvQ1gztOu4KQd1aAZg7UtAow9R6mD0A9h5eGsG1FZ3EpRS5FtdVn3miscKb6r9oRFWJmB/ uFIqJcY1VujZgDsw464/pefW1VTlyOQEzYb8ZX/c3dvhVOwT6jKw0fcM6Pq3XwqfgIWk9rHG5Hl PKTWQNLoXfVjyF36+7ThJxeg0R4x3B5RB/mnW26A112r/eQ0LQrR6E3v+l4jMPxPPwMvrL9OnVd 4DKn8gWtiKmj+CldQM1ofcVwr6/MBzHvBweZa4v3ekKQdqKJ8nljjLsgB+wsX2W+1cYTw8C1d+m meufCPpzwxfgZ4L9nL1Jz+vSGASedHamxhjHhCj9fcseNFltLw8J33lb4Gax1M5tSlN/osXUw94 YQBG8B0PoOMD5eL+sgd1gjUhGVkvsdgd9bzXAIlbvj X-Received: by 2002:a05:6a00:1ca2:b0:82c:e60c:f36d with SMTP id d2e1a72fcca58-834ddc93c18mr8123810b3a.48.1777488900900; Wed, 29 Apr 2026 11:55:00 -0700 (PDT) Received: from vini ([2401:4900:8fcb:3097:5ced:f23a:32ed:4b52]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-834ed5cd3cdsm2683785b3a.19.2026.04.29.11.54.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Apr 2026 11:55:00 -0700 (PDT) From: Vineet Agarwal To: akpm@linux-foundation.org, hannes@cmpxchg.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kasong@tencent.com, qi.zheng@linux.dev, shakeel.butt@linux.dev, baohua@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, david@kernel.org, mhocko@kernel.org, ljs@kernel.org, linuszeng@tencent.com, Vineet Agarwal Subject: [PATCH] mm/vmscan: fix delayed flusher wakeup in MGLRU Date: Thu, 30 Apr 2026 00:24:41 +0530 Message-ID: <20260429185441.486804-1-agarwal.vineet2006@gmail.com> X-Mailer: git-send-email 2.54.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit MGLRU currently decides whether to wake flusher threads in try_to_shrink_lruvec() using cumulative reclaim counters: sc->nr.unqueued_dirty == sc->nr.file_taken However, these counters are accumulated across multiple evict_folios() passes before the check is performed. This can delay or suppress flusher wakeup when an earlier reclaim batch isolates only dirty file folios, but a later batch isolates clean file folios before try_to_shrink_lruvec() performs the final comparison. For example: batch 1: file_taken = 100, unqueued_dirty = 100 batch 2: file_taken += 60, unqueued_dirty += 0 Final check becomes 100 != 160 and flusher wakeup is skipped, even though reclaim was already blocked by dirty file folios in batch 1. Classic reclaim avoids this by using per-batch values: stat.nr_unqueued_dirty == nr_taken and waking flushers immediately when the condition is met. Make MGLRU use the same per-batch flusher wakeup behavior as classic reclaim by moving the flusher wakeup into evict_folios(), using batch-local isolation results from scan_folios() instead of the cumulative counters checked later in try_to_shrink_lruvec(). This avoids missed flusher wakeups and makes dirty folio reclaim behavior consistent with classic reclaim. Fixes: 1bc542c6a0d14 ("mm/vmscan: wake up flushers conditionally to avoid cgroup OOM") Signed-off-by: Vineet Agarwal --- mm/vmscan.c | 46 ++++++++++++++++++++-------------------------- 1 file changed, 20 insertions(+), 26 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index bd1b1aa12581..f9b6cc146a3d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4680,7 +4680,8 @@ static bool isolate_folio(struct lruvec *lruvec, struct folio *folio, struct sca static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int type, int tier, - struct list_head *list) + struct list_head *list, + unsigned long *file_taken) { int i; int gen; @@ -4749,7 +4750,7 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, scanned, skipped, isolated, type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON); if (type == LRU_GEN_FILE) - sc->nr.file_taken += isolated; + *file_taken += isolated; /* * There might not be eligible folios due to reclaim_idx. Check the * remaining to prevent livelock if it's not making progress. @@ -4798,7 +4799,8 @@ static int get_type_to_scan(struct lruvec *lruvec, int swappiness) static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int swappiness, - int *type_scanned, struct list_head *list) + int *type_scanned, struct list_head *list, + unsigned long *file_taken) { int i; int type = get_type_to_scan(lruvec, swappiness); @@ -4809,7 +4811,8 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, *type_scanned = type; - scanned = scan_folios(nr_to_scan, lruvec, sc, type, tier, list); + scanned = scan_folios(nr_to_scan, lruvec, sc, type, tier, + list, file_taken); if (scanned) return scanned; @@ -4825,6 +4828,7 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, int type; int scanned; int reclaimed; + unsigned long file_taken = 0; LIST_HEAD(list); LIST_HEAD(clean); struct folio *folio; @@ -4839,8 +4843,8 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, lruvec_lock_irq(lruvec); - scanned = isolate_folios(nr_to_scan, lruvec, sc, swappiness, &type, &list); - + scanned = isolate_folios(nr_to_scan, lruvec, sc, swappiness, + &type, &list, &file_taken); scanned += try_to_inc_min_seq(lruvec, swappiness); if (evictable_min_seq(lrugen->min_seq, swappiness) + MIN_NR_GENS > lrugen->max_seq) @@ -4852,6 +4856,14 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, return scanned; retry: reclaimed = shrink_folio_list(&list, pgdat, sc, &stat, false, memcg); + + if (stat.nr_unqueued_dirty && stat.nr_unqueued_dirty == file_taken) { + wakeup_flusher_threads(WB_REASON_VMSCAN); + + if (!writeback_throttling_sane(sc)) + reclaim_throttle(pgdat, VMSCAN_THROTTLE_WRITEBACK); + } + sc->nr.file_taken += file_taken; sc->nr.unqueued_dirty += stat.nr_unqueued_dirty; sc->nr_reclaimed += reclaimed; trace_mm_vmscan_lru_shrink_inactive(pgdat->node_id, @@ -5021,27 +5033,9 @@ static bool try_to_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc) } /* - * If too many file cache in the coldest generation can't be evicted - * due to being dirty, wake up the flusher. + * Flusher wakeup and writeback throttling are handled in + * evict_folios() based on per-batch reclaim results. */ - if (sc->nr.unqueued_dirty && sc->nr.unqueued_dirty == sc->nr.file_taken) { - struct pglist_data *pgdat = lruvec_pgdat(lruvec); - - wakeup_flusher_threads(WB_REASON_VMSCAN); - - /* - * For cgroupv1 dirty throttling is achieved by waking up - * the kernel flusher here and later waiting on folios - * which are in writeback to finish (see shrink_folio_list()). - * - * Flusher may not be able to issue writeback quickly - * enough for cgroupv1 writeback throttling to work - * on a large system. - */ - if (!writeback_throttling_sane(sc)) - reclaim_throttle(pgdat, VMSCAN_THROTTLE_WRITEBACK); - } - /* whether this lruvec should be rotated */ return nr_to_scan < 0; } -- 2.54.0