From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A53926ACC for ; Sat, 11 Jul 2026 04:46:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783745165; cv=none; b=KKmxhCb3I0xHvSK8uI07FQsswW3jripNtqWqjimCwXnTTB3z1Q7z9gm7oSD+338VuFAKJk6psOYVaU94O/sIoQPv0ToK+0pwG+lNPHUdvSleBlPvqguhnVH1ZimAXqWnxCv2xwmIpPKglcotCO+x21N3IZZQdrX6todS6jSuCVw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783745165; c=relaxed/simple; bh=p6JmzB+BP0JbalDPQwxArcMzoVMxMW1XGYau9GKrVII=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=CTUiDZua8kU5x+TjUWc9/zPT94r5ljDfWS570sU6cAYnk1zaJ39TAFyUDzfDnp0WAOymDUvp/qhWk/FjfcI+HB7u75JdJhAo9SiC3Gpp08tesPpGo76mPGU40Kxj2KOjraHuU2p2wtLXaXNr7z8ONKu3oZ5pVf3XsvNduiQgD6A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=S48vlUul; arc=none smtp.client-ip=209.85.128.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S48vlUul" Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-81e763fb7bbso21216397b3.0 for ; Fri, 10 Jul 2026 21:46:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783745162; x=1784349962; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=rwiqhuS1dgr/o//ceIUYjpzsCv14pH4erOSkAkROOoo=; b=S48vlUulccexubCTgLoKG/n/MXBSEmZe2CzhWN0oXbyn8i4rpnYAqgk+qRS4BRKs4m ANBcd8s5UAGWknz33aDC01xMNg/ZpZGPw9IuHQ9AvavWSE3O9ScfVo6RRfeIojU4X7ok IcdhvDRt3bC9jsK3X8txNLNX5yZ3AnqrYlochEm0nOJkmT1NlZP8GY1vhsyas7Z8zSf4 yMlFiyeucs0BRwPgt3QOvO2Rl809xRscpwBDiYX5N2G5dT6eQbl3+awxFXQ5nFq/iH/R iwM7e5sq9CAjzJwZtBSyzG4pb/+k2FbUBdbz4zgiCPz+ogk3ycPDEuQxChg+z/JjWdrz XqrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783745162; x=1784349962; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=rwiqhuS1dgr/o//ceIUYjpzsCv14pH4erOSkAkROOoo=; b=KuwmZ11TChwZ6xrY5VRUwrpQvc/sUkpGuJKwv6ipM4Bovh74SfLFIf1FSyld6zQ+iv kcWRgQWH2VuziS0XzapzEz2iFw1Ew7OTcNqxbh8RM2VGAVlffN1drLJdF9o9sNiagGrU j2rgjDn2d4hyrElQJHPLyC1g8BnqaMaHKS178lSDjmcYP7/W3weypj0CUkH7u9MCXWiB cg1i5ClLxt0rFuwc9UEilCMYOduWouVatsJBPvlnLgAXu7JW/0fU69ivF6D/+Q2Q/ZiV P8jrwHpTPKqdWbpEea+SMn4z2dirWNRvXqpdwCb2KyBNKxS4HVzvMcnCmynMVv9k7haq U8TA== X-Forwarded-Encrypted: i=1; AHgh+Rpw5l1Hm6Q1rcppHKU5RY6hZcWLmqplGT0JiSDfA4fftqFphtEG0e8Veo+3/JhHzkju9B+Cs5ZoaVWjkRk=@vger.kernel.org X-Gm-Message-State: AOJu0Yxoov16YaD2EuAjkFB22RjajHdSvs+TCZl3a8Zk30yqcGjdiRar FQrbFNV64wYJ7A9KWLAKApc7yBpR/IIzlefy5qki3dMfiNnE01xjcqOp X-Gm-Gg: AfdE7cl81RSPGQd6hDKzI5E2yB3OOeaPCoAekB9fSwkqF2sc1bJXJNqAOgAi1EhKR2F rTnwSwZfOuo82iaEa0gbgoRz5UP7fzXaXY/WeZtIUdL9zngLFwlbx0D3z1o7ye5at1nRTuMc1n1 sKjCWfrBLTam7S+j9PA5sFvpFYhSxgD+odlWSeQjU7wuXKhyQ2lyxUbDVtbGCCs6KyMGY3iJC1E bHbzi6NGDH+dqdiefph9uLaNkmUMoTzrn/KJHK+23TA8EL/ZnIgZf3fzO3+d2X5pKEBSDE72WZD Teq/BPlGvMrp4j5GfUWvRLs9U5DxWnOrzFnjRyfZhT7hituBXKYQ17iotWYRurjRAjE6nYIIbE5 oDcW08khBKeLNt5g3+W27kv/bBoSt/XhRauxLdXO4ItSh+xOl4LRGTRRfrmTqrIGQh0xU0amuCz gAWJVhKs82tKNHIYhx X-Received: by 2002:a05:690c:688e:b0:81e:79f0:4898 with SMTP id 00721157ae682-81e9000683dmr11835697b3.15.1783745161966; Fri, 10 Jul 2026 21:46:01 -0700 (PDT) Received: from host.arpa ([2600:1700:7c:96e0:d1e9:c04d:a97e:9636]) by smtp.gmail.com with ESMTPSA id 00721157ae682-81e950567eesm2640477b3.13.2026.07.10.21.45.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jul 2026 21:46:00 -0700 (PDT) From: Altan Hacigumus To: Andrew Morton , David Hildenbrand , Johannes Weiner Cc: Kairui Song , Qi Zheng , Shakeel Butt , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jiayuan Chen , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2] mm/vmscan: clear hopeless kswapd when global direct reclaim makes progress Date: Fri, 10 Jul 2026 21:39:46 -0700 Message-ID: <20260711043946.68421-1-ahacigu.linux@gmail.com> X-Mailer: git-send-email 2.55.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Direct reclaim clears the hopeless kswapd state only once the node becomes balanced - added by commit dc9fe9b7056a ("mm/vmscan: mitigate spurious kswapd_failures reset from direct reclaim") so that cgroup memory.high reclaim cannot repeatedly revive a kswapd that is unable to balance the node. Before it, any reclaim progress revived kswapd. However, this restriction also prevents global direct reclaim from reviving kswapd. Under sustained memory pressure, global direct reclaim may continue making progress without the node ever reaching the high watermark, leaving reclaim to allocating tasks. The effect is visible on systems where a database workload mlocks most of memory and the remainder is under continuous pressure: allocations stall long enough that ordinary tasks (e.g. ssh) become slow or unresponsive, while kswapd sits idle. Unlike memcg reclaim, global direct reclaim follows the same node-wide reclaim path as kswapd. Clear the hopeless state when global direct reclaim makes progress, while continuing to require a balanced node for memcg reclaim. kswapd_try_clear_hopeless() becomes static since struct scan_control is private to vmscan.c. On a workload with most memory mlocked and the remainder under sustained churn, with swap enabled, comparing the same 60s window: base patched allocstall (all zones) 413441 6532 pgsteal_direct 15889552 255619 pgsteal_kswapd 0 26079382 PSI memory full avg60 13.10% 9.37% Direct reclaim was already reclaiming many pages in the baseline; this change lets kswapd resume doing that work asynchronously. Signed-off-by: Altan Hacigumus --- v2: fold the impact and the origin commit into the changelog v1: https://lore.kernel.org/all/20260710024429.70923-1-ahacigu.linux@gmail.com/ --- include/linux/mmzone.h | 2 -- mm/vmscan.c | 23 ++++++++++++++--------- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index ca2712187147..1db6f8dac927 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1630,8 +1630,6 @@ enum kswapd_clear_hopeless_reason { void wakeup_kswapd(struct zone *zone, gfp_t gfp_mask, int order, enum zone_type highest_zoneidx); -void kswapd_try_clear_hopeless(struct pglist_data *pgdat, - unsigned int order, int highest_zoneidx); void kswapd_clear_hopeless(pg_data_t *pgdat, enum kswapd_clear_hopeless_reason reason); bool kswapd_test_hopeless(pg_data_t *pgdat); diff --git a/mm/vmscan.c b/mm/vmscan.c index 35c3bb15ae96..78fd35a11eee 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -198,6 +198,9 @@ struct scan_control { */ int vm_swappiness = 60; +static void kswapd_try_clear_hopeless(struct pglist_data *pgdat, + struct scan_control *sc); + #ifdef CONFIG_MEMCG /* Returns true for reclaim through cgroup limits or cgroup interfaces. */ @@ -5172,7 +5175,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * blk_finish_plug(&plug); done: if (sc->nr_reclaimed > reclaimed) - kswapd_try_clear_hopeless(pgdat, sc->order, sc->reclaim_idx); + kswapd_try_clear_hopeless(pgdat, sc); } /****************************************************************************** @@ -6251,7 +6254,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) * successful direct reclaim run will revive a dormant kswapd. */ if (reclaimable) - kswapd_try_clear_hopeless(pgdat, sc->order, sc->reclaim_idx); + kswapd_try_clear_hopeless(pgdat, sc); else if (sc->cache_trim_mode) sc->cache_trim_mode_failed = 1; } @@ -7530,15 +7533,17 @@ void kswapd_clear_hopeless(pg_data_t *pgdat, enum kswapd_clear_hopeless_reason r } /* - * Reset kswapd_failures only when the node is balanced. Without this - * check, successful direct reclaim (e.g., from cgroup memory.high - * throttling) can keep resetting kswapd_failures even when the node - * cannot be balanced, causing kswapd to run endlessly. + * Reset kswapd_failures when the node is balanced, or when global + * direct reclaim makes progress - then kswapd can make progress too. + * Memcg reclaim can succeed where kswapd cannot (memcg protection is + * not enforced against the reclaim target), so its progress resets + * kswapd_failures only when the node is balanced. */ -void kswapd_try_clear_hopeless(struct pglist_data *pgdat, - unsigned int order, int highest_zoneidx) +static void kswapd_try_clear_hopeless(struct pglist_data *pgdat, + struct scan_control *sc) { - if (pgdat_balanced(pgdat, order, highest_zoneidx)) + if ((!current_is_kswapd() && !cgroup_reclaim(sc)) || + pgdat_balanced(pgdat, sc->order, sc->reclaim_idx)) kswapd_clear_hopeless(pgdat, current_is_kswapd() ? KSWAPD_CLEAR_HOPELESS_KSWAPD : KSWAPD_CLEAR_HOPELESS_DIRECT); } -- 2.55.0