From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yx1-f54.google.com (mail-yx1-f54.google.com [74.125.224.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4A3979DA for ; Fri, 10 Jul 2026 02:47:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.224.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783651624; cv=none; b=lY4W6lAzuVV9K7/MY7zV3i+5GFCCGy9HcgFrswKjI260jo/9aznBe19oC0ouZI213sTzXK3XL/4EEquYXSD5hWr2Gc43Dk9NqNuUCTL8QCgpywC7oqfV6GvVBX0yWGJLz4dbMW3nczVt+xs0DrmEVsWBAZBRvChffRw4i4VlIr0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783651624; c=relaxed/simple; bh=z0kAfrJZtKVLm8Wb1E/YcMJrMA4Chu7AnNqZtTmSYsw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=MYF2ciGnQIDPn4WA+YvvOjQK4f2+cKV/241v63FXEuWE0jLiMCh9tZLbdL1YEvpDHeUPCn71L4iyiPeiELHVHAaWATDBX0G85n6k86UIp27jPOV47uvuJpjGNOXRzYngKd3L11BtoeTZoeJ2NNiTo1t7UXn1QrEu5hLP0OrbmoQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=c81RBHdJ; arc=none smtp.client-ip=74.125.224.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c81RBHdJ" Received: by mail-yx1-f54.google.com with SMTP id 956f58d0204a3-664b3831a20so786643d50.3 for ; Thu, 09 Jul 2026 19:47:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783651622; x=1784256422; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=p6nnrWnfSd1WjruGGyvcLkntIHQ6kTbFpN5Ah/hDWRg=; b=c81RBHdJ+HM4jHLYbIN3zuGWNu5Tgr9mof94erzQj7eZZIIsIFk8MgU8iBKBsEc2iX A60Q1Hn/uDyWHVok9VNWJ5cKDwExFDqh0V9ytGn9Be/V+sEbn76ZAvb2bBWKSLdixM37 6qHBTXwatnHUrrWr1RH1GnjeC0O/wol/VwvemdOga07JoLnw9WTdgSq4FKyR3vOmjh73 0BTX2tOh6mcS6dxK/c8jqKo3LO9hqBB/W5jx4ZO8texahdcSsUkPq1bfvbvJfJwy99cR znw5yrmhwp8D5ZF2ihg+SSro7vQLau8pVLbPWEdfNknSSiEBzr7Vypd3wcbxw+BqUxzz Scqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783651622; x=1784256422; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=p6nnrWnfSd1WjruGGyvcLkntIHQ6kTbFpN5Ah/hDWRg=; b=qdVY6sXOI3K9qOThzWwPNLMJigDRjyl2YcNCDDAcUpM/dhgNh3cxcVx4p522/OomdS Kya5lCGWcB2NGkK+nRG4PQJLmJmBu7n7RTcE74F8UO8RJshREzJSwc5grvTVOaitvE2f 1BCFrR8Xye/psLgQDBv71r6fJCvehTBsngnkZsEZuUeCD0pUtMUFXP3xZM3bkbMXA8VB 1Fu7qxs59IzKFer53nkO40u8Bn44iqCZCjcu6Qgjaa6ecUMV1ThYRLxwWUU75HS6gihn GY2aeOnr4OZpaDnTT6YLYRFwqgrOD/eFlYtY27IxCFBtiRZjc4KInw9eCcLe52GGV5Gx GLyw== X-Forwarded-Encrypted: i=1; AHgh+RooqlSJA3X+e4DqtDae62nxQfIeakEwe3d8lwTYh7NRiyx8uYHv9BEKcbTN2PQctc4eyyWYHSL+Ao4UD/4=@vger.kernel.org X-Gm-Message-State: AOJu0YzVyXAMQPo/+kUbXiGAEKe7K4H4OFykyrrk/b3J0PvrE5yuIoKc OczmeUxfw6qbMkzSSyA5C2dFPwv8e2gE1/tJmhga5b41TA2uAZjFdKzv X-Gm-Gg: AfdE7cmqkkfxw+p9npi6r5nbVnW2pud2xqtzNfz2PZLXdc0r43/R3yUfAzqbPRZnzmt 2LJ09MAftkFo26KBl1T0IfCvKlRVg7M5CGl11uAzDZELC4Z76KQ7oodAnbVr4bi6o9vh8QxruC5 YHgCz+NXzHV9LTkc+DTK5m7Xd6c0m3WFdNJh5l7n2pmDupQdTNYCkU1bvGc+EW/OwJbie2xM43B nCNq4M8OfSwh+cEsWik0k/QTBWvOOJR/BMwdvn7AjkMiUYCo7iv31GdcN+ey0k4MkJe59sa5bRT Z22eSIxbGWOXQSU86gTyz1NEs14ix/P8LJ4kHj7wTBIUfSYZ+vUhN9pVPZOe020n2bwJgHUPKlj UZY81FkXND5F2etf92gbJQr+G8E99pIi7yL4CbWjTkInc2RFOyqtDZ/dPi3n6JTGMqG3l9UccNP /6OeRHf72K4OytAFTV X-Received: by 2002:a05:690c:9989:b0:81d:75b4:925f with SMTP id 00721157ae682-81dbe9587ffmr78809617b3.19.1783651621543; Thu, 09 Jul 2026 19:47:01 -0700 (PDT) Received: from host.arpa ([2600:1700:7c:96e0:2b12:f605:51ed:d586]) by smtp.gmail.com with ESMTPSA id 00721157ae682-81e6be8f5f0sm35546697b3.8.2026.07.09.19.47.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jul 2026 19:47:01 -0700 (PDT) From: Altan Hacigumus To: Andrew Morton , David Hildenbrand , Johannes Weiner Cc: Kairui Song , Qi Zheng , Shakeel Butt , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jiayuan Chen , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH] mm/vmscan: clear hopeless kswapd when global direct reclaim makes progress Date: Thu, 9 Jul 2026 19:44:29 -0700 Message-ID: <20260710024429.70923-1-ahacigu.linux@gmail.com> X-Mailer: git-send-email 2.55.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Direct reclaim clears the hopeless kswapd state only once the node becomes balanced. This avoids cgroup memory.high reclaim repeatedly reviving a kswapd that cannot balance the node. However, it also prevents global direct reclaim from reviving kswapd. Under sustained memory pressure, global direct reclaim may continue making progress without the node ever reaching the high watermark, leaving reclaim to allocating tasks. Unlike memcg reclaim, global direct reclaim follows the same node-wide reclaim path as kswapd. Clear the hopeless state when global direct reclaim makes progress, while continuing to require a balanced node for memcg reclaim. kswapd_try_clear_hopeless() becomes static since struct scan_control is private to vmscan.c. On a workload with most memory mlocked and the remainder under sustained churn, with swap enabled, comparing the same 60s window: base patched allocstall (all zones) 413441 6532 pgsteal_direct 15889552 255619 pgsteal_kswapd 0 26079382 PSI memory full avg60 13.10% 9.37% Direct reclaim was already reclaiming many pages in the baseline; this change lets kswapd resume doing that work asynchronously. Signed-off-by: Altan Hacigumus --- include/linux/mmzone.h | 2 -- mm/vmscan.c | 23 ++++++++++++++--------- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index ca2712187147..1db6f8dac927 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1630,8 +1630,6 @@ enum kswapd_clear_hopeless_reason { void wakeup_kswapd(struct zone *zone, gfp_t gfp_mask, int order, enum zone_type highest_zoneidx); -void kswapd_try_clear_hopeless(struct pglist_data *pgdat, - unsigned int order, int highest_zoneidx); void kswapd_clear_hopeless(pg_data_t *pgdat, enum kswapd_clear_hopeless_reason reason); bool kswapd_test_hopeless(pg_data_t *pgdat); diff --git a/mm/vmscan.c b/mm/vmscan.c index 35c3bb15ae96..78fd35a11eee 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -198,6 +198,9 @@ struct scan_control { */ int vm_swappiness = 60; +static void kswapd_try_clear_hopeless(struct pglist_data *pgdat, + struct scan_control *sc); + #ifdef CONFIG_MEMCG /* Returns true for reclaim through cgroup limits or cgroup interfaces. */ @@ -5172,7 +5175,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * blk_finish_plug(&plug); done: if (sc->nr_reclaimed > reclaimed) - kswapd_try_clear_hopeless(pgdat, sc->order, sc->reclaim_idx); + kswapd_try_clear_hopeless(pgdat, sc); } /****************************************************************************** @@ -6251,7 +6254,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) * successful direct reclaim run will revive a dormant kswapd. */ if (reclaimable) - kswapd_try_clear_hopeless(pgdat, sc->order, sc->reclaim_idx); + kswapd_try_clear_hopeless(pgdat, sc); else if (sc->cache_trim_mode) sc->cache_trim_mode_failed = 1; } @@ -7530,15 +7533,17 @@ void kswapd_clear_hopeless(pg_data_t *pgdat, enum kswapd_clear_hopeless_reason r } /* - * Reset kswapd_failures only when the node is balanced. Without this - * check, successful direct reclaim (e.g., from cgroup memory.high - * throttling) can keep resetting kswapd_failures even when the node - * cannot be balanced, causing kswapd to run endlessly. + * Reset kswapd_failures when the node is balanced, or when global + * direct reclaim makes progress - then kswapd can make progress too. + * Memcg reclaim can succeed where kswapd cannot (memcg protection is + * not enforced against the reclaim target), so its progress resets + * kswapd_failures only when the node is balanced. */ -void kswapd_try_clear_hopeless(struct pglist_data *pgdat, - unsigned int order, int highest_zoneidx) +static void kswapd_try_clear_hopeless(struct pglist_data *pgdat, + struct scan_control *sc) { - if (pgdat_balanced(pgdat, order, highest_zoneidx)) + if ((!current_is_kswapd() && !cgroup_reclaim(sc)) || + pgdat_balanced(pgdat, sc->order, sc->reclaim_idx)) kswapd_clear_hopeless(pgdat, current_is_kswapd() ? KSWAPD_CLEAR_HOPELESS_KSWAPD : KSWAPD_CLEAR_HOPELESS_DIRECT); } -- 2.55.0