From: Kairui Song <ryncsn@gmail.com>
To: Shakeel Butt <shakeel.butt@linux.dev>
Cc: kasong@tencent.com, linux-mm@kvack.org,
Andrew Morton <akpm@linux-foundation.org>,
Axel Rasmussen <axelrasmussen@google.com>,
Yuanchu Xie <yuanchu@google.com>, Wei Xu <weixugc@google.com>,
Johannes Weiner <hannes@cmpxchg.org>,
David Hildenbrand <david@kernel.org>,
Michal Hocko <mhocko@kernel.org>,
Qi Zheng <zhengqi.arch@bytedance.com>,
Lorenzo Stoakes <ljs@kernel.org>, Barry Song <baohua@kernel.org>,
David Stevens <stevensd@google.com>,
Chen Ridong <chenridong@huaweicloud.com>,
Leno Hou <lenohou@gmail.com>, Yafang Shao <laoar.shao@gmail.com>,
Yu Zhao <yuzhao@google.com>,
Zicheng Wang <wangzicheng@honor.com>,
Kalesh Singh <kaleshsingh@google.com>,
Suren Baghdasaryan <surenb@google.com>,
Chris Li <chrisl@kernel.org>, Vernon Yang <vernon2gm@gmail.com>,
linux-kernel@vger.kernel.org, Qi Zheng <qi.zheng@linux.dev>,
Baolin Wang <baolin.wang@linux.alibaba.com>
Subject: Re: [PATCH v2 08/12] mm/mglru: simplify and improve dirty writeback handling
Date: Thu, 2 Apr 2026 19:44:47 +0800 [thread overview]
Message-ID: <ac44BVOvOm8lhVvj@KASONG-MC4> (raw)
In-Reply-To: <ac2rO_ovnIQIeYZr@linux.dev>
On Wed, Apr 01, 2026 at 04:37:14PM +0800, Shakeel Butt wrote:
> On Sun, Mar 29, 2026 at 03:52:34AM +0800, Kairui Song via B4 Relay wrote:
> > From: Kairui Song <kasong@tencent.com>
> >
> > The current handling of dirty writeback folios is not working well for
> > file page heavy workloads: Dirty folios are protected and move to next
> > gen upon isolation of getting throttled or reactivation upon pageout
> > (shrink_folio_list).
> >
> > This might help to reduce the LRU lock contention slightly, but as a
> > result, the ping-pong effect of folios between head and tail of last two
> > gens is serious as the shrinker will run into protected dirty writeback
> > folios more frequently compared to activation. The dirty flush wakeup
> > condition is also much more passive compared to active/inactive LRU.
> > Active / inactve LRU wakes the flusher if one batch of folios passed to
> > shrink_folio_list is unevictable due to under writeback, but MGLRU
> > instead has to check this after the whole reclaim loop is done, and then
> > count the isolation protection number compared to the total reclaim
> > number.
>
> I was just ranting about this on Baolin's patch and thanks for unifying them.
>
> >
> > And we previously saw OOM problems with it, too, which were fixed but
> > still not perfect [1].
> >
> > So instead, just drop the special handling for dirty writeback, just
> > re-activate it like active / inactive LRU. And also move the dirty flush
> > wake up check right after shrink_folio_list. This should improve both
> > throttling and performance.
>
> Please divide this patch into two separate ones. One for moving the flusher
> waker (& v1 throttling) within evict_folios() and second the above heuristic of
> dirty writeback.
OK, but throttling is not handled by this commit, it handled by the last
commit. And using the common routine in shrink_folio_list and activate the
folio is suppose to be done before moving the flusher wakeup and throttle,
as I observed some inefficient reclaim or over aggressive / passive if we
don't do that first. We will run into these folios again and again very
frequently and shrink_folio_list also have better dirty / writeback
detection.
I tested these two changes separately again in case I remembered it
wrongly, using the MongoDB YCSB case:
Before this series or commit, it's similar:
Throughput(ops/sec), 63414.891930455
Apply only the remove folio_inc_gen and use shrink_folio_list
to active folio part in this commit:
Throughput(ops/sec), 68580.83394294075
Skip the folio_inc_gen part but apply other part:
Throughput(ops/sec), 61614.29451632779
After the two fixes together (apply this commit fully):
Throughput(ops/sec), 80857.08510208207
And the whole series:
Throughput(ops/sec), 79760.71784646061
The test is a bit noisy, but after the whole series the throttling
seems is already slightly slowing down the workload, still accetable
IMO, this is also why activate the folios here is a good idea or we
will run into problematic throttling.
I think this can be further improved later, as I observed previously with
the LFU alike rework I mentioned, it will help promote folios more
proactively to younger gen and it will have a even better performance:
https://lore.kernel.org/linux-mm/CAMgjq7BoekNjg-Ra3C8M7=8=75su38w=HD782T5E_cxyeCeH_g@mail.gmail.com/
For now I can split this into two in V3, first a commit to use the
common routine for activating the folio, then move then fluster wakeup.
next prev parent reply other threads:[~2026-04-02 11:45 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-28 19:52 [PATCH v2 00/12] mm/mglru: improve reclaim loop and dirty folio handling Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-28 19:52 ` [PATCH v2 01/12] mm/mglru: consolidate common code for retrieving evitable size Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-28 19:52 ` [PATCH v2 02/12] mm/mglru: rename variables related to aging and rotation Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-30 1:57 ` Chen Ridong
2026-03-30 7:59 ` Baolin Wang
2026-04-01 0:00 ` Barry Song
2026-03-28 19:52 ` [PATCH v2 03/12] mm/mglru: relocate the LRU scan batch limit to callers Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-30 8:14 ` Baolin Wang
2026-04-01 0:20 ` Barry Song
2026-03-28 19:52 ` [PATCH v2 04/12] mm/mglru: restructure the reclaim loop Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-29 6:47 ` Kairui Song
2026-03-28 19:52 ` [PATCH v2 05/12] mm/mglru: scan and count the exact number of folios Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-31 8:04 ` Baolin Wang
2026-03-31 9:01 ` Kairui Song
2026-03-31 9:52 ` Baolin Wang
2026-03-28 19:52 ` [PATCH v2 06/12] mm/mglru: use a smaller batch for reclaim Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-31 8:08 ` Baolin Wang
2026-03-28 19:52 ` [PATCH v2 07/12] mm/mglru: don't abort scan immediately right after aging Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-28 19:52 ` [PATCH v2 08/12] mm/mglru: simplify and improve dirty writeback handling Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-29 8:21 ` Kairui Song
2026-03-29 8:46 ` Kairui Song
2026-03-31 8:42 ` Baolin Wang
2026-03-31 9:18 ` Kairui Song
2026-04-01 2:52 ` Baolin Wang
2026-04-01 4:57 ` Kairui Song
2026-04-02 0:11 ` Barry Song
2026-04-07 2:52 ` Chen Ridong
2026-04-01 23:37 ` Shakeel Butt
2026-04-02 11:44 ` Kairui Song [this message]
2026-03-28 19:52 ` [PATCH v2 09/12] mm/mglru: remove no longer used reclaim argument for folio protection Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-28 19:52 ` [PATCH v2 10/12] mm/vmscan: remove sc->file_taken Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-31 8:49 ` Baolin Wang
2026-03-28 19:52 ` [PATCH v2 11/12] mm/vmscan: remove sc->unqueued_dirty Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-31 8:51 ` Baolin Wang
2026-03-28 19:52 ` [PATCH v2 12/12] mm/vmscan: unify writeback reclaim statistic and throttling Kairui Song
2026-03-28 19:52 ` Kairui Song via B4 Relay
2026-03-31 9:24 ` Baolin Wang
2026-03-31 9:29 ` Kairui Song
2026-03-31 9:36 ` Baolin Wang
2026-03-31 9:40 ` Kairui Song
2026-04-01 5:01 ` Leno Hou
2026-04-02 2:39 ` Shakeel Butt
2026-04-02 2:56 ` Kairui Song
2026-04-02 3:17 ` Shakeel Butt
2026-04-01 5:18 ` [PATCH v2 00/12] mm/mglru: improve reclaim loop and dirty folio handling Leno Hou
2026-04-01 7:36 ` Kairui Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ac44BVOvOm8lhVvj@KASONG-MC4 \
--to=ryncsn@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=axelrasmussen@google.com \
--cc=baohua@kernel.org \
--cc=baolin.wang@linux.alibaba.com \
--cc=chenridong@huaweicloud.com \
--cc=chrisl@kernel.org \
--cc=david@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=kaleshsingh@google.com \
--cc=kasong@tencent.com \
--cc=laoar.shao@gmail.com \
--cc=lenohou@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mhocko@kernel.org \
--cc=qi.zheng@linux.dev \
--cc=shakeel.butt@linux.dev \
--cc=stevensd@google.com \
--cc=surenb@google.com \
--cc=vernon2gm@gmail.com \
--cc=wangzicheng@honor.com \
--cc=weixugc@google.com \
--cc=yuanchu@google.com \
--cc=yuzhao@google.com \
--cc=zhengqi.arch@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.