[PATCH] mm/mglru: deactivate anon folios into the oldest generation

[PATCH] mm/mglru: deactivate anon folios into the oldest generation

[Ridong Chen]

From: Ridong Chen <chenridong@xiaomi.com>

folio_deactivate() is used by MADV_COLD and DAMOS_LRU_DEPRIO to move a
folio towards eviction. lru_deactivate() does that by deleting the folio
from its LRU list, clearing PG_active and PG_referenced, and adding it
back with lruvec_add_folio().

With MGLRU, lru_gen_folio_seq() places a swap-backed folio that is not
in the swap cache, i.e. an anon or shmem folio, into max_seq - 1,
regardless of its refs. That generation is active per
lru_gen_is_active(), so such a folio never reaches an inactive
generation, and a folio that has aged into min_seq + 1 is moved to a
younger generation instead. PGDEACTIVATE is still counted. Other folios
are not affected: with their refs cleared they are placed into the
oldest generation.

On a 4-generation memcg, 64MB of aged anon memory in min_seq + 1 ends up
in max_seq - 1 after MADV_COLD, while 64MB of file memory in the same
generation ends up in min_seq.

As a result, MADV_COLD can make anon memory less likely to be reclaimed.
It can be reproduced with a single process in a memcg with swap enabled:

  1) map two 64MB anon regions A and B and fault them in, A first;
  2) age the memcg twice through /sys/kernel/debug/lru_gen, so that both
     regions are in the same generation, min_seq + 1;
  3) apply MADV_COLD to B only, A is the control;
  4) reclaim the size of one region from anon only:
     echo "64M swappiness=max" > memory.reclaim;
  5) count the swapped out pages of A and B from /proc/self/pagemap.

B is expected to be swapped out instead of A. Three runs on mm-new:

                     A swapped out        B (MADV_COLD) swapped out
  before             16384 pages          15 ~ 128 pages
  after              0 ~ 157 pages        16384 pages

Use lruvec_add_folio_tail() for MGLRU, which treats the folio as
reclaiming and adds it to the tail of the oldest generation, the same
place lru_deactivate_file() puts a clean file folio. The other folios,
which were added to the head of the oldest generation, are now added to
its tail as well, so they are reclaimed first as MADV_COLD intends.

Fixes: 081488051d28 ("mm/mglru: fix underprotected page cache")
Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
---

This patch was developed with the help of an AI coding assistant
(Claude).

The reproducer, run as root in a memcg with swap enabled and debugfs
mounted, e.g. "mkdir /sys/fs/cgroup/t; ./reclaim-order /sys/fs/cgroup/t":

  // SPDX-License-Identifier: GPL-2.0
  /*
   * Two 64MB anon regions, A (hot) and B (MADV_COLD). Reclaim 64MB via
   * memory.reclaim (swappiness=max -> anon only). Report how much of each
   * region was swapped out (mincore can't see swap; use /proc/self/pagemap).
   */
  #define _GNU_SOURCE
  #include <sched.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <stdint.h>
  #include <string.h>
  #include <unistd.h>
  #include <fcntl.h>
  #include <sys/mman.h>

  static unsigned long long memcg_id;
  static const char *cg_name;

  /* force one aging step on node 0 of our memcg via debugfs */
  static void aging(void)
  {
  	FILE *f = fopen("/sys/kernel/debug/lru_gen", "r");
  	char line[512], cmd[128];
  	unsigned long max = 0, s, x1, x2, x3;
  	int in = 0, fd, len;

  	while (fgets(line, sizeof(line), f)) {
  		unsigned long long id; char path[256];
  		if (sscanf(line, "memcg %llu %255s", &id, path) == 2) {
  			in = !strcmp(path, cg_name);
  			if (in) memcg_id = id;
  			continue;
  		}
  		if (in && sscanf(line, " %lu %lu %lu %lu", &s, &x1, &x2, &x3) == 4)
  			max = s;
  	}
  	fclose(f);
  	len = snprintf(cmd, sizeof(cmd), "+ %llu 0 %lu 200 1\n", memcg_id, max);
  	fd = open("/sys/kernel/debug/lru_gen", O_WRONLY);
  	if (write(fd, cmd, len) != len) perror("aging");
  	close(fd);
  }

  static long swapped(char *p, size_t sz)
  {
  	int fd = open("/proc/self/pagemap", O_RDONLY);
  	long n = 0;
  	for (size_t off = 0; off < sz; off += 4096) {
  		uint64_t e;
  		pread(fd, &e, 8, ((uintptr_t)(p + off) / 4096) * 8);
  		if (e & (1ULL << 62))
  			n++;
  	}
  	close(fd);
  	return n;
  }

  int main(int argc, char **argv)
  {
  	const char *cg = argv[1];
  	cg_name = strrchr(cg, '/');
  	size_t sz = 64UL << 20;
  	char buf[256], *a, *b, *d;
  	cpu_set_t set;
  	int fd;

  	CPU_ZERO(&set); CPU_SET(0, &set);
  	sched_setaffinity(0, sizeof(set), &set);
  	snprintf(buf, sizeof(buf), "%s/cgroup.procs", cg);
  	fd = open(buf, O_WRONLY); dprintf(fd, "%d\n", getpid()); close(fd);

  	a = mmap(NULL, sz, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
  	b = mmap(NULL, sz, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
  	d = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
  	madvise(a, sz, MADV_NOHUGEPAGE); madvise(b, sz, MADV_NOHUGEPAGE);
  	/* A is faulted in first, so without MADV_COLD it is the older one */
  	for (size_t i = 0; i < sz; i += 4096) a[i] = 1;
  	for (size_t i = 0; i < sz; i += 4096) b[i] = 1;
  	d[0] = 1;
  	madvise(d, 4096, MADV_COLD);	/* drain lru_add */
  	/* two agings: clear young bits of both A and B, both end at min_seq+1 */
  	aging();
  	aging();
  	madvise(b, sz, MADV_COLD);
  	madvise(d, 4096, MADV_COLD);

  	snprintf(buf, sizeof(buf), "%s/memory.reclaim", cg);
  	fd = open(buf, O_WRONLY);
  	if (dprintf(fd, "64M swappiness=max\n") < 0)
  		perror("memory.reclaim");
  	close(fd);

  	printf("hot A swapped: %5ld pages   cold B swapped: %5ld pages  (of %zu)\n",
  	       swapped(a, sz), swapped(b, sz), sz >> 12);
  	return 0;
  }

 mm/folio.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/mm/folio.c b/mm/folio.c
index ea3adb6a89d7..c412d9b398e4 100644
--- a/mm/folio.c
+++ b/mm/folio.c
@@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
 	lruvec_del_folio(lruvec, folio);
 	folio_clear_active(folio);
 	folio_clear_referenced(folio);
-	lruvec_add_folio(lruvec, folio);
+	/*
+	 * For MGLRU, lruvec_add_folio() places anon folios that are not in
+	 * the swap cache into the second youngest generation, which is an
+	 * active one. Add the folio to the oldest generation instead, like
+	 * lru_deactivate_file() does for clean folios.
+	 */
+	if (lru_gen_enabled())
+		lruvec_add_folio_tail(lruvec, folio);
+	else
+		lruvec_add_folio(lruvec, folio);
 
 	__count_vm_events(PGDEACTIVATE, nr_pages);
 	count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_pages);

base-commit: 2dc0136b1fdac153434ab02778fcdb32db5d0dae
-- 
2.34.1

Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation

[Kairui Song]

On Thu, Nov 12, 2026 at 04:19:28PM +0800, Ridong Chen wrote:
> From: Ridong Chen <chenridong@xiaomi.com>
> 
> folio_deactivate() is used by MADV_COLD and DAMOS_LRU_DEPRIO to move a
> folio towards eviction. lru_deactivate() does that by deleting the folio
> from its LRU list, clearing PG_active and PG_referenced, and adding it
> back with lruvec_add_folio().
> 
> With MGLRU, lru_gen_folio_seq() places a swap-backed folio that is not
> in the swap cache, i.e. an anon or shmem folio, into max_seq - 1,
> regardless of its refs. That generation is active per
> lru_gen_is_active(), so such a folio never reaches an inactive
> generation, and a folio that has aged into min_seq + 1 is moved to a
> younger generation instead. PGDEACTIVATE is still counted. Other folios
> are not affected: with their refs cleared they are placed into the
> oldest generation.
> 
> On a 4-generation memcg, 64MB of aged anon memory in min_seq + 1 ends up
> in max_seq - 1 after MADV_COLD, while 64MB of file memory in the same
> generation ends up in min_seq.
> 
> As a result, MADV_COLD can make anon memory less likely to be reclaimed.
> It can be reproduced with a single process in a memcg with swap enabled:
> 
>   1) map two 64MB anon regions A and B and fault them in, A first;
>   2) age the memcg twice through /sys/kernel/debug/lru_gen, so that both
>      regions are in the same generation, min_seq + 1;
>   3) apply MADV_COLD to B only, A is the control;
>   4) reclaim the size of one region from anon only:
>      echo "64M swappiness=max" > memory.reclaim;
>   5) count the swapped out pages of A and B from /proc/self/pagemap.
> 
> B is expected to be swapped out instead of A. Three runs on mm-new:
> 
>                      A swapped out        B (MADV_COLD) swapped out
>   before             16384 pages          15 ~ 128 pages
>   after              0 ~ 157 pages        16384 pages
> 
> Use lruvec_add_folio_tail() for MGLRU, which treats the folio as
> reclaiming and adds it to the tail of the oldest generation, the same
> place lru_deactivate_file() puts a clean file folio. The other folios,
> which were added to the head of the oldest generation, are now added to
> its tail as well, so they are reclaimed first as MADV_COLD intends.
> 
> Fixes: 081488051d28 ("mm/mglru: fix underprotected page cache")
> Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
> ---

...

> 
>  mm/folio.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/folio.c b/mm/folio.c
> index ea3adb6a89d7..c412d9b398e4 100644
> --- a/mm/folio.c
> +++ b/mm/folio.c
> @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
>  	lruvec_del_folio(lruvec, folio);
>  	folio_clear_active(folio);
>  	folio_clear_referenced(folio);
> -	lruvec_add_folio(lruvec, folio);
> +	/*
> +	 * For MGLRU, lruvec_add_folio() places anon folios that are not in
> +	 * the swap cache into the second youngest generation, which is an
> +	 * active one. Add the folio to the oldest generation instead, like
> +	 * lru_deactivate_file() does for clean folios.
> +	 */
> +	if (lru_gen_enabled())
> +		lruvec_add_folio_tail(lruvec, folio);
> +	else
> +		lruvec_add_folio(lruvec, folio);

Hi Ridong,

One question here, why MGLRU/CLRU have to use two different ways to
add the folio back?  Will this cause over aggressive reclaim of the
folio?

Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation

[Barry Song]

On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
[...]
> >  mm/folio.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/folio.c b/mm/folio.c
> > index ea3adb6a89d7..c412d9b398e4 100644
> > --- a/mm/folio.c
> > +++ b/mm/folio.c
> > @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> >       lruvec_del_folio(lruvec, folio);
> >       folio_clear_active(folio);
> >       folio_clear_referenced(folio);
> > -     lruvec_add_folio(lruvec, folio);
> > +     /*
> > +      * For MGLRU, lruvec_add_folio() places anon folios that are not in
> > +      * the swap cache into the second youngest generation, which is an
> > +      * active one. Add the folio to the oldest generation instead, like
> > +      * lru_deactivate_file() does for clean folios.
> > +      */
> > +     if (lru_gen_enabled())
> > +             lruvec_add_folio_tail(lruvec, folio);
> > +     else
> > +             lruvec_add_folio(lruvec, folio);
>
> Hi Ridong,
>
> One question here, why MGLRU/CLRU have to use two different ways to
> add the folio back?  Will this cause over aggressive reclaim of the
> folio?

I guess that is because on mglru, that is lacing a way to add the folio to
the head of the oldest generation for anon folios while CLRU can do
that :-)

Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation

[Kairui Song]

On Thu, Nov 12, 2026 at 4:42 PM Barry Song <baohua@kernel.org> wrote:
>
> On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> [...]
> > >  mm/folio.c | 11 ++++++++++-
> > >  1 file changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/mm/folio.c b/mm/folio.c
> > > index ea3adb6a89d7..c412d9b398e4 100644
> > > --- a/mm/folio.c
> > > +++ b/mm/folio.c
> > > @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> > >       lruvec_del_folio(lruvec, folio);
> > >       folio_clear_active(folio);
> > >       folio_clear_referenced(folio);
> > > -     lruvec_add_folio(lruvec, folio);
> > > +     /*
> > > +      * For MGLRU, lruvec_add_folio() places anon folios that are not in
> > > +      * the swap cache into the second youngest generation, which is an
> > > +      * active one. Add the folio to the oldest generation instead, like
> > > +      * lru_deactivate_file() does for clean folios.
> > > +      */
> > > +     if (lru_gen_enabled())
> > > +             lruvec_add_folio_tail(lruvec, folio);
> > > +     else
> > > +             lruvec_add_folio(lruvec, folio);
> >
> > Hi Ridong,
> >
> > One question here, why MGLRU/CLRU have to use two different ways to
> > add the folio back?  Will this cause over aggressive reclaim of the
> > folio?
>
> I guess that is because on mglru, that is lacing a way to add the folio to
> the head of the oldest generation for anon folios while CLRU can do
> that :-)

Right, but lruvec_add_folio_tail also added the folio to the very tail
of the inactive / oldest list, could it be a gain or problem if we do
that too for CLRU?

Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation

[Barry Song]

On Thu, Nov 12, 2026 at 4:47 PM Kairui Song <ryncsn@gmail.com> wrote:
>
> On Thu, Nov 12, 2026 at 4:42 PM Barry Song <baohua@kernel.org> wrote:
> >
> > On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> > [...]
> > > >  mm/folio.c | 11 ++++++++++-
> > > >  1 file changed, 10 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/mm/folio.c b/mm/folio.c
> > > > index ea3adb6a89d7..c412d9b398e4 100644
> > > > --- a/mm/folio.c
> > > > +++ b/mm/folio.c
> > > > @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> > > >       lruvec_del_folio(lruvec, folio);
> > > >       folio_clear_active(folio);
> > > >       folio_clear_referenced(folio);
> > > > -     lruvec_add_folio(lruvec, folio);
> > > > +     /*
> > > > +      * For MGLRU, lruvec_add_folio() places anon folios that are not in
> > > > +      * the swap cache into the second youngest generation, which is an
> > > > +      * active one. Add the folio to the oldest generation instead, like
> > > > +      * lru_deactivate_file() does for clean folios.
> > > > +      */
> > > > +     if (lru_gen_enabled())
> > > > +             lruvec_add_folio_tail(lruvec, folio);
> > > > +     else
> > > > +             lruvec_add_folio(lruvec, folio);
> > >
> > > Hi Ridong,
> > >
> > > One question here, why MGLRU/CLRU have to use two different ways to
> > > add the folio back?  Will this cause over aggressive reclaim of the
> > > folio?
> >
> > I guess that is because on mglru, that is lacing a way to add the folio to
> > the head of the oldest generation for anon folios while CLRU can do
> > that :-)
>
> Right, but lruvec_add_folio_tail also added the folio to the very tail
> of the inactive / oldest list, could it be a gain or problem if we do
> that too for CLRU?

My gut feeling is that MADV_COLD says the memory is cold. it seems to
make sense to do it all the same for both mglru and clru - to the tail
of inactive(CLRU) or oldest gen(MGLRU).

not quite sure if userspace is expecting things like this. I feel it should
expect this if it says it is cold?

Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation

[Ridong Chen]

On 11/12/2026 4:41 PM, Barry Song wrote:
> On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> [...]
>>>   mm/folio.c | 11 ++++++++++-
>>>   1 file changed, 10 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/mm/folio.c b/mm/folio.c
>>> index ea3adb6a89d7..c412d9b398e4 100644
>>> --- a/mm/folio.c
>>> +++ b/mm/folio.c
>>> @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
>>>        lruvec_del_folio(lruvec, folio);
>>>        folio_clear_active(folio);
>>>        folio_clear_referenced(folio);
>>> -     lruvec_add_folio(lruvec, folio);
>>> +     /*
>>> +      * For MGLRU, lruvec_add_folio() places anon folios that are not in
>>> +      * the swap cache into the second youngest generation, which is an
>>> +      * active one. Add the folio to the oldest generation instead, like
>>> +      * lru_deactivate_file() does for clean folios.
>>> +      */
>>> +     if (lru_gen_enabled())
>>> +             lruvec_add_folio_tail(lruvec, folio);
>>> +     else
>>> +             lruvec_add_folio(lruvec, folio);
>>
>> Hi Ridong,
>>
>> One question here, why MGLRU/CLRU have to use two different ways to
>> add the folio back?  Will this cause over aggressive reclaim of the
>> folio?
> 
> I guess that is because on mglru, that is lacing a way to add the folio to
> the head of the oldest generation for anon folios while CLRU can do
> that :-)

Hi Kairui, Barry,

Thanks for taking a look.

Barry is right about the reason. With the classic LRU,
lruvec_add_folio() adds the deactivated folio to the head of the
inactive list. With MGLRU, lruvec_add_folio() can't put an anon folio
into the oldest generation at all: lru_gen_folio_seq() places a
swap-backed folio that is not in the swap cache into max_seq - 1,
regardless of its refs. The only way to reach the oldest generation is
the reclaiming path of lruvec_add_folio_tail(), and that path adds the
folio to the tail.

And you are right that this makes it more aggressive than the classic
LRU. With this patch, a deactivated folio is evicted before the folios
already in the oldest generation, while the classic LRU puts it behind
the folios already on the inactive list. It also changes file folios:
before this patch they are added to the head of the oldest generation,
the same as the classic LRU, and now they go to the tail.

So the problem in the current code is that a deactivated anon folio is
promoted into an active generation, and with this patch it goes one
step further than the classic LRU in the other direction.

-- 
Best regards
Ridong