[BUG/RFC] mm/madvise: MADV_WILLNEED skips swapped file-backed COW pages

[BUG/RFC] mm/madvise: MADV_WILLNEED skips swapped file-backed COW pages

[Mike Kaplinskiy]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

Hi,

I'm seeing a strange behavior when using MADV_WILLNEED to schedule
swap page-in. It seems MADV_WILLNEED does not schedule swap reads for
swapped-out COW pages in an ordinary file-backed MAP_PRIVATE mapping.

I reproduced this on Linux 7.0.14 on aarch64, but I think this code
hasn't changed in a while. mm/madvise.c:madvise_willneed walks swap
PTEs only when vma->vm_file is NULL, and sends other file-backed
mappings to vfs_fadvise(POSIX_FADV_WILLNEED). This skips the case of
MAP_PRIVATE mappings with changes, which frequently happens for
libraries/binaries with relocations and/or writable globals.

The code is at the top of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/madvise.c#n282
.

Minimal reproducer attached. Would there be interest in changing this
path to prefetch private copies in addition to the readahead?

Thanks,
Mike

[-- Attachment #2: madvise-willneed-cow-repro.c --]
[-- Type: text/x-csrc, Size: 1766 bytes --]

#define _GNU_SOURCE
#include <assert.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>

#define SIZE (64UL << 20)

static size_t swap_kib(void *mapping)
{
	FILE *f = fopen("/proc/self/smaps", "r");
	char line[256], perms[5];
	unsigned long start, end, value = 0;
	int ours = 0;

	assert(f);
	while (fgets(line, sizeof(line), f)) {
		if (sscanf(line, "%lx-%lx %4s", &start, &end, perms) == 3)
			ours = start == (unsigned long)mapping;
		else if (ours && sscanf(line, "Swap: %lu kB", &value) == 1)
			break;
	}
	fclose(f);
	return value;
}

static unsigned long pswpin(void)
{
	FILE *f = fopen("/proc/vmstat", "r");
	char name[32];
	unsigned long value = 0;

	assert(f);
	while (fscanf(f, "%31s %lu", name, &value) == 2)
		if (!strcmp(name, "pswpin"))
			break;
	fclose(f);
	return value;
}

int main(void)
{
	char path[] = "/tmp/madvise-cow-XXXXXX";
	int fd = mkstemp(path);
	char *p;
	unsigned long before;
	volatile unsigned long sum = 0;

	assert(fd >= 0 && !ftruncate(fd, SIZE));
	p = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
	assert(p != MAP_FAILED);
	unlink(path);

	for (size_t i = 0; i < SIZE; i += getpagesize())
		p[i] = i / getpagesize() + 1; /* create private COW pages */
	assert(!madvise(p, SIZE, MADV_PAGEOUT));
	usleep(500000);
	printf("after PAGEOUT:  Swap=%zu kB\n", swap_kib(p));

	before = pswpin();
	assert(!madvise(p, SIZE, MADV_WILLNEED));
	sleep(2); /* WILLNEED is asynchronous */
	printf("after WILLNEED: Swap=%zu kB, pswpin=+%lu\n",
	       swap_kib(p), pswpin() - before);

	before = pswpin();
	for (size_t i = 0; i < SIZE; i += getpagesize())
		sum += p[i];
	printf("after touch:    pswpin=+%lu (sum=%lu)\n",
	       pswpin() - before, sum);
}

Re: [BUG/RFC] mm/madvise: MADV_WILLNEED skips swapped file-backed COW pages

[zhaozhengzhuo]

Hi Mike,

Thanks for the report. I reproduced this on x86-64 with the current
mm-unstable tree at:

  e3b5239afe1b ("mm: gup: cleanup the gup_fast_*() call chain")

The test used a 64 MiB ordinary file-backed MAP_PRIVATE mapping and
disk-backed swap. Since the test machine has 128 GiB of RAM, the pages
initially remained in swap cache after MADV_PAGEOUT. I therefore added
controlled memory pressure between MADV_PAGEOUT and MADV_WILLNEED so
that the swap-cache folios were actually reclaimed.

Across three runs, the pswpin deltas (in 4 KiB pages) were:

                        run 1     run 2     run 3
  after MADV_WILLNEED       2         0         0
  during subsequent touch 16327     15811     15676

The mapping still reported about 64 MiB in Swap after MADV_WILLNEED,
and almost all swap reads occurred only when the pages were touched.

This appears to confirm that madvise_willneed() skips the private swap
entries because the VMA still has vm_file and therefore takes the file
readahead path. vfs_fadvise(POSIX_FADV_WILLNEED) can prefetch the
underlying file contents, but not the modified private contents stored
in swap.

Are you already working on a fix? If testing or other help would be
useful, I would be happy to help.

Thanks,
zhaozhengzhuo

Re: [BUG/RFC] mm/madvise: MADV_WILLNEED skips swapped file-backed COW pages

[Mike Kaplinskiy]

[-- Attachment #1: Type: text/plain, Size: 1730 bytes --]

Hi zhaozhengzhuo,

I am not currently working on a fix, so if you would like to take it on, I
definitely do not mind! I don't have a kernel development environment set
up, so it may take me a while to get this done. Let me know if you'd prefer
I make a patch here!

Mike.


On Thu, Sep 3, 2026 at 12:07 AM zhaozhengzhuo <zhaozhengzhuo@uniontech.com>
wrote:

> Hi Mike,
>
> Thanks for the report. I reproduced this on x86-64 with the current
> mm-unstable tree at:
>
>   e3b5239afe1b ("mm: gup: cleanup the gup_fast_*() call chain")
>
> The test used a 64 MiB ordinary file-backed MAP_PRIVATE mapping and
> disk-backed swap. Since the test machine has 128 GiB of RAM, the pages
> initially remained in swap cache after MADV_PAGEOUT. I therefore added
> controlled memory pressure between MADV_PAGEOUT and MADV_WILLNEED so
> that the swap-cache folios were actually reclaimed.
>
> Across three runs, the pswpin deltas (in 4 KiB pages) were:
>
>                         run 1     run 2     run 3
>   after MADV_WILLNEED       2         0         0
>   during subsequent touch 16327     15811     15676
>
> The mapping still reported about 64 MiB in Swap after MADV_WILLNEED,
> and almost all swap reads occurred only when the pages were touched.
>
> This appears to confirm that madvise_willneed() skips the private swap
> entries because the VMA still has vm_file and therefore takes the file
> readahead path. vfs_fadvise(POSIX_FADV_WILLNEED) can prefetch the
> underlying file contents, but not the modified private contents stored
> in swap.
>
> Are you already working on a fix? If testing or other help would be
> useful, I would be happy to help.
>
> Thanks,
> zhaozhengzhuo
>

[-- Attachment #2: Type: text/html, Size: 2176 bytes --]

Re: [BUG/RFC] mm/madvise: MADV_WILLNEED skips swapped file-backed COW pages

[Lorenzo Stoakes (ARM)]

On Mon, Aug 31, 2026 at 11:24:48AM -0700, Mike Kaplinskiy wrote:
> Hi,
>
> I'm seeing a strange behavior when using MADV_WILLNEED to schedule
> swap page-in. It seems MADV_WILLNEED does not schedule swap reads for
> swapped-out COW pages in an ordinary file-backed MAP_PRIVATE mapping.
>
> I reproduced this on Linux 7.0.14 on aarch64, but I think this code
> hasn't changed in a while. mm/madvise.c:madvise_willneed walks swap

It's more a known limitation of MADV_WILLNEED that has existed forever.

The issue is that every single MAP_PRIVATE-file backed mapping would then
need to be walked twice, once via page tables -> swap cache and once via
readahead.

But you could figure out if it was CoW'd...

Something like:

diff --git a/mm/madvise.c b/mm/madvise.c
index bc6a7dc73021..33ff3ba69c7f 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -297,10 +297,12 @@ static long madvise_willneed(struct madvise_behavior *madv_behavior)
        loff_t offset;

 #ifdef CONFIG_SWAP
-       if (!file) {
+       if (!file || (vma_is_cow_mapping(vma) && vma->anon_vma))
                walk_page_range_vma(vma, start, end, &swapin_walk_ops, vma);
                lru_add_drain(); /* Push any new pages onto the LRU now */
-               return 0;
+               if (!file)
+                       return 0;
        }

        if (shmem_mapping(file->f_mapping)) {

(This also happens to fix a bug there with MAP_PRIVATE-/dev/zero though
that'll get fixed with my upcoming series anyway :)

The vma->anon_vma check ensures CoW'd pages have actually been mapped in.

But then you'd have to do two walks for every single CoW'd MAP_PRIVATE-file
backed mapping.

The majority of the anon walk would be a no-op also.

> PTEs only when vma->vm_file is NULL, and sends other file-backed
> mappings to vfs_fadvise(POSIX_FADV_WILLNEED). This skips the case of
> MAP_PRIVATE mappings with changes, which frequently happens for
> libraries/binaries with relocations and/or writable globals.
>
> The code is at the top of
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/madvise.c#n282

That code is skipping non-swap entries for a swapped out shmem folio? I
think that's irrelevant to this.

> .
>
> Minimal reproducer attached. Would there be interest in changing this
> path to prefetch private copies in addition to the readahead?

I mean I'd like to hear from others, if OK I can send the patch above.

Are we concerned about the inefficiency of this?

Or dropping the mmap lock right after and faulting in the file-backed bits?

I guess if you're doing an MADV_WILLNEED you are fine with it taking a bit
of extra time to swap stuff in.

>
> Thanks,
> Mike

--
Cheers, Lorenzo

[RFC PATCH v1] mm/madvise: prefetch private file COW swap entries

[zhaozhengzhuo]

Hi Mike,

Thanks for the report and for offering to let me take a first pass at the
fix.  I reproduced the behaviour and prepared the RFC below.  I would
appreciate feedback from you and the mm maintainers on both the scope and the
locking model before I prepare a non-RFC revision.

MADV_WILLNEED currently uses file readahead whenever a VMA has vm_file.  For
a MAP_PRIVATE file mapping, however, a write fault creates anonymous COW
folios whose swap entries live only in the PTE.  File readahead cannot
prefetch those modified contents, so the pages are still read synchronously
when the application later touches them.

This RFC walks PTEs for private file VMAs that have an anon_vma and schedules
reads for their swap entries before continuing with the existing file or
shmem path.  The anon_vma check avoids an additional walk for private file
mappings that have never taken a COW fault.  The existing shmem XArray path
is retained; private shmem COW PTEs are covered by the same walk.

The change also avoids draining the current CPU's LRU pagevec when the new
PTE walk did not find a folio.  The existing shmem path is otherwise left
unchanged in this RFC.

One detail from testing is worth calling out.  Mike's original reproducer
uses a file under /tmp, and /tmp is tmpfs (shmem) on the test VM.  I initially
tested a version that excluded shmem, but that version did not fix the
original reproducer: after a private shmem write fault, the modified
anonymous COW page has a swap entry only in the PTE, while the original
shmem page's swap entry is tracked in the shmem XArray.  Consequently, the
PTE walk is needed for private shmem COW pages even though
shmem_swapin_range() must remain for pages tracked by the shmem XArray.  The
RFC therefore covers private shmem as well as regular file mappings, while
leaving the existing shmem path in place.

Test setup
----------

I tested on x86-64 with the mm-unstable tree at e3b5239afe1b, using the
following workload:

  * 64 MiB MAP_PRIVATE mapping;
  * write one byte in every page to create private COW folios;
  * MADV_PAGEOUT, followed by approximately 1 GiB of memory pressure;
  * MADV_WILLNEED, then a page-by-page read of the mapping;
  * 2.3 GiB VM RAM and a 2.5 GiB disk-backed swap device.

The reproducer is based on Mike's original program.  I changed the backing
file to a regular XFS file for the first comparison, and added timing and
pswpin counters.  I also ran the same workload with a memfd to exercise the
private-shmem case.  The complete source is not repeated here because it is
already present in the original thread.

Results
-------

For one representative regular-file run, the counters and wall-clock times
were:

                         MADV_WILLNEED       subsequent touch
  ---------------------  -----------------  -----------------
  baseline pswpin        +2,578             +16,718
  RFC v1 pswpin          +17,610            +0
  baseline time          16.5 ms            752 ms
  RFC v1 time            274 ms             55 ms

The exact values vary with swap pressure and asynchronous I/O completion.
Across additional runs, the RFC version prefetched approximately 17,000 to
19,000 pages during MADV_WILLNEED, while the subsequent touch usually
performed no additional swap-in.

The private-shmem test also moved the majority of the COW swap-in into the
MADV_WILLNEED phase, while retaining shmem_swapin_range() for pages tracked
only by the shmem XArray.

For mappings with no swap entries, repeated advice calls showed no measurable
extra cost for private-none or private-clean mappings because of the
anon_vma guard.  A private-COW mapping paid for the additional PTE walk; the
observed extra wall time was roughly 20 us for 64 MiB, 0.1--0.2 ms for
256 MiB, and 0.5--0.6 ms for 1 GiB on this VM.

On an x86-64 VM with a 64 MiB mapping, 2.3 GiB RAM and 2.5 GiB swap, the
unpatched kernel read 2,578 pages during MADV_WILLNEED and 16,718 pages on
the subsequent touch.  This version read 17,610 pages during MADV_WILLNEED
and none on the touch in the corresponding run.  The advice/touch wall time
was 16.5/752 ms before and 274/55 ms after; values vary with swap pressure.
The same test with a memfd (private shmem) also prefetched the COW pages,
while retaining the existing shmem XArray path.

Locking and open questions
--------------------------

The PTE walk keeps the mmap read lock, as the existing anonymous
MADV_WILLNEED path does.  Swap reads for ordinary block devices are normally
submitted asynchronously, but folio allocation, zswap, and synchronous swap
devices can still make this path block.  I have not attempted to collect
addresses, drop mmap_lock, and perform the reads later: doing that safely
would require pinning or revalidating the VMA, PTEs, swap entries, and NUMA
policy after the unlock.

Could the maintainers please advise on the following points?

  1. Is reusing the existing mmap-lock and PTE-walk model acceptable for this
     small fix, or should this wait for a more general swap-in redesign?
  2. Should private shmem COW PTEs be included in this change, or should the
     first version be limited to regular file mappings?
  3. Is the anon_vma guard and conditional LRU drain the right trade-off, or
     would you prefer a smaller change that keeps the existing drain calls?

I have built mm/madvise.o, run git diff --check, and run checkpatch.pl --strict
on this version.  Any review of the implementation, test methodology, or
the proposed scope would be very helpful before I send a v2.

Fixes: 1998cc048901 ("mm: make madvise(MADV_WILLNEED) support swap file prefetch")
Reported-by: Mike Kaplinskiy <mike@recall.ai>
Link: https://lore.kernel.org/linux-mm/CABeknB_S2XJSHFgnHdgnN0rjzHhH4oQJs_APq9fvxHztQ_pgiA@mail.gmail.com/
Signed-off-by: zhaozhengzhuo <zhaozhengzhuo@uniontech.com>
---
 mm/madvise.c | 32 ++++++++++++++++++++++++++------
 1 file changed, 26 insertions(+), 6 deletions(-)

diff --git a/mm/madvise.c b/mm/madvise.c
index 73c2901b9adb..96cbce6c7f8b 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -193,10 +193,16 @@ static int madvise_update_vma(vm_flags_t new_flags,
 }
 
 #ifdef CONFIG_SWAP
+struct swapin_walk_ctx {
+	struct vm_area_struct *vma;
+	bool swapped;
+};
+
 static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
 		unsigned long end, struct mm_walk *walk)
 {
-	struct vm_area_struct *vma = walk->private;
+	struct swapin_walk_ctx *swc = walk->private;
+	struct vm_area_struct *vma = swc->vma;
 	struct swap_io_ctx ctx = {};
 	pte_t *ptep = NULL;
 	spinlock_t *ptl;
@@ -223,8 +229,10 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
 
 		folio = read_swap_cache_async(&ctx, entry, GFP_HIGHUSER_MOVABLE,
 					vma, addr);
-		if (folio)
+		if (folio) {
+			swc->swapped = true;
 			folio_put(folio);
+		}
 	}
 
 	if (ptep)
@@ -297,10 +305,22 @@ static long madvise_willneed(struct madvise_behavior *madv_behavior)
 	loff_t offset;
 
 #ifdef CONFIG_SWAP
-	if (!file) {
-		walk_page_range_vma(vma, start, end, &swapin_walk_ops, vma);
-		lru_add_drain(); /* Push any new pages onto the LRU now */
-		return 0;
+	bool private_file = file && !(vma->vm_flags & VM_SHARED);
+
+	/*
+	 * A private file mapping can contain anonymous COW pages. Once such
+	 * pages are swapped out, their PTEs contain swap entries even though
+	 * the VMA still has vm_file set. Prefetch those pages as well; file
+	 * readahead can only fetch the original file contents.
+	 */
+	if (!file || (private_file && vma->anon_vma)) {
+		struct swapin_walk_ctx ctx = { .vma = vma };
+
+		walk_page_range_vma(vma, start, end, &swapin_walk_ops, &ctx);
+		if (ctx.swapped)
+			lru_add_drain(); /* Push any new pages onto the LRU now */
+		if (!file)
+			return 0;
 	}
 
 	if (shmem_mapping(file->f_mapping)) {
-- 
2.43.0

Re: [RFC PATCH v1] mm/madvise: prefetch private file COW swap entries

[Lorenzo Stoakes (ARM)]

On Fri, Sep 04, 2026 at 06:09:19PM +0800, zhaozhengzhuo wrote:
...


> Fixes: 1998cc048901 ("mm: make madvise(MADV_WILLNEED) support swap file prefetch")

Fixes totally inappropriate.

> Reported-by: Mike Kaplinskiy <mike@recall.ai>

Missing Closes tag.

> Link: https://lore.kernel.org/linux-mm/CABeknB_S2XJSHFgnHdgnN0rjzHhH4oQJs_APq9fvxHztQ_pgiA@mail.gmail.com/
> Signed-off-by: zhaozhengzhuo <zhaozhengzhuo@uniontech.com>

(Very likely) missing Assisted-by tag.

Please follow kernel procedure.

https://docs.kernel.org/process/coding-assistants.html
https://docs.kernel.org/process/generated-content.html

I already sent at patch at
https://lore.kernel.org/linux-mm/apm68kgHC0NK2zVI@gremlin/ before this one, so
I'm going to take care of this thanks.

> ---
>  mm/madvise.c | 32 ++++++++++++++++++++++++++------
>  1 file changed, 26 insertions(+), 6 deletions(-)
>
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 73c2901b9adb..96cbce6c7f8b 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -193,10 +193,16 @@ static int madvise_update_vma(vm_flags_t new_flags,
>  }
>
>  #ifdef CONFIG_SWAP
> +struct swapin_walk_ctx {
> +	struct vm_area_struct *vma;
> +	bool swapped;
> +};

This is silly.

> +
>  static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
>  		unsigned long end, struct mm_walk *walk)
>  {
> -	struct vm_area_struct *vma = walk->private;
> +	struct swapin_walk_ctx *swc = walk->private;
> +	struct vm_area_struct *vma = swc->vma;
>  	struct swap_io_ctx ctx = {};
>  	pte_t *ptep = NULL;
>  	spinlock_t *ptl;
> @@ -223,8 +229,10 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
>
>  		folio = read_swap_cache_async(&ctx, entry, GFP_HIGHUSER_MOVABLE,
>  					vma, addr);
> -		if (folio)
> +		if (folio) {
> +			swc->swapped = true;
>  			folio_put(folio);
> +		}

This is pointless, it's not a good trade off.

>  	}
>
>  	if (ptep)
> @@ -297,10 +305,22 @@ static long madvise_willneed(struct madvise_behavior *madv_behavior)
>  	loff_t offset;
>
>  #ifdef CONFIG_SWAP
> -	if (!file) {
> -		walk_page_range_vma(vma, start, end, &swapin_walk_ops, vma);
> -		lru_add_drain(); /* Push any new pages onto the LRU now */
> -		return 0;
> +	bool private_file = file && !(vma->vm_flags & VM_SHARED);

This is not correct. My version uses the correct CoW predicate.

> +
> +	/*
> +	 * A private file mapping can contain anonymous COW pages. Once such
> +	 * pages are swapped out, their PTEs contain swap entries even though
> +	 * the VMA still has vm_file set. Prefetch those pages as well; file
> +	 * readahead can only fetch the original file contents.
> +	 */
> +	if (!file || (private_file && vma->anon_vma)) {
> +		struct swapin_walk_ctx ctx = { .vma = vma };
> +
> +		walk_page_range_vma(vma, start, end, &swapin_walk_ops, &ctx);
> +		if (ctx.swapped)
> +			lru_add_drain(); /* Push any new pages onto the LRU now */
> +		if (!file)
> +			return 0;

General structure is OK, but again I already sent a patch.

So please no v2, I will handle this thanks!

>  	}
>
>  	if (shmem_mapping(file->f_mapping)) {
> --
> 2.43.0

--
Cheers, Lorenzo