[PATCH 0/3] mm/swap, zswap: fix stale zswap data and swap-in bugs

[PATCH 0/3] mm/swap, zswap: fix stale zswap data and swap-in bugs

[Kairui Song via B4 Relay]

Hi all,

There are some bug fixes found by AI and collected by David, mostly
originally from Anthropic.  After filtering and dropping a few that are
already being patched up by the community [1] [2], these are the 3 we
kept.  I've reviewed them and they all look good to me, so please help
have a look and we can get them merged soon, they are pretty simple and
clean bug fixes patching real issues.

The zswap bug can return stale data instead of the zeroes that were
written, so it is data corruption and seems critical.  The extent fix is
more urgent than it might look, as it can also read in unexpected data,
but the setup could be really rare. The accounting fix is minor.
The zswap and extent fixes affect stable kernels (v6.12+) and are marked
for stable. The accounting fix only affects v7.3+.

For the extent fix, in the long term we might actually want to detect
extent contiguity and allow mTHP swap-in for single-extent swap files.
We already disable mTHP allocation for swap files, so the current fix is
trivial. It seems the most minimal and correct one for now.
Optimization and cleanups can come later.

Link: https://lore.kernel.org/linux-mm/20261026-swapoff-writeok-v1-1-4f4cd420aa33@columbia.edu/ [1]
Link: https://lore.kernel.org/linux-mm/20260811132209.2862708-3-youngjun.park@lge.com/ [2]

Signed-off-by: Kairui Song <kasong@tencent.com>
---
Chris Mason (1):
      mm, swap: fix large swap-in across swap file extents

Josef Bacik (2):
      mm/zswap: drop the stale zswap entry of a zero folio
      mm/swap: account synchronous swap reads as memory stall

 mm/memory.c  | 16 +++++++++++++---
 mm/page_io.c | 17 +++++++++++++++++
 2 files changed, 30 insertions(+), 3 deletions(-)
---
base-commit: 24d45a1475dd4c671feb1df705f67ef2fc293969
change-id: 20261030-swap-fixes-1aa6c2c1e7f9

Best regards,
--  
Kairui Song <kasong@tencent.com>

[PATCH 0/3] mm/swap, zswap: fix stale zswap data and swap-in bugs

[Kairui Song]

Hi all,

There are some bug fixes found by AI and collected by David, mostly
originally from Anthropic.  After filtering and dropping a few that are
already being patched up by the community [1] [2], these are the 3 we
kept.  I've reviewed them and they all look good to me, so please help
have a look and we can get them merged soon, they are pretty simple and
clean bug fixes patching real issues.

The zswap bug can return stale data instead of the zeroes that were
written, so it is data corruption and seems critical.  The extent fix is
more urgent than it might look, as it can also read in unexpected data,
but the setup could be really rare. The accounting fix is minor.
The zswap and extent fixes affect stable kernels (v6.12+) and are marked
for stable. The accounting fix only affects v7.3+.

For the extent fix, in the long term we might actually want to detect
extent contiguity and allow mTHP swap-in for single-extent swap files.
We already disable mTHP allocation for swap files, so the current fix is
trivial. It seems the most minimal and correct one for now.
Optimization and cleanups can come later.

Link: https://lore.kernel.org/linux-mm/20261026-swapoff-writeok-v1-1-4f4cd420aa33@columbia.edu/ [1]
Link: https://lore.kernel.org/linux-mm/20260811132209.2862708-3-youngjun.park@lge.com/ [2]

Signed-off-by: Kairui Song <kasong@tencent.com>
---
Chris Mason (1):
      mm, swap: fix large swap-in across swap file extents

Josef Bacik (2):
      mm/zswap: drop the stale zswap entry of a zero folio
      mm/swap: account synchronous swap reads as memory stall

 mm/memory.c  | 16 +++++++++++++---
 mm/page_io.c | 17 +++++++++++++++++
 2 files changed, 30 insertions(+), 3 deletions(-)
---
base-commit: 24d45a1475dd4c671feb1df705f67ef2fc293969
change-id: 20261030-swap-fixes-1aa6c2c1e7f9

Best regards,
--  
Kairui Song <kasong@tencent.com>

[PATCH 1/3] mm/zswap: drop the stale zswap entry of a zero folio

[Kairui Song via B4 Relay]

From: Josef Bacik <josef@toxicpanda.com>

A folio can stay in the swap cache after zswap_store() has stored it,
when a swapin fault gets a reference on it before reclaim can remove it.
do_swap_page() maps an exclusive folio writable and, for a read fault,
keeps it in the swap cache, so it can be modified and swapped out again
to the same slot. zswap_store() handles this, it replaces the old entry,
or drops it if the store fails or zswap got disabled, "otherwise,
writeback could overwrite the new data in the swapfile".

swap_writeout() does not get as far as zswap_store() if the folio is
zero-filled now. It only sets the zero flag of the slot, and the zswap
entry with the previous contents stays in the tree and on the LRU:

  reclaim            fault              zswap shrinker
  zswap_store(D1)
                     swap cache hit,
                     map, write zeroes
  swap_writeout()
    zero flag set
  folio reclaimed
                                        zswap_writeback_entry()
                                          swap_cache_alloc_folio()
                                          decompress D1, uptodate
                     swap cache hit:
                     maps D1

Swapin looks at the zero flag first, but only when it has to read the
folio. The folio that zswap writeback leaves in the swap cache is up to
date as far as anybody can tell, so the process gets its old data back
instead of the zeroes it has written, until that folio is reclaimed.

Invalidate the zswap entries in the zero-filled case as well. The folio
is locked and in the swap cache, which keeps zswap writeback away.

Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
[kasong@tencent.com: fix zswap_invalidate() arguments]
Signed-off-by: Kairui Song <kasong@tencent.com>
---
 mm/page_io.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/mm/page_io.c b/mm/page_io.c
index c6824fcd483e..65681eed9c80 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -224,7 +224,17 @@ int swap_writeout(struct swap_io_ctx *ctx, struct folio *folio)
 	 * acquired internally by swap_zeromap_folio_set/clear.
 	 */
 	if (is_folio_zero_filled(folio)) {
+		swp_entry_t entry = folio->swap;
+
 		swap_zeromap_folio_set(folio);
+
+		/*
+		 * The folio may have been stored in zswap before and dirtied
+		 * again while it stayed in the swap cache. Drop the old copy,
+		 * or zswap writeback would bring it back into the swap cache.
+		 */
+		zswap_invalidate(swp_type(entry), swp_offset(entry),
+				 folio_nr_pages(folio));
 		goto out_unlock;
 	}
 

-- 
2.56.0

[PATCH 1/3] mm/zswap: drop the stale zswap entry of a zero folio

[Kairui Song]

From: Josef Bacik <josef@toxicpanda.com>

A folio can stay in the swap cache after zswap_store() has stored it,
when a swapin fault gets a reference on it before reclaim can remove it.
do_swap_page() maps an exclusive folio writable and, for a read fault,
keeps it in the swap cache, so it can be modified and swapped out again
to the same slot. zswap_store() handles this, it replaces the old entry,
or drops it if the store fails or zswap got disabled, "otherwise,
writeback could overwrite the new data in the swapfile".

swap_writeout() does not get as far as zswap_store() if the folio is
zero-filled now. It only sets the zero flag of the slot, and the zswap
entry with the previous contents stays in the tree and on the LRU:

  reclaim            fault              zswap shrinker
  zswap_store(D1)
                     swap cache hit,
                     map, write zeroes
  swap_writeout()
    zero flag set
  folio reclaimed
                                        zswap_writeback_entry()
                                          swap_cache_alloc_folio()
                                          decompress D1, uptodate
                     swap cache hit:
                     maps D1

Swapin looks at the zero flag first, but only when it has to read the
folio. The folio that zswap writeback leaves in the swap cache is up to
date as far as anybody can tell, so the process gets its old data back
instead of the zeroes it has written, until that folio is reclaimed.

Invalidate the zswap entries in the zero-filled case as well. The folio
is locked and in the swap cache, which keeps zswap writeback away.

Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
[kasong@tencent.com: fix zswap_invalidate() arguments]
Signed-off-by: Kairui Song <kasong@tencent.com>
---
 mm/page_io.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/mm/page_io.c b/mm/page_io.c
index c6824fcd483e..65681eed9c80 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -224,7 +224,17 @@ int swap_writeout(struct swap_io_ctx *ctx, struct folio *folio)
 	 * acquired internally by swap_zeromap_folio_set/clear.
 	 */
 	if (is_folio_zero_filled(folio)) {
+		swp_entry_t entry = folio->swap;
+
 		swap_zeromap_folio_set(folio);
+
+		/*
+		 * The folio may have been stored in zswap before and dirtied
+		 * again while it stayed in the swap cache. Drop the old copy,
+		 * or zswap writeback would bring it back into the swap cache.
+		 */
+		zswap_invalidate(swp_type(entry), swp_offset(entry),
+				 folio_nr_pages(folio));
 		goto out_unlock;
 	}
 

-- 
2.56.0

[PATCH 2/3] mm/swap: account synchronous swap reads as memory stall

[Kairui Song via B4 Relay]

From: Josef Bacik <josef@toxicpanda.com>

Swap-ins from SWP_SYNCHRONOUS_IO devices such as zram are no longer
counted in PSI memory pressure or in the swapin and thrashing delays of
delay accounting.  Refaults of the workingset from zram, the main input
to userspace OOM handlers like lmkd and systemd-oomd on zram systems,
become invisible in /proc/pressure/memory.

swap_read_folio() opens the accounting window with
psi_memstall_enter() and delayacct_swapin_start() and, per its comment,
means to count the submission time.  For synchronous devices the
submission is the entire read: swap_bdev_submit_read() does
submit_bio_wait() and completes the folio.  Since reads are batched in
a swap_iocb, swap_read_folio() only queues the folio with
swap_add_folio(), and the bio is submitted later by swap_read_submit()
in the caller, e.g. swapin_sync(), after the window has been closed.
By then the folio is uptodate and unlocked, so the fault does not wait
on the folio lock either and the read time is not counted anywhere.

Found by code inspection; not reproduced.

Fix by submitting synchronous reads from within swap_read_folio(), as
was done before the batching, rather than from the caller.  This gives
up batching of readahead for synchronous devices, which do their I/O
inline anyway, in exchange for keeping the accounting simple and in one
place.

Fixes: dda8fb68b590 ("mm/swap: also use struct swap_iocb for block I/O")
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Kairui Song <kasong@tencent.com>
---
 mm/page_io.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/page_io.c b/mm/page_io.c
index 65681eed9c80..83ed0246ebfc 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -500,6 +500,13 @@ void swap_read_folio(struct swap_io_ctx *ctx, struct folio *folio)
 	zswap_folio_swapin(folio);
 	swap_add_folio(ctx, folio, READ);
 
+	/*
+	 * Synchronous devices do the whole read at submission time, submit
+	 * it here so that it is accounted as memory stall and delay.
+	 */
+	if (synchronous)
+		swap_read_submit(ctx);
+
 finish:
 	if (workingset) {
 		delayacct_thrashing_end(&in_thrashing);

-- 
2.56.0

[PATCH 2/3] mm/swap: account synchronous swap reads as memory stall

[Kairui Song]

From: Josef Bacik <josef@toxicpanda.com>

Swap-ins from SWP_SYNCHRONOUS_IO devices such as zram are no longer
counted in PSI memory pressure or in the swapin and thrashing delays of
delay accounting.  Refaults of the workingset from zram, the main input
to userspace OOM handlers like lmkd and systemd-oomd on zram systems,
become invisible in /proc/pressure/memory.

swap_read_folio() opens the accounting window with
psi_memstall_enter() and delayacct_swapin_start() and, per its comment,
means to count the submission time.  For synchronous devices the
submission is the entire read: swap_bdev_submit_read() does
submit_bio_wait() and completes the folio.  Since reads are batched in
a swap_iocb, swap_read_folio() only queues the folio with
swap_add_folio(), and the bio is submitted later by swap_read_submit()
in the caller, e.g. swapin_sync(), after the window has been closed.
By then the folio is uptodate and unlocked, so the fault does not wait
on the folio lock either and the read time is not counted anywhere.

Found by code inspection; not reproduced.

Fix by submitting synchronous reads from within swap_read_folio(), as
was done before the batching, rather than from the caller.  This gives
up batching of readahead for synchronous devices, which do their I/O
inline anyway, in exchange for keeping the accounting simple and in one
place.

Fixes: dda8fb68b590 ("mm/swap: also use struct swap_iocb for block I/O")
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Kairui Song <kasong@tencent.com>
---
 mm/page_io.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/page_io.c b/mm/page_io.c
index 65681eed9c80..83ed0246ebfc 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -500,6 +500,13 @@ void swap_read_folio(struct swap_io_ctx *ctx, struct folio *folio)
 	zswap_folio_swapin(folio);
 	swap_add_folio(ctx, folio, READ);
 
+	/*
+	 * Synchronous devices do the whole read at submission time, submit
+	 * it here so that it is accounted as memory stall and delay.
+	 */
+	if (synchronous)
+		swap_read_submit(ctx);
+
 finish:
 	if (workingset) {
 		delayacct_thrashing_end(&in_thrashing);

-- 
2.56.0

[PATCH 3/3] mm, swap: fix large swap-in across swap file extents

[Kairui Song via B4 Relay]

From: Chris Mason <mason@kernel.org>

A page fault can give an unprivileged process blocks of other users'
files instead of its own swapped-out data. This affects systems with a
swap file on a synchronous block device, such as brd or zram. For an
anonymous mapping it also takes a huge page size below the PMD size
(mTHP), such as 64 KiB, that is enabled for anonymous memory.

A swap area is divided into slots of one page each. A swap file is a
regular file, so adjacent swap slots can lie in two extents, far apart
on disk. A folio is read from swap with a single bio that starts at the
sector of the folio's first slot. So all slots of a large folio must be
contiguous on disk.

The swap allocator knows this: it only hands out a run of slots for a
large folio if the swap area is a block device (SWP_BLKDEV).

The swap-in path does not. On a synchronous device (SWP_SYNCHRONOUS_IO),
one that always completes I/O in the submitter's context, do_swap_page()
can read several swapped-out pages as one large folio (a large swap-in).
thp_swapin_suitable_orders() makes that decision, and looks only at the
swap offsets (the slot numbers) in the page table.

Pages that were swapped out one at a time can end up in consecutive,
aligned slots, and are then read as one large folio. If the extent of
the first slot ends inside that folio, the rest of the folio is filled
from the blocks that follow the extent on disk. These blocks can belong
to other files.

Reproduced with a fragmented swap file on brd and 64 KiB folios enabled.
After single pages were swapped out, a fault returned blocks of a
root-owned file with mode 0600.

Fix this by allowing a large swap-in only when SWP_BLKDEV is set.

Known gaps:
- A swap file on zram, brd, pmem or btt is now read one page at a time,
  even where its slots lie in one extent. Its swpin counters in
  /sys/kernel/mm/transparent_hugepage/hugepages-*/stats stay at 0.
- With the posted "mm, swap: enable THP swapin for xswap entries"
  applied as well, the large swap-in that it adds never happens. That
  patch gives its swap device SWP_SYNCHRONOUS_IO but not SWP_BLKDEV, so
  the new check would have to let that device pass.

Fixes: 242d12c98174 ("mm: support large folios swap-in for sync io devices")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Chris Mason <mason@kernel.org>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Kairui Song <kasong@tencent.com>
---
 mm/memory.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index b65b56fe9714..944f57153c89 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4863,7 +4863,8 @@ static inline unsigned long thp_swap_suitable_orders(pgoff_t swp_offset,
 	return orders;
 }
 
-static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
+static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
+						struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	unsigned long orders;
@@ -4888,6 +4889,14 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 	if (!zswap_never_enabled())
 		return 0;
 
+	/*
+	 * Neighbouring slots of a swap file can lie in two extents, and a folio
+	 * is read with one bio. The flag is set in swapon() and stays, so the
+	 * race is benign.
+	 */
+	if (!data_race(si->flags & SWP_BLKDEV))
+		return 0;
+
 	entry = softleaf_from_pte(vmf->orig_pte);
 	/*
 	 * Get a list of all the (large) orders below PMD_ORDER that are enabled
@@ -4924,7 +4933,8 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 	return orders;
 }
 #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
-static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
+static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
+						struct vm_fault *vmf)
 {
 	return 0;
 }
@@ -5070,7 +5080,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 		/* Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices */
 		if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
 			folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
-					    thp_swapin_suitable_orders(vmf) | BIT(0),
+					    thp_swapin_suitable_orders(si, vmf) | BIT(0),
 					    vmf, NULL, 0);
 		else
 			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);

-- 
2.56.0

[PATCH 3/3] mm, swap: fix large swap-in across swap file extents

[Kairui Song]

From: Chris Mason <mason@kernel.org>

A page fault can give an unprivileged process blocks of other users'
files instead of its own swapped-out data. This affects systems with a
swap file on a synchronous block device, such as brd or zram. For an
anonymous mapping it also takes a huge page size below the PMD size
(mTHP), such as 64 KiB, that is enabled for anonymous memory.

A swap area is divided into slots of one page each. A swap file is a
regular file, so adjacent swap slots can lie in two extents, far apart
on disk. A folio is read from swap with a single bio that starts at the
sector of the folio's first slot. So all slots of a large folio must be
contiguous on disk.

The swap allocator knows this: it only hands out a run of slots for a
large folio if the swap area is a block device (SWP_BLKDEV).

The swap-in path does not. On a synchronous device (SWP_SYNCHRONOUS_IO),
one that always completes I/O in the submitter's context, do_swap_page()
can read several swapped-out pages as one large folio (a large swap-in).
thp_swapin_suitable_orders() makes that decision, and looks only at the
swap offsets (the slot numbers) in the page table.

Pages that were swapped out one at a time can end up in consecutive,
aligned slots, and are then read as one large folio. If the extent of
the first slot ends inside that folio, the rest of the folio is filled
from the blocks that follow the extent on disk. These blocks can belong
to other files.

Reproduced with a fragmented swap file on brd and 64 KiB folios enabled.
After single pages were swapped out, a fault returned blocks of a
root-owned file with mode 0600.

Fix this by allowing a large swap-in only when SWP_BLKDEV is set.

Known gaps:
- A swap file on zram, brd, pmem or btt is now read one page at a time,
  even where its slots lie in one extent. Its swpin counters in
  /sys/kernel/mm/transparent_hugepage/hugepages-*/stats stay at 0.
- With the posted "mm, swap: enable THP swapin for xswap entries"
  applied as well, the large swap-in that it adds never happens. That
  patch gives its swap device SWP_SYNCHRONOUS_IO but not SWP_BLKDEV, so
  the new check would have to let that device pass.

Fixes: 242d12c98174 ("mm: support large folios swap-in for sync io devices")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Chris Mason <mason@kernel.org>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Kairui Song <kasong@tencent.com>
---
 mm/memory.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index b65b56fe9714..944f57153c89 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4863,7 +4863,8 @@ static inline unsigned long thp_swap_suitable_orders(pgoff_t swp_offset,
 	return orders;
 }
 
-static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
+static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
+						struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	unsigned long orders;
@@ -4888,6 +4889,14 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 	if (!zswap_never_enabled())
 		return 0;
 
+	/*
+	 * Neighbouring slots of a swap file can lie in two extents, and a folio
+	 * is read with one bio. The flag is set in swapon() and stays, so the
+	 * race is benign.
+	 */
+	if (!data_race(si->flags & SWP_BLKDEV))
+		return 0;
+
 	entry = softleaf_from_pte(vmf->orig_pte);
 	/*
 	 * Get a list of all the (large) orders below PMD_ORDER that are enabled
@@ -4924,7 +4933,8 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 	return orders;
 }
 #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
-static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
+static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
+						struct vm_fault *vmf)
 {
 	return 0;
 }
@@ -5070,7 +5080,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 		/* Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices */
 		if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
 			folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
-					    thp_swapin_suitable_orders(vmf) | BIT(0),
+					    thp_swapin_suitable_orders(si, vmf) | BIT(0),
 					    vmf, NULL, 0);
 		else
 			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);

-- 
2.56.0

Re: [PATCH 1/3] mm/zswap: drop the stale zswap entry of a zero folio

[Johannes Weiner]

On Fri, Oct 30, 2026 at 02:31:41AM +0800, Kairui Song via B4 Relay wrote:
> From: Josef Bacik <josef@toxicpanda.com>
> 
> A folio can stay in the swap cache after zswap_store() has stored it,
> when a swapin fault gets a reference on it before reclaim can remove it.
> do_swap_page() maps an exclusive folio writable and, for a read fault,
> keeps it in the swap cache, so it can be modified and swapped out again
> to the same slot. zswap_store() handles this, it replaces the old entry,
> or drops it if the store fails or zswap got disabled, "otherwise,
> writeback could overwrite the new data in the swapfile".
> 
> swap_writeout() does not get as far as zswap_store() if the folio is
> zero-filled now. It only sets the zero flag of the slot, and the zswap
> entry with the previous contents stays in the tree and on the LRU:
> 
>   reclaim            fault              zswap shrinker
>   zswap_store(D1)
>                      swap cache hit,
>                      map, write zeroes
>   swap_writeout()
>     zero flag set
>   folio reclaimed
>                                         zswap_writeback_entry()
>                                           swap_cache_alloc_folio()
>                                           decompress D1, uptodate
>                      swap cache hit:
>                      maps D1
> 
> Swapin looks at the zero flag first, but only when it has to read the
> folio. The folio that zswap writeback leaves in the swap cache is up to
> date as far as anybody can tell, so the process gets its old data back
> instead of the zeroes it has written, until that folio is reclaimed.
> 
> Invalidate the zswap entries in the zero-filled case as well. The folio
> is locked and in the swap cache, which keeps zswap writeback away.
> 
> Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> [kasong@tencent.com: fix zswap_invalidate() arguments]
> Signed-off-by: Kairui Song <kasong@tencent.com>

Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>

Re: [PATCH 2/3] mm/swap: account synchronous swap reads as memory stall

[Johannes Weiner]

On Fri, Oct 30, 2026 at 02:31:42AM +0800, Kairui Song via B4 Relay wrote:
> From: Josef Bacik <josef@toxicpanda.com>
> 
> Swap-ins from SWP_SYNCHRONOUS_IO devices such as zram are no longer
> counted in PSI memory pressure or in the swapin and thrashing delays of
> delay accounting.  Refaults of the workingset from zram, the main input
> to userspace OOM handlers like lmkd and systemd-oomd on zram systems,
> become invisible in /proc/pressure/memory.
> 
> swap_read_folio() opens the accounting window with
> psi_memstall_enter() and delayacct_swapin_start() and, per its comment,
> means to count the submission time.  For synchronous devices the
> submission is the entire read: swap_bdev_submit_read() does
> submit_bio_wait() and completes the folio.  Since reads are batched in
> a swap_iocb, swap_read_folio() only queues the folio with
> swap_add_folio(), and the bio is submitted later by swap_read_submit()
> in the caller, e.g. swapin_sync(), after the window has been closed.
> By then the folio is uptodate and unlocked, so the fault does not wait
> on the folio lock either and the read time is not counted anywhere.
> 
> Found by code inspection; not reproduced.
> 
> Fix by submitting synchronous reads from within swap_read_folio(), as
> was done before the batching, rather than from the caller.  This gives
> up batching of readahead for synchronous devices, which do their I/O
> inline anyway, in exchange for keeping the accounting simple and in one
> place.
> 
> Fixes: dda8fb68b590 ("mm/swap: also use struct swap_iocb for block I/O")
> Assisted-by: LLM
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> Signed-off-by: Kairui Song <kasong@tencent.com>

Hm, this isn't a complete fix to the problem described. IO throttling
happens at submission time, so there can be significant delays on
async devices too that are no longer tracked since the culprit change.

How about keeping the accounting window open across the batch? Opens
on first read, closes on submission. A batch with at least one
workingset folio is "contaminated" and the whole sequence and
submission becomes a stall. Would need to pull the !can_merge flush
out of the batch-add function to get the accounting window right but
that seems reasonable.

Something like this:

diff --git a/include/linux/swap_ops.h b/include/linux/swap_ops.h
index 57ac6c703f68..6e45c9204482 100644
--- a/include/linux/swap_ops.h
+++ b/include/linux/swap_ops.h
@@ -17,6 +17,10 @@ struct swap_iocb {
 struct swap_io_ctx {
 	struct swap_iocb	*sio;
 	struct swap_info_struct	*sis;
+	bool			in_batch;
+	bool			workingset;
+	bool			in_thrashing;
+	unsigned long		in_memstall;
 };
 
 /*
diff --git a/mm/page_io.c b/mm/page_io.c
index c6824fcd483e..22e5ac1d298f 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -197,6 +197,19 @@ static void swap_zeromap_folio_clear(struct folio *folio)
 	swap_cluster_unlock(ci);
 }
 
+static bool swap_can_merge(struct swap_io_ctx *ctx, struct folio *folio,
+		int rw)
+{
+	struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
+	struct bio_vec *last_bv = &ctx->sio->bvecs[ctx->sio->nr_bvecs - 1];
+	struct folio *prev_folio = bvec_folio(last_bv);
+	size_t prev_folio_size = folio_size(prev_folio);
+
+	if (ctx->sis != sis)
+		return false;
+	return sis->ops->can_merge(folio, prev_folio, prev_folio_size, rw);
+}
+
 /*
  * We may have stale swap cache pages in memory: notice
  * them here and get rid of the unnecessary final write.
@@ -218,6 +231,13 @@ int swap_writeout(struct swap_io_ctx *ctx, struct folio *folio)
 		goto out_unlock;
 	}
 
+	/*
+	 * Don't hold up a batch this folio can't join for the zero
+	 * check and possible compression. Submit it now.
+	 */
+	if (ctx->sio && !swap_can_merge(ctx, folio, WRITE))
+		swap_write_submit(ctx);
+
 	/*
 	 * Use the swap table zero mark to avoid doing IO for zero-filled
 	 * folios. The zero mark is protected by the cluster lock, which is
@@ -322,32 +342,15 @@ int sio_pool_init(void)
 	return 0;
 }
 
-static bool swap_can_merge(struct swap_io_ctx *ctx, struct folio *folio,
-		int rw)
-{
-	struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
-	struct bio_vec *last_bv = &ctx->sio->bvecs[ctx->sio->nr_bvecs - 1];
-	struct folio *prev_folio = bvec_folio(last_bv);
-	size_t prev_folio_size = folio_size(prev_folio);
-
-	if (ctx->sis != sis)
-		return false;
-	return sis->ops->can_merge(folio, prev_folio, prev_folio_size, rw);
-}
-
+/*
+ * Callers submit a pending batch that @folio can't join before adding
+ * it, see swap_writeout() and swap_read_folio().
+ */
 static void swap_add_folio(struct swap_io_ctx *ctx, struct folio *folio, int rw)
 {
 	struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
 	struct swap_iocb *sio = ctx->sio;
 
-	if (sio && !swap_can_merge(ctx, folio, rw)) {
-		if (rw == WRITE)
-			swap_write_submit(ctx);
-		else
-			swap_read_submit(ctx);
-		sio = ctx->sio;
-	}
-
 	if (!sio) {
 		ctx->sis = sis;
 		ctx->sio = sio = mempool_alloc(sio_pool, GFP_NOIO);
@@ -460,42 +463,45 @@ void swap_read_folio(struct swap_io_ctx *ctx, struct folio *folio)
 	struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
 	bool synchronous = sis->flags & SWP_SYNCHRONOUS_IO;
 	bool workingset = folio_test_workingset(folio);
-	unsigned long pflags;
-	bool in_thrashing;
 
 	VM_BUG_ON_FOLIO(!folio_test_swapcache(folio) && !synchronous, folio);
 	VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
 	VM_BUG_ON_FOLIO(folio_test_uptodate(folio), folio);
 
+	/*
+	 * Don't hold up a batch this folio can't join for its lookup
+	 * and possible decompression. Submit it now.
+	 */
+	if (ctx->sio && !swap_can_merge(ctx, folio, READ))
+		swap_read_submit(ctx);
+
 	/*
 	 * Count submission time as memory stall and delay. When the device
 	 * is congested, or the submitting cgroup IO-throttled, submission
-	 * can be a significant part of overall IO time.
+	 * can be a significant part of overall IO time. Reads are batched
+	 * and submitted later, so the windows close in swap_read_submit().
 	 */
-	if (workingset) {
-		delayacct_thrashing_start(&in_thrashing);
-		psi_memstall_enter(&pflags);
+	if (!ctx->in_batch) {
+		delayacct_swapin_start();
+		ctx->in_batch = true;
+	}
+	if (workingset && !ctx->workingset) {
+		delayacct_thrashing_start(&ctx->in_thrashing);
+		psi_memstall_enter(&ctx->in_memstall);
+		ctx->workingset = true;
 	}
-	delayacct_swapin_start();
 
 	if (swap_read_folio_zeromap(folio)) {
 		folio_unlock(folio);
-		goto finish;
+		return;
 	}
 
 	if (zswap_load(folio) != -ENOENT)
-		goto finish;
+		return;
 
 	/* We have to read from slower devices. Increase zswap protection. */
 	zswap_folio_swapin(folio);
 	swap_add_folio(ctx, folio, READ);
-
-finish:
-	if (workingset) {
-		delayacct_thrashing_end(&in_thrashing);
-		psi_memstall_leave(&pflags);
-	}
-	delayacct_swapin_end();
 }
 
 static void swap_write_end(struct swap_iocb *sio, bool failed)
@@ -709,9 +715,19 @@ void swap_write_submit(struct swap_io_ctx *ctx)
 void swap_read_submit(struct swap_io_ctx *ctx)
 {
 	if (!ctx->sio)
-		return;
+		goto out;
 	count_vm_events(NRSWPIN, 1);
 	ctx->sis->ops->submit_read(ctx);
 	ctx->sio = NULL;
 	ctx->sis = NULL;
+out:
+	if (ctx->workingset) {
+		delayacct_thrashing_end(&ctx->in_thrashing);
+		psi_memstall_leave(&ctx->in_memstall);
+		ctx->workingset = false;
+	}
+	if (ctx->in_batch) {
+		delayacct_swapin_end();
+		ctx->in_batch = false;
+	}
 }

Re: [PATCH 1/3] mm/zswap: drop the stale zswap entry of a zero folio

[Nhat Pham]

On Thu, Oct 29, 2026 at 11:32 AM Kairui Song via B4 Relay
<devnull+kasong.tencent.com@kernel.org> wrote:
>
> From: Josef Bacik <josef@toxicpanda.com>
>
> A folio can stay in the swap cache after zswap_store() has stored it,
> when a swapin fault gets a reference on it before reclaim can remove it.
> do_swap_page() maps an exclusive folio writable and, for a read fault,
> keeps it in the swap cache, so it can be modified and swapped out again
> to the same slot. zswap_store() handles this, it replaces the old entry,
> or drops it if the store fails or zswap got disabled, "otherwise,
> writeback could overwrite the new data in the swapfile".
>
> swap_writeout() does not get as far as zswap_store() if the folio is
> zero-filled now. It only sets the zero flag of the slot, and the zswap
> entry with the previous contents stays in the tree and on the LRU:
>
>   reclaim            fault              zswap shrinker
>   zswap_store(D1)
>                      swap cache hit,
>                      map, write zeroes
>   swap_writeout()
>     zero flag set
>   folio reclaimed
>                                         zswap_writeback_entry()
>                                           swap_cache_alloc_folio()
>                                           decompress D1, uptodate
>                      swap cache hit:
>                      maps D1
>
> Swapin looks at the zero flag first, but only when it has to read the
> folio. The folio that zswap writeback leaves in the swap cache is up to
> date as far as anybody can tell, so the process gets its old data back
> instead of the zeroes it has written, until that folio is reclaimed.
>
> Invalidate the zswap entries in the zero-filled case as well. The folio
> is locked and in the swap cache, which keeps zswap writeback away.
>
> Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> [kasong@tencent.com: fix zswap_invalidate() arguments]
> Signed-off-by: Kairui Song <kasong@tencent.com>

Reviewed-by: Nhat Pham <nphamcs@gmail.com>

Re: [PATCH 1/3] mm/zswap: drop the stale zswap entry of a zero folio

[Yosry Ahmed]

On Thu, Oct 29, 2026 at 11:32 AM Kairui Song via B4 Relay
<devnull+kasong.tencent.com@kernel.org> wrote:
>
> From: Josef Bacik <josef@toxicpanda.com>
>
> A folio can stay in the swap cache after zswap_store() has stored it,
> when a swapin fault gets a reference on it before reclaim can remove it.
> do_swap_page() maps an exclusive folio writable and, for a read fault,
> keeps it in the swap cache, so it can be modified and swapped out again
> to the same slot. zswap_store() handles this, it replaces the old entry,
> or drops it if the store fails or zswap got disabled, "otherwise,
> writeback could overwrite the new data in the swapfile".
>
> swap_writeout() does not get as far as zswap_store() if the folio is
> zero-filled now. It only sets the zero flag of the slot, and the zswap
> entry with the previous contents stays in the tree and on the LRU:
>
>   reclaim            fault              zswap shrinker
>   zswap_store(D1)
>                      swap cache hit,
>                      map, write zeroes
>   swap_writeout()
>     zero flag set
>   folio reclaimed
>                                         zswap_writeback_entry()
>                                           swap_cache_alloc_folio()
>                                           decompress D1, uptodate
>                      swap cache hit:
>                      maps D1
>
> Swapin looks at the zero flag first, but only when it has to read the
> folio. The folio that zswap writeback leaves in the swap cache is up to
> date as far as anybody can tell, so the process gets its old data back
> instead of the zeroes it has written, until that folio is reclaimed.
>
> Invalidate the zswap entries in the zero-filled case as well. The folio
> is locked and in the swap cache, which keeps zswap writeback away.
>
> Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> [kasong@tencent.com: fix zswap_invalidate() arguments]
> Signed-off-by: Kairui Song <kasong@tencent.com>

Reviewed-by: Yosry Ahmed <yosry@kernel.org>

Re: [PATCH 3/3] mm, swap: fix large swap-in across swap file extents

[Nhat Pham]

On Thu, Oct 29, 2026 at 11:32 AM Kairui Song via B4 Relay
<devnull+kasong.tencent.com@kernel.org> wrote:
>
> From: Chris Mason <mason@kernel.org>
>
> A page fault can give an unprivileged process blocks of other users'
> files instead of its own swapped-out data. This affects systems with a
> swap file on a synchronous block device, such as brd or zram. For an
> anonymous mapping it also takes a huge page size below the PMD size
> (mTHP), such as 64 KiB, that is enabled for anonymous memory.
>
> A swap area is divided into slots of one page each. A swap file is a
> regular file, so adjacent swap slots can lie in two extents, far apart
> on disk. A folio is read from swap with a single bio that starts at the
> sector of the folio's first slot. So all slots of a large folio must be
> contiguous on disk.
>
> The swap allocator knows this: it only hands out a run of slots for a
> large folio if the swap area is a block device (SWP_BLKDEV).
>
> The swap-in path does not. On a synchronous device (SWP_SYNCHRONOUS_IO),
> one that always completes I/O in the submitter's context, do_swap_page()
> can read several swapped-out pages as one large folio (a large swap-in).
> thp_swapin_suitable_orders() makes that decision, and looks only at the
> swap offsets (the slot numbers) in the page table.
>
> Pages that were swapped out one at a time can end up in consecutive,
> aligned slots, and are then read as one large folio. If the extent of
> the first slot ends inside that folio, the rest of the folio is filled
> from the blocks that follow the extent on disk. These blocks can belong
> to other files.
>
> Reproduced with a fragmented swap file on brd and 64 KiB folios enabled.
> After single pages were swapped out, a fault returned blocks of a
> root-owned file with mode 0600.
>
> Fix this by allowing a large swap-in only when SWP_BLKDEV is set.
>
> Known gaps:
> - A swap file on zram, brd, pmem or btt is now read one page at a time,
>   even where its slots lie in one extent. Its swpin counters in
>   /sys/kernel/mm/transparent_hugepage/hugepages-*/stats stay at 0.


I guess we can check for this right? Look for the swap extent of the
first offset, then get extent's size (swap_extent->nr_pages).

Not sure if these users use swap file here though.

> - With the posted "mm, swap: enable THP swapin for xswap entries"
>   applied as well, the large swap-in that it adds never happens. That
>   patch gives its swap device SWP_SYNCHRONOUS_IO but not SWP_BLKDEV, so
>   the new check would have to let that device pass.

Mentioning unmerged patches in changelog? ;)

>
> Fixes: 242d12c98174 ("mm: support large folios swap-in for sync io devices")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Chris Mason <mason@kernel.org>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> Signed-off-by: Kairui Song <kasong@tencent.com>
> ---
>  mm/memory.c | 16 +++++++++++++---
>  1 file changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/mm/memory.c b/mm/memory.c
> index b65b56fe9714..944f57153c89 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -4863,7 +4863,8 @@ static inline unsigned long thp_swap_suitable_orders(pgoff_t swp_offset,
>         return orders;
>  }
>
> -static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
> +static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
> +                                               struct vm_fault *vmf)
>  {
>         struct vm_area_struct *vma = vmf->vma;
>         unsigned long orders;
> @@ -4888,6 +4889,14 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>         if (!zswap_never_enabled())
>                 return 0;
>
> +       /*
> +        * Neighbouring slots of a swap file can lie in two extents, and a folio
> +        * is read with one bio. The flag is set in swapon() and stays, so the
> +        * race is benign.
> +        */
> +       if (!data_race(si->flags & SWP_BLKDEV))
> +               return 0;
> +
>         entry = softleaf_from_pte(vmf->orig_pte);
>         /*
>          * Get a list of all the (large) orders below PMD_ORDER that are enabled
> @@ -4924,7 +4933,8 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>         return orders;
>  }
>  #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> -static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
> +static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
> +                                               struct vm_fault *vmf)

Nice, I think Sashiko is complaining about it on one of my vswap
patches lol, and I was wondering how on earth is correct in non-vswap
case.

Fix LGTM:

Reviewed-by: Nhat Pham <nphamcs@gmail.com>

Re: [PATCH 3/3] mm, swap: fix large swap-in across swap file extents

[Kairui Song]

On Fri, Oct 30, 2026 at 5:21 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Thu, Oct 29, 2026 at 11:32 AM Kairui Song via B4 Relay
> <devnull+kasong.tencent.com@kernel.org> wrote:
> >
> > From: Chris Mason <mason@kernel.org>
> >
> > A page fault can give an unprivileged process blocks of other users'
> > files instead of its own swapped-out data. This affects systems with a
> > swap file on a synchronous block device, such as brd or zram. For an
> > anonymous mapping it also takes a huge page size below the PMD size
> > (mTHP), such as 64 KiB, that is enabled for anonymous memory.
> >
> > A swap area is divided into slots of one page each. A swap file is a
> > regular file, so adjacent swap slots can lie in two extents, far apart
> > on disk. A folio is read from swap with a single bio that starts at the
> > sector of the folio's first slot. So all slots of a large folio must be
> > contiguous on disk.
> >
> > The swap allocator knows this: it only hands out a run of slots for a
> > large folio if the swap area is a block device (SWP_BLKDEV).
> >
> > The swap-in path does not. On a synchronous device (SWP_SYNCHRONOUS_IO),
> > one that always completes I/O in the submitter's context, do_swap_page()
> > can read several swapped-out pages as one large folio (a large swap-in).
> > thp_swapin_suitable_orders() makes that decision, and looks only at the
> > swap offsets (the slot numbers) in the page table.
> >
> > Pages that were swapped out one at a time can end up in consecutive,
> > aligned slots, and are then read as one large folio. If the extent of
> > the first slot ends inside that folio, the rest of the folio is filled
> > from the blocks that follow the extent on disk. These blocks can belong
> > to other files.
> >
> > Reproduced with a fragmented swap file on brd and 64 KiB folios enabled.
> > After single pages were swapped out, a fault returned blocks of a
> > root-owned file with mode 0600.
> >
> > Fix this by allowing a large swap-in only when SWP_BLKDEV is set.
> >
> > Known gaps:
> > - A swap file on zram, brd, pmem or btt is now read one page at a time,
> >   even where its slots lie in one extent. Its swpin counters in
> >   /sys/kernel/mm/transparent_hugepage/hugepages-*/stats stay at 0.
>
>
> I guess we can check for this right? Look for the swap extent of the
> first offset, then get extent's size (swap_extent->nr_pages).
>
> Not sure if these users use swap file here though.

Thanks for the review!

I think it's really not a good idea to use swap over FS on a
zram/brd/pmem device, and we already disable mTHP allocation for any
such setups, and extents is a burden there. Without high-order
allocation, there is no high-order swapout, so high-order swapin seems
much less helpful.

So keeping the fix simple is a good first step, IMO. We may revisit
this after mTHP allocation is doable on these devices, and get rid of
the extent completely.

> > - With the posted "mm, swap: enable THP swapin for xswap entries"
> >   applied as well, the large swap-in that it adds never happens. That
> >   patch gives its swap device SWP_SYNCHRONOUS_IO but not SWP_BLKDEV, so
> >   the new check would have to let that device pass.
>
> Mentioning unmerged patches in changelog? ;)

I can help simplify that part.

>
> >
> > Fixes: 242d12c98174 ("mm: support large folios swap-in for sync io devices")
> > Cc: stable@vger.kernel.org
> > Assisted-by: LLM
> > Signed-off-by: Chris Mason <mason@kernel.org>
> > Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> > Signed-off-by: Kairui Song <kasong@tencent.com>
> > ---
> >  mm/memory.c | 16 +++++++++++++---
> >  1 file changed, 13 insertions(+), 3 deletions(-)
> >
> > diff --git a/mm/memory.c b/mm/memory.c
> > index b65b56fe9714..944f57153c89 100644
> > --- a/mm/memory.c
> > +++ b/mm/memory.c
> > @@ -4863,7 +4863,8 @@ static inline unsigned long thp_swap_suitable_orders(pgoff_t swp_offset,
> >         return orders;
> >  }
> >
> > -static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
> > +static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
> > +                                               struct vm_fault *vmf)
> >  {
> >         struct vm_area_struct *vma = vmf->vma;
> >         unsigned long orders;
> > @@ -4888,6 +4889,14 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
> >         if (!zswap_never_enabled())
> >                 return 0;
> >
> > +       /*
> > +        * Neighbouring slots of a swap file can lie in two extents, and a folio
> > +        * is read with one bio. The flag is set in swapon() and stays, so the
> > +        * race is benign.
> > +        */
> > +       if (!data_race(si->flags & SWP_BLKDEV))
> > +               return 0;
> > +
> >         entry = softleaf_from_pte(vmf->orig_pte);
> >         /*
> >          * Get a list of all the (large) orders below PMD_ORDER that are enabled
> > @@ -4924,7 +4933,8 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
> >         return orders;
> >  }
> >  #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> > -static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
> > +static unsigned long thp_swapin_suitable_orders(struct swap_info_struct *si,
> > +                                               struct vm_fault *vmf)
>
> Nice, I think Sashiko is complaining about it on one of my vswap
> patches lol, and I was wondering how on earth is correct in non-vswap
> case.
>
> Fix LGTM:
>
> Reviewed-by: Nhat Pham <nphamcs@gmail.com>

Thanks!

Re: [PATCH 2/3] mm/swap: account synchronous swap reads as memory stall

[Kairui Song]

On Fri, Oct 30, 2026 at 5:00 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
>
> On Fri, Oct 30, 2026 at 02:31:42AM +0800, Kairui Song via B4 Relay wrote:
> > From: Josef Bacik <josef@toxicpanda.com>
> >
> > Swap-ins from SWP_SYNCHRONOUS_IO devices such as zram are no longer
> > counted in PSI memory pressure or in the swapin and thrashing delays of
> > delay accounting.  Refaults of the workingset from zram, the main input
> > to userspace OOM handlers like lmkd and systemd-oomd on zram systems,
> > become invisible in /proc/pressure/memory.
> >
> > swap_read_folio() opens the accounting window with
> > psi_memstall_enter() and delayacct_swapin_start() and, per its comment,
> > means to count the submission time.  For synchronous devices the
> > submission is the entire read: swap_bdev_submit_read() does
> > submit_bio_wait() and completes the folio.  Since reads are batched in
> > a swap_iocb, swap_read_folio() only queues the folio with
> > swap_add_folio(), and the bio is submitted later by swap_read_submit()
> > in the caller, e.g. swapin_sync(), after the window has been closed.
> > By then the folio is uptodate and unlocked, so the fault does not wait
> > on the folio lock either and the read time is not counted anywhere.
> >
> > Found by code inspection; not reproduced.
> >
> > Fix by submitting synchronous reads from within swap_read_folio(), as
> > was done before the batching, rather than from the caller.  This gives
> > up batching of readahead for synchronous devices, which do their I/O
> > inline anyway, in exchange for keeping the accounting simple and in one
> > place.
> >
> > Fixes: dda8fb68b590 ("mm/swap: also use struct swap_iocb for block I/O")
> > Assisted-by: LLM
> > Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> > Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> > Signed-off-by: Kairui Song <kasong@tencent.com>
>
> Hm, this isn't a complete fix to the problem described. IO throttling
> happens at submission time, so there can be significant delays on
> async devices too that are no longer tracked since the culprit change.

Yeah you are right! Thanks for the heads up, not only a sync isssue
indeed, async devices are affected too :(, throttled swap fault also
went unaccounted.

> How about keeping the accounting window open across the batch? Opens
> on first read, closes on submission. A batch with at least one
> workingset folio is "contaminated" and the whole sequence and
> submission becomes a stall. Would need to pull the !can_merge flush
> out of the batch-add function to get the accounting window right but
> that seems reasonable.

Won't this cause an amplified PSI reading if only one folio in a batch
is workingset? Maybe not a big issue though.

>
> Something like this:
>
> diff --git a/include/linux/swap_ops.h b/include/linux/swap_ops.h
> index 57ac6c703f68..6e45c9204482 100644
> --- a/include/linux/swap_ops.h
> +++ b/include/linux/swap_ops.h
> @@ -17,6 +17,10 @@ struct swap_iocb {
>  struct swap_io_ctx {
>         struct swap_iocb        *sio;
>         struct swap_info_struct *sis;
> +       bool                    in_batch;
> +       bool                    workingset;
> +       bool                    in_thrashing;
> +       unsigned long           in_memstall;
>  };
>
>  /*
> diff --git a/mm/page_io.c b/mm/page_io.c
> index c6824fcd483e..22e5ac1d298f 100644
> --- a/mm/page_io.c
> +++ b/mm/page_io.c
> @@ -197,6 +197,19 @@ static void swap_zeromap_folio_clear(struct folio *folio)
>         swap_cluster_unlock(ci);
>  }
>
> +static bool swap_can_merge(struct swap_io_ctx *ctx, struct folio *folio,
> +               int rw)
> +{
> +       struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
> +       struct bio_vec *last_bv = &ctx->sio->bvecs[ctx->sio->nr_bvecs - 1];
> +       struct folio *prev_folio = bvec_folio(last_bv);
> +       size_t prev_folio_size = folio_size(prev_folio);
> +
> +       if (ctx->sis != sis)
> +               return false;
> +       return sis->ops->can_merge(folio, prev_folio, prev_folio_size, rw);
> +}
> +
>  /*
>   * We may have stale swap cache pages in memory: notice
>   * them here and get rid of the unnecessary final write.
> @@ -218,6 +231,13 @@ int swap_writeout(struct swap_io_ctx *ctx, struct folio *folio)
>                 goto out_unlock;
>         }
>
> +       /*
> +        * Don't hold up a batch this folio can't join for the zero
> +        * check and possible compression. Submit it now.
> +        */
> +       if (ctx->sio && !swap_can_merge(ctx, folio, WRITE))
> +               swap_write_submit(ctx);
> +
>         /*
>          * Use the swap table zero mark to avoid doing IO for zero-filled
>          * folios. The zero mark is protected by the cluster lock, which is
> @@ -322,32 +342,15 @@ int sio_pool_init(void)
>         return 0;
>  }
>
> -static bool swap_can_merge(struct swap_io_ctx *ctx, struct folio *folio,
> -               int rw)
> -{
> -       struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
> -       struct bio_vec *last_bv = &ctx->sio->bvecs[ctx->sio->nr_bvecs - 1];
> -       struct folio *prev_folio = bvec_folio(last_bv);
> -       size_t prev_folio_size = folio_size(prev_folio);
> -
> -       if (ctx->sis != sis)
> -               return false;
> -       return sis->ops->can_merge(folio, prev_folio, prev_folio_size, rw);
> -}
> -
> +/*
> + * Callers submit a pending batch that @folio can't join before adding
> + * it, see swap_writeout() and swap_read_folio().
> + */
>  static void swap_add_folio(struct swap_io_ctx *ctx, struct folio *folio, int rw)
>  {
>         struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
>         struct swap_iocb *sio = ctx->sio;
>
> -       if (sio && !swap_can_merge(ctx, folio, rw)) {
> -               if (rw == WRITE)
> -                       swap_write_submit(ctx);
> -               else
> -                       swap_read_submit(ctx);
> -               sio = ctx->sio;
> -       }
> -
>         if (!sio) {
>                 ctx->sis = sis;
>                 ctx->sio = sio = mempool_alloc(sio_pool, GFP_NOIO);
> @@ -460,42 +463,45 @@ void swap_read_folio(struct swap_io_ctx *ctx, struct folio *folio)
>         struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
>         bool synchronous = sis->flags & SWP_SYNCHRONOUS_IO;
>         bool workingset = folio_test_workingset(folio);
> -       unsigned long pflags;
> -       bool in_thrashing;
>
>         VM_BUG_ON_FOLIO(!folio_test_swapcache(folio) && !synchronous, folio);
>         VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
>         VM_BUG_ON_FOLIO(folio_test_uptodate(folio), folio);
>
> +       /*
> +        * Don't hold up a batch this folio can't join for its lookup
> +        * and possible decompression. Submit it now.
> +        */
> +       if (ctx->sio && !swap_can_merge(ctx, folio, READ))
> +               swap_read_submit(ctx);
> +
>         /*
>          * Count submission time as memory stall and delay. When the device
>          * is congested, or the submitting cgroup IO-throttled, submission
> -        * can be a significant part of overall IO time.
> +        * can be a significant part of overall IO time. Reads are batched
> +        * and submitted later, so the windows close in swap_read_submit().
>          */
> -       if (workingset) {
> -               delayacct_thrashing_start(&in_thrashing);
> -               psi_memstall_enter(&pflags);
> +       if (!ctx->in_batch) {
> +               delayacct_swapin_start();
> +               ctx->in_batch = true;
> +       }
> +       if (workingset && !ctx->workingset) {
> +               delayacct_thrashing_start(&ctx->in_thrashing);
> +               psi_memstall_enter(&ctx->in_memstall);
> +               ctx->workingset = true;
>         }
> -       delayacct_swapin_start();
>
>         if (swap_read_folio_zeromap(folio)) {
>                 folio_unlock(folio);
> -               goto finish;
> +               return;
>         }
>
>         if (zswap_load(folio) != -ENOENT)
> -               goto finish;
> +               return;
>
>         /* We have to read from slower devices. Increase zswap protection. */
>         zswap_folio_swapin(folio);
>         swap_add_folio(ctx, folio, READ);
> -
> -finish:
> -       if (workingset) {
> -               delayacct_thrashing_end(&in_thrashing);
> -               psi_memstall_leave(&pflags);
> -       }
> -       delayacct_swapin_end();
>  }
>
>  static void swap_write_end(struct swap_iocb *sio, bool failed)
> @@ -709,9 +715,19 @@ void swap_write_submit(struct swap_io_ctx *ctx)
>  void swap_read_submit(struct swap_io_ctx *ctx)
>  {
>         if (!ctx->sio)
> -               return;
> +               goto out;
>         count_vm_events(NRSWPIN, 1);
>         ctx->sis->ops->submit_read(ctx);
>         ctx->sio = NULL;
>         ctx->sis = NULL;
> +out:
> +       if (ctx->workingset) {
> +               delayacct_thrashing_end(&ctx->in_thrashing);
> +               psi_memstall_leave(&ctx->in_memstall);
> +               ctx->workingset = false;
> +       }
> +       if (ctx->in_batch) {
> +               delayacct_swapin_end();
> +               ctx->in_batch = false;
> +       }
>  }

Nice suggestion, and I forgot to Cc Christoph on this though,
worth getting more attention.

Re: [PATCH 2/3] mm/swap: account synchronous swap reads as memory stall

[Johannes Weiner]

On Fri, Oct 30, 2026 at 07:16:51PM +0800, Kairui Song wrote:
> On Fri, Oct 30, 2026 at 5:00 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
> >
> > On Fri, Oct 30, 2026 at 02:31:42AM +0800, Kairui Song via B4 Relay wrote:
> > > From: Josef Bacik <josef@toxicpanda.com>
> > >
> > > Swap-ins from SWP_SYNCHRONOUS_IO devices such as zram are no longer
> > > counted in PSI memory pressure or in the swapin and thrashing delays of
> > > delay accounting.  Refaults of the workingset from zram, the main input
> > > to userspace OOM handlers like lmkd and systemd-oomd on zram systems,
> > > become invisible in /proc/pressure/memory.
> > >
> > > swap_read_folio() opens the accounting window with
> > > psi_memstall_enter() and delayacct_swapin_start() and, per its comment,
> > > means to count the submission time.  For synchronous devices the
> > > submission is the entire read: swap_bdev_submit_read() does
> > > submit_bio_wait() and completes the folio.  Since reads are batched in
> > > a swap_iocb, swap_read_folio() only queues the folio with
> > > swap_add_folio(), and the bio is submitted later by swap_read_submit()
> > > in the caller, e.g. swapin_sync(), after the window has been closed.
> > > By then the folio is uptodate and unlocked, so the fault does not wait
> > > on the folio lock either and the read time is not counted anywhere.
> > >
> > > Found by code inspection; not reproduced.
> > >
> > > Fix by submitting synchronous reads from within swap_read_folio(), as
> > > was done before the batching, rather than from the caller.  This gives
> > > up batching of readahead for synchronous devices, which do their I/O
> > > inline anyway, in exchange for keeping the accounting simple and in one
> > > place.
> > >
> > > Fixes: dda8fb68b590 ("mm/swap: also use struct swap_iocb for block I/O")
> > > Assisted-by: LLM
> > > Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> > > Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> > > Signed-off-by: Kairui Song <kasong@tencent.com>
> >
> > Hm, this isn't a complete fix to the problem described. IO throttling
> > happens at submission time, so there can be significant delays on
> > async devices too that are no longer tracked since the culprit change.
> 
> Yeah you are right! Thanks for the heads up, not only a sync isssue
> indeed, async devices are affected too :(, throttled swap fault also
> went unaccounted.
> 
> > How about keeping the accounting window open across the batch? Opens
> > on first read, closes on submission. A batch with at least one
> > workingset folio is "contaminated" and the whole sequence and
> > submission becomes a stall. Would need to pull the !can_merge flush
> > out of the batch-add function to get the accounting window right but
> > that seems reasonable.
> 
> Won't this cause an amplified PSI reading if only one folio in a batch
> is workingset? Maybe not a big issue though.

Hm, I think the only option to avoid this would be to split the batch
and flush more frequently, which I'm not sure is worth it. The batch
size is only 32 folios...

diff --git a/mm/page_io.c b/mm/page_io.c
index 22e5ac1d298f..806cfe633a95 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -207,6 +207,8 @@ static bool swap_can_merge(struct swap_io_ctx *ctx, struct folio *folio,
 
 	if (ctx->sis != sis)
 		return false;
+	if (rw == READ && ctx->workingset != folio_test_workingset(folio))
+		return false;
 	return sis->ops->can_merge(folio, prev_folio, prev_folio_size, rw);
 }