[PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[David Carlier]

sio_read_complete() uses sio->pages to account global PSWPIN vm events,
but sio->pages tracks the number of bvec entries (folios), not base
pages. For large folios this undercounts compared to the per-memcg path
which correctly uses folio_nr_pages(), and compared to the bdev read
paths which also use folio_nr_pages().

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
count since sio->len is accumulated via folio_size(folio).

Fixes: a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
Signed-off-by: David Carlier <devnexen@gmail.com>
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/page_io.c b/mm/page_io.c
index 63b262f4c5a9..1389cd57ca88 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
 			folio_mark_uptodate(folio);
 			folio_unlock(folio);
 		}
-		count_vm_events(PSWPIN, sio->pages);
+		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
 	} else {
 		for (p = 0; p < sio->pages; p++) {
 			struct folio *folio = page_folio(sio->bvec[p].bv_page);
-- 
2.53.0

Re: [PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[Andrew Morton]

On Mon, 30 Mar 2026 08:12:29 +0100 David Carlier <devnexen@gmail.com> wrote:

> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> but sio->pages tracks the number of bvec entries (folios), not base
> pages. For large folios this undercounts compared to the per-memcg path
> which correctly uses folio_nr_pages(), and compared to the bdev read
> paths which also use folio_nr_pages().
> 
> Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> count since sio->len is accumulated via folio_size(folio).

Thanks.  What are the userspace-visible runtime effects of the bug? 
Just /proc/vmstat wrongness?

Seems pretty minor, and it's been in there for several years.  But I
guess I'll slip a cc:stable in there, let it trickle back at a later
time.

> --- a/mm/page_io.c
> +++ b/mm/page_io.c
> @@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
>  			folio_mark_uptodate(folio);
>  			folio_unlock(folio);
>  		}
> -		count_vm_events(PSWPIN, sio->pages);
> +		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
>  	} else {
>  		for (p = 0; p < sio->pages; p++) {
>  			struct folio *folio = page_folio(sio->bvec[p].bv_page);
> -- 
> 2.53.0
> 

Re: [PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[Barry Song]

On Mon, Mar 30, 2026 at 3:12 PM David Carlier <devnexen@gmail.com> wrote:
>
> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> but sio->pages tracks the number of bvec entries (folios), not base
> pages. For large folios this undercounts compared to the per-memcg path
> which correctly uses folio_nr_pages(), and compared to the bdev read
> paths which also use folio_nr_pages().
>
> Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> count since sio->len is accumulated via folio_size(folio).
>
> Fixes: a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
> Signed-off-by: David Carlier <devnexen@gmail.com>

The patch seems theoretically correct, but I’m wondering
where we can swap in mTHP for filesystem-based swap？

In both do_swap_page() and shmem_swapin_folio(), we check
data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
large folios. Am I missing something?

> ---
>  mm/page_io.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/page_io.c b/mm/page_io.c
> index 63b262f4c5a9..1389cd57ca88 100644
> --- a/mm/page_io.c
> +++ b/mm/page_io.c
> @@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
>                         folio_mark_uptodate(folio);
>                         folio_unlock(folio);
>                 }
> -               count_vm_events(PSWPIN, sio->pages);
> +               count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
>         } else {
>                 for (p = 0; p < sio->pages; p++) {
>                         struct folio *folio = page_folio(sio->bvec[p].bv_page);
> --
> 2.53.0
>

Thanks
Barry

Re: [PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[David CARLIER]

On Tue, 31 Mar 2026 at 23:33, Barry Song <21cnbao@gmail.com> wrote:
>
> On Mon, Mar 30, 2026 at 3:12 PM David Carlier <devnexen@gmail.com> wrote:
> >
> > sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> > but sio->pages tracks the number of bvec entries (folios), not base
> > pages. For large folios this undercounts compared to the per-memcg path
> > which correctly uses folio_nr_pages(), and compared to the bdev read
> > paths which also use folio_nr_pages().
> >
> > Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> > count since sio->len is accumulated via folio_size(folio).
> >
> > Fixes: a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
> > Signed-off-by: David Carlier <devnexen@gmail.com>
>
> The patch seems theoretically correct, but I’m wondering
> where we can swap in mTHP for filesystem-based swap？
>
> In both do_swap_page() and shmem_swapin_folio(), we check
> data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
> large folios. Am I missing something?

▎ The patch seems theoretically correct, but I'm wondering
  ▎ where we can swap in mTHP for filesystem-based swap?

  ▎ In both do_swap_page() and shmem_swapin_folio(), we check
  ▎ data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
  ▎ large folios. Am I missing something?

  You're right, I missed that. SWP_FS_OPS is only set by NFS and
  SMB which have no bdev, so SWP_SYNCHRONOUS_IO can never be set
  alongside it. Large folios can't currently reach this path since
  both do_swap_page() and shmem_swapin_folio() gate mTHP allocation
  on SWP_SYNCHRONOUS_IO.

  That said, sio_read_complete() already calls count_mthp_stat()
  and the per-memcg accounting uses folio_nr_pages(), so the code
  seems written with large folios in mind even if the path is
  currently unreachable. Using sio->pages (bvec entry count) for
  a base-page count is still semantically wrong, but I understand
  the practical impact is nil today.

  Happy to either drop this or keep it as a correctness cleanup,
  whatever you and Andrew prefer.

Cheers !
>
> > ---
> >  mm/page_io.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/mm/page_io.c b/mm/page_io.c
> > index 63b262f4c5a9..1389cd57ca88 100644
> > --- a/mm/page_io.c
> > +++ b/mm/page_io.c
> > @@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
> >                         folio_mark_uptodate(folio);
> >                         folio_unlock(folio);
> >                 }
> > -               count_vm_events(PSWPIN, sio->pages);
> > +               count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
> >         } else {
> >                 for (p = 0; p < sio->pages; p++) {
> >                         struct folio *folio = page_folio(sio->bvec[p].bv_page);
> > --
> > 2.53.0
> >
>
> Thanks
> Barry

Re: [PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[David Hildenbrand (Arm)]

On 4/1/26 09:10, David CARLIER wrote:
> On Tue, 31 Mar 2026 at 23:33, Barry Song <21cnbao@gmail.com> wrote:
>>
>> On Mon, Mar 30, 2026 at 3:12 PM David Carlier <devnexen@gmail.com> wrote:
>>>
>>> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
>>> but sio->pages tracks the number of bvec entries (folios), not base
>>> pages. For large folios this undercounts compared to the per-memcg path
>>> which correctly uses folio_nr_pages(), and compared to the bdev read
>>> paths which also use folio_nr_pages().
>>>
>>> Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
>>> count since sio->len is accumulated via folio_size(folio).
>>>
>>> Fixes: a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
>>> Signed-off-by: David Carlier <devnexen@gmail.com>
>>
>> The patch seems theoretically correct, but I’m wondering
>> where we can swap in mTHP for filesystem-based swap？
>>
>> In both do_swap_page() and shmem_swapin_folio(), we check
>> data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
>> large folios. Am I missing something?
> 
> ▎ The patch seems theoretically correct, but I'm wondering
>   ▎ where we can swap in mTHP for filesystem-based swap?
> 
>   ▎ In both do_swap_page() and shmem_swapin_folio(), we check
>   ▎ data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
>   ▎ large folios. Am I missing something?
> 
>   You're right, I missed that. SWP_FS_OPS is only set by NFS and
>   SMB which have no bdev, so SWP_SYNCHRONOUS_IO can never be set
>   alongside it. Large folios can't currently reach this path since
>   both do_swap_page() and shmem_swapin_folio() gate mTHP allocation
>   on SWP_SYNCHRONOUS_IO.
> 
>   That said, sio_read_complete() already calls count_mthp_stat()
>   and the per-memcg accounting uses folio_nr_pages(), so the code
>   seems written with large folios in mind even if the path is
>   currently unreachable. Using sio->pages (bvec entry count) for
>   a base-page count is still semantically wrong, but I understand
>   the practical impact is nil today.
> 
>   Happy to either drop this or keep it as a correctness cleanup,
>   whatever you and Andrew prefer.

Best to rework the patch description (clarify that it's a cleanup) and
drop the Fixes: I guess.

-- 
Cheers,

David

Re: [PATCH v2] mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

[Barry Song]

On Wed, Apr 1, 2026 at 3:10 PM David CARLIER <devnexen@gmail.com> wrote:
>
> On Tue, 31 Mar 2026 at 23:33, Barry Song <21cnbao@gmail.com> wrote:
> >
> > On Mon, Mar 30, 2026 at 3:12 PM David Carlier <devnexen@gmail.com> wrote:
> > >
> > > sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> > > but sio->pages tracks the number of bvec entries (folios), not base
> > > pages. For large folios this undercounts compared to the per-memcg path
> > > which correctly uses folio_nr_pages(), and compared to the bdev read
> > > paths which also use folio_nr_pages().
> > >
> > > Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> > > count since sio->len is accumulated via folio_size(folio).
> > >
> > > Fixes: a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
> > > Signed-off-by: David Carlier <devnexen@gmail.com>
> >
> > The patch seems theoretically correct, but I’m wondering
> > where we can swap in mTHP for filesystem-based swap？
> >
> > In both do_swap_page() and shmem_swapin_folio(), we check
> > data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
> > large folios. Am I missing something?
>
> ▎ The patch seems theoretically correct, but I'm wondering
>   ▎ where we can swap in mTHP for filesystem-based swap?
>
>   ▎ In both do_swap_page() and shmem_swapin_folio(), we check
>   ▎ data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating
>   ▎ large folios. Am I missing something?
>
>   You're right, I missed that. SWP_FS_OPS is only set by NFS and
>   SMB which have no bdev, so SWP_SYNCHRONOUS_IO can never be set
>   alongside it. Large folios can't currently reach this path since
>   both do_swap_page() and shmem_swapin_folio() gate mTHP allocation
>   on SWP_SYNCHRONOUS_IO.
>
>   That said, sio_read_complete() already calls count_mthp_stat()
>   and the per-memcg accounting uses folio_nr_pages(), so the code
>   seems written with large folios in mind even if the path is
>   currently unreachable. Using sio->pages (bvec entry count) for
>   a base-page count is still semantically wrong, but I understand
>   the practical impact is nil today.
>
>   Happy to either drop this or keep it as a correctness cleanup,
>   whatever you and Andrew prefer.
>

The patch looks correct, so I’d personally support keeping it
after cleaning up the description, as David suggested.

Best Regards
Barry

[PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[David Carlier]

sio_read_complete() uses sio->pages to account global PSWPIN vm events,
but sio->pages tracks the number of bvec entries (folios), not base
pages.

While large folios cannot currently reach this path (SWP_FS_OPS and
SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation
is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically
inconsistent with the per-memcg path which correctly uses
folio_nr_pages().

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
count since sio->len is accumulated via folio_size(folio).

Signed-off-by: David Carlier <devnexen@gmail.com>
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/page_io.c b/mm/page_io.c
index 63b262f4c5a9..1389cd57ca88 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
 			folio_mark_uptodate(folio);
 			folio_unlock(folio);
 		}
-		count_vm_events(PSWPIN, sio->pages);
+		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
 	} else {
 		for (p = 0; p < sio->pages; p++) {
 			struct folio *folio = page_folio(sio->bvec[p].bv_page);
-- 
2.53.0

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[David Hildenbrand (Arm)]

On 4/1/26 09:47, David Carlier wrote:
> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> but sio->pages tracks the number of bvec entries (folios), not base
> pages.
> 
> While large folios cannot currently reach this path (SWP_FS_OPS and
> SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation
> is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically
> inconsistent with the per-memcg path which correctly uses
> folio_nr_pages().
> 
> Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> count since sio->len is accumulated via folio_size(folio).
> 
> Signed-off-by: David Carlier <devnexen@gmail.com>

For the next time: Please don't send new revisions as reply to other
revisions :)

> ---
>  mm/page_io.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/page_io.c b/mm/page_io.c
> index 63b262f4c5a9..1389cd57ca88 100644
> --- a/mm/page_io.c
> +++ b/mm/page_io.c
> @@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
>  			folio_mark_uptodate(folio);
>  			folio_unlock(folio);
>  		}
> -		count_vm_events(PSWPIN, sio->pages);
> +		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
>  	} else {
>  		for (p = 0; p < sio->pages; p++) {
>  			struct folio *folio = page_folio(sio->bvec[p].bv_page);

sio->len always covers full pages as processed in swap_read_folio_fs(),
so there should not be any difference.


Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[Andrew Morton]

On Wed, 1 Apr 2026 10:32:30 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote:

> On 4/1/26 09:47, David Carlier wrote:
> > sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> > but sio->pages tracks the number of bvec entries (folios), not base
> > pages.
> > 
> > While large folios cannot currently reach this path (SWP_FS_OPS and
> > SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation
> > is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically
> > inconsistent with the per-memcg path which correctly uses
> > folio_nr_pages().
> > 
> > Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> > count since sio->len is accumulated via folio_size(folio).
> > 
> > Signed-off-by: David Carlier <devnexen@gmail.com>
> 
> For the next time: Please don't send new revisions as reply to other
> revisions :)

Also, altering the Subject: causes akpm confusion (not hard).  But
having accurate titles is more important than avoiding akpm confusion,
so there is that.

A little "what i changed since v2" note after the "---" separator is
helpful, please.

But I figured it out!   Thanks, I'll pop this into my after-rc1 pile.

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[Barry Song]

On Wed, Apr 1, 2026 at 3:47 PM David Carlier <devnexen@gmail.com> wrote:
>
> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> but sio->pages tracks the number of bvec entries (folios), not base
> pages.
>
> While large folios cannot currently reach this path (SWP_FS_OPS and
> SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation
> is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically
> inconsistent with the per-memcg path which correctly uses
> folio_nr_pages().
>
> Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> count since sio->len is accumulated via folio_size(folio).
>
> Signed-off-by: David Carlier <devnexen@gmail.com>

LGTM,

Reviewed-by: Barry Song <baohua@kernel.org>

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[Chris Li]

On Wed, Apr 1, 2026 at 12:48 AM David Carlier <devnexen@gmail.com> wrote:
>
> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> but sio->pages tracks the number of bvec entries (folios), not base
> pages.
>
> While large folios cannot currently reach this path (SWP_FS_OPS and
> SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation
> is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically
> inconsistent with the per-memcg path which correctly uses
> folio_nr_pages().
>
> Use sio->len >> PAGE_SHIFT instead, which gives the correct base page
> count since sio->len is accumulated via folio_size(folio).
>
> Signed-off-by: David Carlier <devnexen@gmail.com>

Acked-by: Chris Li <chrisl@kernel.org>

Chris

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[Matthew Wilcox]

On Wed, Apr 01, 2026 at 08:47:53AM +0100, David Carlier wrote:
> sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> but sio->pages tracks the number of bvec entries (folios), not base
> pages.

Since you're messing with this file anyway, do you want to submit a
third patch to rename swap_iocb->pages to ->bvecs?

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[Barry Song]

On Thu, Apr 2, 2026 at 12:11 PM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Wed, Apr 01, 2026 at 08:47:53AM +0100, David Carlier wrote:
> > sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> > but sio->pages tracks the number of bvec entries (folios), not base
> > pages.
>
> Since you're messing with this file anyway, do you want to submit a
> third patch to rename swap_iocb->pages to ->bvecs?

+1, that would be great.

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

[David CARLIER]

Sure, will send as a follow-up patch.

Cheers.

On Thu, 2 Apr 2026 at 05:11, Matthew Wilcox <willy@infradead.org> wrote:
>
> On Wed, Apr 01, 2026 at 08:47:53AM +0100, David Carlier wrote:
> > sio_read_complete() uses sio->pages to account global PSWPIN vm events,
> > but sio->pages tracks the number of bvec entries (folios), not base
> > pages.
>
> Since you're messing with this file anyway, do you want to submit a
> third patch to rename swap_iocb->pages to ->bvecs?