[PATCH v2 0/2] mm: folio conversion and refactoring in do_swap_page()

[PATCH v2 0/2] mm: folio conversion and refactoring in do_swap_page()

[Hongfu Li]

From: Hongfu Li <lihongfu@kylinos.cn>

Convert the device-private migration path in do_swap_page() to use
folio APIs, then extract all non-swap entry handling into a new
do_non_swap_page() helper to improve readability and reduce the
size of the main swap fault function.

Hongfu Li (2):
  mm: Use a folio in the softleaf_is_device_private path
  mm: Extract non-swap page handling to do_non_swap_page()

 mm/memory.c | 119 ++++++++++++++++++++++++++++++----------------------
 1 file changed, 69 insertions(+), 50 deletions(-)

---
v2:
- Drop fault_folio, reuse the existing folio variable instead
- Refresh commit messages
- Update comment to use "folio" instead of "page" in the
  device_private path
- Add Acked-by: David Hildenbrand
- Add patch 2/2 to extract do_non_swap_page() from do_swap_page()
- Link to v1: https://lore.kernel.org/all/20260721013347.65698-1-hongfu.li@linux.dev/
-- 
2.54.0

[PATCH v2 1/2] mm: Use a folio in the softleaf_is_device_private path

[Hongfu Li]

From: Hongfu Li <lihongfu@kylinos.cn>

Use the folio APIs in the device_private migration path of do_swap_page(),
replacing four calls to compound_head() with one.

Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
---
 mm/memory.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index d5e87624f692..ae521baa8c84 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4871,18 +4871,19 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 				goto unlock;
 
 			/*
-			 * Get a page reference while we know the page can't be
+			 * Get a folio reference while we know the folio can't be
 			 * freed.
 			 */
-			if (trylock_page(vmf->page)) {
+			folio = page_folio(vmf->page);
+			if (folio_trylock(folio)) {
 				struct dev_pagemap *pgmap;
 
-				get_page(vmf->page);
+				folio_get(folio);
 				pte_unmap_unlock(vmf->pte, vmf->ptl);
 				pgmap = page_pgmap(vmf->page);
 				ret = pgmap->ops->migrate_to_ram(vmf);
-				unlock_page(vmf->page);
-				put_page(vmf->page);
+				folio_unlock(folio);
+				folio_put(folio);
 			} else {
 				pte_unmap(vmf->pte);
 				softleaf_entry_wait_on_locked(entry, vmf->ptl);
-- 
2.54.0

[PATCH v2 2/2] mm: Extract non-swap page handling to do_non_swap_page()

[Hongfu Li]

From: Hongfu Li <lihongfu@kylinos.cn>

Factor non-swap entry logic out of bloated do_swap_page() to simplify
the main swap fault flow and improve readability.

The new do_non_swap_page() handles migration, device private/exclusive,
hwpoison, marker and bad PTE entries.

Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
---
 mm/memory.c | 120 ++++++++++++++++++++++++++++++----------------------
 1 file changed, 69 insertions(+), 51 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index ae521baa8c84..50d8e92aef91 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4815,6 +4815,74 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
 	} while (--nr_pages);
 }
 
+static vm_fault_t do_non_swap_page(struct vm_fault *vmf)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	struct folio *folio;
+	softleaf_t entry;
+	vm_fault_t ret = 0;
+
+	entry = softleaf_from_pte(vmf->orig_pte);
+	if (softleaf_is_migration(entry)) {
+		migration_entry_wait(vma->vm_mm, vmf->pmd,
+				     vmf->address);
+	} else if (softleaf_is_device_exclusive(entry)) {
+		vmf->page = softleaf_to_page(entry);
+		ret = remove_device_exclusive_entry(vmf);
+	} else if (softleaf_is_device_private(entry)) {
+
+		if (vmf->flags & FAULT_FLAG_VMA_LOCK) {
+			/*
+			 * migrate_to_ram is not yet ready to operate
+			 * under VMA lock.
+			 */
+			vma_end_read(vma);
+			return VM_FAULT_RETRY;
+		}
+
+		vmf->page = softleaf_to_page(entry);
+		vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
+						vmf->address, &vmf->ptl);
+		if (unlikely(!vmf->pte ||
+			     !pte_same(ptep_get(vmf->pte),
+					vmf->orig_pte)))
+			goto unlock;
+
+		/*
+		 * Get a folio reference while we know the folio can't be
+		 * freed.
+		 */
+		folio = page_folio(vmf->page);
+		if (folio_trylock(folio)) {
+			struct dev_pagemap *pgmap;
+
+			folio_get(folio);
+			pte_unmap_unlock(vmf->pte, vmf->ptl);
+			pgmap = page_pgmap(vmf->page);
+			ret = pgmap->ops->migrate_to_ram(vmf);
+			folio_unlock(folio);
+			folio_put(folio);
+		} else {
+			pte_unmap(vmf->pte);
+			softleaf_entry_wait_on_locked(entry, vmf->ptl);
+		}
+	} else if (softleaf_is_hwpoison(entry)) {
+		ret = VM_FAULT_HWPOISON;
+	} else if (softleaf_is_marker(entry)) {
+		ret = handle_pte_marker(vmf);
+	} else {
+		print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL);
+		ret = VM_FAULT_SIGBUS;
+	}
+
+	return ret;
+
+unlock:
+	if (vmf->pte)
+		pte_unmap_unlock(vmf->pte, vmf->ptl);
+	return ret;
+}
+
 /*
  * We enter with either the VMA lock or the mmap_lock held (see
  * FAULT_FLAG_VMA_LOCK), and pte mapped but not yet locked.
@@ -4845,57 +4913,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 
 	entry = softleaf_from_pte(vmf->orig_pte);
 	if (unlikely(!softleaf_is_swap(entry))) {
-		if (softleaf_is_migration(entry)) {
-			migration_entry_wait(vma->vm_mm, vmf->pmd,
-					     vmf->address);
-		} else if (softleaf_is_device_exclusive(entry)) {
-			vmf->page = softleaf_to_page(entry);
-			ret = remove_device_exclusive_entry(vmf);
-		} else if (softleaf_is_device_private(entry)) {
-			if (vmf->flags & FAULT_FLAG_VMA_LOCK) {
-				/*
-				 * migrate_to_ram is not yet ready to operate
-				 * under VMA lock.
-				 */
-				vma_end_read(vma);
-				ret = VM_FAULT_RETRY;
-				goto out;
-			}
-
-			vmf->page = softleaf_to_page(entry);
-			vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-					vmf->address, &vmf->ptl);
-			if (unlikely(!vmf->pte ||
-				     !pte_same(ptep_get(vmf->pte),
-							vmf->orig_pte)))
-				goto unlock;
-
-			/*
-			 * Get a folio reference while we know the folio can't be
-			 * freed.
-			 */
-			folio = page_folio(vmf->page);
-			if (folio_trylock(folio)) {
-				struct dev_pagemap *pgmap;
-
-				folio_get(folio);
-				pte_unmap_unlock(vmf->pte, vmf->ptl);
-				pgmap = page_pgmap(vmf->page);
-				ret = pgmap->ops->migrate_to_ram(vmf);
-				folio_unlock(folio);
-				folio_put(folio);
-			} else {
-				pte_unmap(vmf->pte);
-				softleaf_entry_wait_on_locked(entry, vmf->ptl);
-			}
-		} else if (softleaf_is_hwpoison(entry)) {
-			ret = VM_FAULT_HWPOISON;
-		} else if (softleaf_is_marker(entry)) {
-			ret = handle_pte_marker(vmf);
-		} else {
-			print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL);
-			ret = VM_FAULT_SIGBUS;
-		}
+		ret = do_non_swap_page(vmf);
 		goto out;
 	}
 
-- 
2.54.0

Re: [PATCH v2 2/2] mm: Extract non-swap page handling to do_non_swap_page()

[David Hildenbrand (Arm)]

On 7/23/26 05:07, Hongfu Li wrote:
> From: Hongfu Li <lihongfu@kylinos.cn>
> 
> Factor non-swap entry logic out of bloated do_swap_page() to simplify
> the main swap fault flow and improve readability.
> 
> The new do_non_swap_page() handles migration, device private/exclusive,
> hwpoison, marker and bad PTE entries.
> 
> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
> ---
>  mm/memory.c | 120 ++++++++++++++++++++++++++++++----------------------
>  1 file changed, 69 insertions(+), 51 deletions(-)
> 
> diff --git a/mm/memory.c b/mm/memory.c
> index ae521baa8c84..50d8e92aef91 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -4815,6 +4815,74 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
>  	} while (--nr_pages);
>  }
>  
> +static vm_fault_t do_non_swap_page(struct vm_fault *vmf)

We're handling softleaf entries that are not swap entries.

do_nonswap_softleaf() or sth like that would be better than talking about "page"
if there is not even a page.

But actually it should be the other way around:

do_softleaf_entry() [currently do_swap_page]
   ... non-swap entries
   do_swap_entry()

Or sth like that. Then, maybe in the future, we can just move do_swap_entry()
out of memory.c

So we're factoring the wrong thing out in this patch.

-- 
Cheers,

David

Re: [PATCH v2 2/2] mm: Extract non-swap page handling to do_non_swap_page()

[Hongfu Li]

> >  mm/memory.c | 120 ++++++++++++++++++++++++++++++----------------------
> >  1 file changed, 69 insertions(+), 51 deletions(-)
> > 
> > diff --git a/mm/memory.c b/mm/memory.c
> > index ae521baa8c84..50d8e92aef91 100644
> > --- a/mm/memory.c
> > +++ b/mm/memory.c
> > @@ -4815,6 +4815,74 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
> >  	} while (--nr_pages);
> >  }
> >  
> > +static vm_fault_t do_non_swap_page(struct vm_fault *vmf)
> 
> We're handling softleaf entries that are not swap entries.
> 
> do_nonswap_softleaf() or sth like that would be better than talking about "page"
> if there is not even a page.
> 
> But actually it should be the other way around:
> 
> do_softleaf_entry() [currently do_swap_page]
>    ... non-swap entries
>    do_swap_entry()
> 
> Or sth like that. Then, maybe in the future, we can just move do_swap_entry()
> out of memory.c
> 
> So we're factoring the wrong thing out in this patch.

Thank you very much for the detailed review, your analysis makes perfect sense to me.
                                                                                                                                                                                                                                                                                
I'm glad to send out v3 shortly for your further review, with 
do_non_swap_page renamed to do_nonswap_softleaf.

Best regards,
Hongfu

Re: [PATCH v2 2/2] mm: Extract non-swap page handling to do_non_swap_page()

[David Hildenbrand (Arm)]

On 7/27/26 14:10, Hongfu Li wrote:
>>>  mm/memory.c | 120 ++++++++++++++++++++++++++++++----------------------
>>>  1 file changed, 69 insertions(+), 51 deletions(-)
>>>
>>> diff --git a/mm/memory.c b/mm/memory.c
>>> index ae521baa8c84..50d8e92aef91 100644
>>> --- a/mm/memory.c
>>> +++ b/mm/memory.c
>>> @@ -4815,6 +4815,74 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
>>>  	} while (--nr_pages);
>>>  }
>>>  
>>> +static vm_fault_t do_non_swap_page(struct vm_fault *vmf)
>>
>> We're handling softleaf entries that are not swap entries.
>>
>> do_nonswap_softleaf() or sth like that would be better than talking about "page"
>> if there is not even a page.
>>
>> But actually it should be the other way around:
>>
>> do_softleaf_entry() [currently do_swap_page]
>>    ... non-swap entries
>>    do_swap_entry()
>>
>> Or sth like that. Then, maybe in the future, we can just move do_swap_entry()
>> out of memory.c
>>
>> So we're factoring the wrong thing out in this patch.
> 
> Thank you very much for the detailed review, your analysis makes perfect sense to me.
>                                                                                                                                                                                                                                                                                 
> I'm glad to send out v3 shortly for your further review, with 
> do_non_swap_page renamed to do_nonswap_softleaf.

Read again, that's not what I meant.

-- 
Cheers,

David

Re: [PATCH v2 2/2] mm: Extract non-swap page handling to do_non_swap_page()

[Lorenzo Stoakes (ARM)]

On Mon, Jul 27, 2026 at 02:16:40PM +0200, David Hildenbrand (Arm) wrote:
> On 7/27/26 14:10, Hongfu Li wrote:
> >>>  mm/memory.c | 120 ++++++++++++++++++++++++++++++----------------------
> >>>  1 file changed, 69 insertions(+), 51 deletions(-)
> >>>
> >>> diff --git a/mm/memory.c b/mm/memory.c
> >>> index ae521baa8c84..50d8e92aef91 100644
> >>> --- a/mm/memory.c
> >>> +++ b/mm/memory.c
> >>> @@ -4815,6 +4815,74 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
> >>>  	} while (--nr_pages);
> >>>  }
> >>>
> >>> +static vm_fault_t do_non_swap_page(struct vm_fault *vmf)
> >>
> >> We're handling softleaf entries that are not swap entries.
> >>
> >> do_nonswap_softleaf() or sth like that would be better than talking about "page"
> >> if there is not even a page.
> >>
> >> But actually it should be the other way around:
> >>
> >> do_softleaf_entry() [currently do_swap_page]
> >>    ... non-swap entries
> >>    do_swap_entry()
> >>
> >> Or sth like that. Then, maybe in the future, we can just move do_swap_entry()
> >> out of memory.c
> >>
> >> So we're factoring the wrong thing out in this patch.
> >
> > Thank you very much for the detailed review, your analysis makes perfect sense to me.
> >
> > I'm glad to send out v3 shortly for your further review, with
> > do_non_swap_page renamed to do_nonswap_softleaf.
>
> Read again, that's not what I meant.

Yeah please do NOT introduce 'nonswap' we already had fun with the nonswap swap
entry before :)

Quoting David:

	do_softleaf_entry() [currently do_swap_page]
		... non-swap entries
		do_swap_entry()

Also, swap entries _are_ softleaf entries.

The whole point of the softleaf change/renaming was to differentiate page
table entries that are purely there for mapping stuff from the hardware's
point of view (non-present because nothing is mapped there, present if
mapping something) vs. something non-present because the entry is
software-defined (hence SOFT-leaf).

Swap entries are software-defined and thus fall into category.

You can see all of the softleaf types trivially in include/linux/leafops.h
in enum softleaf_type:

enum softleaf_type {
	/* Fundamental types. */
	SOFTLEAF_NONE,
	SOFTLEAF_SWAP,
	/* Migration types. */
	SOFTLEAF_MIGRATION_READ,
	SOFTLEAF_MIGRATION_READ_EXCLUSIVE,
	SOFTLEAF_MIGRATION_WRITE,
	/* Device types. */
	SOFTLEAF_DEVICE_PRIVATE_READ,
	SOFTLEAF_DEVICE_PRIVATE_WRITE,
	SOFTLEAF_DEVICE_EXCLUSIVE,
	/* H/W posion types. */
	SOFTLEAF_HWPOISON,
	/* Marker types. */
	SOFTLEAF_MARKER,
};

>
> --
> Cheers,
>
> David

Cheers, Lorenzo

P.S. I do remember suggesting that you _slow down_ :) we are into the quiet
period now so let's maybe reduce the pace of series + respins, thanks.