[PATCH 1/2] mm: memfd_luo: always make all folios uptodate

[PATCH 1/2] mm: memfd_luo: always make all folios uptodate

[Pratyush Yadav]

From: "Pratyush Yadav (Google)" <pratyush@kernel.org>

When a folio is added to a shmem file via fallocate, it is not zeroed on
allocation. This is done as a performance optimization since it is
possible the folio will never end up being used at all. When the folio
is used, shmem checks for the uptodate flag, and if absent, zeroes the
folio (and sets the flag) before returning to user.

With LUO, the flags of each folio are saved at preserve time. It is
possible to have a memfd with some folios fallocated but not uptodate.
For those, the uptodate flag doesn't get saved. The folios might later
end up being used and become uptodate. They would get passed to the next
kernel via KHO correctly since they did get preserved. But they won't
have the MEMFD_LUO_FOLIO_UPTODATE flag.

This means that when the memfd is retrieved, the folios will be added to
the shmem file without the uptodate flag. They will be zeroed before
first use, losing the data in those folios.

Since we take a big performance hit in allocating, zeroing, and pinning
all folios at prepare time anyway, take some more and zero all
non-uptodate ones too.

Later when there is a stronger need to make prepare faster, this can be
optimized.

To avoid racing with another uptodate operation, take the folio lock.

Fixes: b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
Cc: stable@vger.kernel.org
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
---
 mm/memfd_luo.c | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/mm/memfd_luo.c b/mm/memfd_luo.c
index a34fccc23b6a..ccbf1337f650 100644
--- a/mm/memfd_luo.c
+++ b/mm/memfd_luo.c
@@ -152,10 +152,31 @@ static int memfd_luo_preserve_folios(struct file *file,
 		if (err)
 			goto err_unpreserve;
 
+		folio_lock(folio);
+
 		if (folio_test_dirty(folio))
 			flags |= MEMFD_LUO_FOLIO_DIRTY;
-		if (folio_test_uptodate(folio))
-			flags |= MEMFD_LUO_FOLIO_UPTODATE;
+
+		/*
+		 * If the folio is not uptodate, it was fallocated but never
+		 * used. Saving this flag at prepare() doesn't work since it
+		 * might change later when someone uses the folio.
+		 *
+		 * Since we have taken the performance penalty of allocating,
+		 * zeroing, and pinning all the folios in the holes, take a bit
+		 * more and zero all non-uptodate folios too.
+		 *
+		 * NOTE: For someone looking to improve preserve performance,
+		 * this is a good place to look.
+		 */
+		if (!folio_test_uptodate(folio)) {
+			folio_zero_range(folio, 0, folio_size(folio));
+			flush_dcache_folio(folio);
+			folio_mark_uptodate(folio);
+		}
+		flags |= MEMFD_LUO_FOLIO_UPTODATE;
+
+		folio_unlock(folio);
 
 		pfolio->pfn = folio_pfn(folio);
 		pfolio->flags = flags;
-- 
2.53.0.371.g1d285c8824-goog

Re: [PATCH 1/2] mm: memfd_luo: always make all folios uptodate

[Mike Rapoport]

On Mon, Feb 23, 2026 at 06:39:28PM +0100, Pratyush Yadav wrote:
> From: "Pratyush Yadav (Google)" <pratyush@kernel.org>
> 
> When a folio is added to a shmem file via fallocate, it is not zeroed on
> allocation. This is done as a performance optimization since it is
> possible the folio will never end up being used at all. When the folio
> is used, shmem checks for the uptodate flag, and if absent, zeroes the
> folio (and sets the flag) before returning to user.
> 
> With LUO, the flags of each folio are saved at preserve time. It is
> possible to have a memfd with some folios fallocated but not uptodate.
> For those, the uptodate flag doesn't get saved. The folios might later
> end up being used and become uptodate. They would get passed to the next
> kernel via KHO correctly since they did get preserved. But they won't
> have the MEMFD_LUO_FOLIO_UPTODATE flag.
> 
> This means that when the memfd is retrieved, the folios will be added to
> the shmem file without the uptodate flag. They will be zeroed before
> first use, losing the data in those folios.
> 
> Since we take a big performance hit in allocating, zeroing, and pinning
> all folios at prepare time anyway, take some more and zero all
> non-uptodate ones too.
> 
> Later when there is a stronger need to make prepare faster, this can be
> optimized.
> 
> To avoid racing with another uptodate operation, take the folio lock.
> 
> Fixes: b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
> Cc: stable@vger.kernel.org
> Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>

Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>

> ---
>  mm/memfd_luo.c | 25 +++++++++++++++++++++++--
>  1 file changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memfd_luo.c b/mm/memfd_luo.c
> index a34fccc23b6a..ccbf1337f650 100644
> --- a/mm/memfd_luo.c
> +++ b/mm/memfd_luo.c
> @@ -152,10 +152,31 @@ static int memfd_luo_preserve_folios(struct file *file,
>  		if (err)
>  			goto err_unpreserve;
>  
> +		folio_lock(folio);
> +
>  		if (folio_test_dirty(folio))
>  			flags |= MEMFD_LUO_FOLIO_DIRTY;
> -		if (folio_test_uptodate(folio))
> -			flags |= MEMFD_LUO_FOLIO_UPTODATE;
> +
> +		/*
> +		 * If the folio is not uptodate, it was fallocated but never
> +		 * used. Saving this flag at prepare() doesn't work since it
> +		 * might change later when someone uses the folio.
> +		 *
> +		 * Since we have taken the performance penalty of allocating,
> +		 * zeroing, and pinning all the folios in the holes, take a bit
> +		 * more and zero all non-uptodate folios too.
> +		 *
> +		 * NOTE: For someone looking to improve preserve performance,
> +		 * this is a good place to look.

I'd add a larger comment above memfd_luo_preserve_folios() that says that
it allocates, pins etc and fold the last two paragraphs of this comment
there.

> +		 */
> +		if (!folio_test_uptodate(folio)) {
> +			folio_zero_range(folio, 0, folio_size(folio));
> +			flush_dcache_folio(folio);
> +			folio_mark_uptodate(folio);
> +		}
> +		flags |= MEMFD_LUO_FOLIO_UPTODATE;
> +
> +		folio_unlock(folio);
>  
>  		pfolio->pfn = folio_pfn(folio);
>  		pfolio->flags = flags;
> -- 
> 2.53.0.371.g1d285c8824-goog
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH 1/2] mm: memfd_luo: always make all folios uptodate

[Pratyush Yadav]

On Wed, Feb 25 2026, Mike Rapoport wrote:

> On Mon, Feb 23, 2026 at 06:39:28PM +0100, Pratyush Yadav wrote:
[...]
>> +
>> +		/*
>> +		 * If the folio is not uptodate, it was fallocated but never
>> +		 * used. Saving this flag at prepare() doesn't work since it
>> +		 * might change later when someone uses the folio.
>> +		 *
>> +		 * Since we have taken the performance penalty of allocating,
>> +		 * zeroing, and pinning all the folios in the holes, take a bit
>> +		 * more and zero all non-uptodate folios too.
>> +		 *
>> +		 * NOTE: For someone looking to improve preserve performance,
>> +		 * this is a good place to look.
>
> I'd add a larger comment above memfd_luo_preserve_folios() that says that
> it allocates, pins etc and fold the last two paragraphs of this comment
> there.

How about this:

	/*
	 * If the folio is not uptodate, it was fallocated but never
	 * used. Saving this flag at prepare() doesn't work since it
	 * might change later when the folio is used. Make it uptodate
	 * now to avoid this problem.
	 */
	if (!folio_test_uptodate(folio)) {

And the comment above memfd_pin_folios() gets this:

	 * NOTE: For someone looking to improve preserve performance, this is a
	 * good place to look. Also look at the folio zeroing below.

[...]

-- 
Regards,
Pratyush Yadav

Re: [PATCH 1/2] mm: memfd_luo: always make all folios uptodate

[Mike Rapoport]

On Thu, Mar 05, 2026 at 09:50:18AM +0100, Pratyush Yadav wrote:
> On Wed, Feb 25 2026, Mike Rapoport wrote:
> 
> > On Mon, Feb 23, 2026 at 06:39:28PM +0100, Pratyush Yadav wrote:
> [...]
> >> +
> >> +		/*
> >> +		 * If the folio is not uptodate, it was fallocated but never
> >> +		 * used. Saving this flag at prepare() doesn't work since it
> >> +		 * might change later when someone uses the folio.
> >> +		 *
> >> +		 * Since we have taken the performance penalty of allocating,
> >> +		 * zeroing, and pinning all the folios in the holes, take a bit
> >> +		 * more and zero all non-uptodate folios too.
> >> +		 *
> >> +		 * NOTE: For someone looking to improve preserve performance,
> >> +		 * this is a good place to look.
> >
> > I'd add a larger comment above memfd_luo_preserve_folios() that says that
> > it allocates, pins etc and fold the last two paragraphs of this comment
> > there.
> 
> How about this:
> 
> 	/*
> 	 * If the folio is not uptodate, it was fallocated but never
> 	 * used. Saving this flag at prepare() doesn't work since it
> 	 * might change later when the folio is used. Make it uptodate
> 	 * now to avoid this problem.
> 	 */
> 	if (!folio_test_uptodate(folio)) {
> 
> And the comment above memfd_pin_folios() gets this:
> 
> 	 * NOTE: For someone looking to improve preserve performance, this is a
> 	 * good place to look. Also look at the folio zeroing below.
 
Looks great to me

> [...]
> 
> -- 
> Regards,
> Pratyush Yadav

-- 
Sincerely yours,
Mike.

[PATCH 2/2] mm: memfd_luo: always dirty all folios

[Pratyush Yadav]

From: "Pratyush Yadav (Google)" <pratyush@kernel.org>

A dirty folio is one which has been written to. A clean folio is its
opposite. Since a clean folio has no user data, it can be freed under
memory pressure.

memfd preservation with LUO saves the flag at preserve(). This is
problematic. The folio might get dirtied later. Saving it at freeze()
also doesn't work, since the dirty bit from PTE is normally synced at
unmap and there might still be mappings of the file at freeze().

To see why this is a problem, say a folio is clean at preserve, but gets
dirtied later. The serialized state of the folio will mark it as clean.
After retrieve, the next kernel will see the folio as clean and might
try to reclaim it under memory pressure. This will result in losing user
data.

Mark all folios of the file as dirty, and always set the
MEMFD_LUO_FOLIO_DIRTY flag. This comes with the side effect of making
all clean folios un-reclaimable. This is a cost that has to be paid for
participants of live update. It is not expected to be a common use case
to preserve a lot of clean folios anyway.

Since the value of pfolio->flags is a constant now, drop the flags
variable and set it directly.

Fixes: b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
Cc: stable@vger.kernel.org
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
---
 mm/memfd_luo.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/mm/memfd_luo.c b/mm/memfd_luo.c
index ccbf1337f650..9eac02d06b5a 100644
--- a/mm/memfd_luo.c
+++ b/mm/memfd_luo.c
@@ -146,7 +146,6 @@ static int memfd_luo_preserve_folios(struct file *file,
 	for (i = 0; i < nr_folios; i++) {
 		struct memfd_luo_folio_ser *pfolio = &folios_ser[i];
 		struct folio *folio = folios[i];
-		unsigned int flags = 0;
 
 		err = kho_preserve_folio(folio);
 		if (err)
@@ -154,8 +153,26 @@ static int memfd_luo_preserve_folios(struct file *file,
 
 		folio_lock(folio);
 
-		if (folio_test_dirty(folio))
-			flags |= MEMFD_LUO_FOLIO_DIRTY;
+		/*
+		 * A dirty folio is one which has been written to. A clean folio
+		 * is its opposite. Since a clean folio does not carry user
+		 * data, it can be freed by page reclaim under memory pressure.
+		 *
+		 * Saving the dirty flag at prepare() time doesn't work since it
+		 * can change later. Saving it at freeze() also won't work
+		 * because the dirty bit is normally synced at unmap and there
+		 * might still be a mapping of the file at freeze().
+		 *
+		 * To see why this is a problem, say a folio is clean at
+		 * preserve, but gets dirtied later. The pfolio flags will mark
+		 * it as clean. After retrieve, the next kernel might try to
+		 * reclaim this folio under memory pressure, losing user data.
+		 *
+		 * Unconditionally mark it dirty to avoid this problem. This
+		 * comes at the cost of making clean folios un-reclaimable after
+		 * live update.
+		 */
+		folio_mark_dirty(folio);
 
 		/*
 		 * If the folio is not uptodate, it was fallocated but never
@@ -174,12 +191,11 @@ static int memfd_luo_preserve_folios(struct file *file,
 			flush_dcache_folio(folio);
 			folio_mark_uptodate(folio);
 		}
-		flags |= MEMFD_LUO_FOLIO_UPTODATE;
 
 		folio_unlock(folio);
 
 		pfolio->pfn = folio_pfn(folio);
-		pfolio->flags = flags;
+		pfolio->flags = MEMFD_LUO_FOLIO_DIRTY | MEMFD_LUO_FOLIO_UPTODATE;
 		pfolio->index = folio->index;
 	}
 
-- 
2.53.0.371.g1d285c8824-goog

Re: [PATCH 2/2] mm: memfd_luo: always dirty all folios

[Mike Rapoport]

On Mon, Feb 23, 2026 at 06:39:29PM +0100, Pratyush Yadav wrote:
> From: "Pratyush Yadav (Google)" <pratyush@kernel.org>
> 
> A dirty folio is one which has been written to. A clean folio is its
> opposite. Since a clean folio has no user data, it can be freed under
> memory pressure.
> 
> memfd preservation with LUO saves the flag at preserve(). This is
> problematic. The folio might get dirtied later. Saving it at freeze()
> also doesn't work, since the dirty bit from PTE is normally synced at
> unmap and there might still be mappings of the file at freeze().
> 
> To see why this is a problem, say a folio is clean at preserve, but gets
> dirtied later. The serialized state of the folio will mark it as clean.
> After retrieve, the next kernel will see the folio as clean and might
> try to reclaim it under memory pressure. This will result in losing user
> data.
> 
> Mark all folios of the file as dirty, and always set the
> MEMFD_LUO_FOLIO_DIRTY flag. This comes with the side effect of making
> all clean folios un-reclaimable. This is a cost that has to be paid for
> participants of live update. It is not expected to be a common use case
> to preserve a lot of clean folios anyway.
> 
> Since the value of pfolio->flags is a constant now, drop the flags
> variable and set it directly.
> 
> Fixes: b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
> Cc: stable@vger.kernel.org
> Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>

Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>

> ---
>  mm/memfd_luo.c | 26 +++++++++++++++++++++-----
>  1 file changed, 21 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/memfd_luo.c b/mm/memfd_luo.c
> index ccbf1337f650..9eac02d06b5a 100644
> --- a/mm/memfd_luo.c
> +++ b/mm/memfd_luo.c
> @@ -146,7 +146,6 @@ static int memfd_luo_preserve_folios(struct file *file,
>  	for (i = 0; i < nr_folios; i++) {
>  		struct memfd_luo_folio_ser *pfolio = &folios_ser[i];
>  		struct folio *folio = folios[i];
> -		unsigned int flags = 0;
>  
>  		err = kho_preserve_folio(folio);
>  		if (err)
> @@ -154,8 +153,26 @@ static int memfd_luo_preserve_folios(struct file *file,
>  
>  		folio_lock(folio);
>  
> -		if (folio_test_dirty(folio))
> -			flags |= MEMFD_LUO_FOLIO_DIRTY;
> +		/*
> +		 * A dirty folio is one which has been written to. A clean folio
> +		 * is its opposite. Since a clean folio does not carry user
> +		 * data, it can be freed by page reclaim under memory pressure.
> +		 *
> +		 * Saving the dirty flag at prepare() time doesn't work since it
> +		 * can change later. Saving it at freeze() also won't work
> +		 * because the dirty bit is normally synced at unmap and there
> +		 * might still be a mapping of the file at freeze().
> +		 *
> +		 * To see why this is a problem, say a folio is clean at
> +		 * preserve, but gets dirtied later. The pfolio flags will mark
> +		 * it as clean. After retrieve, the next kernel might try to
> +		 * reclaim this folio under memory pressure, losing user data.
> +		 *
> +		 * Unconditionally mark it dirty to avoid this problem. This
> +		 * comes at the cost of making clean folios un-reclaimable after
> +		 * live update.
> +		 */

Can we make the comment here shorter to only contain the gist of the issue?

> +		folio_mark_dirty(folio);
>  
>  		/*
>  		 * If the folio is not uptodate, it was fallocated but never
> @@ -174,12 +191,11 @@ static int memfd_luo_preserve_folios(struct file *file,
>  			flush_dcache_folio(folio);
>  			folio_mark_uptodate(folio);
>  		}
> -		flags |= MEMFD_LUO_FOLIO_UPTODATE;
>  
>  		folio_unlock(folio);
>  
>  		pfolio->pfn = folio_pfn(folio);
> -		pfolio->flags = flags;
> +		pfolio->flags = MEMFD_LUO_FOLIO_DIRTY | MEMFD_LUO_FOLIO_UPTODATE;
>  		pfolio->index = folio->index;
>  	}
>  
> -- 
> 2.53.0.371.g1d285c8824-goog
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH 2/2] mm: memfd_luo: always dirty all folios

[Pratyush Yadav]

Hi Mike,

On Wed, Feb 25 2026, Mike Rapoport wrote:

> On Mon, Feb 23, 2026 at 06:39:29PM +0100, Pratyush Yadav wrote:
[...]
>> -		if (folio_test_dirty(folio))
>> -			flags |= MEMFD_LUO_FOLIO_DIRTY;
>> +		/*
>> +		 * A dirty folio is one which has been written to. A clean folio
>> +		 * is its opposite. Since a clean folio does not carry user
>> +		 * data, it can be freed by page reclaim under memory pressure.
>> +		 *
>> +		 * Saving the dirty flag at prepare() time doesn't work since it
>> +		 * can change later. Saving it at freeze() also won't work
>> +		 * because the dirty bit is normally synced at unmap and there
>> +		 * might still be a mapping of the file at freeze().
>> +		 *
>> +		 * To see why this is a problem, say a folio is clean at
>> +		 * preserve, but gets dirtied later. The pfolio flags will mark
>> +		 * it as clean. After retrieve, the next kernel might try to
>> +		 * reclaim this folio under memory pressure, losing user data.
>> +		 *
>> +		 * Unconditionally mark it dirty to avoid this problem. This
>> +		 * comes at the cost of making clean folios un-reclaimable after
>> +		 * live update.
>> +		 */
>
> Can we make the comment here shorter to only contain the gist of the issue?

Is this any better? Or should I try to make it shorter still?

	/*
	 * Tracking the dirty flag of the folio is difficult since it is
	 * normally synced at unmap and there might still be mappings of
	 * the file alive.
	 *
	 * Not tracking it correctly can cause a dirty folio to be
	 * restored as clean after KHO. The next kernel might then try
	 * to reclaim it, losing user data.
	 *
	 * Unconditionally mark the folio dirty to avoid this. This
	 * comes at the cost of making clean folios un-reclaimable.
	 */

[...]

-- 
Regards,
Pratyush Yadav

Re: [PATCH 2/2] mm: memfd_luo: always dirty all folios

[Mike Rapoport]

On Thu, Mar 05, 2026 at 09:44:27AM +0100, Pratyush Yadav wrote:
> Hi Mike,
> 
> On Wed, Feb 25 2026, Mike Rapoport wrote:
> 
> > On Mon, Feb 23, 2026 at 06:39:29PM +0100, Pratyush Yadav wrote:
> [...]
> >> -		if (folio_test_dirty(folio))
> >> -			flags |= MEMFD_LUO_FOLIO_DIRTY;
> >> +		/*
> >> +		 * A dirty folio is one which has been written to. A clean folio
> >> +		 * is its opposite. Since a clean folio does not carry user
> >> +		 * data, it can be freed by page reclaim under memory pressure.
> >> +		 *
> >> +		 * Saving the dirty flag at prepare() time doesn't work since it
> >> +		 * can change later. Saving it at freeze() also won't work
> >> +		 * because the dirty bit is normally synced at unmap and there
> >> +		 * might still be a mapping of the file at freeze().
> >> +		 *
> >> +		 * To see why this is a problem, say a folio is clean at
> >> +		 * preserve, but gets dirtied later. The pfolio flags will mark
> >> +		 * it as clean. After retrieve, the next kernel might try to
> >> +		 * reclaim this folio under memory pressure, losing user data.
> >> +		 *
> >> +		 * Unconditionally mark it dirty to avoid this problem. This
> >> +		 * comes at the cost of making clean folios un-reclaimable after
> >> +		 * live update.
> >> +		 */
> >
> > Can we make the comment here shorter to only contain the gist of the issue?
> 
> Is this any better? Or should I try to make it shorter still?

Yep, thanks! :)

I don't think it can be made shorter without loosing information. What we
might do is to add a larger comment in what state we preserve folios on top
of memfd_luo_preserve_folios() and leave the code inside the function
alone. Can't say I have strong feelings about it.
 
> 	/*
> 	 * Tracking the dirty flag of the folio is difficult since it is
> 	 * normally synced at unmap and there might still be mappings of
> 	 * the file alive.
> 	 *
> 	 * Not tracking it correctly can cause a dirty folio to be
> 	 * restored as clean after KHO. The next kernel might then try
> 	 * to reclaim it, losing user data.
> 	 *
> 	 * Unconditionally mark the folio dirty to avoid this. This
> 	 * comes at the cost of making clean folios un-reclaimable.
> 	 */
> 
> [...]
> 
> -- 
> Regards,
> Pratyush Yadav

-- 
Sincerely yours,
Mike.