[PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Kees Cook]

Hi,

This fixes a performance regression[1] with vrealloc(). This needs to
get into v6.15, which is where the regression originates, and then it'll
get backport to the -stable releases as well.

Thanks!

-Kees

[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/

Kees Cook (2):
  mm: vmalloc: Actually use the in-place vrealloc region
  mm: vmalloc: Only zero-init on vrealloc shrink

 mm/vmalloc.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

-- 
2.34.1

[PATCH 1/2] mm: vmalloc: Actually use the in-place vrealloc region

[Kees Cook]

The refactoring to not build a new vmalloc region only actually worked
when shrinking. Actually return the resized area when it grows. Ugh.

Reported-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Closes: https://lore.kernel.org/all/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg
Tested-by: Eduard Zingerman <eddyz87@gmail.com>
Tested-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Signed-off-by: Kees Cook <kees@kernel.org>
---
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: <linux-mm@kvack.org>
---
 mm/vmalloc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 2d7511654831..74bd00fd734d 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -4111,6 +4111,7 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 		if (want_init_on_alloc(flags))
 			memset((void *)p + old_size, 0, size - old_size);
 		vm->requested_size = size;
+		return (void *)p;
 	}
 
 	/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
-- 
2.34.1

[PATCH 2/2] mm: vmalloc: Only zero-init on vrealloc shrink

[Kees Cook]

The common case is to grow reallocations, and since init_on_alloc will
have already zeroed the whole allocation, we only need to zero when
shrinking the allocation.

Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Tested-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Signed-off-by: Kees Cook <kees@kernel.org>
---
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: <linux-mm@kvack.org>
---
 mm/vmalloc.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 74bd00fd734d..00cf1b575c89 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -4093,8 +4093,8 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 	 * would be a good heuristic for when to shrink the vm_area?
 	 */
 	if (size <= old_size) {
-		/* Zero out "freed" memory. */
-		if (want_init_on_free())
+		/* Zero out "freed" memory, potentially for future realloc. */
+		if (want_init_on_free() || want_init_on_alloc(flags))
 			memset((void *)p + size, 0, old_size - size);
 		vm->requested_size = size;
 		kasan_poison_vmalloc(p + size, old_size - size);
@@ -4107,9 +4107,11 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 	if (size <= alloced_size) {
 		kasan_unpoison_vmalloc(p + old_size, size - old_size,
 				       KASAN_VMALLOC_PROT_NORMAL);
-		/* Zero out "alloced" memory. */
-		if (want_init_on_alloc(flags))
-			memset((void *)p + old_size, 0, size - old_size);
+		/*
+		 * No need to zero memory here, as unused memory will have
+		 * already been zeroed at initial allocation time or during
+		 * realloc shrink time.
+		 */
 		vm->requested_size = size;
 		return (void *)p;
 	}
-- 
2.34.1

Re: [PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Shung-Hsi Yu]

On Thu, May 15, 2025 at 02:42:14PM -0700, Kees Cook wrote:
> Hi,
> 
> This fixes a performance regression[1] with vrealloc(). This needs to
> get into v6.15, which is where the regression originates, and then it'll
> get backport to the -stable releases as well.
> 
> Thanks!
> 
> -Kees
> 
> [1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/
> 
> Kees Cook (2):
>   mm: vmalloc: Actually use the in-place vrealloc region
>   mm: vmalloc: Only zero-init on vrealloc shrink

Thank you for the prompt fix! I'll remember to include a more thorough
note on reproducing the issue next time.

With the patchset applied, BPF selftests on both 6.15-rc6 and 6.14.7-rc2
passes successfully.

Tested-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>

Re: [PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Uladzislau Rezki]

On Thu, May 15, 2025 at 02:42:14PM -0700, Kees Cook wrote:
> Hi,
> 
> This fixes a performance regression[1] with vrealloc(). This needs to
> get into v6.15, which is where the regression originates, and then it'll
> get backport to the -stable releases as well.
> 
> Thanks!
> 
> -Kees
> 
> [1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/
> 
> Kees Cook (2):
>   mm: vmalloc: Actually use the in-place vrealloc region
>   mm: vmalloc: Only zero-init on vrealloc shrink
> 
>  mm/vmalloc.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
> 
Looks good to me both.

Reviewed-by: "Uladzislau Rezki (Sony)" <urezki@gmail.com>

--
Uladzislau Rezki

Re: [PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Danilo Krummrich]

On Thu, May 15, 2025 at 02:42:14PM -0700, Kees Cook wrote:
> Hi,
> 
> This fixes a performance regression[1] with vrealloc(). This needs to
> get into v6.15, which is where the regression originates, and then it'll
> get backport to the -stable releases as well.

Reviewed-by: Danilo Krummrich <dakr@kernel.org>

Re: [PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Kees Cook]

On Thu, May 15, 2025 at 02:42:14PM -0700, Kees Cook wrote:
> This fixes a performance regression[1] with vrealloc(). This needs to
> get into v6.15, which is where the regression originates, and then it'll
> get backport to the -stable releases as well.

Andrew, can you get these to Linus this week? I can also send them his
way if you'd rather?

-Kees

-- 
Kees Cook

Re: [PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Andrew Morton]

On Mon, 19 May 2025 12:18:42 -0700 Kees Cook <kees@kernel.org> wrote:

> On Thu, May 15, 2025 at 02:42:14PM -0700, Kees Cook wrote:
> > This fixes a performance regression[1] with vrealloc(). This needs to
> > get into v6.15, which is where the regression originates, and then it'll
> > get backport to the -stable releases as well.

No -stable backporting will be needed?

> Andrew, can you get these to Linus this week?

Sure.

Re: [PATCH 0/2] mm: vmalloc: Actually use the in-place vrealloc region

[Kees Cook]

On Mon, May 19, 2025 at 05:06:07PM -0700, Andrew Morton wrote:
> On Mon, 19 May 2025 12:18:42 -0700 Kees Cook <kees@kernel.org> wrote:
> 
> > On Thu, May 15, 2025 at 02:42:14PM -0700, Kees Cook wrote:
> > > This fixes a performance regression[1] with vrealloc(). This needs to
> > > get into v6.15, which is where the regression originates, and then it'll
> > > get backport to the -stable releases as well.
> 
> No -stable backporting will be needed?

I think it will, since the vrealloc patches were backported
automatically, e.g. in v6.14.y: 0b391a520b4e ("mm: vmalloc: support more
granular vrealloc() sizing")

> > Andrew, can you get these to Linus this week?
> 
> Sure.

Thanks!

-- 
Kees Cook