[REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Shung-Hsi Yu]

Hi,

There is an observable slowdown when running BPF selftests on 6.15-rc6
kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
Overall the BPF selftests now takes 2x time to run (from ~25m to ~50m),
and for the verif_scale_loop3_fail it went from single digit seconds to
6 minutes.

Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
support more granular vrealloc() sizing"[2]. To further zoom in the
issue, I tried removing the only kvrealloc() call in kernel/bpf/ by
reverting commit 96a30e469ca1 "bpf: use common instruction history
across all states", so _krealloc()_ was used instead of kvrealloc(), and
observe that there is _no_ slowdown[3]. While the bisect and the revert
is done on 6.14.7-rc2, I think it should stll be pretty representitive.

In short, the follow were tested:
- 6.15-rc6 (has a0309faf1cb0) -> slowdown
- 6.14.7-rc2 (has a0309faf1cb0) -> slowdown
- 6.14.7-rc2 (has a0309faf1cb0, call to kvrealloc in
  kernel/bpf/verifier.c replaced with krealloc) -> _no_ slowdown

And the vrealloc() change is causing slowdown in kvrealloc() call within
push_insn_history().

  /* for any branch, call, exit record the history of jmps in the given state */
  static int push_insn_history(struct bpf_verifier_env *env, struct bpf_verifier_state *cur,
  			     int insn_flags, u64 linked_regs)
  {
  	struct bpf_insn_hist_entry *p;
  	size_t alloc_size;
  	...
  	if (cur->insn_hist_end + 1 > env->insn_hist_cap) {
  		alloc_size = size_mul(cur->insn_hist_end + 1, sizeof(*p));
  		p = kvrealloc(env->insn_hist, alloc_size, GFP_USER);
  		if (!p)
  			return -ENOMEM;
  		env->insn_hist = p;
  		env->insn_hist_cap = alloc_size / sizeof(*p);
  	}
  
  	p = &env->insn_hist[cur->insn_hist_end];
  	p->idx = env->insn_idx;
  	p->prev_idx = env->prev_insn_idx;
  	p->flags = insn_flags;
  	p->linked_regs = linked_regs;
  
  	cur->insn_hist_end++;
  	env->cur_hist_ent = p;
  
  	return 0;
  }

BPF CI probably hasn't hit this yet because bpf-next have only got to
6.15-rc4.

Shung-Hsi

#regzbot introduced: a0309faf1cb0622cac7c820150b7abf2024acff5

1: https://github.com/shunghsiyu/libbpf/actions/runs/15038992168/job/42266125686
2: https://lore.kernel.org/stable/20250515041659.smhllyarxdwp7cav@desk/
3: https://github.com/shunghsiyu/libbpf/actions/runs/15043433548/job/42280277024

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
>Hi,
>
>There is an observable slowdown when running BPF selftests on 6.15-rc6
>kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
>Overall the BPF selftests now takes 2x time to run (from ~25m to ~50m),
>and for the verif_scale_loop3_fail it went from single digit seconds to
>6 minutes.
>
>Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
>support more granular vrealloc() sizing"[2]. To further zoom in the
>issue, I tried removing the only kvrealloc() call in kernel/bpf/ by
>reverting commit 96a30e469ca1 "bpf: use common instruction history
>across all states", so _krealloc()_ was used instead of kvrealloc(), and
>observe that there is _no_ slowdown[3]. While the bisect and the revert
>is done on 6.14.7-rc2, I think it should stll be pretty representitive.
>
>In short, the follow were tested:
>- 6.15-rc6 (has a0309faf1cb0) -> slowdown
>- 6.14.7-rc2 (has a0309faf1cb0) -> slowdown
>- 6.14.7-rc2 (has a0309faf1cb0, call to kvrealloc in
>  kernel/bpf/verifier.c replaced with krealloc) -> _no_ slowdown
>
>And the vrealloc() change is causing slowdown in kvrealloc() call within
>push_insn_history().

This is very strange! The vrealloc change should make things faster -- it removes potentially unneeded vmalloc and full object copies when it isn't needed.

Where can I find the .config for the slow runs?

And how do I run the test myself directly?

-Kees

>
>  /* for any branch, call, exit record the history of jmps in the given state */
>  static int push_insn_history(struct bpf_verifier_env *env, struct bpf_verifier_state *cur,
>  			     int insn_flags, u64 linked_regs)
>  {
>  	struct bpf_insn_hist_entry *p;
>  	size_t alloc_size;
>  	...
>  	if (cur->insn_hist_end + 1 > env->insn_hist_cap) {
>  		alloc_size = size_mul(cur->insn_hist_end + 1, sizeof(*p));
>  		p = kvrealloc(env->insn_hist, alloc_size, GFP_USER);
>  		if (!p)
>  			return -ENOMEM;
>  		env->insn_hist = p;
>  		env->insn_hist_cap = alloc_size / sizeof(*p);
>  	}
>  
>  	p = &env->insn_hist[cur->insn_hist_end];
>  	p->idx = env->insn_idx;
>  	p->prev_idx = env->prev_insn_idx;
>  	p->flags = insn_flags;
>  	p->linked_regs = linked_regs;
>  
>  	cur->insn_hist_end++;
>  	env->cur_hist_ent = p;
>  
>  	return 0;
>  }
>
>BPF CI probably hasn't hit this yet because bpf-next have only got to
>6.15-rc4.
>
>Shung-Hsi
>
>#regzbot introduced: a0309faf1cb0622cac7c820150b7abf2024acff5
>
>1: https://github.com/shunghsiyu/libbpf/actions/runs/15038992168/job/42266125686
>2: https://lore.kernel.org/stable/20250515041659.smhllyarxdwp7cav@desk/
>3: https://github.com/shunghsiyu/libbpf/actions/runs/15043433548/job/42280277024

-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> >There is an observable slowdown when running BPF selftests on 6.15-rc6
> >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> [...]
> Where can I find the .config for the slow runs?

Oops, I can read. :) Doing a build now...

> And how do I run the test myself directly?

I found:
https://docs.kernel.org/bpf/bpf_devel_QA.html

But it doesn't seem to cover a bunch of stuff (no way to prebuild the
tests, no info on building the test modules).

This seems to be needed:

make O=regression-bug -C tools/testing/selftests/bpf/test_kmods

But then the booted kernel doesn't load it (missing signatures?)

Anyway, I'll keep digging...

-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Pawan Gupta]

On Thu, May 15, 2025 at 09:51:15AM -0700, Kees Cook wrote:
> On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> > On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> > >There is an observable slowdown when running BPF selftests on 6.15-rc6
> > >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> > [...]
> > Where can I find the .config for the slow runs?
> 
> Oops, I can read. :) Doing a build now...
> 
> > And how do I run the test myself directly?
> 
> I found:
> https://docs.kernel.org/bpf/bpf_devel_QA.html
> 
> But it doesn't seem to cover a bunch of stuff (no way to prebuild the
> tests, no info on building the test modules).
> 
> This seems to be needed:
> 
> make O=regression-bug -C tools/testing/selftests/bpf/test_kmods
> 
> But then the booted kernel doesn't load it (missing signatures?)
> 
> Anyway, I'll keep digging...

After struggling with this for a while, I figured vmtest.sh is the easiest
way to test bpf:

./tools/testing/selftests/bpf/vmtest.sh -i ./test_progs

To test just the failing case, you can run:

./tools/testing/selftests/bpf/vmtest.sh -i -- timeout 20 ./test_progs -t verif_scale_loop3_fail

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 10:18:21AM -0700, Pawan Gupta wrote:
> On Thu, May 15, 2025 at 09:51:15AM -0700, Kees Cook wrote:
> > On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> > > On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> > > >There is an observable slowdown when running BPF selftests on 6.15-rc6
> > > >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> > > [...]
> > > Where can I find the .config for the slow runs?
> > 
> > Oops, I can read. :) Doing a build now...
> > 
> > > And how do I run the test myself directly?
> > 
> > I found:
> > https://docs.kernel.org/bpf/bpf_devel_QA.html
> > 
> > But it doesn't seem to cover a bunch of stuff (no way to prebuild the
> > tests, no info on building the test modules).
> > 
> > This seems to be needed:
> > 
> > make O=regression-bug -C tools/testing/selftests/bpf/test_kmods
> > 
> > But then the booted kernel doesn't load it (missing signatures?)
> > 
> > Anyway, I'll keep digging...
> 
> After struggling with this for a while, I figured vmtest.sh is the easiest
> way to test bpf:
> 
> ./tools/testing/selftests/bpf/vmtest.sh -i ./test_progs

I can't even build the test_progs. :(

$ make test_progs
...
  CLNG-BPF [test_progs] bpf_iter_tasks.bpf.o
progs/bpf_iter_tasks.c:98:8: error: call to undeclared function 'bpf_copy_from_user_task_str'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
   98 |         ret = bpf_copy_from_user_task_str((char *)task_str1, sizeof(task_str1), ptr, task, 0
);
      |               ^
1 error generated.


-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Pawan Gupta]

On Thu, May 15, 2025 at 10:41:41AM -0700, Kees Cook wrote:
> On Thu, May 15, 2025 at 10:18:21AM -0700, Pawan Gupta wrote:
> > On Thu, May 15, 2025 at 09:51:15AM -0700, Kees Cook wrote:
> > > On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> > > > On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> > > > >There is an observable slowdown when running BPF selftests on 6.15-rc6
> > > > >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> > > > [...]
> > > > Where can I find the .config for the slow runs?
> > > 
> > > Oops, I can read. :) Doing a build now...
> > > 
> > > > And how do I run the test myself directly?
> > > 
> > > I found:
> > > https://docs.kernel.org/bpf/bpf_devel_QA.html
> > > 
> > > But it doesn't seem to cover a bunch of stuff (no way to prebuild the
> > > tests, no info on building the test modules).
> > > 
> > > This seems to be needed:
> > > 
> > > make O=regression-bug -C tools/testing/selftests/bpf/test_kmods
> > > 
> > > But then the booted kernel doesn't load it (missing signatures?)
> > > 
> > > Anyway, I'll keep digging...
> > 
> > After struggling with this for a while, I figured vmtest.sh is the easiest
> > way to test bpf:
> > 
> > ./tools/testing/selftests/bpf/vmtest.sh -i ./test_progs
> 
> I can't even build the test_progs. :(
> 
> $ make test_progs
> ...
>   CLNG-BPF [test_progs] bpf_iter_tasks.bpf.o
> progs/bpf_iter_tasks.c:98:8: error: call to undeclared function 'bpf_copy_from_user_task_str'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
>    98 |         ret = bpf_copy_from_user_task_str((char *)task_str1, sizeof(task_str1), ptr, task, 0
> );
>       |               ^
> 1 error generated.

I just tried on the latest upstream, and I am getting the same error. My
earlier bisection was on a stable-rc for 6.14.y:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/commit/?h=linux-6.14.y

... where it was first reported:

https://lore.kernel.org/stable/20250515041659.smhllyarxdwp7cav@desk/

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Pawan Gupta]

On Thu, May 15, 2025 at 10:52:13AM -0700, Pawan Gupta wrote:
> On Thu, May 15, 2025 at 10:41:41AM -0700, Kees Cook wrote:
> > On Thu, May 15, 2025 at 10:18:21AM -0700, Pawan Gupta wrote:
> > > On Thu, May 15, 2025 at 09:51:15AM -0700, Kees Cook wrote:
> > > > On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> > > > > On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> > > > > >There is an observable slowdown when running BPF selftests on 6.15-rc6
> > > > > >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> > > > > [...]
> > > > > Where can I find the .config for the slow runs?
> > > > 
> > > > Oops, I can read. :) Doing a build now...
> > > > 
> > > > > And how do I run the test myself directly?
> > > > 
> > > > I found:
> > > > https://docs.kernel.org/bpf/bpf_devel_QA.html
> > > > 
> > > > But it doesn't seem to cover a bunch of stuff (no way to prebuild the
> > > > tests, no info on building the test modules).
> > > > 
> > > > This seems to be needed:
> > > > 
> > > > make O=regression-bug -C tools/testing/selftests/bpf/test_kmods
> > > > 
> > > > But then the booted kernel doesn't load it (missing signatures?)
> > > > 
> > > > Anyway, I'll keep digging...
> > > 
> > > After struggling with this for a while, I figured vmtest.sh is the easiest
> > > way to test bpf:
> > > 
> > > ./tools/testing/selftests/bpf/vmtest.sh -i ./test_progs
> > 
> > I can't even build the test_progs. :(
> > 
> > $ make test_progs
> > ...
> >   CLNG-BPF [test_progs] bpf_iter_tasks.bpf.o
> > progs/bpf_iter_tasks.c:98:8: error: call to undeclared function 'bpf_copy_from_user_task_str'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
> >    98 |         ret = bpf_copy_from_user_task_str((char *)task_str1, sizeof(task_str1), ptr, task, 0
> > );
> >       |               ^
> > 1 error generated.
> 
> I just tried on the latest upstream, and I am getting the same error. My
> earlier bisection was on a stable-rc for 6.14.y:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/commit/?h=linux-6.14.y
> 
> ... where it was first reported:
> 
> https://lore.kernel.org/stable/20250515041659.smhllyarxdwp7cav@desk/

I can confirm on v6.14.7-rc2 the test verif_scale_loop3_fail is now
passing(in <10 secs) after applying your first patch (mm: vmalloc: Actually
use the in-place vrealloc region). The the test passes after applying your
second patch also.

For some reason I am unable to build the bpf selftests on latest upstream.
I may be missing something that is required on latest upstream :-(

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Andrii Nakryiko]

On Thu, May 15, 2025 at 10:41 AM Kees Cook <kees@kernel.org> wrote:
>
> On Thu, May 15, 2025 at 10:18:21AM -0700, Pawan Gupta wrote:
> > On Thu, May 15, 2025 at 09:51:15AM -0700, Kees Cook wrote:
> > > On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> > > > On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> > > > >There is an observable slowdown when running BPF selftests on 6.15-rc6
> > > > >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> > > > [...]
> > > > Where can I find the .config for the slow runs?
> > >
> > > Oops, I can read. :) Doing a build now...
> > >
> > > > And how do I run the test myself directly?
> > >
> > > I found:
> > > https://docs.kernel.org/bpf/bpf_devel_QA.html
> > >
> > > But it doesn't seem to cover a bunch of stuff (no way to prebuild the
> > > tests, no info on building the test modules).
> > >
> > > This seems to be needed:
> > >
> > > make O=regression-bug -C tools/testing/selftests/bpf/test_kmods
> > >
> > > But then the booted kernel doesn't load it (missing signatures?)
> > >
> > > Anyway, I'll keep digging...
> >
> > After struggling with this for a while, I figured vmtest.sh is the easiest
> > way to test bpf:
> >
> > ./tools/testing/selftests/bpf/vmtest.sh -i ./test_progs
>
> I can't even build the test_progs. :(
>
> $ make test_progs
> ...
>   CLNG-BPF [test_progs] bpf_iter_tasks.bpf.o
> progs/bpf_iter_tasks.c:98:8: error: call to undeclared function 'bpf_copy_from_user_task_str'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
>    98 |         ret = bpf_copy_from_user_task_str((char *)task_str1, sizeof(task_str1), ptr, task, 0
> );
>       |               ^
> 1 error generated.
>

BPF selftests expect that there was a successful kernel build done
before that. So generally speaking:

0) cd <linux/repo/path>
1) export O=/path/to/build
2) cat tools/testing/selftests/bpf/config >> /path/to/build/.config
3) make O=/path/to/build -j$(nproc) oldefconfig all
4) cd tools/testing/selftests/bpf # everything is built within this
directory, we don't support KBUILD_PATH or O for BPF selftests build
artifacts
5) make O=/path/to/build -j$(nproc)

step #5 will search for vmlinux image across O, KBUILD_PATH or in
current linux source repo (in that order), and will generate necessary
vmlinux.h header out of BTF information embedded in vmlinux, which is
necessary for tests

You might need some dependent packages (libelf-devel, zlib-devel,
maybe some other, don't remember) to build everything, but if you are
on recent enough Clang (19? might work with a bit older one), you
should be good.

But tbh, if the above causes you problems, I don't think you need to
spend that much time trying to build BPF selftests, given you know
what the issue is and you are fixing it.

>
> --
> Kees Cook
>

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 10:53:10AM -0700, Andrii Nakryiko wrote:
> On Thu, May 15, 2025 at 10:41 AM Kees Cook <kees@kernel.org> wrote:
> >
> > On Thu, May 15, 2025 at 10:18:21AM -0700, Pawan Gupta wrote:
> > > On Thu, May 15, 2025 at 09:51:15AM -0700, Kees Cook wrote:
> > > > On Thu, May 15, 2025 at 07:51:26AM -0700, Kees Cook wrote:
> > > > > On May 15, 2025 6:12:25 AM PDT, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> > > > > >There is an observable slowdown when running BPF selftests on 6.15-rc6
> > > > > >kernel[1] built with tools/testing/selftests/bpf/{config,config.x86_64}.
> > > > > [...]
> > > > > Where can I find the .config for the slow runs?
> > > >
> > > > Oops, I can read. :) Doing a build now...
> > > >
> > > > > And how do I run the test myself directly?
> > > >
> > > > I found:
> > > > https://docs.kernel.org/bpf/bpf_devel_QA.html
> > > >
> > > > But it doesn't seem to cover a bunch of stuff (no way to prebuild the
> > > > tests, no info on building the test modules).
> > > >
> > > > This seems to be needed:
> > > >
> > > > make O=regression-bug -C tools/testing/selftests/bpf/test_kmods
> > > >
> > > > But then the booted kernel doesn't load it (missing signatures?)
> > > >
> > > > Anyway, I'll keep digging...
> > >
> > > After struggling with this for a while, I figured vmtest.sh is the easiest
> > > way to test bpf:
> > >
> > > ./tools/testing/selftests/bpf/vmtest.sh -i ./test_progs
> >
> > I can't even build the test_progs. :(
> >
> > $ make test_progs
> > ...
> >   CLNG-BPF [test_progs] bpf_iter_tasks.bpf.o
> > progs/bpf_iter_tasks.c:98:8: error: call to undeclared function 'bpf_copy_from_user_task_str'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
> >    98 |         ret = bpf_copy_from_user_task_str((char *)task_str1, sizeof(task_str1), ptr, task, 0
> > );
> >       |               ^
> > 1 error generated.
> >
> 
> BPF selftests expect that there was a successful kernel build done
> before that. So generally speaking:
> 
> 0) cd <linux/repo/path>
> 1) export O=/path/to/build
> 2) cat tools/testing/selftests/bpf/config >> /path/to/build/.config
> 3) make O=/path/to/build -j$(nproc) oldefconfig all
> 4) cd tools/testing/selftests/bpf # everything is built within this
> directory, we don't support KBUILD_PATH or O for BPF selftests build
> artifacts
> 5) make O=/path/to/build -j$(nproc)

Linux ToT fails to build, -next fails to build. v6.14.6 fails build,
each in different ways. :(

> But tbh, if the above causes you problems, I don't think you need to
> spend that much time trying to build BPF selftests, given you know
> what the issue is and you are fixing it.

Well, nothing I've proposed makes any sense as far as something that would
double execution time, so I don't really know what the issue is yet. :P

-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Eduard Zingerman]

On Thu, 2025-05-15 at 11:24 -0700, Kees Cook wrote:

[...]

> Linux ToT fails to build, -next fails to build. v6.14.6 fails build,
> each in different ways. :(

I posted recipe [1] some time ago.
Checked it right now, everything worked w/o modifications.
Can be used as a baseline for build environment.

[1] https://lore.kernel.org/bpf/62b54401510477eebdb6e1272ba4308ee121c215.camel@gmail.com/

[...]

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 09:12:25PM +0800, Shung-Hsi Yu wrote:
> Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
> support more granular vrealloc() sizing"[2]. To further zoom in the

Can you try this patch? It's a clear bug fix, but if it doesn't improve
things, I have another idea to rearrange the memset.


From e96a0e2519b1c5b50f45bd05bf60e6117d1132b2 Mon Sep 17 00:00:00 2001
From: Kees Cook <kees@kernel.org>
Date: Thu, 15 May 2025 08:43:12 -0700
Subject: [PATCH] mm: vmalloc: Actually use the in-place vrealloc region

The refactoring to not build a new vmalloc region only actually worked
when shrinking. Actually return the resized area when it grows. Ugh.

Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Signed-off-by: Kees Cook <kees@kernel.org>
---
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: <linux-mm@kvack.org>
---
 mm/vmalloc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 2d7511654831..74bd00fd734d 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -4111,6 +4111,7 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 		if (want_init_on_alloc(flags))
 			memset((void *)p + old_size, 0, size - old_size);
 		vm->requested_size = size;
+		return (void *)p;
 	}
 
 	/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
-- 
2.34.1


-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 08:47:47AM -0700, Kees Cook wrote:
> On Thu, May 15, 2025 at 09:12:25PM +0800, Shung-Hsi Yu wrote:
> > Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
> > support more granular vrealloc() sizing"[2]. To further zoom in the
> 
> Can you try this patch? It's a clear bug fix, but if it doesn't improve
> things, I have another idea to rearrange the memset.

Here's the patch (on top of the prior one) that relocates the memset:


From 0bc71b78603500705aca77f82de8ed1fc595c4c3 Mon Sep 17 00:00:00 2001
From: Kees Cook <kees@kernel.org>
Date: Thu, 15 May 2025 08:48:24 -0700
Subject: [PATCH] mm: vmalloc: Only zero-init on vrealloc shrink

The common case is to grow reallocations, and since init_on_alloc will
have already zeroed the whole allocation, we only need to zero when
shrinking the allocation.

Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Signed-off-by: Kees Cook <kees@kernel.org>
---
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: <linux-mm@kvack.org>
---
 mm/vmalloc.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 74bd00fd734d..83bedb1559ac 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -4093,8 +4093,8 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 	 * would be a good heuristic for when to shrink the vm_area?
 	 */
 	if (size <= old_size) {
-		/* Zero out "freed" memory. */
-		if (want_init_on_free())
+		/* Zero out "freed" memory, potentially for future realloc. */
+		if (want_init_on_free() || want_init_on_alloc(flags))
 			memset((void *)p + size, 0, old_size - size);
 		vm->requested_size = size;
 		kasan_poison_vmalloc(p + size, old_size - size);
@@ -4107,9 +4107,11 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 	if (size <= alloced_size) {
 		kasan_unpoison_vmalloc(p + old_size, size - old_size,
 				       KASAN_VMALLOC_PROT_NORMAL);
-		/* Zero out "alloced" memory. */
-		if (want_init_on_alloc(flags))
-			memset((void *)p + old_size, 0, size - old_size);
+		/*
+		 * No need to zero memory here, as unused memory will have
+		 * already been zeroed at initial allocation time or during
+		 * realloc shrink time.
+		 */
 		vm->requested_size = size;
 		return (void *)p;
 	}
-- 
2.34.1



-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Andrii Nakryiko]

On Thu, May 15, 2025 at 8:53 AM Kees Cook <kees@kernel.org> wrote:
>
> On Thu, May 15, 2025 at 08:47:47AM -0700, Kees Cook wrote:
> > On Thu, May 15, 2025 at 09:12:25PM +0800, Shung-Hsi Yu wrote:
> > > Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
> > > support more granular vrealloc() sizing"[2]. To further zoom in the
> >
> > Can you try this patch? It's a clear bug fix, but if it doesn't improve
> > things, I have another idea to rearrange the memset.
>
> Here's the patch (on top of the prior one) that relocates the memset:
>
>
> From 0bc71b78603500705aca77f82de8ed1fc595c4c3 Mon Sep 17 00:00:00 2001
> From: Kees Cook <kees@kernel.org>
> Date: Thu, 15 May 2025 08:48:24 -0700
> Subject: [PATCH] mm: vmalloc: Only zero-init on vrealloc shrink
>
> The common case is to grow reallocations, and since init_on_alloc will
> have already zeroed the whole allocation, we only need to zero when
> shrinking the allocation.
>
> Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Uladzislau Rezki <urezki@gmail.com>
> Cc: <linux-mm@kvack.org>
> ---
>  mm/vmalloc.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 74bd00fd734d..83bedb1559ac 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -4093,8 +4093,8 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
>          * would be a good heuristic for when to shrink the vm_area?
>          */
>         if (size <= old_size) {
> -               /* Zero out "freed" memory. */
> -               if (want_init_on_free())
> +               /* Zero out "freed" memory, potentially for future realloc. */
> +               if (want_init_on_free() || want_init_on_alloc(flags))
>                         memset((void *)p + size, 0, old_size - size);
>                 vm->requested_size = size;
>                 kasan_poison_vmalloc(p + size, old_size - size);
> @@ -4107,9 +4107,11 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
>         if (size <= alloced_size) {
>                 kasan_unpoison_vmalloc(p + old_size, size - old_size,
>                                        KASAN_VMALLOC_PROT_NORMAL);
> -               /* Zero out "alloced" memory. */
> -               if (want_init_on_alloc(flags))
> -                       memset((void *)p + old_size, 0, size - old_size);
> +               /*
> +                * No need to zero memory here, as unused memory will have
> +                * already been zeroed at initial allocation time or during
> +                * realloc shrink time.
> +                */
>                 vm->requested_size = size;

This vm->requested_size change you are adding should also fix the
kasan issue reported by syzbot ([0]).

  [0] https://lore.kernel.org/bpf/68213ddf.050a0220.f2294.0045.GAE@google.com/

>                 return (void *)p;
>         }
> --
> 2.34.1
>
>
>
> --
> Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 08:55:47AM -0700, Andrii Nakryiko wrote:
> On Thu, May 15, 2025 at 8:53 AM Kees Cook <kees@kernel.org> wrote:
> >
> > On Thu, May 15, 2025 at 08:47:47AM -0700, Kees Cook wrote:
> > > On Thu, May 15, 2025 at 09:12:25PM +0800, Shung-Hsi Yu wrote:
> > > > Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
> > > > support more granular vrealloc() sizing"[2]. To further zoom in the
> > >
> > > Can you try this patch? It's a clear bug fix, but if it doesn't improve
> > > things, I have another idea to rearrange the memset.
> >
> > Here's the patch (on top of the prior one) that relocates the memset:
> >
> >
> > From 0bc71b78603500705aca77f82de8ed1fc595c4c3 Mon Sep 17 00:00:00 2001
> > From: Kees Cook <kees@kernel.org>
> > Date: Thu, 15 May 2025 08:48:24 -0700
> > Subject: [PATCH] mm: vmalloc: Only zero-init on vrealloc shrink
> >
> > The common case is to grow reallocations, and since init_on_alloc will
> > have already zeroed the whole allocation, we only need to zero when
> > shrinking the allocation.
> >
> > Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
> > Signed-off-by: Kees Cook <kees@kernel.org>
> > ---
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Uladzislau Rezki <urezki@gmail.com>
> > Cc: <linux-mm@kvack.org>
> > ---
> >  mm/vmalloc.c | 12 +++++++-----
> >  1 file changed, 7 insertions(+), 5 deletions(-)
> >
> > diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> > index 74bd00fd734d..83bedb1559ac 100644
> > --- a/mm/vmalloc.c
> > +++ b/mm/vmalloc.c
> > @@ -4093,8 +4093,8 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
> >          * would be a good heuristic for when to shrink the vm_area?
> >          */
> >         if (size <= old_size) {
> > -               /* Zero out "freed" memory. */
> > -               if (want_init_on_free())
> > +               /* Zero out "freed" memory, potentially for future realloc. */
> > +               if (want_init_on_free() || want_init_on_alloc(flags))
> >                         memset((void *)p + size, 0, old_size - size);
> >                 vm->requested_size = size;
> >                 kasan_poison_vmalloc(p + size, old_size - size);
> > @@ -4107,9 +4107,11 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
> >         if (size <= alloced_size) {
> >                 kasan_unpoison_vmalloc(p + old_size, size - old_size,
> >                                        KASAN_VMALLOC_PROT_NORMAL);
> > -               /* Zero out "alloced" memory. */
> > -               if (want_init_on_alloc(flags))
> > -                       memset((void *)p + old_size, 0, size - old_size);
> > +               /*
> > +                * No need to zero memory here, as unused memory will have
> > +                * already been zeroed at initial allocation time or during
> > +                * realloc shrink time.
> > +                */
> >                 vm->requested_size = size;
> 
> This vm->requested_size change you are adding should also fix the
> kasan issue reported by syzbot ([0]).
> 
>   [0] https://lore.kernel.org/bpf/68213ddf.050a0220.f2294.0045.GAE@google.com/

Yes, this looks very much like the kasan oops that motivated the initial
patch:

https://lore.kernel.org/all/20250408192503.6149a816@outsider.home/

-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Eduard Zingerman]

On Thu, 2025-05-15 at 08:47 -0700, Kees Cook wrote:
> On Thu, May 15, 2025 at 09:12:25PM +0800, Shung-Hsi Yu wrote:
> > Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
> > support more granular vrealloc() sizing"[2]. To further zoom in the
> 
> Can you try this patch? It's a clear bug fix, but if it doesn't improve
> things, I have another idea to rearrange the memset.

I tried this patch on top of the commit 82f2b0b97b36 ("Linux 6.15-rc6").
W/o the patch I observe the slowdown, test times out after 120 seconds,
with the patch test finishes in 3 seconds.

[...]

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Kees Cook]

On Thu, May 15, 2025 at 11:31:09AM -0700, Eduard Zingerman wrote:
> On Thu, 2025-05-15 at 08:47 -0700, Kees Cook wrote:
> > On Thu, May 15, 2025 at 09:12:25PM +0800, Shung-Hsi Yu wrote:
> > > Bisect was done by Pawan and got to commit a0309faf1cb0 "mm: vmalloc:
> > > support more granular vrealloc() sizing"[2]. To further zoom in the
> > 
> > Can you try this patch? It's a clear bug fix, but if it doesn't improve
> > things, I have another idea to rearrange the memset.
> 
> I tried this patch on top of the commit 82f2b0b97b36 ("Linux 6.15-rc6").
> W/o the patch I observe the slowdown, test times out after 120 seconds,
> with the patch test finishes in 3 seconds.

Well that's pretty definitive! Thank you! I'll send out a properly
formatted patch. May I add your Tested-by?

-- 
Kees Cook

Re: [REGRESSION] bpf verifier slowdown due to vrealloc() change since 6.15-rc6

[Eduard Zingerman]

On Thu, 2025-05-15 at 14:36 -0700, Kees Cook wrote:

[...]

> Well that's pretty definitive! Thank you! I'll send out a properly
> formatted patch. May I add your Tested-by?

Sure, thank you.