[PATCHv2 bpf-next 0/2] bpf: Disable private stacks for uprobe programs

[PATCHv2 bpf-next 0/2] bpf: Disable private stacks for uprobe programs

[Jiri Olsa]

hi,
uprobe programs can be preempted and re-entered on the same CPU. Since
the private stack is shared per program and CPU, a nested invocation can
overwrite the preempted invocation's stack.

This patch disables private stacks for KPROBE programs except kprobe multi
and kprobe-session programs, identified by the BPF_TRACE_KPROBE_MULTI and
BPF_TRACE_KPROBE_SESSION attach types. Standard kprobes lack an attach type,
and uprobes use different attach types or none, so they run without private
stacks.

It also rejects multi and session programs through the perf-event attach
path, which lacks their required run context and could allow a uprobe to
reach a private-stack tail-call target [1].

thanks,
jirka


v1: https://lore.kernel.org/bpf/20261103093400.112178-1-jolsa@kernel.org/

v2 changes:
- drop the BPF_TRACE_UPROBE attach type and fail perf event kprobe
  attach for kprobe-multi/uprobe-multi [Alexei]


[1] https://lore.kernel.org/bpf/aus-b8Lo2oCw_cGd@krava/
---
Jiri Olsa (2):
      bpf: Reject multi kprobe programs in perf event attach
      bpf: Disable private stacks for untagged kprobe programs

 kernel/bpf/verifier.c                                      |  9 +++++++
 kernel/events/core.c                                       | 13 ++++++++--
 tools/testing/selftests/bpf/progs/verifier_large_stack.c   |  2 +-
 tools/testing/selftests/bpf/progs/verifier_private_stack.c | 38 +++++++++++++++++++++++++++--
 4 files changed, 57 insertions(+), 5 deletions(-)

[PATCHv2 bpf-next 1/2] bpf: Reject multi kprobe programs in perf event attach

[Jiri Olsa]

PERF_EVENT_IOC_SET_BPF allows attaching multi and session kprobe or
uprobe programs through the standard perf event path. These programs
require attach-specific run context, which this path does not provide.
Attaching them can also let a uprobe reach a private-stack program
through a tail call.

Reject multi and session programs in __perf_event_set_bpf_prog().
Keep the existing sleepable-program check for standard kprobe and
uprobe programs.

Suggested-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 kernel/events/core.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 457d75ea7142..919f43cdb473 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -11835,9 +11835,18 @@ static int __perf_event_set_bpf_prog(struct perf_event *event,
 	    (is_syscall_tp && prog->type != BPF_PROG_TYPE_TRACEPOINT))
 		return -EINVAL;
 
-	if (prog->type == BPF_PROG_TYPE_KPROBE && prog->sleepable && !is_uprobe)
+	if (prog->type == BPF_PROG_TYPE_KPROBE) {
 		/* only uprobe programs are allowed to be sleepable */
-		return -EINVAL;
+		if (prog->sleepable && !is_uprobe)
+			return -EINVAL;
+
+		/* only standard kprobe/uprobe programs are allowed to attach */
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ||
+		    prog->expected_attach_type == BPF_TRACE_KPROBE_SESSION ||
+		    prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI ||
+		    prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION)
+			return -EINVAL;
+	}
 
 	if (prog->type == BPF_PROG_TYPE_TRACEPOINT && prog->sleepable) {
 		/*
-- 
2.43.0

[PATCHv2 bpf-next 2/2] bpf: Disable private stacks for untagged kprobe programs

[Jiri Olsa]

Uprobe programs can be preempted and re-entered on the same CPU. Since
the private stack is shared per program and CPU, another invocation can
overwrite the preempted invocation's stack.

Programs without a multi or session attach type may be standard kprobes
or uprobes. Disable private stacks for them, and retain private stacks
only for kprobe multi and session programs, which have attach types that
identify them as kprobes.

Update verifier tests that rely on private stacks to use kprobe.multi,
and add coverage that untagged kprobe and uprobe programs do not use
private stacks.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 kernel/bpf/verifier.c                         |  9 +++++
 .../bpf/progs/verifier_large_stack.c          |  2 +-
 .../bpf/progs/verifier_private_stack.c        | 38 ++++++++++++++++++-
 3 files changed, 46 insertions(+), 3 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b7329d164771..1b58a1796a2e 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5749,6 +5749,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
 	 */
 	switch (prog->type) {
 	case BPF_PROG_TYPE_KPROBE:
+		/*
+		 * Uprobes are preemptible and lack recursion protection. Classic
+		 * kprobe and uprobe progs have no expected attach type, so enable
+		 * private stacks only for kprobe.multi and kprobe.session progs.
+		 */
+		if (prog->expected_attach_type != BPF_TRACE_KPROBE_MULTI &&
+		    prog->expected_attach_type != BPF_TRACE_KPROBE_SESSION)
+			return NO_PRIV_STACK;
+		fallthrough;
 	case BPF_PROG_TYPE_TRACEPOINT:
 	case BPF_PROG_TYPE_PERF_EVENT:
 	case BPF_PROG_TYPE_RAW_TRACEPOINT:
diff --git a/tools/testing/selftests/bpf/progs/verifier_large_stack.c b/tools/testing/selftests/bpf/progs/verifier_large_stack.c
index 7e4f5e3463a2..cbabb41d973c 100644
--- a/tools/testing/selftests/bpf/progs/verifier_large_stack.c
+++ b/tools/testing/selftests/bpf/progs/verifier_large_stack.c
@@ -243,7 +243,7 @@ static void priv_stack_frame_2048(void)
 "	::: __clobber_all);
 }
 
-SEC("kprobe")
+SEC("kprobe.multi")
 __description("private stack: two frames of 2048 bytes")
 __load_if_large_stack()
 __arch_x86_64
diff --git a/tools/testing/selftests/bpf/progs/verifier_private_stack.c b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
index aa15361bcca5..0ce07bf6b566 100644
--- a/tools/testing/selftests/bpf/progs/verifier_private_stack.c
+++ b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
@@ -22,7 +22,7 @@ struct {
 	__type(value, struct elem);
 } array SEC(".maps");
 
-SEC("kprobe")
+SEC("kprobe.multi")
 __description("Private stack, single prog")
 __success
 __arch_x86_64
@@ -110,6 +110,40 @@ int no_private_stack_stack_args(void)
 }
 #endif
 
+SEC("kprobe")
+__description("No private stack, untagged kprobe")
+__success
+__arch_x86_64
+__jited("	subq	$0x100, %rsp")
+__arch_arm64
+__jited("	sub	sp, sp, #0x100")
+__naked void no_private_stack_kprobe(void)
+{
+	asm volatile ("			\
+	r1 = 42;			\
+	*(u64 *)(r10 - 256) = r1;	\
+	r0 = 0;				\
+	exit;				\
+"	::: __clobber_all);
+}
+
+SEC("uprobe")
+__description("No private stack, uprobe")
+__success
+__arch_x86_64
+__jited("	subq	$0x100, %rsp")
+__arch_arm64
+__jited("	sub	sp, sp, #0x100")
+__naked void no_private_stack_uprobe(void)
+{
+	asm volatile ("			\
+	r1 = 42;			\
+	*(u64 *)(r10 - 256) = r1;	\
+	r0 = 0;				\
+	exit;				\
+"	::: __clobber_all);
+}
+
 __used
 __naked static void cumulative_stack_depth_subprog(void)
 {
@@ -123,7 +157,7 @@ __naked static void cumulative_stack_depth_subprog(void)
 	: __clobber_all);
 }
 
-SEC("kprobe")
+SEC("kprobe.multi")
 __description("Private stack, subtree > MAX_BPF_STACK")
 __success
 __log_level(4) __msg("stack depth max 512")
-- 
2.43.0

Re: [PATCHv2 bpf-next 2/2] bpf: Disable private stacks for untagged kprobe programs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] bpf: bpf_enable_priv_stack() allows private stack on uprobe
  via IOC_SET_BPF
- [Low] bpf: missing Fixes trailer for commit a76ab5731e32 in commit
  message
--

commit f0768285bc0b64d913853bd20bc1d61e4c54eed4
Author: Jiri Olsa <jolsa@kernel.org>

> bpf: Disable private stacks for untagged kprobe programs
>
> Uprobe programs can be preempted and re-entered on the same CPU. Since
> the private stack is shared per program and CPU, another invocation can
> overwrite the preempted invocation's stack.
>
> Programs without a multi or session attach type may be standard kprobes
> or uprobes. Disable private stacks for them, and retain private stacks
> only for kprobe multi and session programs, which have attach types that
> identify them as kprobes.
>
> Update verifier tests that rely on private stacks to use kprobe.multi,
> and add coverage that untagged kprobe and uprobe programs do not use
> private stacks.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>

[Severity: Low]
Should this commit include a Fixes tag for a76ab5731e32 ("bpf: Find
eligible subprogs for private stack support")?

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index b7329d164771c..1b58a1796a2e4 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -5749,6 +5749,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
>  	 */
>  	switch (prog->type) {
>  	case BPF_PROG_TYPE_KPROBE:
> +		/*
> +		 * Uprobes are preemptible and lack recursion protection. Classic
> +		 * kprobe and uprobe progs have no expected attach type, so enable
> +		 * private stacks only for kprobe.multi and kprobe.session progs.
> +		 */
> +		if (prog->expected_attach_type != BPF_TRACE_KPROBE_MULTI &&
> +		    prog->expected_attach_type != BPF_TRACE_KPROBE_SESSION)
> +			return NO_PRIV_STACK;

[Severity: Medium]
Can a BPF_PROG_TYPE_KPROBE program with a private stack still be attached
to a classic uprobe through PERF_EVENT_IOC_SET_BPF?

At load time, bpf_prog_load_check_attach() accepts BPF_TRACE_KPROBE_MULTI
or BPF_TRACE_KPROBE_SESSION on any BPF_PROG_TYPE_KPROBE program (including
sleepable ones, since can_be_sleepable() only checks prog->type ==
BPF_PROG_TYPE_KPROBE).  While BPF_LINK_CREATE checks expected_attach_type
via bpf_prog_attach_check_attach_type(), attaching via
ioctl(perf_event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd) ->
__perf_event_set_bpf_prog() only checks prog->type:

kernel/events/core.c:__perf_event_set_bpf_prog() {
    ...
	if (((is_kprobe || is_uprobe) && prog->type != BPF_PROG_TYPE_KPROBE) ||
	    (is_tracepoint && prog->type != BPF_PROG_TYPE_TRACEPOINT) ||
	    (is_syscall_tp && prog->type != BPF_PROG_TYPE_TRACEPOINT))
		return -EINVAL;
    ...
}

A BPF_PROG_TYPE_KPROBE program loaded with BPF_TRACE_KPROBE_MULTI or
BPF_TRACE_KPROBE_SESSION therefore still gets a per-CPU private stack and
can be attached to a uprobe perf_event, where __uprobe_perf_func() ->
bpf_prog_run_array_uprobe() runs it in preemptible (and optionally
sleepable) context without recursion protection, allowing preemption and
re-entry on the same CPU to overwrite the active invocation's private
stack.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106164043.546879-3-jolsa@kernel.org?part=1

Re: [PATCHv2 bpf-next 2/2] bpf: Disable private stacks for untagged kprobe programs

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 3195 bytes --]

> bpf: Disable private stacks for untagged kprobe programs

TL;DR: Disabling private stacks for classic kprobe programs may make
previously loadable programs fail verification (-EACCES) once frames count
combined against the stack limit, e.g. on powerpc64; the commit message
doesn't mention it.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> @@ -5749,6 +5749,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
>  	 */
>  	switch (prog->type) {
>  	case BPF_PROG_TYPE_KPROBE:
> +		/*
> +		 * Uprobes are preemptible and lack recursion protection. Classic
> +		 * kprobe and uprobe progs have no expected attach type, so enable
> +		 * private stacks only for kprobe.multi and kprobe.session progs.
> +		 */
> +		if (prog->expected_attach_type != BPF_TRACE_KPROBE_MULTI &&
> +		    prog->expected_attach_type != BPF_TRACE_KPROBE_SESSION)
> +			return NO_PRIV_STACK;
> +		fallthrough;

Classic SEC("kprobe"), kretprobe and ksyscall programs run through
trace_call_bpf(), which has cant_sleep() and the per-CPU bpf_prog_active
guard, so they never had the re-entry problem.  They lose private stacks
only because they cannot be told apart from uprobes.

Does this change which classic kprobe programs the verifier accepts?
check_call_depth() treats a frame separately when private stacks are on:

    if (si[s].priv_stack_mode == PRIV_STACK_ADAPTIVE) {
            depth = round_up_stack_depth(env, si[s].stack_depth);
            ...
            continue;
    }
    depth = dinfo[s].depth + subprog_stack_size(env, s);
    ...
    if (depth > env->stack_limit) {
            verbose(env, "combined stack size of %d calls is %d. Too large\n", ...

subprog_stack_size() returns 0 for an ADAPTIVE frame, so before this patch
the frames of a kprobe program were not added together.  Now they are, and
a program that loaded before can fail with -EACCES.

For example, on powerpc64 bpf_jit_supports_private_stack() returns
IS_ENABLED(CONFIG_PPC64) and there is no bpf_jit_supports_large_stack(), so
bpf_prog_stack_limit() returns MAX_BPF_STACK (512).  A SEC("kprobe")
program with a 512-byte main frame calling a subprog with a 32-byte frame
used to load, and would now fail with "combined stack size of 2 calls is
544. Too large".  This is the shape of the "Private stack, subtree >
MAX_BPF_STACK" selftest that this patch moves to kprobe.multi.

On x86-64 and arm64 the limit becomes MAX_BPF_STACK_JIT (2048) combined,
where it used to be 2048 per frame.  If this is backported to a tree
without MAX_BPF_STACK_JIT, x86-64 and arm64 would fall back to 512
combined for these programs.

The commit message does not mention that existing classic kprobe programs
can now fail verification, or that users need to move to kprobe.multi or
kprobe.session to keep the larger stack budget.  Is this rejection
intended?  If so, could the changelog say so?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42256932632

Re: [PATCHv2 bpf-next 0/2] bpf: Disable private stacks for uprobe programs

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (master)
by Alexei Starovoitov <ast@kernel.org>:

On Fri,  6 Nov 2026 17:40:41 +0100 you wrote:
> hi,
> uprobe programs can be preempted and re-entered on the same CPU. Since
> the private stack is shared per program and CPU, a nested invocation can
> overwrite the preempted invocation's stack.
> 
> This patch disables private stacks for KPROBE programs except kprobe multi
> and kprobe-session programs, identified by the BPF_TRACE_KPROBE_MULTI and
> BPF_TRACE_KPROBE_SESSION attach types. Standard kprobes lack an attach type,
> and uprobes use different attach types or none, so they run without private
> stacks.
> 
> [...]

Here is the summary with links:
  - [PATCHv2,bpf-next,1/2] bpf: Reject multi kprobe programs in perf event attach
    https://git.kernel.org/bpf/bpf-next/c/6a6beb501ee7
  - [PATCHv2,bpf-next,2/2] bpf: Disable private stacks for untagged kprobe programs
    https://git.kernel.org/bpf/bpf-next/c/0eaeda0d2075

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html