[PATCH bpf v3 0/2] disable private stack for sleepable programs

[PATCH bpf v3 0/2] disable private stack for sleepable programs

[Christian Simon]

A JITed BPF program can use one private stack per program and CPU.
Sleepable programs can be preempted, allowing another task to run the
same program on the same CPU. The second invocation then reuses and can
overwrite the first invocation's private stack.

This patch series disables private stacks for sleepable programs and
adds a selftest covering that stack corruption.

---
Changes in v3:
- Target sleepable only.
- Add the powerpc Fixes tag.
- Split fix and selftests

v2: https://lore.kernel.org/bpf/20260818203234.1142913-1-simon@swine.de/

Christian Simon (2):
  bpf: disable private stack for sleepable programs
  selftests/bpf: verify preemptible uprobes avoid private stack

 kernel/bpf/verifier.c                         |   9 +
 .../bpf/prog_tests/uprobe_sleepable_stack.c   | 165 ++++++++++++++++++
 .../bpf/progs/uprobe_sleepable_stack.c        |  89 ++++++++++
 3 files changed, 263 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c


base-commit: 75b0a6db4300e4c2c9e97a0848deaa7acfb42fb7
-- 
2.54.0

[PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Christian Simon]

A JITed BPF program can use one private stack per program and CPU.
Sleepable programs can be preempted, allowing another task to run the
same program on the same CPU. The second invocation then reuses and can
overwrite the first invocation's private stack.

Disable private stack for sleepable programs so they use the regular kernel
stack, which handles preemption correctly. This change is intentionally
limited to programs marked sleepable; preemptible non-sleepable dispatch
paths require separate protection.

Fixes: 7d1cd70d4b16 ("bpf, x86: Support private stack in jit")
Fixes: 6c17a882d380 ("bpf, arm64: JIT support for private stack")
Fixes: 156d985123b6 ("powerpc64/bpf: Implement JIT support for private stack")
Cc: stable@vger.kernel.org
Signed-off-by: Christian Simon <simon@swine.de>
---
 kernel/bpf/verifier.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 5e37ca75e5c4..038753ef07a9 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5237,6 +5237,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
 	if (!bpf_jit_supports_private_stack())
 		return NO_PRIV_STACK;
 
+	/*
+	 * Sleepable programs can be preempted, allowing another task to run
+	 * the same program on the same CPU. Since private stack is per-CPU
+	 * and per-program, the second invocation would corrupt the first's
+	 * stack. Disable private stack for sleepable programs.
+	 */
+	if (prog->sleepable)
+		return NO_PRIV_STACK;
+
 	/* bpf_prog_check_recur() checks all prog types that use bpf trampoline
 	 * while kprobe/tp/perf_event/raw_tp don't use trampoline hence checked
 	 * explicitly.
-- 
2.54.0

[PATCH bpf v3 2/2] selftests/bpf: verify preemptible uprobes avoid private stack

[Christian Simon]

Pin two threads to one CPU and overlap invocations of the same uprobe
program. Verify that both invocations execute and that the first
invocation's stack frame is not corrupted by the second.

Cover sleepable classic and multi-uprobe programs with both 64-byte
stack frames, which select private stack when eligible, and small stack
frames as controls. Skip the test when preemption, JIT, or architecture
private-stack support is unavailable.

Signed-off-by: Christian Simon <simon@swine.de>
---

I have noticed the "volatile" warning, I do think it is fine for this
selftests purposes.


 .../bpf/prog_tests/uprobe_sleepable_stack.c   | 165 ++++++++++++++++++
 .../bpf/progs/uprobe_sleepable_stack.c        |  89 ++++++++++
 2 files changed, 254 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c b/tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c
new file mode 100644
index 000000000000..a18970d2c584
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test that preemptible uprobe programs don't use private stack.
+ *
+ * Private stack is per-CPU and per-program, so preemption during a program's
+ * execution would allow another task to corrupt the stack. The verifier must
+ * disable private stack for programs whose invocation can be preempted.
+ *
+ * This test overlaps two invocations of the same uprobe program on the same
+ * CPU and verifies:
+ * 1. Both invocations execute
+ * 2. No stack corruption occurs (each task has its own stack frame)
+ */
+#include <pthread.h>
+#include <sched.h>
+#include <unistd.h>
+
+#include <test_progs.h>
+#include "uprobe_sleepable_stack.skel.h"
+
+static noinline void uprobe_sleepable_stack_trigger(void)
+{
+	asm volatile("");
+}
+
+static void *trigger_uprobe(void *arg)
+{
+	uprobe_sleepable_stack_trigger();
+	return NULL;
+}
+
+static void reset_state(struct uprobe_sleepable_stack *skel)
+{
+	skel->bss->ready = 0;
+	skel->bss->release = 0;
+	skel->bss->executions = 0;
+	skel->bss->corruptions = 0;
+	skel->bss->loop_exhausted = 0;
+}
+
+static void assert_results(struct uprobe_sleepable_stack *skel)
+{
+	ASSERT_EQ(skel->bss->loop_exhausted, 0, "loop_exhausted");
+	ASSERT_EQ(skel->bss->executions, 2, "executions");
+	ASSERT_EQ(skel->bss->corruptions, 0, "corruptions");
+}
+
+static void run_test(struct uprobe_sleepable_stack *skel, bool multi,
+		     bool large_stack)
+{
+	LIBBPF_OPTS(bpf_uprobe_opts, opts);
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, multi_opts);
+	const char *binary = "/proc/self/exe";
+	struct bpf_program *prog;
+	struct bpf_link *link;
+	pthread_t thread;
+	long link_err;
+	int err, i;
+
+	reset_state(skel);
+
+	if (multi) {
+		prog = large_stack ? skel->progs.uprobe_multi_sleepable_large_stack :
+					     skel->progs.uprobe_multi_sleepable_small_stack;
+		link = bpf_program__attach_uprobe_multi(
+			prog, 0, binary, "uprobe_sleepable_stack_trigger",
+			&multi_opts);
+	} else {
+		opts.func_name = "uprobe_sleepable_stack_trigger";
+		prog = large_stack ? skel->progs.uprobe_sleepable_large_stack :
+					     skel->progs.uprobe_sleepable_small_stack;
+		link = bpf_program__attach_uprobe_opts(prog, 0, binary, 0,
+						       &opts);
+	}
+
+	link_err = libbpf_get_error(link);
+	if (link_err == -EOPNOTSUPP) {
+		test__skip();
+		return;
+	}
+	if (!ASSERT_OK_PTR(link, "attach_uprobe"))
+		return;
+
+	err = pthread_create(&thread, NULL, trigger_uprobe, NULL);
+	if (!ASSERT_OK(err, "pthread_create"))
+		goto cleanup;
+
+	for (i = 0; i < 10000; i++) {
+		if (__atomic_load_n(&skel->bss->ready, __ATOMIC_ACQUIRE))
+			break;
+		usleep(1000);
+	}
+
+	if (ASSERT_LT(i, 10000, "first_uprobe_ready"))
+		uprobe_sleepable_stack_trigger();
+	__atomic_store_n(&skel->bss->release, 1, __ATOMIC_RELEASE);
+
+	err = pthread_join(thread, NULL);
+	if (!ASSERT_OK(err, "pthread_join"))
+		goto cleanup;
+
+	assert_results(skel);
+
+cleanup:
+	bpf_link__destroy(link);
+}
+
+void test_uprobe_sleepable_stack(void)
+{
+	struct uprobe_sleepable_stack *skel = NULL;
+	cpu_set_t old_mask, mask;
+	bool affinity_set = false;
+	int cpu;
+
+#if !defined(__x86_64__) && !defined(__aarch64__) && !defined(__powerpc64__)
+	test__skip();
+	return;
+#endif
+	if (!env.jit_enabled) {
+		test__skip();
+		return;
+	}
+
+	if (!ASSERT_OK(sched_getaffinity(0, sizeof(old_mask), &old_mask),
+		       "get_affinity"))
+		return;
+
+	CPU_ZERO(&mask);
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+		if (CPU_ISSET(cpu, &old_mask)) {
+			CPU_SET(cpu, &mask);
+			break;
+		}
+	}
+	if (!ASSERT_LT(cpu, CPU_SETSIZE, "available_cpu"))
+		return;
+	/* Both triggers must run on the same CPU to test stack sharing */
+	if (!ASSERT_OK(sched_setaffinity(0, sizeof(mask), &mask),
+		       "set_affinity"))
+		return;
+	affinity_set = true;
+
+	skel = uprobe_sleepable_stack__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "skel_open"))
+		goto cleanup;
+	if (!skel->kconfig->CONFIG_PREEMPTION) {
+		test__skip();
+		goto cleanup;
+	}
+
+	if (test__start_subtest("sleepable_classic_large"))
+		run_test(skel, false, true);
+	if (test__start_subtest("sleepable_classic_small"))
+		run_test(skel, false, false);
+	if (test__start_subtest("sleepable_multi_large"))
+		run_test(skel, true, true);
+	if (test__start_subtest("sleepable_multi_small"))
+		run_test(skel, true, false);
+
+cleanup:
+	uprobe_sleepable_stack__destroy(skel);
+	if (affinity_set)
+		ASSERT_OK(sched_setaffinity(0, sizeof(old_mask), &old_mask),
+			  "restore_affinity");
+}
diff --git a/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c b/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c
new file mode 100644
index 000000000000..07e0ff548720
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c
@@ -0,0 +1,89 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+extern bool CONFIG_PREEMPTION __kconfig __weak;
+
+volatile int ready;
+volatile int release;
+volatile int executions;
+volatile int corruptions;
+int loop_exhausted;
+
+static long wait_for_release(__u32 index, void *ctx)
+{
+	return release ? 1 : 0;
+}
+
+/*
+ * Test that preemptible uprobes don't use private stack. A 64-byte frame would
+ * normally trigger private stack selection, but programs whose invocation can
+ * be preempted must not use it. Both invocations must execute without
+ * corruption.
+ */
+static __always_inline int run_stack_test(volatile __u64 *stack)
+{
+	__u64 id = bpf_get_current_pid_tgid();
+	int seq;
+
+	if (!CONFIG_PREEMPTION)
+		return 0;
+
+	stack[0] = id;
+	seq = executions;
+	executions = seq + 1;
+	/*
+	 * If private stack were used, an unguarded second invocation would
+	 * overwrite the first invocation's frame. With regular stack, each task
+	 * has its own stack frame.
+	 */
+	if (seq) {
+		stack[0] = ~id;
+		return 0;
+	}
+
+	ready = 1;
+	bpf_loop(1 << 23, wait_for_release, NULL, 0);
+	if (!release)
+		loop_exhausted = 1;
+	if (stack[0] != id)
+		corruptions++;
+
+	return 0;
+}
+
+SEC("uprobe.s//proc/self/exe:uprobe_sleepable_stack_trigger")
+int uprobe_sleepable_large_stack(struct pt_regs *ctx)
+{
+	/* A 64-byte frame would select private stack for non-sleepable progs */
+	volatile __u64 stack[8] = {};
+
+	return run_stack_test(stack);
+}
+
+SEC("uprobe.s//proc/self/exe:uprobe_sleepable_stack_trigger")
+int uprobe_sleepable_small_stack(struct pt_regs *ctx)
+{
+	/* Small stack as control - never triggers private stack */
+	volatile __u64 stack[1] = {};
+
+	return run_stack_test(stack);
+}
+
+SEC("uprobe.multi.s//proc/self/exe:uprobe_sleepable_stack_trigger")
+int uprobe_multi_sleepable_large_stack(struct pt_regs *ctx)
+{
+	volatile __u64 stack[8] = {};
+
+	return run_stack_test(stack);
+}
+
+SEC("uprobe.multi.s//proc/self/exe:uprobe_sleepable_stack_trigger")
+int uprobe_multi_sleepable_small_stack(struct pt_regs *ctx)
+{
+	volatile __u64 stack[1] = {};
+
+	return run_stack_test(stack);
+}
+
+char LICENSE[] SEC("license") = "GPL";
-- 
2.54.0

Re: [PATCH bpf v3 2/2] selftests/bpf: verify preemptible uprobes avoid private stack

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Spurious test failure (flakiness) due to insufficiently small timeout limit in bpf_loop.
--

--- Patch [2]: [PATCH bpf v3 2/2] selftests/bpf: verify preemptible uprobes avoid private stack ---
commit c17909e48f5484c8e9bf3341a1fc87b98ad29988
Author: Christian Simon <simon@swine.de>

selftests/bpf: verify preemptible uprobes avoid private stack
    
Pin two threads to one CPU and overlap invocations of the same uprobe
program. Verify that both invocations execute and that the first
invocation's stack frame is not corrupted by the second.

> diff --git a/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c b/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c
> index 0000000000000..07e0ff5487208
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c
[ ... ]
> +static __always_inline int run_stack_test(volatile __u64 *stack)
> +{
[ ... ]
> +	ready = 1;
> +	bpf_loop(1 << 23, wait_for_release, NULL, 0);
> +	if (!release)
> +		loop_exhausted = 1;

[Severity: Medium]
Could this iteration limit cause spurious test failures under load?

In a heavily loaded CI environment, the main thread's usleep(1000) can be
delayed significantly (for example, if timer granularity is 10ms or due to
hypervisor latency). On a fast CPU, 8.38M iterations of this trivial callback
might complete in just a few milliseconds, exhausting the loop before the
main thread wakes up and sets release.

If this happens, loop_exhausted is set to 1 and the selftest fails.
Is it possible to safely extend this wait time by wrapping the bpf_loop call
in an outer bounded loop?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260822225444.2774461-1-simon@swine.de?part=2

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Non-sleepable BPF uprobe programs remain vulnerable to per-CPU private stack corruption.
--

--- Patch [1]: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs ---
commit 5b27c96c855b5a6df31f1d2543a92540c69a5d98
Author: Christian Simon <simon@swine.de>

bpf: disable private stack for sleepable programs

This commit disables the use of private stacks for sleepable BPF programs
since they can be preempted, which would allow a second invocation of the
program on the same CPU to corrupt the first invocation's stack. The fix
forces sleepable programs to use the regular kernel stack which handles
preemption correctly.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5e37ca75e5c4a..038753ef07a99 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -5237,6 +5237,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
>  	if (!bpf_jit_supports_private_stack())
>  		return NO_PRIV_STACK;
>  
> +	/*
> +	 * Sleepable programs can be preempted, allowing another task to run
> +	 * the same program on the same CPU. Since private stack is per-CPU
> +	 * and per-program, the second invocation would corrupt the first's
> +	 * stack. Disable private stack for sleepable programs.
> +	 */
> +	if (prog->sleepable)
> +		return NO_PRIV_STACK;
> +
>  	/* bpf_prog_check_recur() checks all prog types that use bpf trampoline
>  	 * while kprobe/tp/perf_event/raw_tp don't use trampoline hence checked
>  	 * explicitly.

[Severity: High]
This is a pre-existing issue and was not introduced by this patch, but does
this leave non-sleepable uprobes vulnerable to the same stack corruption?

The patch correctly fixes sleepable programs, but the commit message mentions
that preemptible non-sleepable dispatch paths require separate protection.
However, non-sleepable uprobes are granted private stacks because they fall
under BPF_PROG_TYPE_KPROBE in the switch statement immediately below this code.

If we look at bpf_prog_run_array_uprobe() in include/linux/bpf.h,
non-sleepable uprobes are executed with only rcu_read_lock():

    if (!prog->sleepable)
        rcu_read_lock();
        
    run_ctx.bpf_cookie = item->bpf_cookie;
    ret &= run_prog(prog, ctx);
    item++;
    
    if (!prog->sleepable)
        rcu_read_unlock();

Under CONFIG_PREEMPT_RCU and CONFIG_PREEMPT, rcu_read_lock() does not disable
preemption. Could a task executing this non-sleepable uprobe be preempted,
allowing another task on the same CPU to trigger the same uprobe and overwrite
its per-CPU private stack?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260822225444.2774461-1-simon@swine.de?part=1

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4332 bytes --]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5e37ca75e5c4..038753ef07a9 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -5237,6 +5237,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
>  	if (!bpf_jit_supports_private_stack())
>  		return NO_PRIV_STACK;
>
> +	/*
> +	 * Sleepable programs can be preempted, allowing another task to run
> +	 * the same program on the same CPU. Since private stack is per-CPU
> +	 * and per-program, the second invocation would corrupt the first's
> +	 * stack. Disable private stack for sleepable programs.
> +	 */

Is this comment accurate for all the program types below?

For BPF_PROG_TYPE_TRACING / LSM / STRUCT_OPS, private stack is only
granted when bpf_prog_check_recur(prog) holds, and that same predicate
makes bpf_trampoline_enter() install __bpf_prog_enter_sleepable_recur(),
which bumps the per-CPU prog->active counter under migrate_disable() and
skips the nested invocation.

Similarly, the sleepable raw-tracepoint runner __bpf_trace_run() calls
bpf_prog_get_recursion_context().

The statement "the second invocation would corrupt the first's stack"
would be true for sleepable uprobe programs run via uprobe_prog_run(),
which takes no recursion context. Could the comment explain why the check
is placed before the switch rather than in the case arm that actually
needs it?

> +	if (prog->sleepable)
> +		return NO_PRIV_STACK;
> +

Does placing this check before the prog-type switch have unintended
consequences?

The check strips the private stack from BPF_PROG_TYPE_TRACING / LSM /
STRUCT_OPS even though those paths are already protected. For these types,
bpf_trampoline_enter() consults bpf_prog_check_recur(prog):

kernel/bpf/trampoline.c:bpf_trampoline_enter() {
    if (bpf_prog_check_recur(prog))
            return sleepable ? __bpf_prog_enter_sleepable_recur :
                    __bpf_prog_enter_recur;

and __bpf_prog_enter_sleepable_recur() does migrate_disable() and then
bpf_prog_get_recursion_context(prog), which uses a per-CPU prog->active
counter. A second task entering the same program on the same CPU sees
active != 1 and is skipped, so it can never reach the private stack.

The real hole appears to be below the new check: BPF_PROG_TYPE_KPROBE /
TRACEPOINT / PERF_EVENT / RAW_TRACEPOINT return PRIV_STACK_ADAPTIVE
unconditionally. For sleepable uprobes (the only sleepable case allowed by
can_be_sleepable()), the uprobe runner takes no recursion context:

kernel/trace/bpf_trace.c:uprobe_prog_run() {
    if (sleepable)
            rcu_read_lock_trace();
    ...
    migrate_disable();
    err = bpf_prog_run(link->link.prog, regs);

so a sleepable uprobe can genuinely be re-entered on the same CPU.

Moving the check into the case arm (BPF_PROG_TYPE_KPROBE through
BPF_PROG_TYPE_RAW_TRACEPOINT) would close the uprobe hole without taking
the private stack away from the trampoline types that already handle
re-entry.

There's also a user-visible side effect: priv_stack_mode decides how
check_max_stack_depth_subprog() budgets stack. PRIV_STACK_ADAPTIVE checks
each subprog on its own against MAX_BPF_STACK; NO_PRIV_STACK accumulates
depth += subprog_depth across bpf2bpf frames and checks the combined total
against MAX_BPF_STACK (512).

Any existing sleepable program whose bpf2bpf subtree exceeds 512 bytes
total - SEC("fentry.s/..."), SEC("fexit.s/..."), SEC("fmod_ret.s/..."),
SEC("uprobe.s/...") - now fails to load with -EACCES and "combined stack
size of %d calls is %d. Too large", where it verified fine before. The
selftest at tools/testing/selftests/bpf/progs/verifier_private_stack.c:86
("Private stack, subtree > MAX_BPF_STACK", 512-byte main + 32-byte
subprog, __success) has exactly this shape; it only survives because it
uses the non-sleepable SEC("kprobe"). With Cc: stable@vger.kernel.org,
this would break working programs on a stable update.

>  	/* bpf_prog_check_recur() checks all prog types that use bpf trampoline
>  	 * while kprobe/tp/perf_event/raw_tp don't use trampoline hence checked
>  	 * explicitly.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32604492082

Re: [PATCH bpf v3 2/2] selftests/bpf: verify preemptible uprobes avoid private stack

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 3109 bytes --]

> diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c b/tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c
> new file mode 100644
> index 000000000000..a18970d2c584
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_sleepable_stack.c

A subsystem pattern flags this as potentially concerning: the BPF
selftests guide asks whether a new test file for a narrow variant of
behavior should instead be a new case in an existing more general test.

The behavior being tested - that the verifier disables private stack for
sleepable uprobes - is a verifier-level decision made in
bpf_enable_priv_stack() when prog->sleepable is set. The test suite
already has tools/testing/selftests/bpf/progs/verifier_private_stack.c
in the same directory, which covers private-stack selection
deterministically by asserting on JIT output patterns with __jited()
annotations.

This patch adds a 254-line, two-file runtime test that depends on thread
timing and CPU affinity. Adding a SEC("uprobe.s") case to the verifier
test with a 64-byte frame would pin the fix by asserting the absence of
the private-frame-pointer setup instruction, without needing threads or
timing dependencies.

The runtime test does exercise something the verifier test cannot -
actual concurrent same-CPU execution - so this is raised as a question:
would extending verifier_private_stack.c have been a simpler way to pin
the verifier change, or is the runtime concurrency test worth the added
complexity?

[ ... ]

> @@ -0,0 +1,165 @@

[ ... ]

> +void test_uprobe_sleepable_stack(void)
> +{
> +	struct uprobe_sleepable_stack *skel = NULL;
> +	cpu_set_t old_mask, mask;
> +	bool affinity_set = false;
> +	int cpu;
> +
> +#if !defined(__x86_64__) && !defined(__aarch64__) && !defined(__powerpc64__)
> +	test__skip();
> +	return;
> +#endif
> +	if (!env.jit_enabled) {
> +		test__skip();
> +		return;
> +	}

[ ... ]

> +	if (!skel->kconfig->CONFIG_PREEMPTION) {
> +		test__skip();
> +		goto cleanup;
> +	}

[ ... ]

> +	link_err = libbpf_get_error(link);
> +	if (link_err == -EOPNOTSUPP) {
> +		test__skip();
> +		return;
> +	}

This isn't a bug, but could the skip paths include diagnostics? All four
skip sites (unsupported architecture, JIT disabled, CONFIG_PREEMPTION
unset, and attach returning -EOPNOTSUPP) call bare test__skip() with no
message, so a skipped result gives the operator no way to tell which
prerequisite was missing.

The test_progs framework's test__skip() takes no arguments, so this
matches prevailing practice across the test suite. Is it worth noting
because the missing prerequisite that matters most - runtime preemption
actually being enabled - would be most helpful to communicate.

> diff --git a/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c b/tools/testing/selftests/bpf/progs/uprobe_sleepable_stack.c

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32604492082

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Alexei Starovoitov]

On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> A JITed BPF program can use one private stack per program and CPU.
> Sleepable programs can be preempted, allowing another task to run the
> same program on the same CPU. The second invocation then reuses and can
> overwrite the first invocation's private stack.

I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
which has per-prog recurison counter. So preemption of the prog
doesn't break private stack.
If the same prog attemps to execute on the same cpu it will be skipped.

syscall prog types go via bpf_prog_run_array_sleepable()
that have per prog recursions counter.

Looks like we're not doing it for bpf_prog_run_array_uprobe().
I'm not sure what the right trade off here.
I feel universally checking for recursion is better
then selectively disabling private stack for uprobe.

> Disable private stack for sleepable programs so they use the regular kernel
> stack, which handles preemption correctly. This change is intentionally
> limited to programs marked sleepable; preemptible non-sleepable dispatch
> paths require separate protection.
>
> Fixes: 7d1cd70d4b16 ("bpf, x86: Support private stack in jit")
> Fixes: 6c17a882d380 ("bpf, arm64: JIT support for private stack")
> Fixes: 156d985123b6 ("powerpc64/bpf: Implement JIT support for private stack")
> Cc: stable@vger.kernel.org
> Signed-off-by: Christian Simon <simon@swine.de>
> ---
>  kernel/bpf/verifier.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5e37ca75e5c4..038753ef07a9 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -5237,6 +5237,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
>  	if (!bpf_jit_supports_private_stack())
>  		return NO_PRIV_STACK;
>  
> +	/*
> +	 * Sleepable programs can be preempted, allowing another task to run
> +	 * the same program on the same CPU. Since private stack is per-CPU
> +	 * and per-program, the second invocation would corrupt the first's
> +	 * stack. Disable private stack for sleepable programs.
> +	 */
> +	if (prog->sleepable)
> +		return NO_PRIV_STACK;

This is not true in genreal. sleepable progs can use priv stack.

pw-bot: cr

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Jiri Olsa]

On Tue, Aug 25, 2026 at 06:20:16PM -0700, Alexei Starovoitov wrote:
> On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> > A JITed BPF program can use one private stack per program and CPU.
> > Sleepable programs can be preempted, allowing another task to run the
> > same program on the same CPU. The second invocation then reuses and can
> > overwrite the first invocation's private stack.
> 
> I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
> which has per-prog recurison counter. So preemption of the prog
> doesn't break private stack.
> If the same prog attemps to execute on the same cpu it will be skipped.
> 
> syscall prog types go via bpf_prog_run_array_sleepable()
> that have per prog recursions counter.
> 
> Looks like we're not doing it for bpf_prog_run_array_uprobe().
> I'm not sure what the right trade off here.
> I feel universally checking for recursion is better
> then selectively disabling private stack for uprobe.

also during the load we can't tell if kprobe program will be attached
as kprobe or uprobe [1] so having a way to disable private stack for
a program would solve this as well

jirka


[1] https://lore.kernel.org/bpf/ao2f-rBgT0SqX6Pw@krava/

> 
> > Disable private stack for sleepable programs so they use the regular kernel
> > stack, which handles preemption correctly. This change is intentionally
> > limited to programs marked sleepable; preemptible non-sleepable dispatch
> > paths require separate protection.
> >
> > Fixes: 7d1cd70d4b16 ("bpf, x86: Support private stack in jit")
> > Fixes: 6c17a882d380 ("bpf, arm64: JIT support for private stack")
> > Fixes: 156d985123b6 ("powerpc64/bpf: Implement JIT support for private stack")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Christian Simon <simon@swine.de>
> > ---
> >  kernel/bpf/verifier.c | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 5e37ca75e5c4..038753ef07a9 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -5237,6 +5237,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
> >  	if (!bpf_jit_supports_private_stack())
> >  		return NO_PRIV_STACK;
> >  
> > +	/*
> > +	 * Sleepable programs can be preempted, allowing another task to run
> > +	 * the same program on the same CPU. Since private stack is per-CPU
> > +	 * and per-program, the second invocation would corrupt the first's
> > +	 * stack. Disable private stack for sleepable programs.
> > +	 */
> > +	if (prog->sleepable)
> > +		return NO_PRIV_STACK;
> 
> This is not true in genreal. sleepable progs can use priv stack.
> 
> pw-bot: cr

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Andrii Nakryiko]

On Tue, Aug 25, 2026 at 6:20 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> > A JITed BPF program can use one private stack per program and CPU.
> > Sleepable programs can be preempted, allowing another task to run the
> > same program on the same CPU. The second invocation then reuses and can
> > overwrite the first invocation's private stack.
>
> I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
> which has per-prog recurison counter. So preemption of the prog
> doesn't break private stack.
> If the same prog attemps to execute on the same cpu it will be skipped.
>
> syscall prog types go via bpf_prog_run_array_sleepable()
> that have per prog recursions counter.
>
> Looks like we're not doing it for bpf_prog_run_array_uprobe().
> I'm not sure what the right trade off here.
> I feel universally checking for recursion is better
> then selectively disabling private stack for uprobe.

I'd really like to avoid adding this "recursion protection" to uprobe.
With uprobes, there is no recursion, it's called from well defined
context in the kernel and you can't have recursive uprobe BPF
programs.

All you can have is a very valid and possible sleepable uprobe
interleaving, which the user cannot prevent or work around, they have
no control over this and it's just a fact of life.

 E.g., a simple scenario, we attach one bpf program (let's call it U)
to some USDT. BPF program U is sleepable and actually can sleep due to
page faults (e.g., unwinding Python stack trace requires sleepable
mode for reliably getting filename strings from Python runtime, which
are not always paged in).

In such a case, you can have thread A and thread B both hitting the
same USDT (e.g., somewhere in memory allocator or whatnot). Let's say
thread A hits it first on CPU X, BPF program U starts executing and
unwinding Python stack, does bpf_copy_from_user() for string contents
and causes page fault, is taken off CPU X. Meanwhile thread B hits
USDT on the same CPU X, kernel runs program U, and it is supposed to
work completely independently and concurrently (no shared state or
whatever) from U's execution in thread A.

Yet, if we add this per-CPU "recursion check", we'll just skip U's
execution for thread B. This is data loss, and it's very bad in
practice because it frequently just invalidates the entire data
collection trustworthiness.

So I think we should disable private stack for uprobes (sleepable or
not) instead. I'm not sure private stack buys us anything for uprobe
cases.

As Jiri mentioned, at verification/jitting time we can't tell kprobe
from uprobe, though, so that's a bit of a problem, but it's a separate
discussion we should have. Perhaps having an extra expected attach
type, or type, or some program flag to designate uprobe program as
such would be appropriate, not sure. Let's discuss that.

But please, let's not make uprobes unreliable for no good reason.

>
> > Disable private stack for sleepable programs so they use the regular kernel
> > stack, which handles preemption correctly. This change is intentionally
> > limited to programs marked sleepable; preemptible non-sleepable dispatch
> > paths require separate protection.
> >
> > Fixes: 7d1cd70d4b16 ("bpf, x86: Support private stack in jit")
> > Fixes: 6c17a882d380 ("bpf, arm64: JIT support for private stack")
> > Fixes: 156d985123b6 ("powerpc64/bpf: Implement JIT support for private stack")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Christian Simon <simon@swine.de>
> > ---
> >  kernel/bpf/verifier.c | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 5e37ca75e5c4..038753ef07a9 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -5237,6 +5237,15 @@ static enum priv_stack_mode bpf_enable_priv_stack(struct bpf_prog *prog)
> >       if (!bpf_jit_supports_private_stack())
> >               return NO_PRIV_STACK;
> >
> > +     /*
> > +      * Sleepable programs can be preempted, allowing another task to run
> > +      * the same program on the same CPU. Since private stack is per-CPU
> > +      * and per-program, the second invocation would corrupt the first's
> > +      * stack. Disable private stack for sleepable programs.
> > +      */
> > +     if (prog->sleepable)
> > +             return NO_PRIV_STACK;
>
> This is not true in genreal. sleepable progs can use priv stack.
>
> pw-bot: cr

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Alexei Starovoitov]

On Thu, Aug 27, 2026 at 7:56 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Aug 25, 2026 at 6:20 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> > > A JITed BPF program can use one private stack per program and CPU.
> > > Sleepable programs can be preempted, allowing another task to run the
> > > same program on the same CPU. The second invocation then reuses and can
> > > overwrite the first invocation's private stack.
> >
> > I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
> > which has per-prog recurison counter. So preemption of the prog
> > doesn't break private stack.
> > If the same prog attemps to execute on the same cpu it will be skipped.
> >
> > syscall prog types go via bpf_prog_run_array_sleepable()
> > that have per prog recursions counter.
> >
> > Looks like we're not doing it for bpf_prog_run_array_uprobe().
> > I'm not sure what the right trade off here.
> > I feel universally checking for recursion is better
> > then selectively disabling private stack for uprobe.
>
> I'd really like to avoid adding this "recursion protection" to uprobe.
> With uprobes, there is no recursion, it's called from well defined
> context in the kernel and you can't have recursive uprobe BPF
> programs.
>
> All you can have is a very valid and possible sleepable uprobe
> interleaving, which the user cannot prevent or work around, they have
> no control over this and it's just a fact of life.
>
>  E.g., a simple scenario, we attach one bpf program (let's call it U)
> to some USDT. BPF program U is sleepable and actually can sleep due to
> page faults (e.g., unwinding Python stack trace requires sleepable
> mode for reliably getting filename strings from Python runtime, which
> are not always paged in).
>
> In such a case, you can have thread A and thread B both hitting the
> same USDT (e.g., somewhere in memory allocator or whatnot). Let's say
> thread A hits it first on CPU X, BPF program U starts executing and
> unwinding Python stack, does bpf_copy_from_user() for string contents
> and causes page fault, is taken off CPU X. Meanwhile thread B hits
> USDT on the same CPU X, kernel runs program U, and it is supposed to
> work completely independently and concurrently (no shared state or
> whatever) from U's execution in thread A.
>
> Yet, if we add this per-CPU "recursion check", we'll just skip U's
> execution for thread B. This is data loss, and it's very bad in
> practice because it frequently just invalidates the entire data
> collection trustworthiness.

ok. fair

> So I think we should disable private stack for uprobes (sleepable or
> not) instead. I'm not sure private stack buys us anything for uprobe
> cases.

why disable priv stack for non-sleepable uprobes?
While non-sleepable bpf prog is executing the same or different
uprobe cannot execute on the same cpu.
So bpf prog can be preempted by kernel execution,
but a user task cannot start preempt bpf prog,
so 2nd uprobe cannot start running,
no?

> As Jiri mentioned, at verification/jitting time we can't tell kprobe
> from uprobe, though, so that's a bit of a problem, but it's a separate
> discussion we should have. Perhaps having an extra expected attach
> type, or type, or some program flag to designate uprobe program as
> such would be appropriate, not sure. Let's discuss that.

we don't have sleepable kprobes.
So disable sleepable && [ku]probe type is enough. no?

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Andrii Nakryiko]

On Thu, Aug 27, 2026 at 9:35 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Aug 27, 2026 at 7:56 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Tue, Aug 25, 2026 at 6:20 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> > > > A JITed BPF program can use one private stack per program and CPU.
> > > > Sleepable programs can be preempted, allowing another task to run the
> > > > same program on the same CPU. The second invocation then reuses and can
> > > > overwrite the first invocation's private stack.
> > >
> > > I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
> > > which has per-prog recurison counter. So preemption of the prog
> > > doesn't break private stack.
> > > If the same prog attemps to execute on the same cpu it will be skipped.
> > >
> > > syscall prog types go via bpf_prog_run_array_sleepable()
> > > that have per prog recursions counter.
> > >
> > > Looks like we're not doing it for bpf_prog_run_array_uprobe().
> > > I'm not sure what the right trade off here.
> > > I feel universally checking for recursion is better
> > > then selectively disabling private stack for uprobe.
> >
> > I'd really like to avoid adding this "recursion protection" to uprobe.
> > With uprobes, there is no recursion, it's called from well defined
> > context in the kernel and you can't have recursive uprobe BPF
> > programs.
> >
> > All you can have is a very valid and possible sleepable uprobe
> > interleaving, which the user cannot prevent or work around, they have
> > no control over this and it's just a fact of life.
> >
> >  E.g., a simple scenario, we attach one bpf program (let's call it U)
> > to some USDT. BPF program U is sleepable and actually can sleep due to
> > page faults (e.g., unwinding Python stack trace requires sleepable
> > mode for reliably getting filename strings from Python runtime, which
> > are not always paged in).
> >
> > In such a case, you can have thread A and thread B both hitting the
> > same USDT (e.g., somewhere in memory allocator or whatnot). Let's say
> > thread A hits it first on CPU X, BPF program U starts executing and
> > unwinding Python stack, does bpf_copy_from_user() for string contents
> > and causes page fault, is taken off CPU X. Meanwhile thread B hits
> > USDT on the same CPU X, kernel runs program U, and it is supposed to
> > work completely independently and concurrently (no shared state or
> > whatever) from U's execution in thread A.
> >
> > Yet, if we add this per-CPU "recursion check", we'll just skip U's
> > execution for thread B. This is data loss, and it's very bad in
> > practice because it frequently just invalidates the entire data
> > collection trustworthiness.
>
> ok. fair
>

great, thanks!

> > So I think we should disable private stack for uprobes (sleepable or
> > not) instead. I'm not sure private stack buys us anything for uprobe
> > cases.
>
> why disable priv stack for non-sleepable uprobes?
> While non-sleepable bpf prog is executing the same or different
> uprobe cannot execute on the same cpu.
> So bpf prog can be preempted by kernel execution,
> but a user task cannot start preempt bpf prog,
> so 2nd uprobe cannot start running,
> no?

I think that changes on preemptible kernels, this was called out in
discussions on previous versions of this patch. So only for that
reason.

For non-preemptible kernels yes, it's only sleepable that is a problem.

>
> > As Jiri mentioned, at verification/jitting time we can't tell kprobe
> > from uprobe, though, so that's a bit of a problem, but it's a separate
> > discussion we should have. Perhaps having an extra expected attach
> > type, or type, or some program flag to designate uprobe program as
> > such would be appropriate, not sure. Let's discuss that.
>
> we don't have sleepable kprobes.
> So disable sleepable && [ku]probe type is enough. no?

Yes, unless we want to also safeguard preemptible kernels. But we can
do that as an extra #ifdef CONFIG_PREEMPT_RT or something?

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Alexei Starovoitov]

On Thu, Aug 27, 2026 at 9:40 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Aug 27, 2026 at 9:35 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Thu, Aug 27, 2026 at 7:56 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Tue, Aug 25, 2026 at 6:20 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> > > > > A JITed BPF program can use one private stack per program and CPU.
> > > > > Sleepable programs can be preempted, allowing another task to run the
> > > > > same program on the same CPU. The second invocation then reuses and can
> > > > > overwrite the first invocation's private stack.
> > > >
> > > > I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
> > > > which has per-prog recurison counter. So preemption of the prog
> > > > doesn't break private stack.
> > > > If the same prog attemps to execute on the same cpu it will be skipped.
> > > >
> > > > syscall prog types go via bpf_prog_run_array_sleepable()
> > > > that have per prog recursions counter.
> > > >
> > > > Looks like we're not doing it for bpf_prog_run_array_uprobe().
> > > > I'm not sure what the right trade off here.
> > > > I feel universally checking for recursion is better
> > > > then selectively disabling private stack for uprobe.
> > >
> > > I'd really like to avoid adding this "recursion protection" to uprobe.
> > > With uprobes, there is no recursion, it's called from well defined
> > > context in the kernel and you can't have recursive uprobe BPF
> > > programs.
> > >
> > > All you can have is a very valid and possible sleepable uprobe
> > > interleaving, which the user cannot prevent or work around, they have
> > > no control over this and it's just a fact of life.
> > >
> > >  E.g., a simple scenario, we attach one bpf program (let's call it U)
> > > to some USDT. BPF program U is sleepable and actually can sleep due to
> > > page faults (e.g., unwinding Python stack trace requires sleepable
> > > mode for reliably getting filename strings from Python runtime, which
> > > are not always paged in).
> > >
> > > In such a case, you can have thread A and thread B both hitting the
> > > same USDT (e.g., somewhere in memory allocator or whatnot). Let's say
> > > thread A hits it first on CPU X, BPF program U starts executing and
> > > unwinding Python stack, does bpf_copy_from_user() for string contents
> > > and causes page fault, is taken off CPU X. Meanwhile thread B hits
> > > USDT on the same CPU X, kernel runs program U, and it is supposed to
> > > work completely independently and concurrently (no shared state or
> > > whatever) from U's execution in thread A.
> > >
> > > Yet, if we add this per-CPU "recursion check", we'll just skip U's
> > > execution for thread B. This is data loss, and it's very bad in
> > > practice because it frequently just invalidates the entire data
> > > collection trustworthiness.
> >
> > ok. fair
> >
>
> great, thanks!
>
> > > So I think we should disable private stack for uprobes (sleepable or
> > > not) instead. I'm not sure private stack buys us anything for uprobe
> > > cases.
> >
> > why disable priv stack for non-sleepable uprobes?
> > While non-sleepable bpf prog is executing the same or different
> > uprobe cannot execute on the same cpu.
> > So bpf prog can be preempted by kernel execution,
> > but a user task cannot start preempt bpf prog,
> > so 2nd uprobe cannot start running,
> > no?
>
> I think that changes on preemptible kernels, this was called out in
> discussions on previous versions of this patch. So only for that
> reason.

my understanding is that preemptable kernel doesn't mean that
bpf prog can be preempted by user space.
only by kernel.

I looked up earlier thread, but don't understand what Jiri meant.

Jiri,
please clarify what problem do you see with non-sleepable uprobes?

Re: [PATCH bpf v3 1/2] bpf: disable private stack for sleepable programs

[Jiri Olsa]

On Thu, Aug 27, 2026 at 09:55:08AM -0700, Alexei Starovoitov wrote:
> On Thu, Aug 27, 2026 at 9:40 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Aug 27, 2026 at 9:35 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Thu, Aug 27, 2026 at 7:56 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Tue, Aug 25, 2026 at 6:20 PM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > On Sat Aug 22, 2026 at 3:54 PM PDT, Christian Simon wrote:
> > > > > > A JITed BPF program can use one private stack per program and CPU.
> > > > > > Sleepable programs can be preempted, allowing another task to run the
> > > > > > same program on the same CPU. The second invocation then reuses and can
> > > > > > overwrite the first invocation's private stack.
> > > > >
> > > > > I'm confused by this. sleepable progs go through __bpf_prog_enter_sleepable_recur()
> > > > > which has per-prog recurison counter. So preemption of the prog
> > > > > doesn't break private stack.
> > > > > If the same prog attemps to execute on the same cpu it will be skipped.
> > > > >
> > > > > syscall prog types go via bpf_prog_run_array_sleepable()
> > > > > that have per prog recursions counter.
> > > > >
> > > > > Looks like we're not doing it for bpf_prog_run_array_uprobe().
> > > > > I'm not sure what the right trade off here.
> > > > > I feel universally checking for recursion is better
> > > > > then selectively disabling private stack for uprobe.
> > > >
> > > > I'd really like to avoid adding this "recursion protection" to uprobe.
> > > > With uprobes, there is no recursion, it's called from well defined
> > > > context in the kernel and you can't have recursive uprobe BPF
> > > > programs.
> > > >
> > > > All you can have is a very valid and possible sleepable uprobe
> > > > interleaving, which the user cannot prevent or work around, they have
> > > > no control over this and it's just a fact of life.
> > > >
> > > >  E.g., a simple scenario, we attach one bpf program (let's call it U)
> > > > to some USDT. BPF program U is sleepable and actually can sleep due to
> > > > page faults (e.g., unwinding Python stack trace requires sleepable
> > > > mode for reliably getting filename strings from Python runtime, which
> > > > are not always paged in).
> > > >
> > > > In such a case, you can have thread A and thread B both hitting the
> > > > same USDT (e.g., somewhere in memory allocator or whatnot). Let's say
> > > > thread A hits it first on CPU X, BPF program U starts executing and
> > > > unwinding Python stack, does bpf_copy_from_user() for string contents
> > > > and causes page fault, is taken off CPU X. Meanwhile thread B hits
> > > > USDT on the same CPU X, kernel runs program U, and it is supposed to
> > > > work completely independently and concurrently (no shared state or
> > > > whatever) from U's execution in thread A.
> > > >
> > > > Yet, if we add this per-CPU "recursion check", we'll just skip U's
> > > > execution for thread B. This is data loss, and it's very bad in
> > > > practice because it frequently just invalidates the entire data
> > > > collection trustworthiness.
> > >
> > > ok. fair
> > >
> >
> > great, thanks!
> >
> > > > So I think we should disable private stack for uprobes (sleepable or
> > > > not) instead. I'm not sure private stack buys us anything for uprobe
> > > > cases.
> > >
> > > why disable priv stack for non-sleepable uprobes?
> > > While non-sleepable bpf prog is executing the same or different
> > > uprobe cannot execute on the same cpu.
> > > So bpf prog can be preempted by kernel execution,
> > > but a user task cannot start preempt bpf prog,
> > > so 2nd uprobe cannot start running,
> > > no?
> >
> > I think that changes on preemptible kernels, this was called out in
> > discussions on previous versions of this patch. So only for that
> > reason.
> 
> my understanding is that preemptable kernel doesn't mean that
> bpf prog can be preempted by user space.
> only by kernel.
> 
> I looked up earlier thread, but don't understand what Jiri meant.
> 
> Jiri,
> please clarify what problem do you see with non-sleepable uprobes?

hum.. non-sleepable uprobe prog is run by bpf_prog_run_array_uprobe
and it disables only task migration, preemption is not disabled and
holds rcu_read_lock (which seems ok for preemption)

so I'm not sure why it wouldn't be preemptible by another task

jirka