[PATCH bpf-next v2 0/2] Fix global subprog verification context

[PATCH bpf-next v2 0/2] Fix global subprog verification context

[Kumar Kartikeya Dwivedi]

Global subprogs can be invoked in both sleepable and non-sleepable
contexts, but are verified based on context assumptions for the program
type as a whole and not the precise context in which they are invoked.
Nicholas identified that this can violate verifier safety assumptions.
Verify global subprogs once for each observed sleepability context and add
selftests covering workqueue and task-work callbacks.

Changelog:
----------
v1 -> v2
v1: https://lore.kernel.org/bpf/20260905034018.2095649-1-memxor@gmail.com

 * Preserve the harmless global subprog calls in the positive test with
   barrier() so LLVM cannot eliminate the intended coverage.
 * Return zero explicitly from workqueue callbacks after global subprog calls.
 * Document cumulative instruction accounting across verification contexts.
 * Fix BPF multi-line comment formatting.

Kumar Kartikeya Dwivedi (2):
  bpf: Verify global subprogs in each sleepability context
  selftests/bpf: Test global subprog callback contexts

 include/linux/bpf.h                           |   5 +-
 kernel/bpf/verifier.c                         |  77 ++++++----
 .../bpf/progs/verifier_async_cb_context.c     | 132 ++++++++++++++++++
 3 files changed, 182 insertions(+), 32 deletions(-)


base-commit: 3ccdb07813829ba9487273e75d1cb238cfa774c1
-- 
2.53.0

[PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[Kumar Kartikeya Dwivedi]

Global subprograms are verified independently with a fresh verifier root.
do_check_common() currently seeds that root's in_sleepable state from the
program, even though a global subprogram can also run from callbacks whose
execution context differs from the program's main entry point.

In particular, workqueue and task-work callbacks are sleepable even when
the containing program is not. A global subprogram of that program is
therefore verified as non-sleepable, making in_rcu_cs() true and allowing
loads of RCU-protected kptrs to produce trusted MEM_RCU pointers. The same
subprogram can then be called from a sleepable callback without a classic
RCU reader. It can retain such a pointer while the object is freed and use
it after free.

The verifier's execution-context predicates are complementary. A state is
sleepable only when in_sleepable is set and no RCU, preemption, IRQ, or lock
region is active. Each condition which prevents sleeping also provides RCU
protection, while in_rcu_cs() treats a non-sleepable state as implicitly
protected.

Use this relationship to represent a global subprogram caller with only the
result of in_sleepable_context(). A protected sleepable caller is normalized
to in_sleepable=false at the independent verification root. This both
prevents sleepable operations and makes in_rcu_cs() true without copying
caller-owned lock state.

Record whether each global subprogram is called with either in_sleepable
value and verify it once for every observed value. Walk global subprograms
in caller-before-callee order so the values propagate through global call
chains, and repeat until every discovered context has been verified to cover
asynchronous callback cycles.

Since a global subprogram may now be verified twice, accumulate both passes
in subprog_info[].insns_total. This makes BPF_LOG_STATS and per-subprogram
veristat output account for both contexts instead of reporting only the last
pass.

This makes an unprotected callback verify the global subprogram as
sleepable, turning its RCU-protected kptr load into an untrusted pointer.
Protected callers and global subprograms which do not depend on implicit RCU
protection remain valid.

Fixes: 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl")
Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 include/linux/bpf.h   |  5 +--
 kernel/bpf/verifier.c | 77 ++++++++++++++++++++++++++-----------------
 2 files changed, 50 insertions(+), 32 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 3a7eb2185c35..66d04244c737 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1650,8 +1650,9 @@ static inline void bpf_trampoline_set_flags(struct bpf_trampoline *tr, u32 flags
 struct bpf_func_info_aux {
 	u16 linkage;
 	bool unreliable;
-	bool called : 1;
-	bool verified : 1;
+	/* Indexed by in_sleepable. */
+	bool called[2];
+	bool verified[2];
 };
 
 enum bpf_jit_poke_reason {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 5b51e7ee1a3f..f759a020c8a5 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -9957,6 +9957,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	if (err == -EFAULT)
 		return err;
 	if (bpf_subprog_is_global(env, subprog)) {
+		struct bpf_func_info_aux *sub_aux = subprog_aux(env, subprog);
 		const char *sub_name = bpf_subprog_name(env, subprog);
 		const char *operation;
 		bool returns_void;
@@ -9988,11 +9989,10 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		if (env->log.level & BPF_LOG_LEVEL)
 			verbose(env, "Func#%d ('%s') is global and assumed valid.\n",
 				subprog, sub_name);
+		sub_aux->called[in_sleepable_context(env)] = true;
 		returns_void = subprog_returns_void(env, subprog);
 		if (env->subprog_info[subprog].changes_pkt_data)
 			clear_all_pkt_pointers(env);
-		/* mark global subprog for verifying after main prog */
-		subprog_aux(env, subprog)->called = true;
 		if (returns_void)
 			bpf_diag_record_scrub(env, &caller->regs[BPF_REG_0], BPF_DIAG_MOD_CALLER_SAVED);
 		else
@@ -10804,7 +10804,12 @@ int bpf_get_helper_proto(struct bpf_verifier_env *env, int func_id,
 	return *ptr && (*ptr)->func ? 0 : -EINVAL;
 }
 
-/* Check if we're in a sleepable context. */
+/*
+ * This predicate is the inverse of in_rcu_cs(): non-sleepable programs and
+ * every condition that prevents sleeping also provide RCU protection. Global
+ * subprog verification relies on this equivalence to represent the caller's
+ * execution context using only the in_sleepable bit.
+ */
 static inline bool in_sleepable_context(struct bpf_verifier_env *env)
 {
 	return !env->cur_state->active_rcu_locks &&
@@ -19560,13 +19565,14 @@ static void free_states(struct bpf_verifier_env *env)
 	}
 }
 
-static int do_check_common(struct bpf_verifier_env *env, int subprog)
+static int do_check_common(struct bpf_verifier_env *env, int subprog, bool in_sleepable)
 {
 	bool pop_log = !(env->log.level & BPF_LOG_LEVEL2);
 	struct bpf_subprog_info *sub = subprog_info(env, subprog);
 	struct bpf_prog_aux *aux = env->prog->aux;
 	struct bpf_verifier_state *state;
 	struct bpf_reg_state *regs;
+	u32 old_insns_total = sub->insns_total;
 	u32 insn_processed = env->insn_processed;
 	int ret, i;
 
@@ -19579,7 +19585,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
 	state->curframe = 0;
 	state->speculative = false;
 	state->branches = 1;
-	state->in_sleepable = env->prog->sleepable;
+	state->in_sleepable = in_sleepable;
 	state->frame[0] = kzalloc_obj(struct bpf_func_state, GFP_KERNEL_ACCOUNT);
 	if (!state->frame[0]) {
 		kfree(state);
@@ -19721,7 +19727,8 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
 	 * Accumulate their total counts as total counts of the main or
 	 * global subprog hosting the async call.
 	 */
-	env->subprog_info[subprog].insns_total = env->insn_processed - insn_processed;
+	env->subprog_info[subprog].insns_total = old_insns_total +
+						(env->insn_processed - insn_processed);
 	return ret;
 }
 
@@ -19749,45 +19756,55 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
 {
 	struct bpf_prog_aux *aux = env->prog->aux;
 	struct bpf_func_info_aux *sub_aux;
-	int i, ret, new_cnt;
+	int context, i, j, ret, new_cnt;
 
 	if (!aux->func_info)
 		return 0;
 
 	/* exception callback is presumed to be always called */
-	if (env->exception_callback_subprog)
-		subprog_aux(env, env->exception_callback_subprog)->called = true;
+	if (env->exception_callback_subprog) {
+		sub_aux = subprog_aux(env, env->exception_callback_subprog);
+		sub_aux->called[env->prog->sleepable] = true;
+	}
 
 again:
 	new_cnt = 0;
-	for (i = 1; i < env->subprog_cnt; i++) {
+	/*
+	 * Walk callers before callees so each global subprog normally sees all
+	 * of its contexts before it is verified. Async callback cycles can add a
+	 * context to an earlier subprog, so repeat until every called context is
+	 * verified.
+	 */
+	for (j = env->subprog_cnt - 1; j >= 0; j--) {
+		i = env->subprog_topo_order[j];
+		if (!i)
+			continue;
 		if (!bpf_subprog_is_global(env, i))
 			continue;
 
 		sub_aux = subprog_aux(env, i);
-		if (!sub_aux->called || sub_aux->verified)
-			continue;
+		for (context = 0; context < ARRAY_SIZE(sub_aux->called); context++) {
+			if (!sub_aux->called[context] || sub_aux->verified[context])
+				continue;
 
-		env->insn_idx = env->subprog_info[i].start;
-		WARN_ON_ONCE(env->insn_idx == 0);
-		ret = do_check_common(env, i);
-		if (ret) {
-			return ret;
-		} else if (env->log.level & BPF_LOG_LEVEL) {
-			verbose(env, "Func#%d ('%s') is safe for any args that match its prototype\n",
-				i, bpf_subprog_name(env, i));
-		}
+			env->insn_idx = env->subprog_info[i].start;
+			WARN_ON_ONCE(env->insn_idx == 0);
+			ret = do_check_common(env, i, context);
+			if (ret)
+				return ret;
+			if (env->log.level & BPF_LOG_LEVEL)
+				verbose(env, "Func#%d ('%s') is safe for any args "
+					"that match its prototype\n",
+					i, bpf_subprog_name(env, i));
 
-		/* We verified new global subprog, it might have called some
-		 * more global subprogs that we haven't verified yet, so we
-		 * need to do another pass over subprogs to verify those.
-		 */
-		sub_aux->verified = true;
-		new_cnt++;
+			sub_aux->verified[context] = true;
+			new_cnt++;
+		}
 	}
 
-	/* We can't loop forever as we verify at least one global subprog on
-	 * each pass.
+	/*
+	 * We can't loop forever as each pass verifies at least one new context,
+	 * and there are only two contexts per global subprog.
 	 */
 	if (new_cnt)
 		goto again;
@@ -19800,7 +19817,7 @@ static int do_check_main(struct bpf_verifier_env *env)
 	int ret;
 
 	env->insn_idx = 0;
-	ret = do_check_common(env, 0);
+	ret = do_check_common(env, 0, env->prog->sleepable);
 	if (!ret)
 		env->prog->aux->stack_depth = env->subprog_info[0].stack_depth;
 	return ret;
-- 
2.53.0

[PATCH bpf-next v2 2/2] selftests/bpf: Test global subprog callback contexts

[Kumar Kartikeya Dwivedi]

Exercise global subprogram verification from workqueue and task-work
callbacks. Both callback types can run in a sleepable context even when the
containing program is not sleepable, so an unprotected callback must not let
the global subprogram use implicit RCU protection inherited from the program.

Add negative cases which load an RCU-protected task kptr in a global
subprogram reached from each callback type. The tests fail on an unfixed
kernel because the programs are incorrectly accepted.

Also cover a workqueue callback protected by an explicit RCU read-side
critical section. Finally, call the same harmless global subprogram directly
from the main program and from an unprotected callback. This requires both
non-sleepable and sleepable verification roots and proves that global calls
from callbacks are not rejected wholesale.

Keep the harmless global opaque to LLVM with barrier() so both call sites
remain in the generated BPF object and the positive case cannot pass
vacuously. Have workqueue callbacks return zero explicitly after calling a
global subprogram, as required by their callback contract, instead of relying
on interprocedural return-value optimization.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 .../bpf/progs/verifier_async_cb_context.c     | 132 ++++++++++++++++++
 1 file changed, 132 insertions(+)

diff --git a/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c b/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c
index 6bf95550a024..987ca9dec6b7 100644
--- a/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c
+++ b/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c
@@ -9,6 +9,11 @@
 
 char _license[] SEC("license") = "GPL";
 
+struct task_struct *bpf_task_acquire(struct task_struct *p) __ksym;
+void bpf_task_release(struct task_struct *p) __ksym;
+void bpf_rcu_read_lock(void) __ksym;
+void bpf_rcu_read_unlock(void) __ksym;
+
 /* Timer tests */
 
 struct timer_elem {
@@ -66,6 +71,7 @@ int timer_sleepable_prog(void *ctx)
 
 struct wq_elem {
 	struct bpf_wq w;
+	struct task_struct __kptr *task;
 };
 
 struct {
@@ -119,6 +125,106 @@ int wq_sleepable_prog(void *ctx)
 	return 0;
 }
 
+__noinline int wq_global_acquire(void)
+{
+	struct task_struct *task, *acquired;
+	struct wq_elem *val;
+	int key = 0;
+
+	val = bpf_map_lookup_elem(&wq_map, &key);
+	if (!val)
+		return 0;
+
+	task = val->task;
+	if (!task)
+		return 0;
+
+	acquired = bpf_task_acquire(task);
+	if (acquired)
+		bpf_task_release(acquired);
+	return 0;
+}
+
+static int wq_global_rcu_cb(void *map, int *key, void *value)
+{
+	wq_global_acquire();
+	return 0;
+}
+
+SEC("fentry/bpf_fentry_test1")
+__failure __msg("R1 must be a rcu pointer")
+int wq_global_rcu_prog(void *ctx)
+{
+	struct wq_elem *val;
+	int key = 0;
+
+	val = bpf_map_lookup_elem(&wq_map, &key);
+	if (!val)
+		return 0;
+
+	bpf_wq_init(&val->w, &wq_map, 0);
+	bpf_wq_set_callback(&val->w, wq_global_rcu_cb, 0);
+	return 0;
+}
+
+static int wq_global_rcu_lock_cb(void *map, int *key, void *value)
+{
+	bpf_rcu_read_lock();
+	wq_global_acquire();
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("fentry/bpf_fentry_test1")
+__success
+int wq_global_rcu_lock_prog(void *ctx)
+{
+	struct wq_elem *val;
+	int key = 0;
+
+	/* Verify the same global subprog in non-sleepable and protected contexts. */
+	wq_global_acquire();
+
+	val = bpf_map_lookup_elem(&wq_map, &key);
+	if (!val)
+		return 0;
+
+	bpf_wq_init(&val->w, &wq_map, 0);
+	bpf_wq_set_callback(&val->w, wq_global_rcu_lock_cb, 0);
+	return 0;
+}
+
+__noinline int wq_global_no_rcu(void)
+{
+	barrier();
+	return 0;
+}
+
+static int wq_global_no_rcu_cb(void *map, int *key, void *value)
+{
+	wq_global_no_rcu();
+	return 0;
+}
+
+SEC("fentry/bpf_fentry_test1")
+__success
+int wq_global_no_rcu_prog(void *ctx)
+{
+	struct wq_elem *val;
+	int key = 0;
+
+	/* Verify the same global in non-sleepable and unprotected contexts. */
+	wq_global_no_rcu();
+
+	val = bpf_map_lookup_elem(&wq_map, &key);
+	if (!val)
+		return 0;
+
+	bpf_wq_init(&val->w, &wq_map, 0);
+	bpf_wq_set_callback(&val->w, wq_global_no_rcu_cb, 0);
+	return 0;
+}
+
 /* Task work tests */
 
 struct task_work_elem {
@@ -179,3 +285,29 @@ int task_work_sleepable_prog(void *ctx)
 	bpf_task_work_schedule_resume(task, &val->tw, &task_work_map, task_work_cb);
 	return 0;
 }
+
+static int task_work_global_rcu_cb(struct bpf_map *map, void *key, void *value)
+{
+	return wq_global_acquire();
+}
+
+SEC("fentry/bpf_fentry_test1")
+__failure __msg("R1 must be a rcu pointer")
+int task_work_global_rcu_prog(void *ctx)
+{
+	struct task_work_elem *val;
+	struct task_struct *task;
+	int key = 0;
+
+	val = bpf_map_lookup_elem(&task_work_map, &key);
+	if (!val)
+		return 0;
+
+	task = bpf_get_current_task_btf();
+	if (!task)
+		return 0;
+
+	bpf_task_work_schedule_resume(task, &val->tw, &task_work_map,
+				      task_work_global_rcu_cb);
+	return 0;
+}
-- 
2.53.0

Re: [PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Global subprograms can bypass caller lock and reference balancing by using bpf_tail_call(), leading to permanent resource leaks.
--

commit abbef478df8567a76eb64989bbf7f08c717afd13
Author: Kumar Kartikeya Dwivedi <memxor@gmail.com>

bpf: Verify global subprogs in each sleepability context

This commit updates global subprogram verification to independently verify
each context (sleepable and non-sleepable) that calls it. This prevents
non-sleepable callbacks from incorrectly treating the context as sleepable
and allowing RCU-protected pointer misuse.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5b51e7ee1a3f9..f759a020c8a5a 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -9988,11 +9989,10 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  		if (env->log.level & BPF_LOG_LEVEL)
>  			verbose(env, "Func#%d ('%s') is global and assumed valid.\n",
>  				subprog, sub_name);
> +		sub_aux->called[in_sleepable_context(env)] = true;

[Severity: High]
This is a pre-existing issue, but while reviewing the global subprogram
execution context changes here, I noticed a potential resource leak
related to tail calls.

When a global subprogram is verified independently with a fresh verifier
root in do_check_common():

kernel/bpf/verifier.c:do_check_common() {
        ...
        state = kzalloc_obj(struct bpf_verifier_state, GFP_KERNEL_ACCOUNT);
        ...
}

The new state starts with zero locks and references. If the global
subprogram executes a bpf_tail_call(), check_resource_leak() inside the
subprogram's verification will succeed.

However, back in the caller's context in check_func_call(), the verifier
seems to evaluate the call assuming it will return normally:

kernel/bpf/verifier.c:check_func_call() {
        if (bpf_subprog_is_global(env, subprog)) {
                ...
                if (env->subprog_info[subprog].might_throw) {
                        branch = push_stack(...);
                        return process_bpf_exit_full(env, NULL, true);
                }
                /* continue with next insn after call */
                return 0;
        }
}

The caller check correctly handles divergence for might_throw, but
appears to silently ignore has_tail_call for global subprograms. If a
user program acquires a resource (e.g. bpf_rcu_read_lock) and then
calls a global subprogram that performs a tail call, does this bypass
the caller's resource cleanup and validations, permanently leaking the
resource?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260905051224.2325381-1-memxor@gmail.com?part=1

Re: [PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[Kumar Kartikeya Dwivedi]

On Sat Sep 5, 2026 at 7:32 AM CEST, sashiko-bot wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
>
> Pre-existing issues:
> - [High] Global subprograms can bypass caller lock and reference balancing by using bpf_tail_call(), leading to permanent resource leaks.
> --
>
> commit abbef478df8567a76eb64989bbf7f08c717afd13
> Author: Kumar Kartikeya Dwivedi <memxor@gmail.com>
>
> bpf: Verify global subprogs in each sleepability context
>
> This commit updates global subprogram verification to independently verify
> each context (sleepable and non-sleepable) that calls it. This prevents
> non-sleepable callbacks from incorrectly treating the context as sleepable
> and allowing RCU-protected pointer misuse.
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index 5b51e7ee1a3f9..f759a020c8a5a 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
>> @@ -9988,11 +9989,10 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>>  		if (env->log.level & BPF_LOG_LEVEL)
>>  			verbose(env, "Func#%d ('%s') is global and assumed valid.\n",
>>  				subprog, sub_name);
>> +		sub_aux->called[in_sleepable_context(env)] = true;
>
> [Severity: High]
> This is a pre-existing issue, but while reviewing the global subprogram
> execution context changes here, I noticed a potential resource leak
> related to tail calls.
>
> When a global subprogram is verified independently with a fresh verifier
> root in do_check_common():
>
> kernel/bpf/verifier.c:do_check_common() {
>         ...
>         state = kzalloc_obj(struct bpf_verifier_state, GFP_KERNEL_ACCOUNT);
>         ...
> }
>
> The new state starts with zero locks and references. If the global
> subprogram executes a bpf_tail_call(), check_resource_leak() inside the
> subprogram's verification will succeed.
>
> However, back in the caller's context in check_func_call(), the verifier
> seems to evaluate the call assuming it will return normally:
>
> kernel/bpf/verifier.c:check_func_call() {
>         if (bpf_subprog_is_global(env, subprog)) {
>                 ...
>                 if (env->subprog_info[subprog].might_throw) {
>                         branch = push_stack(...);
>                         return process_bpf_exit_full(env, NULL, true);
>                 }
>                 /* continue with next insn after call */
>                 return 0;
>         }
> }
>
> The caller check correctly handles divergence for might_throw, but
> appears to silently ignore has_tail_call for global subprograms. If a
> user program acquires a resource (e.g. bpf_rcu_read_lock) and then
> calls a global subprogram that performs a tail call, does this bypass
> the caller's resource cleanup and validations, permanently leaking the
> resource?

I don't think this report is valid. Even if a tail call happens in a global
subprog, it should still appear to return like a normal global subprog, and
the existing checks should ensure resource leak safety. More specifically,
the tail call will replace the global subprog frame, but not the one for the
caller, so it should be fine. Throwing exceptions is different.

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Test global subprog callback contexts

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1339 bytes --]

> diff --git a/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c b/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c
> index 6bf95550a024..987ca9dec6b7 100644
> --- a/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c
> +++ b/tools/testing/selftests/bpf/progs/verifier_async_cb_context.c

[ ... ]

> +__noinline int wq_global_acquire(void)
> +{
> +	struct task_struct *task, *acquired;
> +	struct wq_elem *val;
> +	int key = 0;
> +
> +	val = bpf_map_lookup_elem(&wq_map, &key);
> +	if (!val)
> +		return 0;
> +
> +	task = val->task;
> +	if (!task)
> +		return 0;
> +
> +	acquired = bpf_task_acquire(task);
> +	if (acquired)
> +		bpf_task_release(acquired);
> +	return 0;
> +}

[ ... ]

> +/* Task work tests */

[ ... ]

> +static int task_work_global_rcu_cb(struct bpf_map *map, void *key, void *value)
> +{
> +	return wq_global_acquire();
> +}
                  ^^^

This isn't a bug, but since wq_global_acquire() is now shared between
the workqueue and task-work test sections, would a neutral name like
global_acquire() read better than the wq_ prefix here?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/33947189666

Re: [PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 8133 bytes --]

> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index 3a7eb2185c35..66d04244c737 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -1650,8 +1650,9 @@ static inline void bpf_trampoline_set_flags(struct bpf_trampoline *tr, u32 flags
>  struct bpf_func_info_aux {
>  	u16 linkage;
>  	bool unreliable;
> -	bool called : 1;
> -	bool verified : 1;
> +	/* Indexed by in_sleepable. */
> +	bool called[2];
> +	bool verified[2];
>  };

[ ... ]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5b51e7ee1a3f..f759a020c8a5 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -9957,6 +9957,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  	if (err == -EFAULT)
>  		return err;
>  	if (bpf_subprog_is_global(env, subprog)) {
> +		struct bpf_func_info_aux *sub_aux = subprog_aux(env, subprog);
>  		const char *sub_name = bpf_subprog_name(env, subprog);
>  		const char *operation;
>  		bool returns_void;
> @@ -9988,11 +9989,10 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  		if (env->log.level & BPF_LOG_LEVEL)
>  			verbose(env, "Func#%d ('%s') is global and assumed valid.\n",
>  				subprog, sub_name);
> +		sub_aux->called[in_sleepable_context(env)] = true;
>  		returns_void = subprog_returns_void(env, subprog);
>  		if (env->subprog_info[subprog].changes_pkt_data)
>  			clear_all_pkt_pointers(env);
> -		/* mark global subprog for verifying after main prog */
> -		subprog_aux(env, subprog)->called = true;
>  		if (returns_void)
>  			bpf_diag_record_scrub(env, &caller->regs[BPF_REG_0], BPF_DIAG_MOD_CALLER_SAVED);
>  		else

[ ... ]

> @@ -19560,13 +19565,14 @@ static void free_states(struct bpf_verifier_env *env)
>  	}
>  }
>
> -static int do_check_common(struct bpf_verifier_env *env, int subprog)
> +static int do_check_common(struct bpf_verifier_env *env, int subprog, bool in_sleepable)
>  {
>  	bool pop_log = !(env->log.level & BPF_LOG_LEVEL2);
>  	struct bpf_subprog_info *sub = subprog_info(env, subprog);
>  	struct bpf_prog_aux *aux = env->prog->aux;
>  	struct bpf_verifier_state *state;
>  	struct bpf_reg_state *regs;
> +	u32 old_insns_total = sub->insns_total;
>  	u32 insn_processed = env->insn_processed;
>  	int ret, i;
>
> @@ -19579,7 +19585,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>  	state->curframe = 0;
>  	state->speculative = false;
>  	state->branches = 1;
> -	state->in_sleepable = env->prog->sleepable;
> +	state->in_sleepable = in_sleepable;
>  	state->frame[0] = kzalloc_obj(struct bpf_func_state, GFP_KERNEL_ACCOUNT);
>  	if (!state->frame[0]) {
>  		kfree(state);

Can this change break check_css_task_iter_allowlist()?

That function currently uses in_sleepable(env) to decide whether to allow
bpf_iter_css_task_new(), per its comment:

  "Any sleepable progs are also safe since bpf_check_attach_target() enforce
   them can only be attached to some specific hook points."

Before this patch, do_check_common() always set state->in_sleepable =
env->prog->sleepable, so in_sleepable(env) matched the program-level
sleepability even for global subprogs. After this patch, when a global
subprog is called from inside bpf_rcu_read_lock() or a similar protected
region in a sleepable program, in_sleepable_context(env) is false in
check_func_call(), so the global subprog is verified with in_sleepable ==
false.

If that global subprog then calls bpf_iter_css_task_new(),
check_css_task_iter_allowlist() sees in_sleepable(env) == false and rejects
it with "css_task_iter is only allowed in bpf_lsm, bpf_iter and sleepable
progs", even though:

1. The iterator itself never sleeps (it is marked only KF_ITER_NEW, not
   KF_SLEEPABLE, so it only takes spin_lock_irq(&css_set_lock));

2. The allowlist check is meant to prove the attach point does not already
   hold css_set_lock, which is a property of the program's attach type, not
   of the current RCU/preempt nesting;

3. The identical call written directly in the main program under the same
   bpf_rcu_read_lock() is still accepted, because there in_sleepable remains
   true. Only the version factored into a global subprog is newly rejected.

Should check_css_task_iter_allowlist() consult env->prog->sleepable directly
instead of in_sleepable(env), so that normalizing a protected sleepable caller
to in_sleepable=false at the global-subprog root does not affect allowlist
decisions?

> @@ -19721,7 +19727,8 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>  	 * Accumulate their total counts as total counts of the main or
>  	 * global subprog hosting the async call.
>  	 */
> -	env->subprog_info[subprog].insns_total = env->insn_processed - insn_processed;
> +	env->subprog_info[subprog].insns_total = old_insns_total +
> +						(env->insn_processed - insn_processed);
>  	return ret;
>  }

[ ... ]

> @@ -19749,45 +19756,55 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
>  {
>  	struct bpf_prog_aux *aux = env->prog->aux;
>  	struct bpf_func_info_aux *sub_aux;
> -	int i, ret, new_cnt;
> +	int context, i, j, ret, new_cnt;
>
>  	if (!aux->func_info)
>  		return 0;
>
>  	/* exception callback is presumed to be always called */
> -	if (env->exception_callback_subprog)
> -		subprog_aux(env, env->exception_callback_subprog)->called = true;
> +	if (env->exception_callback_subprog) {
> +		sub_aux = subprog_aux(env, env->exception_callback_subprog);
> +		sub_aux->called[env->prog->sleepable] = true;
> +	}
>
>  again:
>  	new_cnt = 0;
> -	for (i = 1; i < env->subprog_cnt; i++) {
> +	/*
> +	 * Walk callers before callees so each global subprog normally sees all
> +	 * of its contexts before it is verified. Async callback cycles can add a
> +	 * context to an earlier subprog, so repeat until every called context is
> +	 * verified.
> +	 */
> +	for (j = env->subprog_cnt - 1; j >= 0; j--) {
> +		i = env->subprog_topo_order[j];
> +		if (!i)
> +			continue;
>  		if (!bpf_subprog_is_global(env, i))
>  			continue;
>
>  		sub_aux = subprog_aux(env, i);
> -		if (!sub_aux->called || sub_aux->verified)
> -			continue;
> +		for (context = 0; context < ARRAY_SIZE(sub_aux->called); context++) {
> +			if (!sub_aux->called[context] || sub_aux->verified[context])
> +				continue;
>
> -		env->insn_idx = env->subprog_info[i].start;
> -		WARN_ON_ONCE(env->insn_idx == 0);
> -		ret = do_check_common(env, i);
> -		if (ret) {
> -			return ret;
> -		} else if (env->log.level & BPF_LOG_LEVEL) {
> -			verbose(env, "Func#%d ('%s') is safe for any args that match its prototype\n",
> -				i, bpf_subprog_name(env, i));
> -		}
> +			env->insn_idx = env->subprog_info[i].start;
> +			WARN_ON_ONCE(env->insn_idx == 0);
> +			ret = do_check_common(env, i, context);
> +			if (ret)
> +				return ret;
> +			if (env->log.level & BPF_LOG_LEVEL)
> +				verbose(env, "Func#%d ('%s') is safe for any args "
> +					"that match its prototype\n",
> +					i, bpf_subprog_name(env, i));
>
> -		/* We verified new global subprog, it might have called some
> -		 * more global subprogs that we haven't verified yet, so we
> -		 * need to do another pass over subprogs to verify those.
> -		 */
> -		sub_aux->verified = true;
> -		new_cnt++;
> +			sub_aux->verified[context] = true;
> +			new_cnt++;
> +		}
>  	}
>
> -	/* We can't loop forever as we verify at least one global subprog on
> -	 * each pass.
> +	/*
> +	 * We can't loop forever as each pass verifies at least one new context,
> +	 * and there are only two contexts per global subprog.
>  	 */
>  	if (new_cnt)
>  		goto again;
> @@ -19800,7 +19817,7 @@ static int do_check_main(struct bpf_verifier_env *env)
>  	int ret;
>
>  	env->insn_idx = 0;
> -	ret = do_check_common(env, 0);
> +	ret = do_check_common(env, 0, env->prog->sleepable);
>  	if (!ret)
>  		env->prog->aux->stack_depth = env->subprog_info[0].stack_depth;
>  	return ret;

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/33947189666

Re: [PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[Kumar Kartikeya Dwivedi]

On Sat Sep 5, 2026 at 8:05 AM CEST, bot+bpf-ci wrote:
>> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
>> index 3a7eb2185c35..66d04244c737 100644
>> --- a/include/linux/bpf.h
>> +++ b/include/linux/bpf.h
>> @@ -1650,8 +1650,9 @@ static inline void bpf_trampoline_set_flags(struct bpf_trampoline *tr, u32 flags
>>  struct bpf_func_info_aux {
>>  	u16 linkage;
>>  	bool unreliable;
>> -	bool called : 1;
>> -	bool verified : 1;
>> +	/* Indexed by in_sleepable. */
>> +	bool called[2];
>> +	bool verified[2];
>>  };
>
> [ ... ]
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index 5b51e7ee1a3f..f759a020c8a5 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
>> @@ -9957,6 +9957,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>>  	if (err == -EFAULT)
>>  		return err;
>>  	if (bpf_subprog_is_global(env, subprog)) {
>> +		struct bpf_func_info_aux *sub_aux = subprog_aux(env, subprog);
>>  		const char *sub_name = bpf_subprog_name(env, subprog);
>>  		const char *operation;
>>  		bool returns_void;
>> @@ -9988,11 +9989,10 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>>  		if (env->log.level & BPF_LOG_LEVEL)
>>  			verbose(env, "Func#%d ('%s') is global and assumed valid.\n",
>>  				subprog, sub_name);
>> +		sub_aux->called[in_sleepable_context(env)] = true;
>>  		returns_void = subprog_returns_void(env, subprog);
>>  		if (env->subprog_info[subprog].changes_pkt_data)
>>  			clear_all_pkt_pointers(env);
>> -		/* mark global subprog for verifying after main prog */
>> -		subprog_aux(env, subprog)->called = true;
>>  		if (returns_void)
>>  			bpf_diag_record_scrub(env, &caller->regs[BPF_REG_0], BPF_DIAG_MOD_CALLER_SAVED);
>>  		else
>
> [ ... ]
>
>> @@ -19560,13 +19565,14 @@ static void free_states(struct bpf_verifier_env *env)
>>  	}
>>  }
>>
>> -static int do_check_common(struct bpf_verifier_env *env, int subprog)
>> +static int do_check_common(struct bpf_verifier_env *env, int subprog, bool in_sleepable)
>>  {
>>  	bool pop_log = !(env->log.level & BPF_LOG_LEVEL2);
>>  	struct bpf_subprog_info *sub = subprog_info(env, subprog);
>>  	struct bpf_prog_aux *aux = env->prog->aux;
>>  	struct bpf_verifier_state *state;
>>  	struct bpf_reg_state *regs;
>> +	u32 old_insns_total = sub->insns_total;
>>  	u32 insn_processed = env->insn_processed;
>>  	int ret, i;
>>
>> @@ -19579,7 +19585,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>>  	state->curframe = 0;
>>  	state->speculative = false;
>>  	state->branches = 1;
>> -	state->in_sleepable = env->prog->sleepable;
>> +	state->in_sleepable = in_sleepable;
>>  	state->frame[0] = kzalloc_obj(struct bpf_func_state, GFP_KERNEL_ACCOUNT);
>>  	if (!state->frame[0]) {
>>  		kfree(state);
>
> Can this change break check_css_task_iter_allowlist()?
>
> That function currently uses in_sleepable(env) to decide whether to allow
> bpf_iter_css_task_new(), per its comment:
>
>   "Any sleepable progs are also safe since bpf_check_attach_target() enforce
>    them can only be attached to some specific hook points."
>
> Before this patch, do_check_common() always set state->in_sleepable =
> env->prog->sleepable, so in_sleepable(env) matched the program-level
> sleepability even for global subprogs. After this patch, when a global
> subprog is called from inside bpf_rcu_read_lock() or a similar protected
> region in a sleepable program, in_sleepable_context(env) is false in
> check_func_call(), so the global subprog is verified with in_sleepable ==
> false.
>
> If that global subprog then calls bpf_iter_css_task_new(),
> check_css_task_iter_allowlist() sees in_sleepable(env) == false and rejects
> it with "css_task_iter is only allowed in bpf_lsm, bpf_iter and sleepable
> progs", even though:
>
> 1. The iterator itself never sleeps (it is marked only KF_ITER_NEW, not
>    KF_SLEEPABLE, so it only takes spin_lock_irq(&css_set_lock));
>
> 2. The allowlist check is meant to prove the attach point does not already
>    hold css_set_lock, which is a property of the program's attach type, not
>    of the current RCU/preempt nesting;
>
> 3. The identical call written directly in the main program under the same
>    bpf_rcu_read_lock() is still accepted, because there in_sleepable remains
>    true. Only the version factored into a global subprog is newly rejected.
>
> Should check_css_task_iter_allowlist() consult env->prog->sleepable directly
> instead of in_sleepable(env), so that normalizing a protected sleepable caller
> to in_sleepable=false at the global-subprog root does not affect allowlist
> decisions?

No, it should not, the behavior in the patch is correct for this case.

>
>> @@ -19721,7 +19727,8 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>>  	 * Accumulate their total counts as total counts of the main or
>>  	 * global subprog hosting the async call.
>>  	 */
>> -	env->subprog_info[subprog].insns_total = env->insn_processed - insn_processed;
>> +	env->subprog_info[subprog].insns_total = old_insns_total +
>> +						(env->insn_processed - insn_processed);
>>  	return ret;
>>  }
>
> [ ... ]
>
>> @@ -19749,45 +19756,55 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
>>  {
>>  	struct bpf_prog_aux *aux = env->prog->aux;
>>  	struct bpf_func_info_aux *sub_aux;
>> -	int i, ret, new_cnt;
>> +	int context, i, j, ret, new_cnt;
>>
>>  	if (!aux->func_info)
>>  		return 0;
>>
>>  	/* exception callback is presumed to be always called */
>> -	if (env->exception_callback_subprog)
>> -		subprog_aux(env, env->exception_callback_subprog)->called = true;
>> +	if (env->exception_callback_subprog) {
>> +		sub_aux = subprog_aux(env, env->exception_callback_subprog);
>> +		sub_aux->called[env->prog->sleepable] = true;
>> +	}
>>
>>  again:
>>  	new_cnt = 0;
>> -	for (i = 1; i < env->subprog_cnt; i++) {
>> +	/*
>> +	 * Walk callers before callees so each global subprog normally sees all
>> +	 * of its contexts before it is verified. Async callback cycles can add a
>> +	 * context to an earlier subprog, so repeat until every called context is
>> +	 * verified.
>> +	 */
>> +	for (j = env->subprog_cnt - 1; j >= 0; j--) {
>> +		i = env->subprog_topo_order[j];
>> +		if (!i)
>> +			continue;
>>  		if (!bpf_subprog_is_global(env, i))
>>  			continue;
>>
>>  		sub_aux = subprog_aux(env, i);
>> -		if (!sub_aux->called || sub_aux->verified)
>> -			continue;
>> +		for (context = 0; context < ARRAY_SIZE(sub_aux->called); context++) {
>> +			if (!sub_aux->called[context] || sub_aux->verified[context])
>> +				continue;
>>
>> -		env->insn_idx = env->subprog_info[i].start;
>> -		WARN_ON_ONCE(env->insn_idx == 0);
>> -		ret = do_check_common(env, i);
>> -		if (ret) {
>> -			return ret;
>> -		} else if (env->log.level & BPF_LOG_LEVEL) {
>> -			verbose(env, "Func#%d ('%s') is safe for any args that match its prototype\n",
>> -				i, bpf_subprog_name(env, i));
>> -		}
>> +			env->insn_idx = env->subprog_info[i].start;
>> +			WARN_ON_ONCE(env->insn_idx == 0);
>> +			ret = do_check_common(env, i, context);
>> +			if (ret)
>> +				return ret;
>> +			if (env->log.level & BPF_LOG_LEVEL)
>> +				verbose(env, "Func#%d ('%s') is safe for any args "
>> +					"that match its prototype\n",
>> +					i, bpf_subprog_name(env, i));
>>
>> -		/* We verified new global subprog, it might have called some
>> -		 * more global subprogs that we haven't verified yet, so we
>> -		 * need to do another pass over subprogs to verify those.
>> -		 */
>> -		sub_aux->verified = true;
>> -		new_cnt++;
>> +			sub_aux->verified[context] = true;
>> +			new_cnt++;
>> +		}
>>  	}
>>
>> -	/* We can't loop forever as we verify at least one global subprog on
>> -	 * each pass.
>> +	/*
>> +	 * We can't loop forever as each pass verifies at least one new context,
>> +	 * and there are only two contexts per global subprog.
>>  	 */
>>  	if (new_cnt)
>>  		goto again;
>> @@ -19800,7 +19817,7 @@ static int do_check_main(struct bpf_verifier_env *env)
>>  	int ret;
>>
>>  	env->insn_idx = 0;
>> -	ret = do_check_common(env, 0);
>> +	ret = do_check_common(env, 0, env->prog->sleepable);
>>  	if (!ret)
>>  		env->prog->aux->stack_depth = env->subprog_info[0].stack_depth;
>>  	return ret;
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/33947189666

Re: [PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[Alexei Starovoitov]

On Fri Sep 4, 2026 at 10:12 PM PDT, Kumar Kartikeya Dwivedi wrote:
>
> Record whether each global subprogram is called with either in_sleepable
> value and verify it once for every observed value.
...

> Since a global subprogram may now be verified twice, accumulate both passes
> in subprog_info[].insns_total.

Only after reading the patch carefully I realized that above is saying 'may'.
Just by reading commit log it sounds that all global progs are now verified twice.

Please reword to make it clear that glob progs are verifier in the specific
context when it's actually reachable.

>  
> -/* Check if we're in a sleepable context. */
> +/*
> + * This predicate is the inverse of in_rcu_cs(): non-sleepable programs and
> + * every condition that prevents sleeping also provide RCU protection. Global
> + * subprog verification relies on this equivalence to represent the caller's
> + * execution context using only the in_sleepable bit.
> + */
>  static inline bool in_sleepable_context(struct bpf_verifier_env *env)
>  {
>  	return !env->cur_state->active_rcu_locks &&

Since it is !in_rcu_cs() let's make it so in the code.

pw-bot: cr

Re: [PATCH bpf-next v2 1/2] bpf: Verify global subprogs in each sleepability context

[Eduard Zingerman]

On Sat, 2026-09-05 at 07:12 +0200, Kumar Kartikeya Dwivedi wrote:
> Global subprograms are verified independently with a fresh verifier root.
> do_check_common() currently seeds that root's in_sleepable state from the
> program, even though a global subprogram can also run from callbacks whose
> execution context differs from the program's main entry point.
> 
> In particular, workqueue and task-work callbacks are sleepable even when
> the containing program is not. A global subprogram of that program is
> therefore verified as non-sleepable, making in_rcu_cs() true and allowing
> loads of RCU-protected kptrs to produce trusted MEM_RCU pointers. The same
> subprogram can then be called from a sleepable callback without a classic
> RCU reader. It can retain such a pointer while the object is freed and use
> it after free.
> 
> The verifier's execution-context predicates are complementary. A state is
> sleepable only when in_sleepable is set and no RCU, preemption, IRQ, or lock
> region is active. Each condition which prevents sleeping also provides RCU
> protection, while in_rcu_cs() treats a non-sleepable state as implicitly
> protected.
> 
> Use this relationship to represent a global subprogram caller with only the
> result of in_sleepable_context(). A protected sleepable caller is normalized
> to in_sleepable=false at the independent verification root. This both
> prevents sleepable operations and makes in_rcu_cs() true without copying
> caller-owned lock state.
> 
> Record whether each global subprogram is called with either in_sleepable
> value and verify it once for every observed value. Walk global subprograms
> in caller-before-callee order so the values propagate through global call
> chains, and repeat until every discovered context has been verified to cover
> asynchronous callback cycles.
> 
> Since a global subprogram may now be verified twice, accumulate both passes
> in subprog_info[].insns_total. This makes BPF_LOG_STATS and per-subprogram
> veristat output account for both contexts instead of reporting only the last
> pass.
> 
> This makes an unprotected callback verify the global subprogram as
> sleepable, turning its RCU-protected kptr load into an untrusted pointer.
> Protected callers and global subprograms which do not depend on implicit RCU
> protection remain valid.
> 
> Fixes: 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl")
> Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs")
> Reported-by: Nicholas Carlini <npc@anthropic.com>
> Suggested-by: Nicholas Carlini <npc@anthropic.com>
> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
> ---

Hi Kartikeya,

Apologies for the delayed review. Overall the code lgtm, please see a
few questions and nits below.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5b51e7ee1a3f..f759a020c8a5 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

...

> @@ -10804,7 +10804,12 @@ int bpf_get_helper_proto(struct bpf_verifier_env *env, int func_id,
>  	return *ptr && (*ptr)->func ? 0 : -EINVAL;
>  }
>  
> -/* Check if we're in a sleepable context. */
> +/*
> + * This predicate is the inverse of in_rcu_cs(): non-sleepable programs and
> + * every condition that prevents sleeping also provide RCU protection. Global
> + * subprog verification relies on this equivalence to represent the caller's
> + * execution context using only the in_sleepable bit.
> + */

Agree with Alexei, let's just write it down as !in_rcu_cs().
I think it's a second time we argue :)
For the purposes of this patch-set one can just drop the comment above.

>  static inline bool in_sleepable_context(struct bpf_verifier_env *env)
>  {
>  	return !env->cur_state->active_rcu_locks &&
> @@ -19560,13 +19565,14 @@ static void free_states(struct bpf_verifier_env *env)
>  	}
>  }
>  
> -static int do_check_common(struct bpf_verifier_env *env, int subprog)
> +static int do_check_common(struct bpf_verifier_env *env, int subprog, bool in_sleepable)
>  {
>  	bool pop_log = !(env->log.level & BPF_LOG_LEVEL2);
>  	struct bpf_subprog_info *sub = subprog_info(env, subprog);
>  	struct bpf_prog_aux *aux = env->prog->aux;
>  	struct bpf_verifier_state *state;
>  	struct bpf_reg_state *regs;
> +	u32 old_insns_total = sub->insns_total;
>  	u32 insn_processed = env->insn_processed;
>  	int ret, i;
>  
> @@ -19579,7 +19585,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>  	state->curframe = 0;
>  	state->speculative = false;
>  	state->branches = 1;
> -	state->in_sleepable = env->prog->sleepable;
> +	state->in_sleepable = in_sleepable;
>  	state->frame[0] = kzalloc_obj(struct bpf_func_state, GFP_KERNEL_ACCOUNT);
>  	if (!state->frame[0]) {
>  		kfree(state);
> @@ -19721,7 +19727,8 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>  	 * Accumulate their total counts as total counts of the main or
>  	 * global subprog hosting the async call.
>  	 */
> -	env->subprog_info[subprog].insns_total = env->insn_processed - insn_processed;
> +	env->subprog_info[subprog].insns_total = old_insns_total +
> +						(env->insn_processed - insn_processed);

Wouldn't += work w/o old_insns_total temporary?

>  	return ret;
>  }
>  
> @@ -19749,45 +19756,55 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
>  {
>  	struct bpf_prog_aux *aux = env->prog->aux;
>  	struct bpf_func_info_aux *sub_aux;
> -	int i, ret, new_cnt;
> +	int context, i, j, ret, new_cnt;
>  
>  	if (!aux->func_info)
>  		return 0;
>  
>  	/* exception callback is presumed to be always called */
> -	if (env->exception_callback_subprog)
> -		subprog_aux(env, env->exception_callback_subprog)->called = true;
> +	if (env->exception_callback_subprog) {
> +		sub_aux = subprog_aux(env, env->exception_callback_subprog);
> +		sub_aux->called[env->prog->sleepable] = true;

Do we allow to call throw from the callbacks?
If we do, do we handle the `called[sleepable|not-sleepable]`
management for exception callbacks?

> +	}
>  
>  again:
>  	new_cnt = 0;
> -	for (i = 1; i < env->subprog_cnt; i++) {
> +	/*
> +	 * Walk callers before callees so each global subprog normally sees all
> +	 * of its contexts before it is verified. Async callback cycles can add a
> +	 * context to an earlier subprog, so repeat until every called context is
> +	 * verified.
> +	 */
> +	for (j = env->subprog_cnt - 1; j >= 0; j--) {
> +		i = env->subprog_topo_order[j];
> +		if (!i)
> +			continue;

Nit: is it really necessary to explicitly change the traversal order here?
     the algorithm already explores the function only when 'called' flag is set.

>  		if (!bpf_subprog_is_global(env, i))
>  			continue;
>  
>  		sub_aux = subprog_aux(env, i);
> -		if (!sub_aux->called || sub_aux->verified)
> -			continue;
> +		for (context = 0; context < ARRAY_SIZE(sub_aux->called); context++) {
> +			if (!sub_aux->called[context] || sub_aux->verified[context])
> +				continue;
>  
> -		env->insn_idx = env->subprog_info[i].start;
> -		WARN_ON_ONCE(env->insn_idx == 0);
> -		ret = do_check_common(env, i);
> -		if (ret) {
> -			return ret;
> -		} else if (env->log.level & BPF_LOG_LEVEL) {
> -			verbose(env, "Func#%d ('%s') is safe for any args that match its prototype\n",
> -				i, bpf_subprog_name(env, i));
> -		}
> +			env->insn_idx = env->subprog_info[i].start;
> +			WARN_ON_ONCE(env->insn_idx == 0);
> +			ret = do_check_common(env, i, context);
> +			if (ret)
> +				return ret;
> +			if (env->log.level & BPF_LOG_LEVEL)
> +				verbose(env, "Func#%d ('%s') is safe for any args "
> +					"that match its prototype\n",
> +					i, bpf_subprog_name(env, i));
>  
> -		/* We verified new global subprog, it might have called some
> -		 * more global subprogs that we haven't verified yet, so we
> -		 * need to do another pass over subprogs to verify those.
> -		 */
> -		sub_aux->verified = true;
> -		new_cnt++;
> +			sub_aux->verified[context] = true;
> +			new_cnt++;
> +		}
>  	}
>  
> -	/* We can't loop forever as we verify at least one global subprog on
> -	 * each pass.
> +	/*
> +	 * We can't loop forever as each pass verifies at least one new context,
> +	 * and there are only two contexts per global subprog.
>  	 */
>  	if (new_cnt)
>  		goto again;

...

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Test global subprog callback contexts

[Eduard Zingerman]

On Sat, 2026-09-05 at 07:12 +0200, Kumar Kartikeya Dwivedi wrote:
> Exercise global subprogram verification from workqueue and task-work
> callbacks. Both callback types can run in a sleepable context even when the
> containing program is not sleepable, so an unprotected callback must not let
> the global subprogram use implicit RCU protection inherited from the program.
> 
> Add negative cases which load an RCU-protected task kptr in a global
> subprogram reached from each callback type. The tests fail on an unfixed
> kernel because the programs are incorrectly accepted.
> 
> Also cover a workqueue callback protected by an explicit RCU read-side
> critical section. Finally, call the same harmless global subprogram directly
> from the main program and from an unprotected callback. This requires both
> non-sleepable and sleepable verification roots and proves that global calls
> from callbacks are not rejected wholesale.
> 
> Keep the harmless global opaque to LLVM with barrier() so both call sites

__weak is another useful trick for such things.

> remain in the generated BPF object and the positive case cannot pass
> vacuously. Have workqueue callbacks return zero explicitly after calling a
> global subprogram, as required by their callback contract, instead of relying
> on interprocedural return-value optimization.
> 
> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

> @@ -179,3 +285,29 @@ int task_work_sleepable_prog(void *ctx)
>  	bpf_task_work_schedule_resume(task, &val->tw, &task_work_map, task_work_cb);
>  	return 0;
>  }
> +
> +static int task_work_global_rcu_cb(struct bpf_map *map, void *key, void *value)
> +{
> +	return wq_global_acquire();
> +}
> +
> +SEC("fentry/bpf_fentry_test1")
> +__failure __msg("R1 must be a rcu pointer")
> +int task_work_global_rcu_prog(void *ctx)
> +{
> +	struct task_work_elem *val;
> +	struct task_struct *task;
> +	int key = 0;
> +
> +	val = bpf_map_lookup_elem(&task_work_map, &key);
> +	if (!val)
> +		return 0;
> +
> +	task = bpf_get_current_task_btf();
> +	if (!task)
> +		return 0;
> +
> +	bpf_task_work_schedule_resume(task, &val->tw, &task_work_map,
> +				      task_work_global_rcu_cb);
> +	return 0;
> +}

I think this test is redundant.