[PATCH bpf-next v6 0/6] Improve stack depth verification stats output

[PATCH bpf-next v6 0/6] Improve stack depth verification stats output

[Kumar Kartikeya Dwivedi]

Some improvements for more clarity in the stack depth verification
statistics output. See commit logs for details.

For example, ./test_progs -t subprogs/subprogs_alone loads prog4,
which has a main program, two static subprograms, and two independently
verified global subprograms. A sample run produces:

  verification time 1765 usec
  stack depth max 48
  subprog 0 (prog4) main insns_own 29 insns_total 51 stack 8
  subprog 1 (get_task_tgid) global insns_own 9 insns_total 9 stack 8
  subprog 2 (sub4) static insns_own 15 insns_total 22 stack 8
  subprog 3 (sub3) static insns_own 7 insns_total 7 stack 0
  subprog 4 (sub1) global insns_own 10 insns_total 10 stack 8
  processed 70 insns (limit 1000000) max_states_per_insn 0 total_states 7 peak_states 7 mark_read 0

The insns_own counts account for every processed instruction exactly once:

  29 + 9 + 15 + 7 + 10 = 70

The main program and global subprograms are independent exploration roots,
so their insns_total counts also account for the full processed budget:

  51 + 9 + 10 = 70

Static subprogram totals provide a nested, top-down breakdown inside their
root. In this example:

  sub4: 22 = 15 own + 7 in sub3
  prog4: 51 = 29 own + 22 in sub4

The global subprogram bodies are accounted in their own root totals rather
than being included in prog4 or the static callees which call them.

Asynchronous callback work is propagated through every scheduling subprogram
in a nested callback chain. Running:

  ./test_progs -t verifier_subprog_insn_stats/stats_async_nested -v

produces the following stats:

  stack depth max 0
  subprog 0 (stats_async_nested) main insns_own 9 insns_total 23 stack 0
  subprog 1 (stats_async_nested_leaf) static insns_own 2 insns_total 2 stack 0
  subprog 2 (stats_async_outer) static insns_own 6 insns_total 8 stack 0
  subprog 3 (stats_async_nested_schedule) static insns_own 6 insns_total 14 stack 0
  processed 23 insns

Here, 9 + 2 + 6 + 6 = 23. The nested callback work is propagated
bottom-up through both scheduling subprograms:

  stats_async_outer: 8 = 6 own + 2 in stats_async_nested_leaf
  stats_async_nested_schedule: 14 = 6 own + 8 in stats_async_outer
  stats_async_nested: 23 = 9 own + 14 in stats_async_nested_schedule

Changelog:
----------
v5 -> v6
v5: https://lore.kernel.org/bpf/20260804081114.3871564-1-memxor@gmail.com

 * Track own and inclusive instruction counts for main, global, and static
   subprograms. (Andrii, Eduard)
 * Keep instruction subtotals path-local across verifier state copies.
 * Propagate async callback budget through nested scheduling chains. (Andrii)
 * Split per-subprogram instruction accounting into a preparatory patch.
 * Add deterministic selftests with exact own, total, and processed counts.

v4 -> v5
v4: https://lore.kernel.org/bpf/20260803072733.191502-1-memxor@gmail.com

 * Change the format to combine instruction counts and stack depths into
   per-program records. (Andrii)
 * Adjust veristat for the new format while retaining support for the legacy format.
 * Explain why the legacy stack parsing buffer is zero-initialized. (BPF CI Bot)

v3 -> v4
v3: https://lore.kernel.org/bpf/20260803031457.3115812-1-memxor@gmail.com

 * Read names from subprog_info directly to avoid an out-of-bounds access
   when func_info validation fails. (BPF CI Bot)

v2 -> v3
v2: https://lore.kernel.org/bpf/20260802225209.2511758-1-memxor@gmail.com

 * Reuse subprog_name() to fetch subprogram names. (BPF CI Bot)

v1 -> v2
v1: https://lore.kernel.org/bpf/20260801230400.850271-1-memxor@gmail.com

 * Use multi-line format. (Eduard)
 * Adjust veristat to work with old and new format.
 * Adjust selftest log_level without new option. (Eduard)

Kumar Kartikeya Dwivedi (6):
  bpf: Track verifier instruction stats for each subprogram
  bpf: Propagate async callback instructions to scheduling subprograms
  bpf: Show more useful info in stack depth stats
  selftests/bpf: Adjust veristat stack depth parsing
  selftests/bpf: Test stack depth stats without BTF subprog names
  selftests/bpf: Test subprogram instruction statistics

 include/linux/bpf_verifier.h                  |   7 +-
 kernel/bpf/verifier.c                         |  88 +++++--
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_basic_stack.c          |   8 +-
 .../bpf/progs/verifier_bpf_fastcall.c         |  38 ++-
 .../bpf/progs/verifier_global_subprogs.c      |   6 +-
 .../bpf/progs/verifier_private_stack.c        |  22 +-
 .../bpf/progs/verifier_subprog_insn_stats.c   | 225 ++++++++++++++++++
 .../selftests/bpf/progs/verifier_var_off.c    |   8 +-
 tools/testing/selftests/bpf/test_verifier.c   |   2 +-
 tools/testing/selftests/bpf/verifier/calls.c  |  12 +-
 tools/testing/selftests/bpf/veristat.c        |  16 +-
 12 files changed, 395 insertions(+), 39 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c


base-commit: 6655c409707ec8ce9ce0850ffe4fe02331fd4d9c
-- 
2.53.0

[PATCH bpf-next v6 1/6] bpf: Track verifier instruction stats for each subprogram

[Kumar Kartikeya Dwivedi]

The verifier currently records one instruction count for the main program
and each global subprogram checked independently. Static subprograms are
explored within callers, so their verification cost cannot be reported
separately.

Track both own and inclusive instruction counts for every subprogram.
Charge each processed instruction as own work to the current subprogram and
to a path-local subtotal in its function frame. When a function returns, add
the callee subtotal to its inclusive count and to its parent subtotal. Fold
any remaining frames when a path terminates or is pruned.

Instruction subtotals are accounting state, not semantic verifier state.
Clear them when a verifier state is copied so work before a path fork is
charged once, rather than again when a saved branch is explored.

This generic frame accounting also records own and inclusive totals when an
asynchronous callback starts as a fresh frame-zero state. It does not yet
charge that independently explored callback path back to the main or global
exploration root which scheduled it. That will be done in subsequent changes.

This does not change the verification statistics output format. It only
prepares the counters for per-subprogram reporting.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 include/linux/bpf_verifier.h |  5 +++-
 kernel/bpf/verifier.c        | 50 +++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 10 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index a2a40caca0a0..9de45ade473b 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -385,6 +385,8 @@ struct bpf_func_state {
 	 *                           | number of simulations is tracked in frame N
 	 */
 	u32 callback_depth;
+	/* Instructions processed in this frame and callees on the current path. */
+	u32 insns_subtotal;
 
 	/* The following fields should be last. See copy_func_state() */
 	/* The state of the stack. Each element of the array describes BPF_REG_SIZE
@@ -803,7 +805,8 @@ struct bpf_subprog_info {
 	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
 	u16 stack_depth; /* max. stack depth used by this function */
 	u16 stack_extra;
-	u32 insn_processed;
+	u32 insns_total;
+	u32 insns_own;
 	/* offsets in range [stack_depth .. fastcall_stack_off)
 	 * are used for bpf_fastcall spills and fills.
 	 */
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 7439afdc851a..88e7ea6fbe73 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -1593,6 +1593,8 @@ static int copy_func_state(struct bpf_func_state *dst,
 			   const struct bpf_func_state *src)
 {
 	memcpy(dst, src, offsetof(struct bpf_func_state, stack));
+	/* Instruction accounting is path-local, not part of verifier state. */
+	dst->insns_subtotal = 0;
 	return copy_stack_state(dst, src);
 }
 
@@ -9807,6 +9809,37 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
 
 static bool is_rbtree_lock_required_kfunc(u32 btf_id);
 
+static void account_processed_insn(struct bpf_verifier_env *env)
+{
+	struct bpf_func_state *frame = cur_func(env);
+
+	env->insn_processed++;
+	frame->insns_subtotal++;
+	env->subprog_info[frame->subprogno].insns_own++;
+}
+
+static void account_processed_insns(struct bpf_verifier_env *env,
+				    struct bpf_func_state *callee,
+				    struct bpf_func_state *caller)
+{
+	u32 insns = callee->insns_subtotal;
+
+	env->subprog_info[callee->subprogno].insns_total += insns;
+	if (caller)
+		caller->insns_subtotal += insns;
+	callee->insns_subtotal = 0;
+}
+
+static void account_current_path(struct bpf_verifier_env *env)
+{
+	struct bpf_verifier_state *state = env->cur_state;
+	int frame;
+
+	for (frame = state->curframe; frame >= 0; frame--)
+		account_processed_insns(env, state->frame[frame],
+					frame ? state->frame[frame - 1] : NULL);
+}
+
 /* Are we currently verifying the callback for a rbtree helper that must
  * be called with lock held? If so, no need to complain about unreleased
  * lock
@@ -9903,6 +9936,7 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
 		verbose(env, "to caller at %d:\n", *insn_idx);
 		print_verifier_state(env, state, caller->frameno, true);
 	}
+	account_processed_insns(env, callee, caller);
 	/* clear everything in the callee. In case of exceptional exits using
 	 * bpf_throw, this will be done by copy_verifier_state for extra frames. */
 	free_func_state(callee);
@@ -17496,7 +17530,9 @@ static int do_check(struct bpf_verifier_env *env)
 		insn = &insns[env->insn_idx];
 		insn_aux = &env->insn_aux_data[env->insn_idx];
 
-		if (++env->insn_processed > BPF_COMPLEXITY_LIMIT_INSNS) {
+		account_processed_insn(env);
+
+		if (env->insn_processed > BPF_COMPLEXITY_LIMIT_INSNS) {
 			verbose(env,
 				"BPF program is too large. Processed %d insn\n",
 				env->insn_processed);
@@ -17636,6 +17672,7 @@ static int do_check(struct bpf_verifier_env *env)
 					    "speculation barrier after jump instruction may not have the desired effect"))
 				return -EFAULT;
 process_bpf_exit:
+			account_current_path(env);
 			mark_verifier_state_scratched(env);
 			err = bpf_update_branch_counts(env, env->cur_state);
 			if (err)
@@ -18680,6 +18717,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
 
 	ret = do_check(env);
 out:
+	account_current_path(env);
 	if (!ret && pop_log)
 		bpf_vlog_reset(&env->log, 0);
 	free_states(env);
@@ -18711,7 +18749,6 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
 	struct bpf_prog_aux *aux = env->prog->aux;
 	struct bpf_func_info_aux *sub_aux;
 	int i, ret, new_cnt;
-	u32 insn_processed;
 
 	if (!aux->func_info)
 		return 0;
@@ -18726,8 +18763,6 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
 		if (!bpf_subprog_is_global(env, i))
 			continue;
 
-		insn_processed = env->insn_processed;
-
 		sub_aux = subprog_aux(env, i);
 		if (!sub_aux->called || sub_aux->verified)
 			continue;
@@ -18735,7 +18770,6 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
 		env->insn_idx = env->subprog_info[i].start;
 		WARN_ON_ONCE(env->insn_idx == 0);
 		ret = do_check_common(env, i);
-		env->subprog_info[i].insn_processed = env->insn_processed - insn_processed;
 		if (ret) {
 			return ret;
 		} else if (env->log.level & BPF_LOG_LEVEL) {
@@ -18762,12 +18796,10 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
 
 static int do_check_main(struct bpf_verifier_env *env)
 {
-	u32 insn_processed = env->insn_processed;
 	int ret;
 
 	env->insn_idx = 0;
 	ret = do_check_common(env, 0);
-	env->subprog_info[0].insn_processed = env->insn_processed - insn_processed;
 	if (!ret)
 		env->prog->aux->stack_depth = env->subprog_info[0].stack_depth;
 	return ret;
@@ -18786,10 +18818,10 @@ static void print_verification_stats(struct bpf_verifier_env *env)
 		for (i = 1; i < subprog_cnt; i++)
 			verbose(env, "+%d", env->subprog_info[i].stack_depth);
 		verbose(env, " max %d\n", env->max_stack_depth);
-		verbose(env, "insns processed %d", env->subprog_info[0].insn_processed);
+		verbose(env, "insns processed %d", env->subprog_info[0].insns_total);
 		for (i = 1; i < subprog_cnt; i++)
 			if (bpf_subprog_is_global(env, i))
-				verbose(env, "+%d", env->subprog_info[i].insn_processed);
+				verbose(env, "+%d", env->subprog_info[i].insns_total);
 		verbose(env, "\n");
 	}
 	verbose(env, "processed %d insns (limit %d) max_states_per_insn %d "
-- 
2.53.0

[PATCH bpf-next v6 2/6] bpf: Propagate async callback instructions to scheduling subprograms

[Kumar Kartikeya Dwivedi]

Asynchronous callbacks are explored as fresh frame-zero verifier states,
so normal callee-to-caller accounting cannot propagate their instruction
budget to the subprograms which scheduled them.

When an async callback is queued, save the active subprogram IDs in a fixed
async_stats_subprog_ids array and record its length. Copy this metadata with
the verifier state. When an async frame-zero path finishes, add its inclusive
subtotal to every saved scheduling subprogram.

If an async callback schedules another callback, preserve its saved IDs before
appending the active call chain. This propagates nested callback work to both
the immediate callback and the original scheduling subprograms.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 include/linux/bpf_verifier.h |  2 ++
 kernel/bpf/verifier.c        | 31 +++++++++++++++++++++++++------
 2 files changed, 27 insertions(+), 6 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 9de45ade473b..42fa464c520f 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -447,6 +447,8 @@ static_assert(MAX_BPF_STACK / 8 <= (1 << 6));
 struct bpf_verifier_state {
 	/* call stack tracking */
 	struct bpf_func_state *frame[MAX_CALL_FRAMES];
+	u32 async_stats_subprog_ids[MAX_CALL_FRAMES];
+	u32 async_stats_subprog_cnt;
 	struct bpf_verifier_state *parent;
 	/* Acquired reference states */
 	struct bpf_reference_state *refs;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 88e7ea6fbe73..47f3791530de 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -1632,6 +1632,9 @@ int bpf_copy_verifier_state(struct bpf_verifier_state *dst_state,
 	dst_state->callback_unroll_depth = src->callback_unroll_depth;
 	dst_state->may_goto_depth = src->may_goto_depth;
 	dst_state->equal_state = src->equal_state;
+	memcpy(dst_state->async_stats_subprog_ids, src->async_stats_subprog_ids,
+	       sizeof(dst_state->async_stats_subprog_ids));
+	dst_state->async_stats_subprog_cnt = src->async_stats_subprog_cnt;
 	for (i = 0; i <= src->curframe; i++) {
 		dst = dst_state->frame[i];
 		if (!dst) {
@@ -2261,6 +2264,8 @@ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env,
 {
 	struct bpf_verifier_stack_elem *elem;
 	struct bpf_func_state *frame;
+	int i;
+	u32 cnt;
 
 	elem = kzalloc_obj(struct bpf_verifier_stack_elem, GFP_KERNEL_ACCOUNT);
 	if (!elem)
@@ -2293,6 +2298,12 @@ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env,
 			0 /* frameno within this callchain */,
 			subprog /* subprog number within this prog */);
 	elem->st.frame[0] = frame;
+	cnt = env->cur_state->async_stats_subprog_cnt;
+	memcpy(elem->st.async_stats_subprog_ids, env->cur_state->async_stats_subprog_ids,
+	       cnt * sizeof(elem->st.async_stats_subprog_ids[0]));
+	for (i = 0; i <= env->cur_state->curframe; i++)
+		elem->st.async_stats_subprog_ids[cnt++] = env->cur_state->frame[i]->subprogno;
+	elem->st.async_stats_subprog_cnt = cnt;
 	return &elem->st;
 }
 
@@ -9818,9 +9829,9 @@ static void account_processed_insn(struct bpf_verifier_env *env)
 	env->subprog_info[frame->subprogno].insns_own++;
 }
 
-static void account_processed_insns(struct bpf_verifier_env *env,
-				    struct bpf_func_state *callee,
-				    struct bpf_func_state *caller)
+static u32 account_processed_insns(struct bpf_verifier_env *env,
+				   struct bpf_func_state *callee,
+				   struct bpf_func_state *caller)
 {
 	u32 insns = callee->insns_subtotal;
 
@@ -9828,16 +9839,24 @@ static void account_processed_insns(struct bpf_verifier_env *env,
 	if (caller)
 		caller->insns_subtotal += insns;
 	callee->insns_subtotal = 0;
+	return insns;
 }
 
 static void account_current_path(struct bpf_verifier_env *env)
 {
 	struct bpf_verifier_state *state = env->cur_state;
-	int frame;
+	u32 insns;
+	int frame, i;
 
 	for (frame = state->curframe; frame >= 0; frame--)
-		account_processed_insns(env, state->frame[frame],
-					frame ? state->frame[frame - 1] : NULL);
+		insns = account_processed_insns(env, state->frame[frame],
+						frame ? state->frame[frame - 1] : NULL);
+
+	if (!state->async_stats_subprog_cnt)
+		return;
+
+	for (i = 0; i < state->async_stats_subprog_cnt; i++)
+		env->subprog_info[state->async_stats_subprog_ids[i]].insns_total += insns;
 }
 
 /* Are we currently verifying the callback for a rbtree helper that must
-- 
2.53.0

[PATCH bpf-next v6 3/6] bpf: Show more useful info in stack depth stats

[Kumar Kartikeya Dwivedi]

Stack depth statistics list captured depths in subprogram-number order,
while per-verification instruction counts are reported separately. Since
libbpf determines subprogram numbers, it is hard to associate either
statistic with its subprogram name or see where verifier work is spent.

Now that own and inclusive instruction counts are available for every
subprogram, keep the combined maximum stack depth on its own line and
print one uniform record for each subprogram. Represent the main program
as subprog 0, then classify each record as main, global, or static before
reporting insns_own, insns_total, and stack depth.

The aggregate processed count is the sum of all own counts, while each
total shows verifier work rooted at that subprogram.

When no subprogram name is available, print <unknown>. Keep the existing
aggregate "processed ... insns" record unchanged for compatibility.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 kernel/bpf/verifier.c                         | 23 ++++++-----
 .../bpf/progs/verifier_basic_stack.c          |  8 +++-
 .../bpf/progs/verifier_bpf_fastcall.c         | 38 +++++++++++++++----
 .../bpf/progs/verifier_global_subprogs.c      |  6 ++-
 .../bpf/progs/verifier_private_stack.c        | 22 +++++++++--
 .../selftests/bpf/progs/verifier_var_off.c    |  8 +++-
 6 files changed, 80 insertions(+), 25 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 47f3791530de..a4ab7ee334c6 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18833,15 +18833,20 @@ static void print_verification_stats(struct bpf_verifier_env *env)
 	if (env->log.level & BPF_LOG_STATS) {
 		verbose(env, "verification time %lld usec\n",
 			div_u64(env->verification_time, 1000));
-		verbose(env, "stack depth %d", env->subprog_info[0].stack_depth);
-		for (i = 1; i < subprog_cnt; i++)
-			verbose(env, "+%d", env->subprog_info[i].stack_depth);
-		verbose(env, " max %d\n", env->max_stack_depth);
-		verbose(env, "insns processed %d", env->subprog_info[0].insns_total);
-		for (i = 1; i < subprog_cnt; i++)
-			if (bpf_subprog_is_global(env, i))
-				verbose(env, "+%d", env->subprog_info[i].insns_total);
-		verbose(env, "\n");
+		verbose(env, "stack depth max %d\n", env->max_stack_depth);
+		for (i = 0; i < subprog_cnt; i++) {
+			const char *name = env->subprog_info[i].name;
+			const char *kind;
+
+			if (!name || !name[0])
+				name = "<unknown>";
+			kind = i == 0 ? "main" :
+			       bpf_subprog_is_global(env, i) ? "global" : "static";
+			verbose(env, "subprog %d (%s) %s insns_own %d insns_total %d stack %d\n",
+				i, name, kind, env->subprog_info[i].insns_own,
+				env->subprog_info[i].insns_total,
+				env->subprog_info[i].stack_depth);
+		}
 	}
 	verbose(env, "processed %d insns (limit %d) max_states_per_insn %d "
 		"total_states %d peak_states %d mark_read %d\n",
diff --git a/tools/testing/selftests/bpf/progs/verifier_basic_stack.c b/tools/testing/selftests/bpf/progs/verifier_basic_stack.c
index fb62e09f2114..8693d959806d 100644
--- a/tools/testing/selftests/bpf/progs/verifier_basic_stack.c
+++ b/tools/testing/selftests/bpf/progs/verifier_basic_stack.c
@@ -27,7 +27,9 @@ __naked void stack_out_of_bounds(void)
 
 SEC("socket")
 __description("uninitialized stack1")
-__success __log_level(4) __msg("stack depth 8")
+__success __log_level(4)
+__msg("subprog 0 (uninitialized_stack1) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 8")
 __failure_unpriv __msg_unpriv("invalid read from stack")
 __naked void uninitialized_stack1(void)
 {
@@ -45,7 +47,9 @@ __naked void uninitialized_stack1(void)
 
 SEC("socket")
 __description("uninitialized stack2")
-__success __log_level(4) __msg("stack depth 8")
+__success __log_level(4)
+__msg("subprog 0 (uninitialized_stack2) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 8")
 __failure_unpriv __msg_unpriv("invalid read from stack")
 __naked void uninitialized_stack2(void)
 {
diff --git a/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c b/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c
index 83707faea049..2d2581fc43a5 100644
--- a/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c
+++ b/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c
@@ -10,7 +10,8 @@
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 8")
+__log_level(4)
+__msg("subprog 0 (simple) main insns_own {{[0-9]+}} insns_total {{[0-9]+}} stack 8")
 __xlated("4: r5 = 5")
 __xlated("5: r0 = ")
 __xlated("6: r0 = &(void __percpu *)(r0)")
@@ -96,7 +97,9 @@ __naked void canary_zero_spills(void)
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 16")
+__log_level(4)
+__msg("subprog 0 (wrong_reg_in_pattern1) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 16")
 __xlated("1: *(u64 *)(r10 -16) = r1")
 __xlated("...")
 __xlated("3: r0 = &(void __percpu *)(r0)")
@@ -598,7 +601,9 @@ __naked static void subprogs_use_independent_offsets_aux(void)
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 8")
+__log_level(4)
+__msg("subprog 0 (helper_call_does_not_prevent_bpf_fastcall) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 8")
 __xlated("2: r0 = &(void __percpu *)(r0)")
 __success
 __naked void helper_call_does_not_prevent_bpf_fastcall(void)
@@ -620,7 +625,9 @@ __naked void helper_call_does_not_prevent_bpf_fastcall(void)
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 24")
+__log_level(4)
+__msg("subprog 0 (may_goto_interaction_x86_64) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 24")
 /* may_goto counter at -24 */
 __xlated("0: *(u64 *)(r10 -24) =")
 /* may_goto timestamp at -16 */
@@ -661,7 +668,9 @@ __naked void may_goto_interaction_x86_64(void)
 SEC("raw_tp")
 __arch_arm64
 __arch_riscv64
-__log_level(4) __msg("stack depth 24")
+__log_level(4)
+__msg("subprog 0 (may_goto_interaction) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 24")
 /* may_goto counter at -24 */
 __xlated("0: *(u64 *)(r10 -24) =")
 /* may_goto timestamp at -16 */
@@ -708,7 +717,11 @@ __naked static void dummy_loop_callback(void)
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 32+0")
+__log_level(4)
+__msg("subprog 0 (bpf_loop_interaction1) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 32")
+__msg("subprog 1 (dummy_loop_callback) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 0")
 __xlated("2: r1 = 1")
 __xlated("3: r0 =")
 __xlated("4: r0 = &(void __percpu *)(r0)")
@@ -756,7 +769,11 @@ __naked int bpf_loop_interaction1(void)
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 40+0")
+__log_level(4)
+__msg("subprog 0 (bpf_loop_interaction2) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 40")
+__msg("subprog 1 (dummy_loop_callback) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 0")
 /* call bpf_get_smp_processor_id */
 __xlated("2: r1 = 42")
 __xlated("3: r0 =")
@@ -800,7 +817,12 @@ __naked int bpf_loop_interaction2(void)
 
 SEC("raw_tp")
 __arch_x86_64
-__log_level(4) __msg("stack depth 512+0 max 512")
+__log_level(4)
+__msg("stack depth max 512")
+__msg("subprog 0 (cumulative_stack_depth) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 512")
+__msg("subprog 1 (cumulative_stack_depth_subprog) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 0")
 /* just to print xlated version when debugging */
 __xlated("r0 = &(void __percpu *)(r0)")
 __success
diff --git a/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c b/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c
index 67dc352addfd..b6555d3095bd 100644
--- a/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c
+++ b/tools/testing/selftests/bpf/progs/verifier_global_subprogs.c
@@ -52,7 +52,11 @@ __msg("('global_calls_good_only') is global and assumed valid.")
 /* eventually global_good() is transitively validated as well */
 __msg("Validating global_good() func")
 __msg("('global_good') is safe for any args that match its prototype")
-__msg("insns processed {{[0-9]+\\+[0-9]+\\+[0-9]+$}}")
+__msg("subprog 0 (chained_global_func_calls_success) main insns_own 7 insns_total 7 stack")
+__msg("subprog {{[0-9]+}} (global_calls_good_only) global "
+      "insns_own 2 insns_total 2 stack")
+__msg("subprog {{[0-9]+}} (global_good) global insns_own 5 insns_total 5 stack")
+__msg("processed 14 insns")
 int chained_global_func_calls_success(void)
 {
 	int sum = 0;
diff --git a/tools/testing/selftests/bpf/progs/verifier_private_stack.c b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
index bb8206e10880..75bf5898283c 100644
--- a/tools/testing/selftests/bpf/progs/verifier_private_stack.c
+++ b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
@@ -86,7 +86,11 @@ __naked static void cumulative_stack_depth_subprog(void)
 SEC("kprobe")
 __description("Private stack, subtree > MAX_BPF_STACK")
 __success
-__log_level(4) __msg("stack depth 512+32 max 512")
+__log_level(4) __msg("stack depth max 512")
+__msg("subprog 0 (private_stack_nested_1) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 512")
+__msg("subprog 1 (cumulative_stack_depth_subprog) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 32")
 __arch_x86_64
 /* private stack fp for the main prog */
 __jited("	movabsq	$0x{{.*}}, %r9")
@@ -331,7 +335,15 @@ SEC("fentry/bpf_fentry_test9")
 __description("Private stack, async callback, potential nesting")
 __success __retval(0)
 __load_if_JITed()
-__log_level(4) __msg("stack depth 8+0+256+0 max 272")
+__log_level(4) __msg("stack depth max 272")
+__msg("subprog 0 (private_stack_async_callback_2) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 8")
+__msg("subprog 1 (timer_cb1) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 0")
+__msg("subprog 2 (subprog1) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 256")
+__msg("subprog 3 (subprog2) static insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 0")
 __arch_x86_64
 __jited("	subq	$0x100, %rsp")
 __arch_arm64
@@ -355,7 +367,11 @@ int private_stack_async_callback_2(void)
 SEC("fentry/bpf_fentry_test9")
 __description("private stack, max stack depth is private stack")
 __success
-__log_level(4) __msg("stack depth 8+256+0 max 256")
+__log_level(4) __msg("stack depth max 256")
+__msg("subprog 0 (private_stack_max_depth) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 8")
+__msg("subprog 1 (subprog1) static insns_own {{[0-9]+}} insns_total {{[0-9]+}} stack 256")
+__msg("subprog 2 (subprog2) static insns_own {{[0-9]+}} insns_total {{[0-9]+}} stack 0")
 int private_stack_max_depth(void)
 {
 	int x = 0;
diff --git a/tools/testing/selftests/bpf/progs/verifier_var_off.c b/tools/testing/selftests/bpf/progs/verifier_var_off.c
index 24cd0a763673..48be0195f85e 100644
--- a/tools/testing/selftests/bpf/progs/verifier_var_off.c
+++ b/tools/testing/selftests/bpf/progs/verifier_var_off.c
@@ -198,7 +198,9 @@ __success
 /* Check that the maximum stack depth is correctly maintained according to the
  * maximum possible variable offset.
  */
-__log_level(4) __msg("stack depth 16")
+__log_level(4)
+__msg("subprog 0 (stack_write_priv_vs_unpriv) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 16")
 __failure_unpriv
 /* Variable stack access is rejected for unprivileged.
  */
@@ -238,7 +240,9 @@ __success
 /* Check that the maximum stack depth is correctly maintained according to the
  * maximum possible variable offset.
  */
-__log_level(4) __msg("stack depth 16")
+__log_level(4)
+__msg("subprog 0 (stack_write_followed_by_read) main insns_own {{[0-9]+}} "
+      "insns_total {{[0-9]+}} stack 16")
 __failure_unpriv
 __msg_unpriv("R2 variable stack access prohibited for !root")
 __retval(0)
-- 
2.53.0

[PATCH bpf-next v6 4/6] selftests/bpf: Adjust veristat stack depth parsing

[Kumar Kartikeya Dwivedi]

The verifier now reports instruction and stack depth statistics using
uniform "subprog <id> (<name>) <kind>" records. Subprogram 0 is
classified as main, while other records are global or static. Each record
carries insns_own, insns_total, and stack depth.

Teach veristat to parse the new records while retaining support for the
legacy one-line stack depth format used by older kernels. Skip both
instruction counts and match only through the stack value so fields can
still be appended without breaking parsing.

Increase the bounded backward scan so it can include all 256
per-subprogram records.

Zero-initialize the legacy stack buffer because logs using the new
format do not populate it before the trailing tokenizer loop. This makes
the loop see an empty string instead of reading uninitialized data.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 tools/testing/selftests/bpf/veristat.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/bpf/veristat.c b/tools/testing/selftests/bpf/veristat.c
index c9c257784ee3..8dcc5c50d311 100644
--- a/tools/testing/selftests/bpf/veristat.c
+++ b/tools/testing/selftests/bpf/veristat.c
@@ -993,13 +993,15 @@ static void free_verif_stats(struct verif_stats *stats, size_t stat_cnt)
 
 static char verif_log_buf[64 * 1024];
 
-#define MAX_PARSED_LOG_LINES 100
+/* Keep room for all 256 subprogram records and trailing statistics. */
+#define MAX_PARSED_LOG_LINES 300
 
 static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats *s)
 {
 	const char *cur;
-	int pos, lines, sub_stack, cnt = 0;
-	char *state = NULL, *token, stack[512];
+	long sub_stack;
+	int pos, lines, cnt = 0;
+	char *state = NULL, *token, stack[512] = {};
 
 	buf[buf_sz - 1] = '\0';
 
@@ -1025,11 +1027,17 @@ static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats *
 				&s->stats[MARK_READ_MAX_LEN]))
 			continue;
 
+		if (1 == sscanf(cur, "stack depth max %ld", &s->stats[MAX_STACK]))
+			continue;
+		if (1 == sscanf(cur, "subprog %*d %*s %*s insns_own %*d insns_total %*d stack %ld", &sub_stack)) {
+			s->stats[STACK] += sub_stack;
+			continue;
+		}
 		if (2 == sscanf(cur, "stack depth %511s max %ld", stack, &s->stats[MAX_STACK]))
 			continue;
 	}
 	while ((token = strtok_r(cnt++ ? NULL : stack, "+", &state))) {
-		if (sscanf(token, "%d", &sub_stack) == 0)
+		if (sscanf(token, "%ld", &sub_stack) == 0)
 			break;
 		s->stats[STACK] += sub_stack;
 	}
-- 
2.53.0

[PATCH bpf-next v6 5/6] selftests/bpf: Test stack depth stats without BTF subprog names

[Kumar Kartikeya Dwivedi]

Test the per-program insns_own, insns_total, and stack depth statistics
emitted when BTF function info does not provide subprogram names. Check
that the subprog 0 main record and static-subprogram records use
<unknown>.

Make VERBOSE_ACCEPT request verifier statistics so the raw-insn test can
validate the output without a test-specific log level.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 tools/testing/selftests/bpf/test_verifier.c  |  2 +-
 tools/testing/selftests/bpf/verifier/calls.c | 12 +++++++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index a8ae03c57bba..bffb7360434c 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -1560,7 +1560,7 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
 
 	opts.expected_attach_type = test->expected_attach_type;
 	if (expected_ret == VERBOSE_ACCEPT)
-		opts.log_level = 2;
+		opts.log_level = 2 | 4;
 	else if (verbose)
 		opts.log_level = verif_log_level | 4; /* force stats */
 	else
diff --git a/tools/testing/selftests/bpf/verifier/calls.c b/tools/testing/selftests/bpf/verifier/calls.c
index 8cd626e04551..d1bc884e44b8 100644
--- a/tools/testing/selftests/bpf/verifier/calls.c
+++ b/tools/testing/selftests/bpf/verifier/calls.c
@@ -1091,7 +1091,17 @@
 	/* stack_main=32, stack_A=256, stack_B=64
 	 * and max(main+A, main+A+B) < 512
 	 */
-	.result = ACCEPT,
+	.result = VERBOSE_ACCEPT,
+	.errstr = "stack depth max 352\t"
+		  "subprog 0 (<unknown>) main insns_own \t"
+		  " insns_total \t"
+		  " stack 32\t"
+		  "subprog 1 (<unknown>) static insns_own \t"
+		  " insns_total \t"
+		  " stack 256\t"
+		  "subprog 2 (<unknown>) static insns_own \t"
+		  " insns_total \t"
+		  " stack 64",
 },
 {
 	"calls: stack depth check using three frames. test2",
-- 
2.53.0

[PATCH bpf-next v6 6/6] selftests/bpf: Test subprogram instruction statistics

[Kumar Kartikeya Dwivedi]

Add small verifier programs with deterministic instruction streams to
exercise per-subprogram own and inclusive instruction accounting. Use
assembly for normal call chains and straight-line callback bodies
containing only moves, calls, and returns or exits, so control-flow
pruning does not make the expected counts unstable.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_subprog_insn_stats.c   | 225 ++++++++++++++++++
 2 files changed, 227 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c

diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index b79bafca68f7..19d03f6d6525 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -100,6 +100,7 @@
 #include "verifier_stack_arg_order.skel.h"
 #include "verifier_stack_ptr.skel.h"
 #include "verifier_store_release.skel.h"
+#include "verifier_subprog_insn_stats.skel.h"
 #include "verifier_subprog_precision.skel.h"
 #include "verifier_subprog_topo.skel.h"
 #include "verifier_subreg.skel.h"
@@ -255,6 +256,7 @@ void test_verifier_stack_arg(void)            { RUN(verifier_stack_arg); }
 void test_verifier_stack_arg_order(void)      { RUN(verifier_stack_arg_order); }
 void test_verifier_stack_ptr(void)            { RUN(verifier_stack_ptr); }
 void test_verifier_store_release(void)        { RUN(verifier_store_release); }
+void test_verifier_subprog_insn_stats(void)   { RUN(verifier_subprog_insn_stats); }
 void test_verifier_subprog_precision(void)    { RUN(verifier_subprog_precision); }
 void test_verifier_subprog_topo(void)        { RUN(verifier_subprog_topo); }
 void test_verifier_subreg(void)               { RUN(verifier_subreg); }
diff --git a/tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c b/tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c
new file mode 100644
index 000000000000..b7ac55f4f136
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c
@@ -0,0 +1,225 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+struct timer_value {
+	struct bpf_timer timer;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__uint(max_entries, 1);
+	__type(key, __u32);
+	__type(value, struct timer_value);
+} timer_map SEC(".maps");
+
+SEC("?raw_tp")
+__success __log_level(4)
+__msg("subprog 0 (stats_main_only) main insns_own 2 insns_total 2 stack 0")
+__msg("processed 2 insns")
+__naked int stats_main_only(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"exit;"
+	);
+}
+
+__naked __noinline __used
+static int stats_chain_leaf(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"exit;"
+	);
+}
+
+__naked __noinline __used
+static int stats_chain_parent(void)
+{
+	asm volatile (
+		"call stats_chain_leaf;"
+		"exit;"
+	);
+}
+
+SEC("?raw_tp")
+__success __log_level(4)
+/*
+ * own: 2 + 2 + 2 = 6
+ * totals: leaf 2, parent 2 + 2 = 4, main 2 + 4 = 6
+ */
+__msg("subprog 0 (stats_static_chain) main insns_own 2 insns_total 6 stack 0")
+__msg("subprog {{[0-9]+}} (stats_chain_parent) static insns_own 2 insns_total 4 stack 0")
+__msg("subprog {{[0-9]+}} (stats_chain_leaf) static insns_own 2 insns_total 2 stack 0")
+__msg("processed 6 insns")
+__naked int stats_static_chain(void)
+{
+	asm volatile (
+		"call stats_chain_parent;"
+		"exit;"
+	);
+}
+
+__naked __noinline __used
+static int stats_shared_leaf(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"exit;"
+	);
+}
+
+__naked __noinline __used
+int stats_global_root(void)
+{
+	asm volatile (
+		"call stats_shared_leaf;"
+		"exit;"
+	);
+}
+
+SEC("?raw_tp")
+__success __log_level(4)
+/*
+ * stats_shared_leaf is explored once under each independent root.
+ * own: main 3 + leaf 4 + global 2 = 9
+ * root totals: main 5 + global 4 = 9
+ */
+__msg("subprog 0 (stats_shared_roots) main insns_own 3 insns_total 5 stack 0")
+__msg("subprog {{[0-9]+}} (stats_shared_leaf) static insns_own 4 insns_total 4 stack 0")
+__msg("subprog {{[0-9]+}} (stats_global_root) global insns_own 2 insns_total 4 stack 0")
+__msg("processed 9 insns")
+__naked int stats_shared_roots(void)
+{
+	asm volatile (
+		"call stats_shared_leaf;"
+		"call stats_global_root;"
+		"exit;"
+	);
+}
+
+__noinline __used
+static int stats_async_leaf(void *map, __u32 *key, struct bpf_timer *timer)
+{
+	return 0;
+}
+
+__noinline __used
+static __u64 stats_async_schedule(struct bpf_map *map, __u32 *key,
+				  struct timer_value *value, void *ctx)
+{
+	asm volatile (
+		"r1 = r3;"
+		"r2 = %[stats_async_leaf];"
+		"call %[bpf_timer_set_callback];"
+		:
+		: __imm_ptr(stats_async_leaf),
+		  __imm(bpf_timer_set_callback)
+		: __clobber_common
+	);
+	return 0;
+}
+
+SEC("?raw_tp")
+__success __log_level(4)
+/*
+ * own: 9 + 6 + 2 = 17
+ * totals: leaf 2, scheduler 6 + 2 = 8, main 17
+ */
+__msg("subprog 0 (stats_async_direct) main insns_own 9 insns_total 17 stack 0")
+__msg("subprog {{[0-9]+}} (stats_async_schedule) static insns_own 6 "
+      "insns_total 8 stack 0")
+__msg("subprog {{[0-9]+}} (stats_async_leaf) static insns_own 2 "
+      "insns_total 2 stack 0")
+__msg("processed 17 insns")
+__naked int stats_async_direct(void)
+{
+	asm volatile (
+		"r1 = %[timer_map] ll;"
+		"r2 = %[stats_async_schedule];"
+		"r3 = 0;"
+		"r4 = 0;"
+		"call %[bpf_for_each_map_elem];"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_addr(timer_map),
+		  __imm_ptr(stats_async_schedule),
+		  __imm(bpf_for_each_map_elem)
+		: __clobber_common
+	);
+}
+
+__noinline __used
+static int stats_async_nested_leaf(void *map, __u32 *key, struct bpf_timer *timer)
+{
+	return 0;
+}
+
+__noinline __used
+static int stats_async_outer(void *map, __u32 *key, struct bpf_timer *timer)
+{
+	asm volatile (
+		"r1 = r3;"
+		"r2 = %[stats_async_nested_leaf];"
+		"call %[bpf_timer_set_callback];"
+		:
+		: __imm_ptr(stats_async_nested_leaf),
+		  __imm(bpf_timer_set_callback)
+		: __clobber_common
+	);
+	return 0;
+}
+
+__noinline __used
+static __u64 stats_async_nested_schedule(struct bpf_map *map, __u32 *key,
+					 struct timer_value *value, void *ctx)
+{
+	asm volatile (
+		"r1 = r3;"
+		"r2 = %[stats_async_outer];"
+		"call %[bpf_timer_set_callback];"
+		:
+		: __imm_ptr(stats_async_outer),
+		  __imm(bpf_timer_set_callback)
+		: __clobber_common
+	);
+	return 0;
+}
+
+SEC("?raw_tp")
+__success __log_level(4)
+/*
+ * own: 9 + 6 + 6 + 2 = 23
+ * totals: leaf 2, outer 8, scheduler 14, main 23
+ */
+__msg("subprog 0 (stats_async_nested) main insns_own 9 insns_total 23 stack 0")
+__msg("subprog {{[0-9]+}} (stats_async_nested_schedule) static insns_own 6 "
+      "insns_total 14 stack 0")
+__msg("subprog {{[0-9]+}} (stats_async_outer) static insns_own 6 "
+      "insns_total 8 stack 0")
+__msg("subprog {{[0-9]+}} (stats_async_nested_leaf) static insns_own 2 "
+      "insns_total 2 stack 0")
+__msg("processed 23 insns")
+__naked int stats_async_nested(void)
+{
+	asm volatile (
+		"r1 = %[timer_map] ll;"
+		"r2 = %[stats_async_nested_schedule];"
+		"r3 = 0;"
+		"r4 = 0;"
+		"call %[bpf_for_each_map_elem];"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_addr(timer_map),
+		  __imm_ptr(stats_async_nested_schedule),
+		  __imm(bpf_for_each_map_elem)
+		: __clobber_common
+	);
+}
+
+char _license[] SEC("license") = "GPL";
-- 
2.53.0

Re: [PATCH bpf-next v6 6/6] selftests/bpf: Test subprogram instruction statistics

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Inline assembly in non-naked C functions implicitly reads argument registers, risking register corruption and test failures due to unstable instruction counts.
--

--- Patch [6]: [PATCH bpf-next v6 6/6] selftests/bpf: Test subprogram instruction statistics ---
commit 567cc6b8313bad87d8db2a9e9b28df5026767381
Author: Kumar Kartikeya Dwivedi <memxor@gmail.com>

selftests/bpf: Test subprogram instruction statistics

Adds small verifier programs with deterministic instruction streams to
exercise per-subprogram own and inclusive instruction accounting.

> diff --git a/tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c b/tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c
> new file mode 100644
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/verifier_subprog_insn_stats.c

[ ... ]

> +__noinline __used
> +static __u64 stats_async_schedule(struct bpf_map *map, __u32 *key,
> +				  struct timer_value *value, void *ctx)
> +{
> +	asm volatile (
> +		"r1 = r3;"

[Severity: Medium]
Because stats_async_schedule() is not marked __naked, could the compiler
emit prologue or epilogue instructions that change the exact instruction
count, causing the strict test assertions (e.g., insns_own 6) to fail?

Also, since r3 is implicitly read here without an input constraint, is
there a risk that the register allocator could clobber r3 before the
assembly block executes?

[ ... ]

> +__noinline __used
> +static int stats_async_outer(void *map, __u32 *key, struct bpf_timer *timer)
> +{
> +	asm volatile (
> +		"r1 = r3;"

[Severity: Medium]
Similarly, does the lack of __naked on stats_async_outer() risk unexpected
prologue instructions breaking the exact instruction counts?

Could the implicit use of r3 without an input constraint result in register
corruption?

[ ... ]

> +__noinline __used
> +static __u64 stats_async_nested_schedule(struct bpf_map *map, __u32 *key,
> +					 struct timer_value *value, void *ctx)
> +{
> +	asm volatile (
> +		"r1 = r3;"

[Severity: Medium]
Could the compiler emit extra instructions here for
stats_async_nested_schedule() since it is not marked __naked, breaking
the expected instruction counts?

Does reading r3 without constraints here also risk register corruption?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260805011517.1717238-1-memxor@gmail.com?part=6

Re: [PATCH bpf-next v6 1/6] bpf: Track verifier instruction stats for each subprogram

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] A NULL pointer dereference occurs in `account_current_path()` when handling an allocation failure during `pop_stack()`.
--

--- Patch [1]: [PATCH bpf-next v6 1/6] bpf: Track verifier instruction stats for each subprogram ---
commit 026daf56a558e1775b6ccd02d21953d3b59b6a28
Author: Kumar Kartikeya Dwivedi <memxor@gmail.com>

bpf: Track verifier instruction stats for each subprogram
    
This patch tracks verifier instruction counts for each subprogram
independently. It records both own and inclusive instruction counts,
updating the subtotals during function returns and when a path terminates
or is pruned.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 7439afdc851a7..88e7ea6fbe73a 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> @@ -9807,6 +9809,37 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
>  
>  static bool is_rbtree_lock_required_kfunc(u32 btf_id);
>  
> +static void account_processed_insn(struct bpf_verifier_env *env)
> +{
> +	struct bpf_func_state *frame = cur_func(env);
> +
> +	env->insn_processed++;
> +	frame->insns_subtotal++;
> +	env->subprog_info[frame->subprogno].insns_own++;
> +}
> +
> +static void account_processed_insns(struct bpf_verifier_env *env,
> +				    struct bpf_func_state *callee,
> +				    struct bpf_func_state *caller)
> +{
> +	u32 insns = callee->insns_subtotal;
                    ^^^^^^^^^^^^^^^^^^^^^^

[Severity: High]
Could `callee` be NULL here on error paths?

If a memory allocation failure happens during `pop_stack()` (for example, if
`kzalloc_obj()` fails while allocating a new frame during the verifier state
copy), the state can be partially copied, leaving `state->frame[i]` as NULL
for some `i <= state->curframe`.

When `do_check_common()` handles this error, it unconditionally calls
`account_current_path()`. The loop in `account_current_path()` iterates up to
`state->curframe` and passes `state->frame[frame]` to
`account_processed_insns()`, which would then unconditionally dereference the
NULL pointer.

Should we check `if (!callee)` before attempting to read its subtotal?

> +
> +	env->subprog_info[callee->subprogno].insns_total += insns;
> +	if (caller)
> +		caller->insns_subtotal += insns;
> +	callee->insns_subtotal = 0;
> +}
> +
> +static void account_current_path(struct bpf_verifier_env *env)
> +{
> +	struct bpf_verifier_state *state = env->cur_state;
> +	int frame;
> +
> +	for (frame = state->curframe; frame >= 0; frame--)
> +		account_processed_insns(env, state->frame[frame],
> +					frame ? state->frame[frame - 1] : NULL);
> +}

[ ... ]

> @@ -18680,6 +18717,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
>  
>  	ret = do_check(env);
>  out:
> +	account_current_path(env);
>  	if (!ret && pop_log)
>  		bpf_vlog_reset(&env->log, 0);
>  	free_states(env);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260805011517.1717238-1-memxor@gmail.com?part=1

Re: [PATCH bpf-next v6 3/6] bpf: Show more useful info in stack depth stats

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The verifier log format change for stack depth breaks `veristat`'s parsing of stack depth statistics.
--

--- Patch [3]: [PATCH bpf-next v6 3/6] bpf: Show more useful info in stack depth stats ---
commit 48703e00e6b412bacd50ea22b6dd5fcca0a7acb6
Author: Kumar Kartikeya Dwivedi <memxor@gmail.com>

bpf: Show more useful info in stack depth stats

This commit changes the verifier stack depth statistics format to keep
the combined maximum stack depth on its own line and print one uniform
record for each subprogram. This helps associate the statistics with
subprogram names and clarifies where verifier work is spent.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 47f3791530ded..a4ab7ee334c6b 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -18833,15 +18833,20 @@ static void print_verification_stats(struct bpf_verifier_env *env)
>  	if (env->log.level & BPF_LOG_STATS) {
>  		verbose(env, "verification time %lld usec\n",
>  			div_u64(env->verification_time, 1000));
> -		verbose(env, "stack depth %d", env->subprog_info[0].stack_depth);
> -		for (i = 1; i < subprog_cnt; i++)
> -			verbose(env, "+%d", env->subprog_info[i].stack_depth);
> -		verbose(env, " max %d\n", env->max_stack_depth);
> -		verbose(env, "insns processed %d", env->subprog_info[0].insns_total);
> -		for (i = 1; i < subprog_cnt; i++)
> -			if (bpf_subprog_is_global(env, i))
> -				verbose(env, "+%d", env->subprog_info[i].insns_total);
> -		verbose(env, "\n");
> +		verbose(env, "stack depth max %d\n", env->max_stack_depth);

[Severity: Medium]
Does this format change break veristat's log parsing?

Looking at parse_verif_log() in tools/testing/selftests/bpf/veristat.c,
the tool relies on the older format to parse these stats:

    if (2 == sscanf(cur, "stack depth %511s max %ld", stack, &s->stats[MAX_STACK]))
        continue;

Because the new format is "stack depth max %d\n", the sscanf parser will
consume "max" into the %511s string buffer and fail to match the rest of
the string.

This causes the sscanf to return 1 instead of 2, and the subsequent
fallback tokenization loop fails as well. Will this cause veristat to
silently report 0 for both STACK and MAX_STACK metrics for all processed
BPF programs?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260805011517.1717238-1-memxor@gmail.com?part=3

Re: [PATCH bpf-next v6 2/6] bpf: Propagate async callback instructions to scheduling subprograms

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] Unbounded `async_stats_subprog_cnt` causes OOB write in `push_async_cb` and OOB read/write in `account_current_path`.
--

--- Patch [2]: [PATCH bpf-next v6 2/6] bpf: Propagate async callback instructions to scheduling subprograms ---
commit e22fc6d02fda1f775515a1bbc9834113d0ba7767
Author: Kumar Kartikeya Dwivedi <memxor@gmail.com>

bpf: Propagate async callback instructions to scheduling subprograms

This commit propagates the instruction budget from asynchronous callback
executions back to the subprograms that originally scheduled them. It tracks
active subprogram IDs when a callback is queued and adds the inclusive
subtotal of the async frame-zero path to the saved scheduling subprograms.

> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h
> @@ -447,6 +447,8 @@ static_assert(MAX_BPF_STACK / 8 <= (1 << 6));
>  struct bpf_verifier_state {
>  	/* call stack tracking */
>  	struct bpf_func_state *frame[MAX_CALL_FRAMES];
> +	u32 async_stats_subprog_ids[MAX_CALL_FRAMES];
> +	u32 async_stats_subprog_cnt;

[Severity: Critical]
Is it possible for the accumulated call chains of all ancestral scheduling
events to exceed MAX_CALL_FRAMES? If a BPF program chains async callbacks
(like a timer callback scheduling another timer callback), the count might
grow indefinitely.

>  	struct bpf_verifier_state *parent;
>  	/* Acquired reference states */
>  	struct bpf_reference_state *refs;

[ ... ]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> @@ -2293,6 +2298,12 @@ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env,
>  			0 /* frameno within this callchain */,
>  			subprog /* subprog number within this prog */);
>  	elem->st.frame[0] = frame;
> +	cnt = env->cur_state->async_stats_subprog_cnt;
> +	memcpy(elem->st.async_stats_subprog_ids, env->cur_state->async_stats_subprog_ids,
> +	       cnt * sizeof(elem->st.async_stats_subprog_ids[0]));
> +	for (i = 0; i <= env->cur_state->curframe; i++)
> +		elem->st.async_stats_subprog_ids[cnt++] = env->cur_state->frame[i]->subprogno;
> +	elem->st.async_stats_subprog_cnt = cnt;

[Severity: Critical]
Can this loop overflow the async_stats_subprog_ids array? Since cnt is
incremented without checking against MAX_CALL_FRAMES, it appears this could
write past the end of the array, corrupting adjacent fields in struct
bpf_verifier_state like the parent pointer.

>  	return &elem->st;
>  }

[ ... ]

> @@ -9828,16 +9839,24 @@ static void account_processed_insns(struct bpf_verifier_env *env,
>  	if (caller)
>  		caller->insns_subtotal += insns;
>  	callee->insns_subtotal = 0;
> +	return insns;
>  }
>  
>  static void account_current_path(struct bpf_verifier_env *env)
>  {
>  	struct bpf_verifier_state *state = env->cur_state;
> -	int frame;
> +	u32 insns;
> +	int frame, i;
>  
>  	for (frame = state->curframe; frame >= 0; frame--)
> -		account_processed_insns(env, state->frame[frame],
> -					frame ? state->frame[frame - 1] : NULL);
> +		insns = account_processed_insns(env, state->frame[frame],
> +						frame ? state->frame[frame - 1] : NULL);
> +
> +	if (!state->async_stats_subprog_cnt)
> +		return;
> +
> +	for (i = 0; i < state->async_stats_subprog_cnt; i++)
> +		env->subprog_info[state->async_stats_subprog_ids[i]].insns_total += insns;

[Severity: Critical]
If async_stats_subprog_cnt was corrupted and grew beyond MAX_CALL_FRAMES, does
this result in an out-of-bounds read from async_stats_subprog_ids? This could
read from the corrupted parent pointer and cause an arbitrary kernel memory
write into env->subprog_info.

>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260805011517.1717238-1-memxor@gmail.com?part=2

Re: [PATCH bpf-next v6 2/6] bpf: Propagate async callback instructions to scheduling subprograms

[Eduard Zingerman]

On Wed, 2026-08-05 at 03:15 +0200, Kumar Kartikeya Dwivedi wrote:

...

> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> index 9de45ade473b..42fa464c520f 100644
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h
> @@ -447,6 +447,8 @@ static_assert(MAX_BPF_STACK / 8 <= (1 << 6));
>  struct bpf_verifier_state {
>  	/* call stack tracking */
>  	struct bpf_func_state *frame[MAX_CALL_FRAMES];
> +	u32 async_stats_subprog_ids[MAX_CALL_FRAMES];
> +	u32 async_stats_subprog_cnt;

One async callback can call another async callback, e.g. see program
'test1' in progs/timer.c. Meaning that this array is not really bound
by MAX_CALL_FRAMES and the code below may overflow it.

Overall, it seems that having an alternative full call stack solely
for the purpose of accounting is an overkill. I'd just show async
subprograms as their own roots, tbh. On the other hand, same
subprogram can be called both as an async and as a regular subprogram :)
so the idea of a separate spine is not w/o it's merit.
If we decide to stick with it, maybe pick a better name?
'verification_call_stack' or something like this?

If we decide to go with this separate call stack, would it be possible
to adapt the code in a way that only this call stack is used for
accounting? (e.g. is filled on regular subprogram calls etc).

Another option is to build an implicit call graph in env while
verifying and count only self instructions and counters on edges.
Then propagate the data over the graph in post-order traversal
(subprog_topo_order is already computed). I think I like this idea
the most.

...

> @@ -2293,6 +2298,12 @@ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env,
>  			0 /* frameno within this callchain */,
>  			subprog /* subprog number within this prog */);
>  	elem->st.frame[0] = frame;
> +	cnt = env->cur_state->async_stats_subprog_cnt;
> +	memcpy(elem->st.async_stats_subprog_ids, env->cur_state->async_stats_subprog_ids,
> +	       cnt * sizeof(elem->st.async_stats_subprog_ids[0]));
> +	for (i = 0; i <= env->cur_state->curframe; i++)
> +		elem->st.async_stats_subprog_ids[cnt++] = env->cur_state->frame[i]->subprogno;
> +	elem->st.async_stats_subprog_cnt = cnt;
>  	return &elem->st;
>  }

...