[PATCH bpf-next v4 06/20] bpf: Explore the landing pads no call site reaches

[Yonghong Song <yonghong.song@linux.dev>]

A cleanup record need not cover a call an exception can unwind out of: a
frontend is free to emit a region around a helper or an ordinary kfunc,
both nounwind here. Nothing marks a call site then, and that record's
landing pad is reached by nothing at all -- leaving bpf_check_cfg() to
refuse the program over code its own frontend had no way not to emit:

   0: call bpf_preempt_disable
   1: call bpf_preempt_enable      record = { begin = 1, end = 2, pad = 4 }
   2: r0 = 0
   3: exit
   4: r1 = pads_ran ll             landing pad
   6: r2 = *(u64 *)(r1 + 0)
   7: r2 |= RAN_NOUNWIND_REC
   8: *(u64 *)(r1 + 0) = r2
   9: call bpf_unwind_resume
  10: exit

The range [1,2) holds one call, and it is a kfunc, so an exception cannot
come out of it. cleanup_mark_call_sites() marks nothing, nothing pushes an
edge to 4, and 4 through 10 are reachable from nothing: "unreachable insn
4". The pad is dead, which is correct -- no exception can ever arrive at it
-- but the program is fine and has to load.

Walk every pad the table names that the edges did not reach, the way the
walk is already re-seeded at an exception callback. From there the pad is
code like any other: do_check() never enters it, because no call site
dispatches to it, so the dead code sweep removes it along with everything
else that was not reached.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/cfg.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index ef02906fd5a8..ba8f9cca2955 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -640,6 +640,7 @@ int bpf_check_cfg(struct bpf_verifier_env *env)
 	int insn_cnt = env->prog->len;
 	int *insn_stack, *insn_state;
 	int ex_insn_beg, i, ret = 0;
+	u32 pad_idx = 0;
 
 	insn_state = env->cfg.insn_state = kvzalloc_objs(int, insn_cnt,
 							 GFP_KERNEL_ACCOUNT);
@@ -695,6 +696,22 @@ int bpf_check_cfg(struct bpf_verifier_env *env)
 		goto walk_cfg;
 	}
 
+	/*
+	 * A landing pad no call site was marked with -- a record whose range
+	 * holds no bpf2bpf call and no bpf_throw() -- is reached by nothing.
+	 * Walk it from here, and let the dead code sweep remove it.
+	 */
+	while (pad_idx < env->cleanup_info_cnt) {
+		u32 pad = env->cleanup_info[pad_idx++].landing_pad_off;
+
+		if (insn_state[pad] != EXPLORED) {
+			insn_state[pad] = DISCOVERED;
+			insn_stack[0] = pad;
+			env->cfg.cur_stack = 1;
+			goto walk_cfg;
+		}
+	}
+
 	for (i = 0; i < insn_cnt; i++) {
 		struct bpf_insn *insn = &env->prog->insnsi[i];
 
-- 
2.53.0-Meta