[PATCH bpf v2 00/11] Misc BPF gotox fixes

[PATCH bpf v2 00/11] Misc BPF gotox fixes

[Daniel Borkmann]

Address various open issues around gotox instruction, see individual
commits for details. Also BPF selftests are added to assert that these
are addressed.

v1 -> v2:
  - Remove jt_spans_subprogs and reject right away (Eduard)
  - Rework jump table to no longer keep it in insn_aux_data (Eduard)
  - Fix up comments in bpf_compute_scc (Eduard)
  - Fix up commit message nit wrt signed_loader (Anton)
  - Added 3 more fixes related to gotox along with BPF selftests

Daniel Borkmann (11):
  bpf: Avoid quadratic successor rescan in bpf_compute_scc
  bpf: Bound the number of indirect jump edges in a program
  bpf: Cache the jump table of a subprogram during CFG discovery
  bpf: Reject indirect jumps that leave their subprogram
  bpf: Don't remove indirect jump targets in the nop removal pass
  bpf: Clear insn array jitted pointers on map reuse
  bpf: Retarget indirect jump targets across prologue prepends
  selftests/bpf: Add tests for the indirect jump edge limit
  selftests/bpf: Add tests for indirect jumps across subprograms
  selftests/bpf: Add tests for gotox nop targets and insn array reuse
  selftests/bpf: Add tests for gotox targets across prologue prepends

 include/linux/bpf.h                           |    2 +-
 include/linux/bpf_verifier.h                  |    6 +-
 kernel/bpf/bpf_insn_array.c                   |   16 +-
 kernel/bpf/cfg.c                              |  208 ++--
 kernel/bpf/fixups.c                           |   63 +-
 kernel/bpf/liveness.c                         |   18 +-
 kernel/bpf/verifier.c                         |   58 +-
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 1061 ++++++++++++++++-
 .../selftests/bpf/progs/verifier_gotox.c      |    2 +-
 9 files changed, 1276 insertions(+), 158 deletions(-)

-- 
2.43.0

[PATCH bpf v2 01/11] bpf: Avoid quadratic successor rescan in bpf_compute_scc

[Daniel Borkmann]

The iterative Tarjan DFS in bpf_compute_scc() emulates recursion with an
explicit 'dfs' stack: when a successor has not been visited yet, the
successor is pushed and the walk restarts at the top of the loop. On the
way back to a vertex the successor scan starts over at index zero, so a
vertex with k successors rescans up to k successors on each of its up to k
descents, i.e. O(k^2) work.

For ordinary instructions k <= 2 and this is irrelevant. For a gotox the
successors are the jump table of the containing subprogram, whose size is
bounded only by the max_entries of the insn_array map, so k can reach the
1M instruction complexity limit. Loading such a program keeps a CPU busy
in the loop for a very long time before verification even begins.

Record in 'dfs_pos' the successor index each frame stopped at and resume
the scan there. Each edge is therefore examined a bounded number of times
and the walk becomes linear in the number of edges.

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
 kernel/bpf/cfg.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 842c7d1eabcc..5cfd02c42afd 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -749,7 +749,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	struct bpf_insn_aux_data *aux = env->insn_aux_data;
 	const u32 insn_cnt = env->prog->len;
 	int stack_sz, dfs_sz, err = 0;
-	u32 *stack, *pre, *low, *dfs;
+	u32 *stack, *pre, *low, *dfs, *dfs_pos;
 	u32 i, j, t, w;
 	u32 next_preorder_num;
 	u32 next_scc_id;
@@ -762,13 +762,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	 * - 'stack' accumulates vertices in DFS order, see invariant comment below;
 	 * - 'pre[t] == p' => preorder number of vertex 't' is 'p';
 	 * - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n';
-	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion.
+	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion;
+	 * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its
+	 *   successors at index 'j'.
 	 */
 	stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	dfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);
-	if (!stack || !pre || !low || !dfs) {
+	dfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);
+	if (!stack || !pre || !low || !dfs || !dfs_pos) {
 		err = -ENOMEM;
 		goto exit;
 	}
@@ -851,6 +854,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 		stack_sz = 0;
 		dfs_sz = 1;
 		dfs[0] = i;
+		dfs_pos[0] = 0;
 dfs_continue:
 		while (dfs_sz) {
 			w = dfs[dfs_sz - 1];
@@ -860,13 +864,22 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 				next_preorder_num++;
 				stack[stack_sz++] = w;
 			}
-			/* Visit 'w' successors */
+			/* Visit remaining 'w' successors */
 			succ = bpf_insn_successors(env, w);
-			for (j = 0; j < succ->cnt; ++j) {
+			for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) {
 				if (pre[succ->items[j]]) {
 					low[w] = min(low[w], low[succ->items[j]]);
 				} else {
-					dfs[dfs_sz++] = succ->items[j];
+					/*
+					 * Once DFS for succ->items[j] is complete,
+					 * pre[succ->items[j]] is non-zero, hence
+					 * resuming at 'j' allows to follow the
+					 * low[w] = min(...) update branch above.
+					 */
+					dfs_pos[dfs_sz - 1] = j;
+					dfs_pos[dfs_sz] = 0;
+					dfs[dfs_sz] = succ->items[j];
+					dfs_sz++;
 					goto dfs_continue;
 				}
 			}
@@ -916,5 +929,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	kvfree(pre);
 	kvfree(low);
 	kvfree(dfs);
+	kvfree(dfs_pos);
 	return err;
 }
-- 
2.43.0

[PATCH bpf v2 02/11] bpf: Bound the number of indirect jump edges in a program

[Daniel Borkmann]

Every gotox instruction gets its own copy of the jump table of the subprog
containing it, and each distinct target in that table is a CFG successor
of the instruction. The number of such edges is therefore the number of
gotox instructions times the number of distinct targets, and neither
factor is bounded by anything except the instruction limit.

What is expensive is a BPF prog whose gotox instructions are themselves
the targets, which makes the edge count quadratic. 1024 such gotox are
already ~1e6 edges and about 4s of CPU to load.

Bound the total across the program at BPF_COMPLEXITY_LIMIT_INSNS, aka
the limit as the number of instructions the verifier processes. Progs
with real switch statements are orders of magnitude below this.

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 include/linux/bpf_verifier.h |  1 +
 kernel/bpf/cfg.c             | 14 ++++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 36b65797877d..04bb8f71cabe 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -977,6 +977,7 @@ struct bpf_verifier_env {
 		int cur_stack;
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
+		u32 gotox_edges;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 5cfd02c42afd..df4513e71c80 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -9,6 +9,8 @@
 
 #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
 
+#define BPF_MAX_GOTOX_EDGES	BPF_COMPLEXITY_LIMIT_INSNS
+
 /* non-recursive DFS pseudo code
  * 1  procedure DFS-iterative(G,v):
  * 2      label v as discovered
@@ -388,6 +390,18 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 			return PTR_ERR(jt);
 
 		env->insn_aux_data[t].jt = jt;
+
+		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+				       &env->cfg.gotox_edges) ||
+		    env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+			verbose(env, "number of indirect jump edges in the program exceeds %u\n",
+				BPF_MAX_GOTOX_EDGES);
+			bpf_diag_program_structure(env, t, "too many indirect jump edges",
+				"Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+				"The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+				BPF_MAX_GOTOX_EDGES);
+			return -E2BIG;
+		}
 	}
 
 	mark_prune_point(env, t);
-- 
2.43.0

Re: [PATCH bpf v2 02/11] bpf: Bound the number of indirect jump edges in a program

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4292 bytes --]

> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> index 36b65797877d0..04bb8f71cabe3 100644
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h
> @@ -977,6 +977,7 @@ struct bpf_verifier_env {
>  		int cur_stack;
>  		/* current position in the insn_postorder vector */
>  		int cur_postorder;
> +		u32 gotox_edges;
>  	} cfg;
>  	struct backtrack_state bt;
>  	struct bpf_jmp_history_entry *cur_hist_ent;
> diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
> index 5cfd02c42afd5..df4513e71c80a 100644
> --- a/kernel/bpf/cfg.c
> +++ b/kernel/bpf/cfg.c
> @@ -9,6 +9,8 @@
>
>  #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
>
> +#define BPF_MAX_GOTOX_EDGES	BPF_COMPLEXITY_LIMIT_INSNS
> +
>  /* non-recursive DFS pseudo code
>   * 1  procedure DFS-iterative(G,v):
>   * 2      label v as discovered
> @@ -388,6 +390,18 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
>  			return PTR_ERR(jt);
>
>  		env->insn_aux_data[t].jt = jt;
> +
> +		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
> +				       &env->cfg.gotox_edges) ||
> +		    env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
> +			verbose(env, "number of indirect jump edges in the program exceeds %u\n",
> +				BPF_MAX_GOTOX_EDGES);
> +			bpf_diag_program_structure(env, t, "too many indirect jump edges",
> +				"Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
> +				"The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
> +				BPF_MAX_GOTOX_EDGES);
> +			return -E2BIG;
> +		}
>  	}
>
>  	mark_prune_point(env, t);

Does this bound prevent the load-time DoS the commit message describes?

The accounting itself is correct: the counter increments only inside the
if (!jt) block, so each gotox contributes jt->cnt exactly once, and the
counter is program-wide across both walk_cfg passes.

However, gotox_edges counts the CFG edges added, not the work that
check_cfg() performs. The expensive operations happen before this check:

  visit_gotox_insn() -> create_jt() -> jt_from_subprog()

jt_from_subprog() loops over every insn_array map in the program, and
for each map it allocates map->max_entries u32s and calls
bpf_copy_insn_array_uniq(), which does one map_lookup_elem() per entry
and then sorts them. This runs once per gotox instruction.

The edges added per gotox are only the distinct targets that survive the
sort and dedup, which can be 1 or 2 no matter how large the maps are.

So the cost is O(gotox_count * total_map_entries * log) while the limit
constrains gotox_count * distinct_targets. A program can sit at a small
fraction of BPF_MAX_GOTOX_EDGES and still take arbitrarily long to reject.

The commit message states "1024 such gotox are already ~1e6 edges and
about 4s of CPU to load" and "Progs with real switch statements are
orders of magnitude below this", which reads as if the 1e6-edge bound
caps load time in the seconds range. It does not.

This is confirmed by the next patch in the series, b94c1f057586
("bpf: Cache the jump table of a subprogram during CFG discovery"),
which measures with this bound already applied: "A program of 4003
instructions with 2000 gotox and one 500k entry map holding two distinct
targets has 4000 indirect jump edges, 0.4% of the limit, and takes 343s
to be rejected. [...] At the insn limit, with a single 1M entry map, the
same shape extrapolates to 41 hours." That patch carries the edge
accounting over verbatim and instead builds each subprogram's jump table
once, which makes check_cfg linear.

For the unprivileged load path, bpf_check() holds bpf_verifier_lock, so
a single such program stalls all other non-privileged program loads for
the duration.

Given the Fixes: tag and Reported-by:, should the commit message clarify
that this patch bounds the edge count only, and that the per-gotox
jump-table construction cost is addressed separately in the next patch?
Otherwise the series might be read as closing the DoS at this commit.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

[PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

[Daniel Borkmann]

create_jt() builds the jump table of the subprogram containing a gotox by
copying out and sorting every insn_array map of the program, and it does
so once per gotox instruction. The cost is therefore the number of gotox
instructions times the number of entries in all of the maps. A program of
4003 instructions with 2000 gotox and one 500k entry map holding two
distinct targets has 4000 indirect jump edges, 0.4% of the limit, and
takes 343s to be rejected. The map costs next to nothing to prepare, as
an unset entry is already a valid target. At the insn limit, with a single
1M entry map, the same shape extrapolates to 41 hours.

All gotox instructions of a subprogram share the same jump table, so
build the table of every subprogram in a single pass over the maps and
let every gotox use it in place: bpf_insn_successors() resolves a gotox
through its containing subprogram, and the tables are kept until the
verifier environment is torn down, so nothing is copied into the
instruction aux data. The edge accounting in visit_gotox_insn() moves
to the first visit of the gotox, marked by the BRANCH bit of its CFG
state.

The only user left of insn_aux_data[].jt is then the two entry table
which visit_abnormal_return_insn() allocates for tail_call and
ld_{abs,ind} insns, so that bpf_insn_successors() reports the hidden
exit from their subprogram. Both are recognisable by their opcode, so
derive that edge in bpf_insn_successors() from the exit_idx of the
containing subprogram instead and drop the field along with
bpf_clear_insn_aux_data(). The instruction aux data then owns no
allocation, thus nothing needs to be freed when insns are removed or
the verifier environment is torn down.

check_cfg() is then linear in the number of map entries plus the number
of indirect jump edges, so what still scales now with the program is what
BPF_MAX_GOTOX_EDGES bounds:

  gotox  map entries  edges     before     after
  ----------------------------------------------
    500       250000   1000     35.34s     0.07s
   1000       250000   2000     82.00s     0.07s
   2000       250000   4000    148.62s     0.07s
   2000       125000   4000     74.54s     0.05s
   2000       500000   4000    342.71s     0.19s

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 include/linux/bpf_verifier.h |   5 +-
 kernel/bpf/cfg.c             | 143 ++++++++++++++++-------------------
 kernel/bpf/fixups.c          |  19 -----
 kernel/bpf/liveness.c        |  18 ++++-
 kernel/bpf/verifier.c        |   3 +-
 5 files changed, 85 insertions(+), 103 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 04bb8f71cabe..754e24d41f47 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
 		/* remember the offset of node field within type to rewrite */
 		u64 insert_off;
 	};
-	struct bpf_iarray *jt;	/* jump table for gotox or bpf_tailcall call instruction */
 	struct btf_struct_meta *kptr_struct_meta;
 	u64 map_key_state; /* constant (32 bit) key tracking for maps */
 	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
@@ -805,6 +804,7 @@ struct bpf_subprog_info {
 	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
 	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
 	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
 	u16 stack_depth; /* max. stack depth used by this function */
 	u16 stack_extra;
 	u32 insns_total;
@@ -978,6 +978,7 @@ struct bpf_verifier_env {
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
 		u32 gotox_edges;
+		bool subprog_jts_ready;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
@@ -1502,6 +1503,7 @@ bool bpf_is_throw_kfunc(struct bpf_insn *insn);
 int bpf_compute_const_regs(struct bpf_verifier_env *env);
 int bpf_prune_dead_branches(struct bpf_verifier_env *env);
 int bpf_check_cfg(struct bpf_verifier_env *env);
+void bpf_free_subprog_jts(struct bpf_verifier_env *env);
 int bpf_compute_postorder(struct bpf_verifier_env *env);
 int bpf_compute_scc(struct bpf_verifier_env *env);
 
@@ -1703,7 +1705,6 @@ struct bpf_kfunc_desc_tab {
 };
 
 /* Functions exported from verifier.c, used by fixups.c */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len);
 void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog);
 bool bpf_allow_tail_call_in_subprogs(struct bpf_verifier_env *env);
 bool bpf_verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm);
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index df4513e71c80..c38d33f0b0ba 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -286,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
 }
 
 /*
- * Find and collect all maps which fit in the subprog. Return the result as one
- * combined jump table in jt->items (allocated with kvcalloc)
+ * Collect the jump table of every subprogram that has one, as the combined
+ * table of all maps whose first target lands inside that subprogram. All gotox
+ * instructions of a subprogram share the same table, so this is done in a
+ * single pass over the maps rather than once per gotox.
  */
-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
-					  int subprog_start, int subprog_end)
+static int compute_subprog_jts(struct bpf_verifier_env *env)
 {
-	struct bpf_iarray *jt = NULL;
+	struct bpf_subprog_info *subprog;
+	struct bpf_iarray *jt, *jt_cur;
 	struct bpf_map *map;
-	struct bpf_iarray *jt_cur;
+	u32 old_cnt;
 	int i;
 
 	for (i = 0; i < env->insn_array_map_cnt; i++) {
@@ -305,56 +307,81 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
 		map = env->insn_array_maps[i];
 
 		jt_cur = jt_from_map(map);
-		if (IS_ERR(jt_cur)) {
-			kvfree(jt);
-			return jt_cur;
-		}
+		if (IS_ERR(jt_cur))
+			return PTR_ERR(jt_cur);
 
 		/*
 		 * This is enough to check one element. The full table is
-		 * checked to fit inside the subprog later in create_jt()
+		 * checked to fit inside the subprog later in subprog_jt()
 		 */
-		if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) {
-			u32 old_cnt = jt ? jt->cnt : 0;
-			jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt);
-			if (!jt) {
-				kvfree(jt_cur);
-				return ERR_PTR(-ENOMEM);
-			}
-			memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+		subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
+		if (!subprog) {
+			kvfree(jt_cur);
+			continue;
 		}
 
+		old_cnt = subprog->jt ? subprog->jt->cnt : 0;
+		jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
+		if (!jt) {
+			subprog->jt = NULL;
+			kvfree(jt_cur);
+			return -ENOMEM;
+		}
+		memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+		subprog->jt = jt;
+
 		kvfree(jt_cur);
 	}
 
-	if (!jt) {
-		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
-		bpf_diag_program_structure(
-			env, subprog_start, "missing jump table",
-			"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
-			"No jump table was found for the subprogram that starts at instruction %u.",
-			subprog_start);
-		return ERR_PTR(-EINVAL);
+	for (i = 0; i < env->subprog_cnt; i++) {
+		jt = env->subprog_info[i].jt;
+		if (jt)
+			jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
 	}
 
-	jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
-	return jt;
+	env->cfg.subprog_jts_ready = true;
+	return 0;
+}
+
+void bpf_free_subprog_jts(struct bpf_verifier_env *env)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
+		kvfree(env->subprog_info[i].jt);
+		env->subprog_info[i].jt = NULL;
+	}
+	env->cfg.subprog_jts_ready = false;
 }
 
 static struct bpf_iarray *
-create_jt(int t, struct bpf_verifier_env *env)
+subprog_jt(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_subprog_info *subprog;
 	int subprog_start, subprog_end;
 	struct bpf_iarray *jt;
-	int i;
+	int i, err;
+
+	if (!env->cfg.subprog_jts_ready) {
+		err = compute_subprog_jts(env);
+		if (err)
+			return ERR_PTR(err);
+	}
 
 	subprog = bpf_find_containing_subprog(env, t);
 	subprog_start = subprog->start;
 	subprog_end = (subprog + 1)->start;
-	jt = jt_from_subprog(env, subprog_start, subprog_end);
-	if (IS_ERR(jt))
-		return jt;
+
+	if (!subprog->jt) {
+		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
+		bpf_diag_program_structure(env, subprog_start, "missing jump table",
+			"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
+			"No jump table was found for the subprogram that starts at instruction %u.",
+			subprog_start);
+		return ERR_PTR(-EINVAL);
+	}
+
+	jt = subprog->jt;
 
 	/* Check that the every element of the jump table fits within the given subprogram */
 	for (i = 0; i < jt->cnt; i++) {
@@ -366,7 +393,6 @@ create_jt(int t, struct bpf_verifier_env *env)
 				"Keep every jump-table target inside the same subprogram.",
 				"The jump table for instruction %d points outside subprogram range [%u,%u).",
 				t, subprog_start, subprog_end);
-			kvfree(jt);
 			return ERR_PTR(-EINVAL);
 		}
 	}
@@ -383,13 +409,12 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 	struct bpf_iarray *jt;
 	int i, w;
 
-	jt = env->insn_aux_data[t].jt;
-	if (!jt) {
-		jt = create_jt(t, env);
-		if (IS_ERR(jt))
-			return PTR_ERR(jt);
+	jt = subprog_jt(t, env);
+	if (IS_ERR(jt))
+		return PTR_ERR(jt);
 
-		env->insn_aux_data[t].jt = jt;
+	if (!(insn_state[t] & BRANCH)) {
+		insn_state[t] |= BRANCH;
 
 		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
 				       &env->cfg.gotox_edges) ||
@@ -435,30 +460,6 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 	return keep_exploring ? KEEP_EXPLORING : DONE_EXPLORING;
 }
 
-/*
- * Instructions that can abnormally return from a subprog (tail_call
- * upon success, ld_{abs,ind} upon load failure) have a hidden exit
- * that the verifier must account for.
- */
-static int visit_abnormal_return_insn(struct bpf_verifier_env *env, int t)
-{
-	struct bpf_subprog_info *subprog;
-	struct bpf_iarray *jt;
-
-	if (env->insn_aux_data[t].jt)
-		return 0;
-
-	jt = bpf_iarray_realloc(NULL, 2);
-	if (!jt)
-		return -ENOMEM;
-
-	subprog = bpf_find_containing_subprog(env, t);
-	jt->items[0] = t + 1;
-	jt->items[1] = subprog->exit_idx;
-	env->insn_aux_data[t].jt = jt;
-	return 0;
-}
-
 /* Visits the instruction at index t and returns one of the following:
  *  < 0 - an error occurred
  *  DONE_EXPLORING - the instruction was fully explored
@@ -475,13 +476,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
 	/* All non-branch instructions have a single fall-through edge. */
 	if (BPF_CLASS(insn->code) != BPF_JMP &&
 	    BPF_CLASS(insn->code) != BPF_JMP32) {
-		if (BPF_CLASS(insn->code) == BPF_LD &&
-		    (BPF_MODE(insn->code) == BPF_ABS ||
-		     BPF_MODE(insn->code) == BPF_IND)) {
-			ret = visit_abnormal_return_insn(env, t);
-			if (ret)
-				return ret;
-		}
 		insn_sz = bpf_is_ldimm64(insn) ? 2 : 1;
 		return push_insn(t, t + insn_sz, FALLTHROUGH, env);
 	}
@@ -526,11 +520,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
 				mark_subprog_might_sleep(env, t);
 			if (bpf_helper_changes_pkt_data(insn->imm))
 				mark_subprog_changes_pkt_data(env, t);
-			if (insn->imm == BPF_FUNC_tail_call) {
-				ret = visit_abnormal_return_insn(env, t);
-				if (ret)
-					return ret;
-			}
 		} else if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
 			struct bpf_call_arg_meta meta;
 
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 52d3cec33672..f3cf66b8147a 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -500,23 +500,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
 	return 0;
 }
 
-/*
- * Clean up dynamically allocated fields of aux data for instructions [start, ...]
- */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
-{
-	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
-	int end = start + len;
-	int i;
-
-	for (i = start; i < end; i++) {
-		if (aux_data[i].jt) {
-			kvfree(aux_data[i].jt);
-			aux_data[i].jt = NULL;
-		}
-	}
-}
-
 static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 {
 	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
@@ -526,8 +509,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 	if (bpf_prog_is_offloaded(env->prog->aux))
 		bpf_prog_offload_remove_insns(env, off, cnt);
 
-	bpf_clear_insn_aux_data(env, off, cnt);
-
 	err = bpf_remove_insns(env->prog, off, cnt);
 	if (err)
 		return err;
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 301fc60bddc4..08592fdb5510 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -248,9 +248,12 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
 	struct bpf_iarray *succ, *jt;
 	int insn_sz;
 
-	jt = env->insn_aux_data[idx].jt;
-	if (unlikely(jt))
-		return jt;
+	/* All gotox of a subprogram share its jump table, see compute_subprog_jts() */
+	if (unlikely(insn_is_gotox(insn))) {
+		jt = bpf_find_containing_subprog(env, idx)->jt;
+		if (jt)
+			return jt;
+	}
 
 	/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */
 	succ = env->succ;
@@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
 	if (opcode_info->can_jump)
 		succ->items[succ->cnt++] = idx + bpf_jmp_offset(insn) + 1;
 
+	/*
+	 * tail_call upon success and ld_{abs,ind} upon load failure return
+	 * from the subprogram, which is a hidden edge to its exit.
+	 */
+	if (unlikely((bpf_helper_call(insn) && insn->imm == BPF_FUNC_tail_call) ||
+		     (BPF_CLASS(insn->code) == BPF_LD &&
+		      (BPF_MODE(insn->code) == BPF_ABS || BPF_MODE(insn->code) == BPF_IND))))
+		succ->items[succ->cnt++] = bpf_find_containing_subprog(env, idx)->exit_idx;
+
 	return succ;
 }
 
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index ddba53eaa333..32157d145a60 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -21477,8 +21477,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
 	release_maps(env);
 	release_btfs(env);
 err_free_env:
-	if (env->insn_aux_data)
-		bpf_clear_insn_aux_data(env, 0, env->insn_aux_data_len);
+	bpf_free_subprog_jts(env);
 	vfree(env->insn_aux_data);
 	kvfree(env->fd_array);
 	bpf_stack_liveness_free(env);
-- 
2.43.0

Re: [PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 5879 bytes --]

> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> index 04bb8f71cabe3..754e24d41f47f 100644
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h
> @@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
>  		/* remember the offset of node field within type to rewrite */
>  		u64 insert_off;
>  	};
> -	struct bpf_iarray *jt;	/* jump table for gotox or bpf_tailcall call instruction */
>  	struct btf_struct_meta *kptr_struct_meta;
>  	u64 map_key_state; /* constant (32 bit) key tracking for maps */
>  	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
> @@ -805,6 +804,7 @@ struct bpf_subprog_info {
>  	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
>  	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
>  	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
> +	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
                                ^^^^
The patch moves jt ownership into bpf_subprog_info but doesn't update the
code that removes instructions. When verifier_remove_insns() deletes a
whole subprogram, adjust_subprog_starts_after_remove() in
kernel/bpf/fixups.c compacts the array with memmove:

    move = env->subprog_cnt + 1 - j;
    memmove(env->subprog_info + i,
            env->subprog_info + j,
            sizeof(*env->subprog_info) * move);
    env->subprog_cnt -= j - i;

If subprogram 3 owns a jump table and subprograms 1 and 2 are removed
(i=1, j=3), slots 3 and 4 are never rewritten, so subprog_info[1].jt and
subprog_info[3].jt both point to the same kvmalloc'd buffer.

Does bpf_free_subprog_jts() then double-free that buffer? Its loop walks
the entire fixed-size array rather than stopping at env->subprog_cnt:

    for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++)
        kvfree(env->subprog_info[i].jt);

so both the moved copy at index 1 and the stale alias at index 3 would be
freed.

Also, when verifier_remove_insns() overwrites subprog_info[i..j), doesn't
that leak the jump table of any subprogram in that range?

Before this patch, verifier_remove_insns() called
bpf_clear_insn_aux_data(env, off, cnt) to free the jt of every removed
instruction and then memmoved the aux array. The new owner needs the same
handling: kvfree the tables of removed subprograms and clear the stale
tail slots left by the memmove, or bound bpf_free_subprog_jts() to
env->subprog_cnt and explicitly zero the tail.

[ ... ]

> diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
> index df4513e71c80a..c38d33f0b0ba5 100644
> --- a/kernel/bpf/cfg.c
> +++ b/kernel/bpf/cfg.c

[ ... ]

> +void bpf_free_subprog_jts(struct bpf_verifier_env *env)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
> +		kvfree(env->subprog_info[i].jt);
> +		env->subprog_info[i].jt = NULL;
> +	}
> +	env->cfg.subprog_jts_ready = false;
> +}

This iterates over all BPF_MAX_SUBPROGS + 2 slots (258 entries), but
adjust_subprog_starts_after_remove() only updates env->subprog_cnt when
subprograms are removed. Slots beyond that count can hold stale pointers
duplicated by the memmove. Is there a risk of freeing the same allocation
twice, once from its new index and once from the old stale slot?

[ ... ]

> diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
> index 52d3cec336727..f3cf66b8147a4 100644
> --- a/kernel/bpf/fixups.c
> +++ b/kernel/bpf/fixups.c
> @@ -500,23 +500,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
>  	return 0;
>  }
>
> -/*
> - * Clean up dynamically allocated fields of aux data for instructions [start, ...]
> - */
> -void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
> -{
> -	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
> -	int end = start + len;
> -	int i;
> -
> -	for (i = start; i < end; i++) {
> -		if (aux_data[i].jt) {
> -			kvfree(aux_data[i].jt);
> -			aux_data[i].jt = NULL;
> -		}
> -	}
> -}
> -
>  static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
>  {
>  	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
> @@ -526,8 +509,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
>  	if (bpf_prog_is_offloaded(env->prog->aux))
>  		bpf_prog_offload_remove_insns(env, off, cnt);
>
> -	bpf_clear_insn_aux_data(env, off, cnt);
> -
                          ^^^^
Removing bpf_clear_insn_aux_data() makes sense for insn_aux_data, since
it no longer owns the jt allocation. But subprog_info[].jt is still an
owned pointer, and adjust_subprog_starts_after_remove() will memmove
those structs when entire subprograms disappear.

Does verifier_remove_insns() need equivalent cleanup for the new owner?
When j - i removed subprograms are dropped, the memmove copies slots
[j, subprog_cnt] down to [i, ...] and decrements subprog_cnt by j - i,
but it doesn't kvfree the jt pointers being overwritten in [i, j) or
NULL the duplicates left in the tail slots.

>  	err = bpf_remove_insns(env->prog, off, cnt);
>  	if (err)
>  		return err;

[ ... ]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index ddba53eaa3331..32157d145a609 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -21477,8 +21477,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
>  	release_maps(env);
>  	release_btfs(env);
>  err_free_env:
> -	if (env->insn_aux_data)
> -		bpf_clear_insn_aux_data(env, 0, env->insn_aux_data_len);
> +	bpf_free_subprog_jts(env);
>  	vfree(env->insn_aux_data);
>  	kvfree(env->fd_array);
>  	bpf_stack_liveness_free(env);


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

Re: [PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

[Daniel Borkmann]

On 9/14/26 8:19 PM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
>> index 04bb8f71cabe3..754e24d41f47f 100644
>> --- a/include/linux/bpf_verifier.h
>> +++ b/include/linux/bpf_verifier.h
>> @@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
>>   		/* remember the offset of node field within type to rewrite */
>>   		u64 insert_off;
>>   	};
>> -	struct bpf_iarray *jt;	/* jump table for gotox or bpf_tailcall call instruction */
>>   	struct btf_struct_meta *kptr_struct_meta;
>>   	u64 map_key_state; /* constant (32 bit) key tracking for maps */
>>   	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
>> @@ -805,6 +804,7 @@ struct bpf_subprog_info {
>>   	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
>>   	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
>>   	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
>> +	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
>                                  ^^^^
> The patch moves jt ownership into bpf_subprog_info but doesn't update the
> code that removes instructions. When verifier_remove_insns() deletes a
> whole subprogram, adjust_subprog_starts_after_remove() in
> kernel/bpf/fixups.c compacts the array with memmove:
Great catch! I'll fix this up in a v3 along with a selftest, and look into either
reordering patch 2 & 3 or tweaking the commit msg a bit.

Re: [PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

[Eduard Zingerman]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:

Hi Daniel,

I think this is a good approach.

...

> diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c

...

> @@ -383,13 +409,12 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
>  	struct bpf_iarray *jt;
>  	int i, w;
>  
> -	jt = env->insn_aux_data[t].jt;
> -	if (!jt) {
> -		jt = create_jt(t, env);
> -		if (IS_ERR(jt))
> -			return PTR_ERR(jt);
> +	jt = subprog_jt(t, env);
> +	if (IS_ERR(jt))
> +		return PTR_ERR(jt);
>  
> -		env->insn_aux_data[t].jt = jt;
> +	if (!(insn_state[t] & BRANCH)) {
> +		insn_state[t] |= BRANCH;

I think it is possible to exit early here if BRANCH is already set.

>  
>  		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
>  				       &env->cfg.gotox_edges) ||

...

> diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c

...

> @@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
>  	if (opcode_info->can_jump)
>  		succ->items[succ->cnt++] = idx + bpf_jmp_offset(insn) + 1;
>  
> +	/*
> +	 * tail_call upon success and ld_{abs,ind} upon load failure return
> +	 * from the subprogram, which is a hidden edge to its exit.
> +	 */
> +	if (unlikely((bpf_helper_call(insn) && insn->imm == BPF_FUNC_tail_call) ||
> +		     (BPF_CLASS(insn->code) == BPF_LD &&
> +		      (BPF_MODE(insn->code) == BPF_ABS || BPF_MODE(insn->code) == BPF_IND))))
> +		succ->items[succ->cnt++] = bpf_find_containing_subprog(env, idx)->exit_idx;

It appears that check_subprogs() assigns exit_idx but does not
guarantee it to be set. Meaning that for the following progam exit_idx
would remain zero:

  0: r0 = 0
  1: r0 = *(u8 *)skb[0]
  2: goto -2

Hence the above would add a spurious edge.
I think check_subprogs() should be updated to demand each subprogram
to have at-least one exit.

> +
>  	return succ;
>  }
>  

...

[PATCH bpf v2 04/11] bpf: Reject indirect jumps that leave their subprogram

[Daniel Borkmann]

The jump table of a subprog is collected in compute_subprog_jts() from the
insn_array maps of the program, and a map is attributed to the subprog that
contains its first entry. check_indirect_jump() instead resolves the targets
from the map the gotox register actually points to, bounded only by the
index range of that register, and never relates them back to the subprog
of the gotox.

The two disagree, so bpf_insn_successors() reports a subset of the edges the
BPF program can take and a gotox can enter a subprog the CFG never walked.

Close both ends: reject a map that reaches past the subprog holding its
first entry right when it is collected in compute_subprog_jts(), so that
every map is confined to one subprog, and confine the resolved targets of
a gotox to its subprog in check_indirect_jump(). A target inside the
subprog is then always in the jump table the CFG walked, which is the
invariant that actually has to hold, and is kept as a verifier_bug_if().

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reported-by: James Burton <jamesburton@meta.com>
Reported-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 kernel/bpf/cfg.c                              | 39 +++++--------
 kernel/bpf/verifier.c                         | 55 +++++++++++++++++++
 .../selftests/bpf/progs/verifier_gotox.c      |  2 +-
 3 files changed, 69 insertions(+), 27 deletions(-)

diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index c38d33f0b0ba..5a91d73d3e3f 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -287,7 +287,7 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
 
 /*
  * Collect the jump table of every subprogram that has one, as the combined
- * table of all maps whose first target lands inside that subprogram. All gotox
+ * table of all maps whose targets land inside that subprogram. All gotox
  * instructions of a subprogram share the same table, so this is done in a
  * single pass over the maps rather than once per gotox.
  */
@@ -310,15 +310,21 @@ static int compute_subprog_jts(struct bpf_verifier_env *env)
 		if (IS_ERR(jt_cur))
 			return PTR_ERR(jt_cur);
 
-		/*
-		 * This is enough to check one element. The full table is
-		 * checked to fit inside the subprog later in subprog_jt()
-		 */
 		subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
 		if (!subprog) {
 			kvfree(jt_cur);
 			continue;
 		}
+		if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) {
+			verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n",
+				subprog->start);
+			bpf_diag_program_structure(env, subprog->start, "jump table spans subprograms",
+				"Keep every entry of a jump table inside one subprogram.",
+				"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.",
+				subprog->start, (subprog + 1)->start);
+			kvfree(jt_cur);
+			return -EINVAL;
+		}
 
 		old_cnt = subprog->jt ? subprog->jt->cnt : 0;
 		jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
@@ -358,9 +364,7 @@ static struct bpf_iarray *
 subprog_jt(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_subprog_info *subprog;
-	int subprog_start, subprog_end;
-	struct bpf_iarray *jt;
-	int i, err;
+	int subprog_start, err;
 
 	if (!env->cfg.subprog_jts_ready) {
 		err = compute_subprog_jts(env);
@@ -370,7 +374,6 @@ subprog_jt(int t, struct bpf_verifier_env *env)
 
 	subprog = bpf_find_containing_subprog(env, t);
 	subprog_start = subprog->start;
-	subprog_end = (subprog + 1)->start;
 
 	if (!subprog->jt) {
 		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
@@ -381,23 +384,7 @@ subprog_jt(int t, struct bpf_verifier_env *env)
 		return ERR_PTR(-EINVAL);
 	}
 
-	jt = subprog->jt;
-
-	/* Check that the every element of the jump table fits within the given subprogram */
-	for (i = 0; i < jt->cnt; i++) {
-		if (jt->items[i] < subprog_start || jt->items[i] >= subprog_end) {
-			verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n",
-					t, subprog_start, subprog_end);
-			bpf_diag_program_structure(
-				env, t, "jump table target out of range",
-				"Keep every jump-table target inside the same subprogram.",
-				"The jump table for instruction %d points outside subprogram range [%u,%u).",
-				t, subprog_start, subprog_end);
-			return ERR_PTR(-EINVAL);
-		}
-	}
-
-	return jt;
+	return subprog->jt;
 }
 
 /* "conditional jump with N edges" */
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 32157d145a60..dc3375fcfab9 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18191,11 +18191,43 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,
 	return 0;
 }
 
+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */
+static bool jt_contains(const struct bpf_iarray *jt, u32 target)
+{
+	int l = 0, r = jt->cnt - 1, m;
+
+	while (l <= r) {
+		m = l + (r - l) / 2;
+		if (jt->items[m] == target)
+			return true;
+		if (jt->items[m] < target)
+			l = m + 1;
+		else
+			r = m - 1;
+	}
+	return false;
+}
+
+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,
+				       u32 subprog_start, u32 subprog_end)
+{
+	verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n",
+		     env->insn_idx, target, subprog_start, subprog_end);
+	bpf_diag_program_structure(env, env->insn_idx, "indirect jump leaves subprogram",
+		"Keep every reachable jump-table target inside the subprogram of the indirect jump.",
+		"Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).",
+		env->insn_idx, target, subprog_start, subprog_end);
+	return -EINVAL;
+}
+
 /* gotox *dst_reg */
 static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)
 {
 	struct bpf_verifier_state *other_branch;
+	struct bpf_subprog_info *subprog;
+	u32 subprog_start, subprog_end;
 	struct bpf_reg_state *dst_reg;
+	struct bpf_iarray *jt;
 	struct bpf_map *map;
 	u32 min_index, max_index;
 	int err = 0;
@@ -18238,6 +18270,29 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
 		return -EINVAL;
 	}
 
+	subprog = bpf_find_containing_subprog(env, env->insn_idx);
+	if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
+		return -EFAULT;
+	subprog_start = subprog->start;
+	subprog_end = (subprog + 1)->start;
+
+	jt = subprog->jt;
+	if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
+		return -EFAULT;
+
+	for (i = 0; i < n; i++) {
+		u32 target = env->gotox_tmp_buf->items[i];
+
+		if (target < subprog_start || target >= subprog_end)
+			return reject_gotox_out_of_subprog(env, target, subprog_start,
+							   subprog_end);
+		/* Maps are confined to a subprog, see compute_subprog_jts() */
+		if (verifier_bug_if(!jt_contains(jt, target), env,
+				    "insn %d target %u is not in the jump table of its subprog",
+				    env->insn_idx, target))
+			return -EFAULT;
+	}
+
 	for (i = 0; i < n - 1; i++) {
 		mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
 		other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index 0e27c2c79c57..4931a77ba0b2 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -366,7 +366,7 @@ __used static int test_subprog(void)
 }
 
 SEC("socket")
-__failure __msg("jump table for insn 4 points outside of the subprog [0,10]")
+__failure __msg("jump table of subprog starting at 0 spans multiple subprogs")
 __naked void jump_table_outside_subprog(void)
 {
 	asm volatile ("						\
-- 
2.43.0

Re: [PATCH bpf v2 04/11] bpf: Reject indirect jumps that leave their subprogram

[Eduard Zingerman]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:
> The jump table of a subprog is collected in compute_subprog_jts() from the
> insn_array maps of the program, and a map is attributed to the subprog that
> contains its first entry. check_indirect_jump() instead resolves the targets
> from the map the gotox register actually points to, bounded only by the
> index range of that register, and never relates them back to the subprog
> of the gotox.
> 
> The two disagree, so bpf_insn_successors() reports a subset of the edges the
> BPF program can take and a gotox can enter a subprog the CFG never walked.
> 
> Close both ends: reject a map that reaches past the subprog holding its
> first entry right when it is collected in compute_subprog_jts(), so that
> every map is confined to one subprog, and confine the resolved targets of
> a gotox to its subprog in check_indirect_jump(). A target inside the
> subprog is then always in the jump table the CFG walked, which is the
> invariant that actually has to hold, and is kept as a verifier_bug_if().
> 
> Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
> Reported-by: James Burton <jamesburton@meta.com>
> Reported-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

> @@ -18238,6 +18270,29 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
>  		return -EINVAL;
>  	}
>  
> +	subprog = bpf_find_containing_subprog(env, env->insn_idx);
> +	if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
> +		return -EFAULT;
> +	subprog_start = subprog->start;
> +	subprog_end = (subprog + 1)->start;
> +
> +	jt = subprog->jt;
> +	if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
> +		return -EFAULT;
> +
> +	for (i = 0; i < n; i++) {
> +		u32 target = env->gotox_tmp_buf->items[i];
> +
> +		if (target < subprog_start || target >= subprog_end)
> +			return reject_gotox_out_of_subprog(env, target, subprog_start,
> +							   subprog_end);
> +		/* Maps are confined to a subprog, see compute_subprog_jts() */
> +		if (verifier_bug_if(!jt_contains(jt, target), env,
> +				    "insn %d target %u is not in the jump table of its subprog",
> +				    env->insn_idx, target))
> +			return -EFAULT;

Nit: I think this check is redundant, it can be dropped together with
     jt_contains(), moving the above subprog_{start,end} checks out of
     the loop.

> +	}
> +
>  	for (i = 0; i < n - 1; i++) {
>  		mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
>  		other_branch = push_stack(env, env->gotox_tmp_buf->items[i],

...

[PATCH bpf v2 05/11] bpf: Don't remove indirect jump targets in the nop removal pass

[Daniel Borkmann]

bpf_opt_remove_nops() drops every ja +0 and may_goto +0 unconditionally.
A gotox target may itself be such a transparent nop: it is a verified
entry of an insn array jump table that is reachable at run time through
the indirect jump. bpf_insn_array_adjust_after_remove() marks it as
INSN_DELETED instead, and bpf_insn_array_ready() skips INSN_DELETED
entries, so the program loads with a hole in its jump table. The x86
JIT, for example, lowers gotox to a raw indirect jump, so executing
that index jumps to the zeroed ips[] slot, that is, to address 0.
Fix by keeping an indirect jump target in place.

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reported-by: Sandipan Roy <saroy@redhat.com>
Reported-by: James Burton <jamesburton@meta.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 kernel/bpf/fixups.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index f3cf66b8147a..aa3fb7b87fb0 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -599,6 +599,7 @@ int bpf_opt_remove_dead_code(struct bpf_verifier_env *env)
 
 int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 {
+	struct bpf_insn_aux_data *aux = env->insn_aux_data;
 	struct bpf_insn *insn = env->prog->insnsi;
 	int insn_cnt = env->prog->len;
 	bool is_may_goto_0, is_ja;
@@ -610,6 +611,8 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 
 		if (!is_may_goto_0 && !is_ja)
 			continue;
+		if (aux[i].indirect_target)
+			continue;
 
 		err = verifier_remove_insns(env, i, 1);
 		if (err)
-- 
2.43.0

Re: [PATCH bpf v2 05/11] bpf: Don't remove indirect jump targets in the nop removal pass

[Eduard Zingerman]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:
> bpf_opt_remove_nops() drops every ja +0 and may_goto +0 unconditionally.
> A gotox target may itself be such a transparent nop: it is a verified
> entry of an insn array jump table that is reachable at run time through
> the indirect jump. bpf_insn_array_adjust_after_remove() marks it as
> INSN_DELETED instead, and bpf_insn_array_ready() skips INSN_DELETED
> entries, so the program loads with a hole in its jump table. The x86
> JIT, for example, lowers gotox to a raw indirect jump, so executing
> that index jumps to the zeroed ips[] slot, that is, to address 0.
> Fix by keeping an indirect jump target in place.
> 
> Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
> Reported-by: Sandipan Roy <saroy@redhat.com>
> Reported-by: James Burton <jamesburton@meta.com>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

[PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

[Daniel Borkmann]

An insn array records, per entry, the xlated offset, the jitted offset and
a jitted target pointer in ips[]. bpf_insn_array_init() runs when the map
is bound to a program and resets only xlated_off; the jitted offset and the
ips[] pointer are left as the previous owner set them. A program can be
verified and JITed and only then fail to load (e.g. bpf_check_tail_call()).
By that point bpf_prog_update_insn_ptrs() has already filled ips[] with
pointers into that program's JIT image. Thus, the map is handed back for
reuse with the stale pointers intact. If the next program reuses the map,
the stale pointer survived the now active program's jump table, pointing
into the previous, freed image. Fix by resetting the jitted offset and the
ips[] pointer alongside the xlated offset.

Fixes: b4ce5923e780 ("bpf, x86: add new map type: instructions array")
Reported-by: Sandipan Roy <saroy@redhat.com>
Reported-by: James Burton <jamesburton@meta.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 kernel/bpf/bpf_insn_array.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index a2f84afe6f7c..5874abd1f743 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
 		return -EBUSY;
 
 	/*
-	 * Reset all the map indexes to the original values.  This is needed,
-	 * e.g., when a replay of verification with different log level should
-	 * be performed.
+	 * Reset the map to its pre-verification state. The xlated and jitted
+	 * offsets and the jitted target pointers are recomputed by the verifier
+	 * and the JIT for this program, so any values left by a previous owner
+	 * must be cleared here.
 	 */
-	for (i = 0; i < map->max_entries; i++)
+	for (i = 0; i < map->max_entries; i++) {
 		values[i].xlated_off = values[i].orig_off;
+		values[i].jitted_off = 0;
+		insn_array->ips[i] = 0;
+	}
 
 	return 0;
 }
-- 
2.43.0

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing release semantics in `bpf_insn_array_release` can allow stale JIT pointers to overwrite the re-initialized array due to CPU store reordering.
--

commit 9b6ea61f193d20b96c453fda13952db5c1dc1506
Author: Daniel Borkmann <daniel@iogearbox.net>

bpf: Clear insn array jitted pointers on map reuse

This commit fixes a bug where stale JIT pointers and jitted offsets are left in
an instruction array map when it is reused. It clears these fields during map
initialization in bpf_insn_array_init().

> diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
> index a2f84afe6f7c6..5874abd1f7438 100644
> --- a/kernel/bpf/bpf_insn_array.c
> +++ b/kernel/bpf/bpf_insn_array.c
> @@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
>  		return -EBUSY;
>  
>  	/*
> -	 * Reset all the map indexes to the original values.  This is needed,
> -	 * e.g., when a replay of verification with different log level should
> -	 * be performed.
> +	 * Reset the map to its pre-verification state. The xlated and jitted
> +	 * offsets and the jitted target pointers are recomputed by the verifier
> +	 * and the JIT for this program, so any values left by a previous owner
> +	 * must be cleared here.
>  	 */
> -	for (i = 0; i < map->max_entries; i++)
> +	for (i = 0; i < map->max_entries; i++) {
>  		values[i].xlated_off = values[i].orig_off;
> +		values[i].jitted_off = 0;
> +		insn_array->ips[i] = 0;
> +	}
>  
>  	return 0;
>  }

[Severity: High]
This isn't a bug introduced by this patch, but while looking at how the map is
reused, I noticed a potential missing release semantics issue in
bpf_insn_array_release():

kernel/bpf/bpf_insn_array.c:bpf_insn_array_release() {
	struct bpf_insn_array *insn_array = cast_insn_array(map);

	atomic_set(&insn_array->used, 0);
}

Is it possible that without release semantics (like atomic_set_release), prior
stores to the map can be reordered after this flag is set?

If two threads concurrently load a BPF program using the same
BPF_MAP_TYPE_INSN_ARRAY map:

Thread 1 JITs the program and writes to ips via bpf_prog_update_insn_ptrs(),
but fails a late verification check like bpf_check_tail_call(). It then
releases the map by setting used = 0 via the relaxed atomic_set().

Thread 2 acquires the map using atomic_xchg() and zeroes ips during
initialization, as added in this patch.

Could CPU reordering on weakly-ordered architectures allow Thread 1's delayed
JIT stores to overwrite the zeros with stale pointers, causing Thread 2 to
subsequently use those stale pointers?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260914170326.249953-1-daniel@iogearbox.net?part=6

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

[Daniel Borkmann]

On 9/14/26 7:29 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] Missing release semantics in `bpf_insn_array_release` can allow stale JIT pointers to overwrite the re-initialized array due to CPU store reordering.
> --
[...]
> [Severity: High]
> This isn't a bug introduced by this patch, but while looking at how the map is
> reused, I noticed a potential missing release semantics issue in
> bpf_insn_array_release():
> 
> kernel/bpf/bpf_insn_array.c:bpf_insn_array_release() {
> 	struct bpf_insn_array *insn_array = cast_insn_array(map);
> 
> 	atomic_set(&insn_array->used, 0);
> }
> 
> Is it possible that without release semantics (like atomic_set_release), prior
> stores to the map can be reordered after this flag is set?
> 
> If two threads concurrently load a BPF program using the same
> BPF_MAP_TYPE_INSN_ARRAY map:
> 
> Thread 1 JITs the program and writes to ips via bpf_prog_update_insn_ptrs(),
> but fails a late verification check like bpf_check_tail_call(). It then
> releases the map by setting used = 0 via the relaxed atomic_set().
> 
> Thread 2 acquires the map using atomic_xchg() and zeroes ips during
> initialization, as added in this patch.
> 
> Could CPU reordering on weakly-ordered architectures allow Thread 1's delayed
> JIT stores to overwrite the zeros with stale pointers, causing Thread 2 to
> subsequently use those stale pointers?

I'll look into that one separately.

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]

> bpf: Clear insn array jitted pointers on map reuse
>
> An insn array records, per entry, the xlated offset, the jitted offset and
> a jitted target pointer in ips[]. bpf_insn_array_init() runs when the map
> is bound to a program and resets only xlated_off; the jitted offset and the
> ips[] pointer are left as the previous owner set them. A program can be
> verified and JITed and only then fail to load (e.g. bpf_check_tail_call()).
> By that point bpf_prog_update_insn_ptrs() has already filled ips[] with
> pointers into that program's JIT image. Thus, the map is handed back for
> reuse with the stale pointers intact. If the next program reuses the map,
> the stale pointer survived the now active program's jump table, pointing
> into the previous, freed image. Fix by resetting the jitted offset and the
> ips[] pointer alongside the xlated offset.

The sentence "the stale pointer survived the now active program's jump
table, pointing into the previous, freed image" is hard to parse.
'Survived' takes the jump table as a direct object, so it reads as if
the pointer outlived the table rather than persisting inside it. Would
"the stale pointer survives in the now active program's jump table" be
clearer? This is the sentence that describes the actual security impact,
so the wording matters for anyone reading the changelog to judge
severity.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

[Eduard Zingerman]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:
> An insn array records, per entry, the xlated offset, the jitted offset and
> a jitted target pointer in ips[]. bpf_insn_array_init() runs when the map
> is bound to a program and resets only xlated_off; the jitted offset and the
> ips[] pointer are left as the previous owner set them. A program can be
> verified and JITed and only then fail to load (e.g. bpf_check_tail_call()).
> By that point bpf_prog_update_insn_ptrs() has already filled ips[] with
> pointers into that program's JIT image. Thus, the map is handed back for
> reuse with the stale pointers intact. If the next program reuses the map,
> the stale pointer survived the now active program's jump table, pointing
> into the previous, freed image. Fix by resetting the jitted offset and the
> ips[] pointer alongside the xlated offset.
> 
> Fixes: b4ce5923e780 ("bpf, x86: add new map type: instructions array")
> Reported-by: Sandipan Roy <saroy@redhat.com>
> Reported-by: James Burton <jamesburton@meta.com>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

[PATCH bpf v2 07/11] bpf: Retarget indirect jump targets across prologue prepends

[Daniel Borkmann]

A few patches do not expand in place, they prepend: gen_epilogue and
gen_prologue splice their instructions in front of insn 0 in
bpf_convert_ctx_accesses(), and bpf_do_misc_fixups() prepends the may_goto
counter init at every subprog start.

They copy the original instruction of 'tgt_idx' into the last slot of the
patch buffer, so it now lives at tgt_idx + delta, and call adjust_jmp_off()
to move direct branches from tgt_idx to tgt_idx + delta.

Nothing does the same for indirect branches, so a BPF_MAP_TYPE_INSN_ARRAY
slot that named tgt_idx keeps naming tgt_idx, which is now the first prepended
instruction, and insn_aux_data[tgt_idx].indirect_target makes the JIT emit
the landing pad there. Add a __bpf_patch_insn_data() variant and pass the
BPF_PREPEND at the affected call-sites to fix up the delta.

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Fixes: 07ae6c130b46 ("bpf: Add helper to detect indirect jump targets")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 include/linux/bpf.h         |  2 +-
 kernel/bpf/bpf_insn_array.c |  4 ++--
 kernel/bpf/fixups.c         | 41 ++++++++++++++++++++++++++-----------
 3 files changed, 32 insertions(+), 15 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e57af902560c..8115f307444a 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);
 int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
 int bpf_insn_array_ready(struct bpf_map *map);
 void bpf_insn_array_release(struct bpf_map *map);
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);
 void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);
 
 #ifdef CONFIG_BPF_SYSCALL
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index 5874abd1f743..350ecea8f1e8 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -235,7 +235,7 @@ void bpf_insn_array_release(struct bpf_map *map)
 	atomic_set(&insn_array->used, 0);
 }
 
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)
 {
 	struct bpf_insn_array *insn_array = cast_insn_array(map);
 	int i;
@@ -244,7 +244,7 @@ void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
 		return;
 
 	for (i = 0; i < map->max_entries; i++) {
-		if (insn_array->values[i].xlated_off <= off)
+		if (insn_array->values[i].xlated_off < first)
 			continue;
 		if (insn_array->values[i].xlated_off == INSN_DELETED)
 			continue;
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index aa3fb7b87fb0..e132bd0600c7 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -213,12 +213,18 @@ static int get_callee_stack_depth(struct bpf_verifier_env *env,
 }
 #endif
 
+enum bpf_patch_mode {
+	BPF_APPEND,
+	BPF_PREPEND,
+};
+
 /* single env->prog->insni[off] instruction was replaced with the range
  * insni[off, off + cnt).  Adjust corresponding insn_aux_data by copying
  * [0, off) and [off, end) to new locations, so the patched range stays zero
  */
 static void adjust_insn_aux_data(struct bpf_verifier_env *env,
-				 struct bpf_prog *new_prog, u32 off, u32 cnt)
+				 struct bpf_prog *new_prog, u32 off, u32 cnt,
+				 enum bpf_patch_mode mode)
 {
 	struct bpf_insn_aux_data *data = env->insn_aux_data;
 	struct bpf_insn *insn = new_prog->insnsi;
@@ -251,9 +257,10 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
 	 * new instructions by the above memmove and memset, but the indirect jump target is
 	 * actually the first instruction, so move it back. This also matches with the behavior
 	 * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
-	 * instruction.
+	 * instruction. For BPF_PREPEND the original instruction is the last one, so the flag
+	 * already sits where needed.
 	 */
-	if (data[off + cnt - 1].indirect_target) {
+	if (mode == BPF_APPEND && data[off + cnt - 1].indirect_target) {
 		data[off].indirect_target = 1;
 		data[off + cnt - 1].indirect_target = 0;
 	}
@@ -273,7 +280,7 @@ static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len
 	}
 }
 
-static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)
+static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 first, u32 len)
 {
 	int i;
 
@@ -281,7 +288,7 @@ static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)
 		return;
 
 	for (i = 0; i < env->insn_array_map_cnt; i++)
-		bpf_insn_array_adjust(env->insn_array_maps[i], off, len);
+		bpf_insn_array_adjust(env->insn_array_maps[i], first, len);
 }
 
 static void adjust_insn_arrays_after_remove(struct bpf_verifier_env *env, u32 off, u32 len)
@@ -306,8 +313,9 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
 	}
 }
 
-struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
-				     const struct bpf_insn *patch, u32 len)
+static struct bpf_prog *__bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+					      const struct bpf_insn *patch, u32 len,
+					      enum bpf_patch_mode mode)
 {
 	struct bpf_prog *new_prog;
 	struct bpf_insn_aux_data *new_data = NULL;
@@ -331,13 +339,19 @@ struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
 				env->insn_aux_data[off].orig_idx);
 		return NULL;
 	}
-	adjust_insn_aux_data(env, new_prog, off, len);
+	adjust_insn_aux_data(env, new_prog, off, len, mode);
 	adjust_subprog_starts(env, off, len);
-	adjust_insn_arrays(env, off, len);
+	adjust_insn_arrays(env, mode == BPF_PREPEND ? off : off + 1, len);
 	adjust_poke_descs(new_prog, off, len);
 	return new_prog;
 }
 
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+				     const struct bpf_insn *patch, u32 len)
+{
+	return __bpf_patch_insn_data(env, off, patch, len, BPF_APPEND);
+}
+
 /*
  * For all jmp insns in a given 'prog' that point to 'tgt_idx' insn adjust the
  * jump offset by 'delta'.
@@ -755,7 +769,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
 			insn_buf[cnt++] = BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1,
 						      -subprogs[0].stack_depth);
 			insn_buf[cnt++] = env->prog->insnsi[0];
-			new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+			new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+							 BPF_PREPEND);
 			if (!new_prog)
 				return -ENOMEM;
 			env->prog = new_prog;
@@ -778,7 +793,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
 			verifier_bug(env, "prologue is too long");
 			return -EFAULT;
 		} else if (cnt) {
-			new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+			new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+							 BPF_PREPEND);
 			if (!new_prog)
 				return -ENOMEM;
 
@@ -2443,7 +2459,8 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env)
 		/* Copy first actual insn to preserve it */
 		insn_buf[cnt++] = env->prog->insnsi[subprog_start];
 
-		new_prog = bpf_patch_insn_data(env, subprog_start, insn_buf, cnt);
+		new_prog = __bpf_patch_insn_data(env, subprog_start, insn_buf, cnt,
+						 BPF_PREPEND);
 		if (!new_prog)
 			return -ENOMEM;
 		env->prog = prog = new_prog;
-- 
2.43.0

Re: [PATCH bpf v2 07/11] bpf: Retarget indirect jump targets across prologue prepends

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4210 bytes --]

> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index e57af902560c3..8115f307444ad 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);
>  int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
>  int bpf_insn_array_ready(struct bpf_map *map);
>  void bpf_insn_array_release(struct bpf_map *map);
> -void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
> +void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);
>  void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);

[ ... ]

> diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
> index aa3fb7b87fb06..e132bd0600c7e 100644
> --- a/kernel/bpf/fixups.c
> +++ b/kernel/bpf/fixups.c
> @@ -213,12 +213,18 @@ static int get_callee_stack_depth(struct bpf_verifier_env *env,
>  }
>  #endif
>
> +enum bpf_patch_mode {
> +	BPF_APPEND,
> +	BPF_PREPEND,
> +};
> +

This isn't a bug, but since BPF_APPEND also covers buffers that prepend
to the original insn (like the BPF_ST_NOSPEC sites where the original
insn is copied to the end of the buffer), would names describing whether
the target index moves be clearer here?

The mode really selects whether the original instruction stays at 'off'
or moves to 'off + len - 1', which is why the BPF_PREPEND sites coincide
exactly with the two adjust_jmp_off() call sites. Something like
BPF_PATCH_KEEP_TARGET vs BPF_PATCH_MOVE_TARGET, or at least
BPF_PATCH_APPEND / BPF_PATCH_PREPEND to match the usual stem pairing in
this subsystem, might read less contradicted by neighbouring APPEND call
sites.

>  /* single env->prog->insni[off] instruction was replaced with the range
>   * insni[off, off + cnt).  Adjust corresponding insn_aux_data by copying
>   * [0, off) and [off, end) to new locations, so the patched range stays zero
>   */
>  static void adjust_insn_aux_data(struct bpf_verifier_env *env,
> -				 struct bpf_prog *new_prog, u32 off, u32 cnt)
> +				 struct bpf_prog *new_prog, u32 off, u32 cnt,
> +				 enum bpf_patch_mode mode)
>  {

[ ... ]

> @@ -755,7 +769,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
>  			insn_buf[cnt++] = BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1,
>  						      -subprogs[0].stack_depth);
>  			insn_buf[cnt++] = env->prog->insnsi[0];
> -			new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
> +			new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
> +							 BPF_PREPEND);
>  			if (!new_prog)
>  				return -ENOMEM;
>  			env->prog = new_prog;
> @@ -778,7 +793,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
>  			verifier_bug(env, "prologue is too long");
>  			return -EFAULT;
>  		} else if (cnt) {
> -			new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
> +			new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
> +							 BPF_PREPEND);
>  			if (!new_prog)
>  				return -ENOMEM;

[ ... ]

>    bpf: Retarget indirect jump targets across prologue prepends
>
>    A few patches do not expand in place, they prepend: gen_epilogue and
>    gen_prologue splice their instructions in front of insn 0 in
>    bpf_convert_ctx_accesses(), and bpf_do_misc_fixups() prepends the may_goto
>    counter init at every subprog start.

This isn't a bug, but would the changelog read more precisely if it said
that for the gen_epilogue case, the ctx-save insn is prepended at insn 0,
and that gen_prologue is the one that splices its own instructions there?

In bpf_convert_ctx_accesses() above, when ops->gen_epilogue is present,
the epilogue body is patched in at the BPF_EXIT insn (still BPF_APPEND).
What is actually prepended at insn 0 is the single ctx-saving
'BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1, ...)' plus the copied
insnsi[0]. A reader trying to match the three converted call sites to
the sentence "gen_epilogue and gen_prologue splice their instructions in
front of insn 0" has to re-derive that distinction.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

Re: [PATCH bpf v2 07/11] bpf: Retarget indirect jump targets across prologue prepends

[Eduard Zingerman]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:
> A few patches do not expand in place, they prepend: gen_epilogue and
> gen_prologue splice their instructions in front of insn 0 in
> bpf_convert_ctx_accesses(), and bpf_do_misc_fixups() prepends the may_goto
> counter init at every subprog start.
> 
> They copy the original instruction of 'tgt_idx' into the last slot of the
> patch buffer, so it now lives at tgt_idx + delta, and call adjust_jmp_off()
> to move direct branches from tgt_idx to tgt_idx + delta.
> 
> Nothing does the same for indirect branches, so a BPF_MAP_TYPE_INSN_ARRAY
> slot that named tgt_idx keeps naming tgt_idx, which is now the first prepended
> instruction, and insn_aux_data[tgt_idx].indirect_target makes the JIT emit
> the landing pad there. Add a __bpf_patch_insn_data() variant and pass the
> BPF_PREPEND at the affected call-sites to fix up the delta.
> 
> Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
> Fixes: 07ae6c130b46 ("bpf: Add helper to detect indirect jump targets")
> Reported-by: Nicholas Carlini <npc@anthropic.com>
> Suggested-by: Nicholas Carlini <npc@anthropic.com>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

[PATCH bpf v2 08/11] selftests/bpf: Add tests for the indirect jump edge limit

[Daniel Borkmann]

Add corner case tests for programs whose gotox instructions are their own
jump table targets.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array
  [...]
  #24/10   bpf_insn_array/too-many-gotox-edges:OK
  #24/11   bpf_insn_array/gotox-edges-at-limit:OK
  #24/12   bpf_insn_array/gotox-edges-across-subprogs:OK
  #24      bpf_insn_array:OK
  Summary: 1/12 PASSED, 0 SKIPPED, 0/0 FAILED

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 266 ++++++++++++++++++
 1 file changed, 266 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 0222a9a5d076..08f5d4f3d64f 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -453,6 +453,263 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
+#define GOTOX_CNT_AT_LIMIT	1000
+#define GOTOX_LOG_SZ		(256 * 1024)
+
+static const char gotox_limit_msg[] =
+	"number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+	/* the run of gotox itself, plus the insn right after it */
+	const __u32 jt_cnt = gotox_cnt + 1;
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < jt_cnt; i++) {
+		val.orig_off = first_gotox + i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int prog_fd;
+
+	log[0] = 0;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = fd_array_cnt;
+	opts.log_buf = log;
+	opts.log_size = GOTOX_LOG_SZ;
+	opts.log_level = 1;
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+	if (prog_fd >= 0) {
+		close(prog_fd);
+		return 0;
+	}
+	return prog_fd;
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	__u32 i;
+
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+	for (i = 1; i <= gotox_cnt; i++)
+		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+	const __u32 insn_cnt = gotox_cnt + 3;
+	struct bpf_insn *insns;
+	bool attempted = false;
+	int map_fd;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		return false;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd = gotox_jt_create(1, gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+	attempted = true;
+free_insns:
+	free(insns);
+	return attempted;
+}
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	if (try_load_gotox_prog(gotox_cnt, log, &err))
+		check_gotox_limit_hit(log, err);
+
+	free(log);
+}
+
+/*
+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
+ * table holds the starts of the blocks that follow plus the exit block,
+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
+ * exactly at the limit and must still load.
+ */
+#define GOTOX_BLOCK_SZ		4
+
+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	struct bpf_insn *at;
+	__u32 k;
+
+	for (k = 0; k < gotox_cnt; k++) {
+		at = insns + k * GOTOX_BLOCK_SZ;
+
+		/* r1 = &jt[0], by index 0 into fd_array */
+		at[0] = (struct bpf_insn) {
+			.code = BPF_LD | BPF_DW | BPF_IMM,
+			.dst_reg = BPF_REG_1,
+			.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+			.imm = 0,
+		};
+		at[1] = (struct bpf_insn) { .imm = 0 };
+		at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
+		at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	}
+
+	insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
+}
+
+static int gotox_chain_jt_create(__u32 gotox_cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < gotox_cnt; i++) {
+		val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static void check_gotox_edges_at_limit(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
+	struct bpf_insn *insns;
+	char *log;
+	int map_fd, err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_chain_fill(insns, gotox_cnt);
+
+	map_fd = gotox_chain_jt_create(gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+
+	if (!ASSERT_OK(err, "program at the edge limit should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
+static void check_gotox_edges_across_subprogs(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
+	const __u32 sub_start = gotox_cnt + 3;
+	const __u32 insn_cnt = 2 * (gotox_cnt + 3);
+	int map_fd[2] = { -1, -1 };
+	struct bpf_insn *insns;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
+					    BPF_PSEUDO_CALL, 0,
+					    sub_start - (gotox_cnt + 1) - 1);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	gotox_run_fill(insns + sub_start, gotox_cnt);
+	insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd[0] = gotox_jt_create(1, gotox_cnt);
+	if (map_fd[0] < 0)
+		goto free_insns;
+	map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
+	check_gotox_limit_hit(log, err);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
 static void check_bpf_side(void)
 {
 	check_bpf_no_lookup();
@@ -490,6 +747,15 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
+
+	if (test__start_subtest("too-many-gotox-edges"))
+		check_too_many_gotox_edges();
+
+	if (test__start_subtest("gotox-edges-at-limit"))
+		check_gotox_edges_at_limit();
+
+	if (test__start_subtest("gotox-edges-across-subprogs"))
+		check_gotox_edges_across_subprogs();
 }
 #else
 static void __test_bpf_insn_array(void)
-- 
2.43.0

Re: [PATCH bpf v2 08/11] selftests/bpf: Add tests for the indirect jump edge limit

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4255 bytes --]

> diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> index 0222a9a5d0761..08f5d4f3d64fb 100644
> --- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> +++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> @@ -453,6 +453,263 @@ static void check_bpf_no_lookup(void)
>  	close(map_fd);
>  }
>
> +#define GOTOX_CNT_AT_LIMIT	1000
> +#define GOTOX_LOG_SZ		(256 * 1024)
> +
> +static const char gotox_limit_msg[] =
> +	"number of indirect jump edges in the program exceeds";
> +
> +static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
> +{
> +	/* the run of gotox itself, plus the insn right after it */
> +	const __u32 jt_cnt = gotox_cnt + 1;
> +	struct bpf_insn_array_value val = {};
> +	int map_fd;
> +	__u32 i;
> +
> +	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
> +	if (!ASSERT_GE(map_fd, 0, "map_create"))
> +		return map_fd;
> +
> +	for (i = 0; i < jt_cnt; i++) {
> +		val.orig_off = first_gotox + i;
> +		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
> +			       "bpf_map_update_elem"))
> +			goto err;
> +	}
> +
> +	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
> +		goto err;
> +
> +	return map_fd;
> +err:
> +	close(map_fd);
> +	return -1;
> +}
> +
> +static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
> +			   int *fd_array, __u32 fd_array_cnt, char *log)
> +{
> +	LIBBPF_OPTS(bpf_prog_load_opts, opts);
> +	int prog_fd;
> +
> +	log[0] = 0;
> +	opts.fd_array = fd_array;
> +	opts.fd_array_cnt = fd_array_cnt;
> +	opts.log_buf = log;
> +	opts.log_size = GOTOX_LOG_SZ;
> +	opts.log_level = 1;
> +
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
> +	if (prog_fd >= 0) {
> +		close(prog_fd);
> +		return 0;
> +	}
> +	return prog_fd;
> +}
> +
> +/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
> +static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
> +{
> +	__u32 i;
> +
> +	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
> +	for (i = 1; i <= gotox_cnt; i++)
> +		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
> +}
> +
> +static void check_gotox_limit_hit(const char *log, int err)
> +{
> +	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
> +	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
> +}
> +
> +static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
> +{
> +	const __u32 insn_cnt = gotox_cnt + 3;
> +	struct bpf_insn *insns;
> +	bool attempted = false;
> +	int map_fd;
> +
> +	insns = calloc(insn_cnt, sizeof(*insns));
> +	if (!ASSERT_OK_PTR(insns, "calloc insns"))
> +		return false;
> +
> +	gotox_run_fill(insns, gotox_cnt);
> +	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
> +	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
> +
> +	map_fd = gotox_jt_create(1, gotox_cnt);
> +	if (map_fd < 0)
> +		goto free_insns;
> +
> +	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
> +	close(map_fd);
> +	attempted = true;
> +free_insns:
> +	free(insns);
> +	return attempted;
> +}

This isn't a bug, but could try_load_gotox_prog() be inlined or simplified?
It has a single caller in check_too_many_gotox_edges(), and the bool
return plus int *err out-parameter split makes that test read differently
from check_gotox_edges_at_limit() and check_gotox_edges_across_subprogs(),
which open-code the identical scaffolding (calloc log, calloc insns, fill
insns, create map, load, close map, free) inline.

> +
> +/*
> + * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
> + * + 1) edges, hence the program is over the limit by gotox_cnt edges.
> + */
> +static void check_too_many_gotox_edges(void)
> +{
> +	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
> +	char *log;
> +	int err;
> +
> +	log = calloc(1, GOTOX_LOG_SZ);
> +	if (!ASSERT_OK_PTR(log, "calloc log"))
> +		return;
> +
> +	if (try_load_gotox_prog(gotox_cnt, log, &err))
> +		check_gotox_limit_hit(log, err);
> +
> +	free(log);
> +}

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

Re: [PATCH bpf v2 08/11] selftests/bpf: Add tests for the indirect jump edge limit

[Eduard Zingerman]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:
> Add corner case tests for programs whose gotox instructions are their own
> jump table targets.
> 
>   # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array
>   [...]
>   #24/10   bpf_insn_array/too-many-gotox-edges:OK
>   #24/11   bpf_insn_array/gotox-edges-at-limit:OK
>   #24/12   bpf_insn_array/gotox-edges-across-subprogs:OK
>   #24      bpf_insn_array:OK
>   Summary: 1/12 PASSED, 0 SKIPPED, 0/0 FAILED
> 
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
> ---

I think it's a bit of an overkill of a test.
Codex generated transformation to inline assembly as attached.
Object file size is 100k, but I think it is significantly easier
to understand in this form. Wdyt?

[-- Attachment #2: inline-asm-gotox.diff --]
[-- Type: text/x-patch, Size: 10629 bytes --]

diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index f102435675c2..70150b54bdaf 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -454,40 +454,8 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
-#define GOTOX_CNT_AT_LIMIT	1000
 #define GOTOX_LOG_SZ		(256 * 1024)
 
-static const char gotox_limit_msg[] =
-	"number of indirect jump edges in the program exceeds";
-
-static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
-{
-	/* the run of gotox itself, plus the insn right after it */
-	const __u32 jt_cnt = gotox_cnt + 1;
-	struct bpf_insn_array_value val = {};
-	int map_fd;
-	__u32 i;
-
-	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
-	if (!ASSERT_GE(map_fd, 0, "map_create"))
-		return map_fd;
-
-	for (i = 0; i < jt_cnt; i++) {
-		val.orig_off = first_gotox + i;
-		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
-			       "bpf_map_update_elem"))
-			goto err;
-	}
-
-	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
-		goto err;
-
-	return map_fd;
-err:
-	close(map_fd);
-	return -1;
-}
-
 static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
 				 int *fd_array, __u32 fd_array_cnt, char *log,
 				 int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
@@ -523,208 +491,6 @@ static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
 				     -1, NULL, 0);
 }
 
-/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
-static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
-{
-	__u32 i;
-
-	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
-	for (i = 1; i <= gotox_cnt; i++)
-		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
-}
-
-static void check_gotox_limit_hit(const char *log, int err)
-{
-	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
-	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
-}
-
-static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
-{
-	const __u32 insn_cnt = gotox_cnt + 3;
-	struct bpf_insn *insns;
-	bool attempted = false;
-	int map_fd;
-
-	insns = calloc(insn_cnt, sizeof(*insns));
-	if (!ASSERT_OK_PTR(insns, "calloc insns"))
-		return false;
-
-	gotox_run_fill(insns, gotox_cnt);
-	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
-	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
-
-	map_fd = gotox_jt_create(1, gotox_cnt);
-	if (map_fd < 0)
-		goto free_insns;
-
-	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
-	close(map_fd);
-	attempted = true;
-free_insns:
-	free(insns);
-	return attempted;
-}
-
-/*
- * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
- * + 1) edges, hence the program is over the limit by gotox_cnt edges.
- */
-static void check_too_many_gotox_edges(void)
-{
-	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
-	char *log;
-	int err;
-
-	log = calloc(1, GOTOX_LOG_SZ);
-	if (!ASSERT_OK_PTR(log, "calloc log"))
-		return;
-
-	if (try_load_gotox_prog(gotox_cnt, log, &err))
-		check_gotox_limit_hit(log, err);
-
-	free(log);
-}
-
-/*
- * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
- * table holds the starts of the blocks that follow plus the exit block,
- * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
- * exactly at the limit and must still load.
- */
-#define GOTOX_BLOCK_SZ		4
-
-static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
-{
-	struct bpf_insn *at;
-	__u32 k;
-
-	for (k = 0; k < gotox_cnt; k++) {
-		at = insns + k * GOTOX_BLOCK_SZ;
-
-		/* r1 = &jt[0], by index 0 into fd_array */
-		at[0] = (struct bpf_insn) {
-			.code = BPF_LD | BPF_DW | BPF_IMM,
-			.dst_reg = BPF_REG_1,
-			.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
-			.imm = 0,
-		};
-		at[1] = (struct bpf_insn) { .imm = 0 };
-		at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
-		at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
-	}
-
-	insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
-	insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
-}
-
-static int gotox_chain_jt_create(__u32 gotox_cnt)
-{
-	struct bpf_insn_array_value val = {};
-	int map_fd;
-	__u32 i;
-
-	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
-	if (!ASSERT_GE(map_fd, 0, "map_create"))
-		return map_fd;
-
-	for (i = 0; i < gotox_cnt; i++) {
-		val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
-		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
-			       "bpf_map_update_elem"))
-			goto err;
-	}
-
-	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
-		goto err;
-
-	return map_fd;
-err:
-	close(map_fd);
-	return -1;
-}
-
-static void check_gotox_edges_at_limit(void)
-{
-	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
-	const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
-	struct bpf_insn *insns;
-	char *log;
-	int map_fd, err;
-
-	log = calloc(1, GOTOX_LOG_SZ);
-	if (!ASSERT_OK_PTR(log, "calloc log"))
-		return;
-
-	insns = calloc(insn_cnt, sizeof(*insns));
-	if (!ASSERT_OK_PTR(insns, "calloc insns"))
-		goto free_log;
-
-	gotox_chain_fill(insns, gotox_cnt);
-
-	map_fd = gotox_chain_jt_create(gotox_cnt);
-	if (map_fd < 0)
-		goto free_insns;
-
-	err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
-	close(map_fd);
-
-	if (!ASSERT_OK(err, "program at the edge limit should load"))
-		fprintf(stderr, "verifier log: %s\n", log);
-
-free_insns:
-	free(insns);
-free_log:
-	free(log);
-}
-
-static void check_gotox_edges_across_subprogs(void)
-{
-	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
-	const __u32 sub_start = gotox_cnt + 3;
-	const __u32 insn_cnt = 2 * (gotox_cnt + 3);
-	int map_fd[2] = { -1, -1 };
-	struct bpf_insn *insns;
-	char *log;
-	int err;
-
-	log = calloc(1, GOTOX_LOG_SZ);
-	if (!ASSERT_OK_PTR(log, "calloc log"))
-		return;
-
-	insns = calloc(insn_cnt, sizeof(*insns));
-	if (!ASSERT_OK_PTR(insns, "calloc insns"))
-		goto free_log;
-
-	gotox_run_fill(insns, gotox_cnt);
-	insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
-					    BPF_PSEUDO_CALL, 0,
-					    sub_start - (gotox_cnt + 1) - 1);
-	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
-
-	gotox_run_fill(insns + sub_start, gotox_cnt);
-	insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
-	insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
-
-	map_fd[0] = gotox_jt_create(1, gotox_cnt);
-	if (map_fd[0] < 0)
-		goto free_insns;
-	map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
-	if (map_fd[1] < 0)
-		goto close_maps;
-
-	err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
-	check_gotox_limit_hit(log, err);
-
-close_maps:
-	close(map_fd[0]);
-	close(map_fd[1]);
-free_insns:
-	free(insns);
-free_log:
-	free(log);
-}
-
 static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
 {
 	struct bpf_insn_array_value val = {};
@@ -1501,12 +1267,6 @@ static void __test_bpf_insn_array(void)
 		check_no_map_reuse();
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
-	if (test__start_subtest("too-many-gotox-edges"))
-		check_too_many_gotox_edges();
-	if (test__start_subtest("gotox-edges-at-limit"))
-		check_gotox_edges_at_limit();
-	if (test__start_subtest("gotox-edges-across-subprogs"))
-		check_gotox_edges_across_subprogs();
 	if (test__start_subtest("gotox-jt-spans-subprogs"))
 		check_gotox_jt_spans_subprogs();
 	if (test__start_subtest("gotox-jt-spans-with-own-table"))
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index 4931a77ba0b2..b917c2e14283 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -457,6 +457,100 @@ __naked void spill_fill_ptr_to_insn(void)
 	: __clobber_all);
 }
 
+/*
+ * Emit each table entry next to the instruction it targets. Numeric labels
+ * can be reused by .rept, while %= keeps table symbols unique per asm block.
+ * BASE is the code section: libbpf relocates section-relative byte offsets
+ * to instruction offsets in the loaded program.
+ */
+#define GOTOX_TABLE_BEGIN(COUNT) \
+	".pushsection .jumptables,\"\",@progbits;" \
+	"jt_%=:;" \
+	".global jt_%=;" \
+	".size jt_%=, 8 * (" #COUNT ");" \
+	".popsection;" \
+	"r1 = jt_%= ll;"
+
+#define GOTOX_TABLE_ENTRY(LABEL, BASE) \
+	".pushsection .jumptables,\"\",@progbits;" \
+	".quad " LABEL " - " BASE ";" \
+	".popsection;"
+
+/* N gotox instructions, each targeting all N gotox plus the final block. */
+#define GOTOX_SELF_TARGETS(N, BASE) \
+	GOTOX_TABLE_BEGIN(N + 1) \
+	"r0 = 0;" \
+	".rept " #N ";" \
+		GOTOX_TABLE_ENTRY("1f", BASE) \
+		"1:; .8byte %[gotox];" \
+	".endr;" \
+	GOTOX_TABLE_ENTRY("2f", BASE) \
+	"2:;"
+
+#define GOTOX_OPERAND \
+	__imm_insn(gotox, BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_0, 0, 0, 0))
+
+/*
+ * Count edges even when their targets are already discovered. These
+ * programs deliberately use a scalar: the CFG edge limit must reject them
+ * before symbolic verification reaches the invalid gotox operand.
+ */
+SEC("socket")
+__description("too-many-gotox-edges")
+__failure __msg("number of indirect jump edges in the program exceeds")
+__naked void too_many_gotox_edges(void)
+{
+	asm volatile (
+		/* 1000 * 1001 = 1,001,000 edges. */
+		GOTOX_SELF_TARGETS(1000, "socket")
+		"r0 = 0; exit;"
+		: : GOTOX_OPERAND : __clobber_all);
+}
+
+SEC("socket")
+__description("gotox-edges-at-limit")
+__success __retval(0)
+__naked void gotox_edges_at_limit(void)
+{
+	asm volatile (
+		/*
+		 * 1000 gotox * 1000 targets = 1,000,000 CFG edges. At run
+		 * time, each block loads the next table entry and jumps to
+		 * the following block, so the program terminates.
+		 */
+		GOTOX_TABLE_BEGIN(1000)
+		".rept 1000;"
+			GOTOX_TABLE_ENTRY("1f", "socket")
+			"r0 = *(u64 *)(r1 + 0);"
+			"r1 += 8;"
+			".8byte %[gotox];"
+			"1:;"
+		".endr;"
+		"r0 = 0; exit;"
+		: : GOTOX_OPERAND : __clobber_all);
+}
+
+static __noinline __used __naked void gotox_edges_subprog(void)
+{
+	asm volatile (
+		GOTOX_SELF_TARGETS(750, ".text")
+		"r0 = 0; exit;"
+		: : GOTOX_OPERAND : __clobber_all);
+}
+
+SEC("socket")
+__description("gotox-edges-across-subprogs")
+__failure __msg("number of indirect jump edges in the program exceeds")
+__naked void gotox_edges_across_subprogs(void)
+{
+	asm volatile (
+		/* Each subprog has 563,250 edges; together, 1,126,500. */
+		GOTOX_SELF_TARGETS(750, "socket")
+		"call gotox_edges_subprog;"
+		"exit;"
+		: : GOTOX_OPERAND : __clobber_all);
+}
+
 #endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 || __TARGET_ARCH_powerpc*/
 
 char _license[] SEC("license") = "GPL";

[PATCH bpf v2 09/11] selftests/bpf: Add tests for indirect jumps across subprograms

[Daniel Borkmann]

Add various gotox corner case tests for indirect jumps across subprograms.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array,verifier_gotox
  [...]
  #24/13   bpf_insn_array/gotox-jt-spans-subprogs:OK
  #24/14   bpf_insn_array/gotox-jt-spans-with-own-table:OK
  #24/15   bpf_insn_array/gotox-target-other-subprog:OK
  #24/16   bpf_insn_array/gotox-jt-per-subprog:OK
  #24/17   bpf_insn_array/gotox-target-subprog-from-main:OK
  #24/18   bpf_insn_array/gotox-index-slice-other-subprog:OK
  #24/19   bpf_insn_array/gotox-target-other-global-subprog:OK
  #24/20   bpf_insn_array/gotox-callback-leaves-subprog:OK
  #24      bpf_insn_array:OK
  [...]
  #618     verifier_gotox:OK
  Summary: 2/49 PASSED, 0 SKIPPED, 0/0 FAILED

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 555 +++++++++++++++++-
 1 file changed, 553 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 08f5d4f3d64f..d6ebb4c37aea 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 
 #include <bpf/bpf.h>
+#include <bpf/btf.h>
 #include <test_progs.h>
 
 #if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)
@@ -487,8 +488,9 @@ static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
 	return -1;
 }
 
-static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
-			   int *fd_array, __u32 fd_array_cnt, char *log)
+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
+				 int *fd_array, __u32 fd_array_cnt, char *log,
+				 int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
 {
 	LIBBPF_OPTS(bpf_prog_load_opts, opts);
 	int prog_fd;
@@ -499,6 +501,12 @@ static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
 	opts.log_buf = log;
 	opts.log_size = GOTOX_LOG_SZ;
 	opts.log_level = 1;
+	if (fi_cnt) {
+		opts.prog_btf_fd = btf_fd;
+		opts.func_info = fi;
+		opts.func_info_cnt = fi_cnt;
+		opts.func_info_rec_size = sizeof(*fi);
+	}
 
 	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
 	if (prog_fd >= 0) {
@@ -508,6 +516,13 @@ static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
 	return prog_fd;
 }
 
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,
+				     -1, NULL, 0);
+}
+
 /* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
 static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
 {
@@ -710,6 +725,518 @@ static void check_gotox_edges_across_subprogs(void)
 	free(log);
 }
 
+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < cnt; i++) {
+		val.orig_off = offs[i];
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+#define GOTOX_SUB_START		4
+#define GOTOX_MAIN_TGT		2
+#define GOTOX_SUB_TGT		8
+#define GOTOX_TWO_INSN_CNT	10
+
+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SUB_START - 1 - 1);
+	insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_EXIT_INSN();
+
+	/* r1 = &jt[0], by index 'jt_idx' into fd_array */
+	insns[GOTOX_SUB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = jt_idx,
+	};
+	insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);
+	insns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[9] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_other_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static void check_gotox_jt_per_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 1, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_FWD_GOTOX		11
+#define GOTOX_FWD_OWN_TGT	12
+#define GOTOX_FWD_SUB_START	14
+#define GOTOX_FWD_INSN_CNT	16
+
+static void gotox_from_main_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_FWD_SUB_START - 1 - 1);
+	insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,
+			       offsetof(struct xdp_md, ingress_ifindex));
+	insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);
+
+	/* r1 = &jt_leaves[0], by index 1 into fd_array */
+	insns[4] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 1,
+	};
+	insns[5] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[7] = BPF_JMP_A(3);
+
+	/* r1 = &jt_own[0], by index 0 into fd_array */
+	insns[8] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+	insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[15] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_subprog_from_main(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The only map of the subprog holding the gotox reaches past that subprog, so
+ * the subprog is left without a jump table at all.
+ */
+static void check_gotox_jt_spans_subprogs(void)
+{
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The subprog holding the gotox has a well formed jump table of its own and
+ * also collects a map that reaches past its end. The spanning map is still
+ * rejected, even though the subprog is not left without a table.
+ */
+static void check_gotox_jt_spans_with_own_table(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_SLICE_SUB_START	6
+#define GOTOX_SLICE_GOTOX	14
+#define GOTOX_SLICE_SUB_TGT	15
+#define GOTOX_SLICE_INSN_CNT	17
+
+static void gotox_slice_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SLICE_SUB_START - 1 - 1);
+	insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[5] = BPF_EXIT_INSN();
+
+	insns[GOTOX_SLICE_SUB_START] =
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+			    offsetof(struct xdp_md, ingress_ifindex));
+	insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);
+	insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);
+	insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[10] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[11] = (struct bpf_insn) { .imm = 0 };
+	insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);
+	insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[16] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_index_slice_other_subprog(void)
+{
+	const __u32 jt_main[] = { 2, 3, 4 };
+	const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };
+	struct bpf_insn insns[GOTOX_SLICE_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_slice_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,
+			    struct bpf_func_info *fi, struct btf **pbtf)
+{
+	int int_id, proto_id, id;
+	struct btf *btf;
+	char name[24];
+	__u32 i;
+
+	btf = btf__new_empty();
+	if (!ASSERT_OK_PTR(btf, "btf__new_empty"))
+		return -1;
+
+	int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED);
+	if (!ASSERT_GT(int_id, 0, "btf__add_int"))
+		goto err;
+
+	proto_id = btf__add_func_proto(btf, int_id);
+	if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto"))
+		goto err;
+
+	for (i = 0; i < cnt; i++) {
+		snprintf(name, sizeof(name), "gotox_f%u", i);
+		id = btf__add_func(btf, name, linkage[i], proto_id);
+		if (!ASSERT_GT(id, 0, "btf__add_func"))
+			goto err;
+		fi[i].insn_off = starts[i];
+		fi[i].type_id = id;
+	}
+
+	if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel"))
+		goto err;
+
+	*pbtf = btf;
+	return btf__fd(btf);
+err:
+	btf__free(btf);
+	return -1;
+}
+
+static void check_gotox_target_other_global_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_SUB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
+#define GOTOX_CB_MAIN_TGT	6
+#define GOTOX_CB_START		8
+#define GOTOX_CB_GOTOX		11
+#define GOTOX_CB_TGT		12
+#define GOTOX_CB_INSN_CNT	14
+
+static void gotox_callback_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);
+	/* r2 = &callback */
+	insns[1] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_2,
+		.src_reg = BPF_PSEUDO_FUNC,
+		.imm = GOTOX_CB_START - 1 - 1,
+	};
+	insns[2] = (struct bpf_insn) { .imm = 0 };
+	insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);
+	insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);
+	insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[7] = BPF_EXIT_INSN();
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[GOTOX_CB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_callback_leaves_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_CB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };
+	const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };
+	const __u32 jt_cb[] = { GOTOX_CB_TGT };
+	struct bpf_insn insns[GOTOX_CB_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_callback_fill(insns);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
 static void check_bpf_side(void)
 {
 	check_bpf_no_lookup();
@@ -756,6 +1283,30 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("gotox-edges-across-subprogs"))
 		check_gotox_edges_across_subprogs();
+
+	if (test__start_subtest("gotox-jt-spans-subprogs"))
+		check_gotox_jt_spans_subprogs();
+
+	if (test__start_subtest("gotox-jt-spans-with-own-table"))
+		check_gotox_jt_spans_with_own_table();
+
+	if (test__start_subtest("gotox-target-other-subprog"))
+		check_gotox_target_other_subprog();
+
+	if (test__start_subtest("gotox-jt-per-subprog"))
+		check_gotox_jt_per_subprog();
+
+	if (test__start_subtest("gotox-target-subprog-from-main"))
+		check_gotox_target_subprog_from_main();
+
+	if (test__start_subtest("gotox-index-slice-other-subprog"))
+		check_gotox_index_slice_other_subprog();
+
+	if (test__start_subtest("gotox-target-other-global-subprog"))
+		check_gotox_target_other_global_subprog();
+
+	if (test__start_subtest("gotox-callback-leaves-subprog"))
+		check_gotox_callback_leaves_subprog();
 }
 #else
 static void __test_bpf_insn_array(void)
-- 
2.43.0

[PATCH bpf v2 10/11] selftests/bpf: Add tests for gotox nop targets and insn array reuse

[Daniel Borkmann]

Add two tests to the bpf_insn_array suite which excercise gotox target
which is a ja +0, and the insn array reuse which initially fails the
post-JIT stage.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array
  [...]
  #24/21   bpf_insn_array/gotox-target-nop:OK
  #24/22   bpf_insn_array/insn-array-stale-reuse:OK
  #24      bpf_insn_array:OK
  Summary: 1/22 PASSED, 0 SKIPPED, 0/0 FAILED

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 140 ++++++++++++++++++
 1 file changed, 140 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index d6ebb4c37aea..9908e538c022 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1242,6 +1242,140 @@ static void check_bpf_side(void)
 	check_bpf_no_lookup();
 }
 
+static void check_gotox_target_nop(void)
+{
+	struct bpf_insn insns[] = {
+		/* r1 = &jt[0] */
+		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: the nop target */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 4;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target must not be INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target must have a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
+static void check_insn_array_stale_reuse(void)
+{
+	/* helper of a different prog type, to set the prog array's owner */
+	struct bpf_insn sf[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	/*
+	 * prog1 binds the insn array (entry points at a real insn, so the JIT
+	 * fills it in) and tail calls a prog array whose owner is now a
+	 * different type, which fails in the post-JIT compatibility check.
+	 */
+	struct bpf_insn prog1[] = {
+		BPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */
+		BPF_MOV64_IMM(BPF_REG_3, 0),			   /* r3 = index */
+		BPF_EMIT_CALL(BPF_FUNC_tail_call),
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),		   /* insn 4: real entry */
+		BPF_EXIT_INSN(),
+	};
+	/*
+	 * prog2 reuses the insn array as a tracker. Its entry points at a nop
+	 * that the nop pass removes, so the entry becomes INSN_DELETED and the
+	 * JIT skips it -- exactly the slot a stale pointer would linger in.
+	 */
+	struct bpf_insn prog2[] = {
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: tracked nop */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;
+	struct bpf_insn_array_value val = {};
+	int fd_array[2];
+	__u32 key = 0;
+
+	jt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(jt_fd, 0, "insn_array create"))
+		return;
+	val.orig_off = 4;
+	if (!ASSERT_EQ(bpf_map_update_elem(jt_fd, &key, &val, 0), 0, "insn_array update"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, "insn_array freeze"))
+		goto cleanup;
+
+	pa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, "pa", 4, 4, 1, NULL);
+	if (!ASSERT_GE(pa_fd, 0, "prog_array create"))
+		goto cleanup;
+	sf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", sf, ARRAY_SIZE(sf), NULL);
+	if (!ASSERT_GE(sf_fd, 0, "owner prog load"))
+		goto cleanup;
+	/* insert a non-XDP prog to pin the prog array owner to another type */
+	if (!ASSERT_EQ(bpf_map_update_elem(pa_fd, &key, &sf_fd, 0), 0, "prog_array update"))
+		goto cleanup;
+
+	fd_array[0] = jt_fd;
+	fd_array[1] = pa_fd;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = 2;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog1, ARRAY_SIZE(prog1), &opts);
+	if (!ASSERT_LT(prog_fd, 0, "prog1 must fail the post-JIT tail call check"))
+		goto cleanup;
+
+	/* prog1 reached the JIT before failing, so the entry is now populated */
+	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog1"))
+		goto cleanup;
+	if (!ASSERT_NEQ(val.jitted_off, 0, "prog1 should have filled the jitted address"))
+		goto cleanup;
+
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog2, ARRAY_SIZE(prog2), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "prog2 reuse load"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog2"))
+		goto cleanup;
+	ASSERT_EQ(val.xlated_off, (__u32)-1, "reused entry should be INSN_DELETED");
+	ASSERT_EQ(val.jitted_off, 0, "stale jitted address must be cleared on reuse");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	if (sf_fd >= 0)
+		close(sf_fd);
+	if (pa_fd >= 0)
+		close(pa_fd);
+	if (jt_fd >= 0)
+		close(jt_fd);
+}
+
 static void __test_bpf_insn_array(void)
 {
 	/* Test if offsets are adjusted properly */
@@ -1307,6 +1441,12 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("gotox-callback-leaves-subprog"))
 		check_gotox_callback_leaves_subprog();
+
+	if (test__start_subtest("gotox-target-nop"))
+		check_gotox_target_nop();
+
+	if (test__start_subtest("insn-array-stale-reuse"))
+		check_insn_array_stale_reuse();
 }
 #else
 static void __test_bpf_insn_array(void)
-- 
2.43.0

Re: [PATCH bpf v2 10/11] selftests/bpf: Add tests for gotox nop targets and insn array reuse

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 7384 bytes --]

> diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> index d6ebb4c37aea7..9908e538c022b 100644
> --- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> +++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c

[ ... ]

> +static void check_gotox_target_nop(void)
> +{
> +	struct bpf_insn insns[] = {
> +		/* r1 = &jt[0] */
> +		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
> +		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),
> +		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */
> +		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: the nop target */
> +		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
> +		BPF_EXIT_INSN(),
> +	};
> +	LIBBPF_OPTS(bpf_prog_load_opts, opts);
> +	struct bpf_insn_array_value val = {};
> +	int prog_fd = -1, map_fd;
> +	__u32 key = 0;
> +
> +	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
> +	if (!ASSERT_GE(map_fd, 0, "map_create"))
> +		return;
> +
> +	val.orig_off = 4;
> +	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
> +		goto cleanup;
> +	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
> +		goto cleanup;

This isn't a bug, but could this setup reuse gotox_jt_create_offs()
from earlier in the file?  The create/update/freeze sequence here is
what gotox_jt_create_offs() already does, so map_fd =
gotox_jt_create_offs((const __u32[]){ 4 }, 1); would express it in
one line.

> +
> +	opts.fd_array = &map_fd;
> +	opts.fd_array_cnt = 1;
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
> +	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
> +		goto cleanup;
> +
> +	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
> +		goto cleanup;
> +	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target must not be INSN_DELETED");
> +	ASSERT_NEQ(val.jitted_off, 0, "gotox target must have a jitted address");
> +cleanup:
> +	if (prog_fd >= 0)
> +		close(prog_fd);
> +	close(map_fd);
> +}

[ ... ]

> +static void check_insn_array_stale_reuse(void)
> +{
> +	/* helper of a different prog type, to set the prog array's owner */
> +	struct bpf_insn sf[] = {
> +		BPF_MOV64_IMM(BPF_REG_0, 0),
> +		BPF_EXIT_INSN(),
> +	};
> +	/*
> +	 * prog1 binds the insn array (entry points at a real insn, so the JIT
> +	 * fills it in) and tail calls a prog array whose owner is now a
> +	 * different type, which fails in the post-JIT compatibility check.
> +	 */
> +	struct bpf_insn prog1[] = {
> +		BPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */
> +		BPF_MOV64_IMM(BPF_REG_3, 0),			   /* r3 = index */
> +		BPF_EMIT_CALL(BPF_FUNC_tail_call),
> +		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),		   /* insn 4: real entry */
> +		BPF_EXIT_INSN(),
> +	};
> +	/*
> +	 * prog2 reuses the insn array as a tracker. Its entry points at a nop
> +	 * that the nop pass removes, so the entry becomes INSN_DELETED and the
> +	 * JIT skips it -- exactly the slot a stale pointer would linger in.
> +	 */
> +	struct bpf_insn prog2[] = {
> +		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
> +		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
> +		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
> +		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
> +		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: tracked nop */
> +		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
> +		BPF_EXIT_INSN(),
> +	};
> +	LIBBPF_OPTS(bpf_prog_load_opts, opts);
> +	int jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;
> +	struct bpf_insn_array_value val = {};
> +	int fd_array[2];
> +	__u32 key = 0;
> +
> +	jt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
> +	if (!ASSERT_GE(jt_fd, 0, "insn_array create"))
> +		return;
> +	val.orig_off = 4;
> +	if (!ASSERT_EQ(bpf_map_update_elem(jt_fd, &key, &val, 0), 0, "insn_array update"))
> +		goto cleanup;
> +	if (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, "insn_array freeze"))
> +		goto cleanup;

This isn't a bug, but could this setup also reuse gotox_jt_create_offs()?
The same create/update/freeze sequence appears here as well.

> +
> +	pa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, "pa", 4, 4, 1, NULL);
> +	if (!ASSERT_GE(pa_fd, 0, "prog_array create"))
> +		goto cleanup;
> +	sf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", sf, ARRAY_SIZE(sf), NULL);
> +	if (!ASSERT_GE(sf_fd, 0, "owner prog load"))
> +		goto cleanup;
> +	/* insert a non-XDP prog to pin the prog array owner to another type */
> +	if (!ASSERT_EQ(bpf_map_update_elem(pa_fd, &key, &sf_fd, 0), 0, "prog_array update"))
> +		goto cleanup;
> +
> +	fd_array[0] = jt_fd;
> +	fd_array[1] = pa_fd;
> +	opts.fd_array = fd_array;
> +	opts.fd_array_cnt = 2;
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog1, ARRAY_SIZE(prog1), &opts);
> +	if (!ASSERT_LT(prog_fd, 0, "prog1 must fail the post-JIT tail call check"))
> +		goto cleanup;
> +
> +	/* prog1 reached the JIT before failing, so the entry is now populated */
> +	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog1"))
> +		goto cleanup;
> +	if (!ASSERT_NEQ(val.jitted_off, 0, "prog1 should have filled the jitted address"))
> +		goto cleanup;

A subsystem pattern flags this as potentially concerning: the negative
load of prog1 is only checked with ASSERT_LT(prog_fd, 0), so the test
accepts any failure.

The assertion immediately after it (ASSERT_NEQ(val.jitted_off, 0)) is
only meaningful if prog1 failed specifically in the post-JIT
bpf_check_tail_call() stage in __bpf_prog_select_runtime(). Any earlier
failure also satisfies ASSERT_LT, and the test then reports "prog1
should have filled the jitted address" instead of the real cause.

For example, if the JIT is off, prog1 hits -ENOTSUPP from:

kernel/bpf/core.c:__bpf_prog_select_runtime() {
    ...
    if (!fp->jited && jit_needed)
        return -ENOTSUPP;
    ...
}

Or if a future change moves the prog_array owner-type check into
check_map_prog_compatibility(), the verifier would reject prog1 before
it reaches the JIT.

Every other negative case in this file pins the code instead. In
check_gotox_target_other_subprog():

ASSERT_EQ(err, -EINVAL, "program should have been rejected")

And in check_no_map_reuse():

ASSERT_EQ(prog_fd, -EINVAL, ...)

Would asserting prog_fd == -EINVAL here keep the test self-diagnosing?

On the tree as of this commit, prog1 does fail exactly at
bpf_check_tail_call(), so the test is correct today. The question is
whether it remains self-diagnosing if the kernel behavior changes.

> +
> +	opts.fd_array_cnt = 1;
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog2, ARRAY_SIZE(prog2), &opts);
> +	if (!ASSERT_GE(prog_fd, 0, "prog2 reuse load"))
> +		goto cleanup;
> +
> +	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog2"))
> +		goto cleanup;
> +	ASSERT_EQ(val.xlated_off, (__u32)-1, "reused entry should be INSN_DELETED");
> +	ASSERT_EQ(val.jitted_off, 0, "stale jitted address must be cleared on reuse");
> +cleanup:
> +	if (prog_fd >= 0)
> +		close(prog_fd);
> +	if (sf_fd >= 0)
> +		close(sf_fd);
> +	if (pa_fd >= 0)
> +		close(pa_fd);
> +	if (jt_fd >= 0)
> +		close(jt_fd);
> +}

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

Re: [PATCH bpf v2 10/11] selftests/bpf: Add tests for gotox nop targets and insn array reuse

[Eduard Zingerman]

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:
> Add two tests to the bpf_insn_array suite which excercise gotox target
> which is a ja +0, and the insn array reuse which initially fails the
> post-JIT stage.
> 
>   # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array
>   [...]
>   #24/21   bpf_insn_array/gotox-target-nop:OK
>   #24/22   bpf_insn_array/insn-array-stale-reuse:OK
>   #24      bpf_insn_array:OK
>   Summary: 1/22 PASSED, 0 SKIPPED, 0/0 FAILED
> 
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
> ---

Tbh, having a full stale ips issure repro seem to be an overkill.
I'd move the first test to inline assembly with __xlated or __jited
matcher and drop the insn-array-stale-reuse. ¯\_(ツ)_/¯

...

[PATCH bpf v2 11/11] selftests/bpf: Add tests for gotox targets across prologue prepends

[Daniel Borkmann]

Add tests to the bpf_insn_array suite for a gotox whose jump table points
at the first instruction of a subprogram.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array
  [...]
  #24/23   bpf_insn_array/gotox-target-prologue-shift:OK
  #24/24   bpf_insn_array/gotox-target-ctx-prologue-shift:OK
  #24      bpf_insn_array:OK
  Summary: 1/24 PASSED, 0 SKIPPED, 0/0 FAILED

As small nit, cleanup the extra whitespaces between subtests.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 130 ++++++++++++++----
 1 file changed, 107 insertions(+), 23 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 9908e538c022..f102435675c2 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1376,77 +1376,161 @@ static void check_insn_array_stale_reuse(void)
 		close(jt_fd);
 }
 
+static void check_gotox_target_prologue_shift(void)
+{
+	struct bpf_insn insns[] = {
+		/* insn 0: gotox target and subprog start */
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		/* may_goto +4 -> exit block, bounds the loop */
+		BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),
+		/* r1 = &jt[0] (insns 2 and 3) */
+		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */
+		/* insn 5: gotox r1 -> insn 0 */
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 0;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
+static void check_gotox_target_ctx_prologue_shift(void)
+{
+	struct bpf_insn insns[] = {
+		/* insn 0: gotox target and subprog start */
+		BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),
+		BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */
+		BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),
+		BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),
+		BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),
+		BPF_MOV64_REG(BPF_REG_4, BPF_REG_2),
+		BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),
+		BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),
+		/* insn 9: direct packet write -> tc unclone prologue prepended at insn 0 */
+		BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),
+		/* r5 = &jt[0] (insns 10 and 11) */
+		BPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */
+		/* insn 13: gotox r5 -> insn 0 */
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),
+		BPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 0;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns,
+				ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
 static void __test_bpf_insn_array(void)
 {
 	/* Test if offsets are adjusted properly */
-
 	if (test__start_subtest("one2one"))
 		check_one_to_one_mapping();
-
 	if (test__start_subtest("simple"))
 		check_simple();
-
 	if (test__start_subtest("deletions"))
 		check_deletions();
-
 	if (test__start_subtest("deletions-with-functions"))
 		check_deletions_with_functions();
-
 	if (test__start_subtest("blindness"))
 		check_blindness();
-
 	/* Check all kinds of operations and related restrictions */
-
 	if (test__start_subtest("incorrect-index"))
 		check_incorrect_index();
-
 	if (test__start_subtest("load-unfrozen-map"))
 		check_load_unfrozen_map();
-
 	if (test__start_subtest("no-map-reuse"))
 		check_no_map_reuse();
-
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
-
 	if (test__start_subtest("too-many-gotox-edges"))
 		check_too_many_gotox_edges();
-
 	if (test__start_subtest("gotox-edges-at-limit"))
 		check_gotox_edges_at_limit();
-
 	if (test__start_subtest("gotox-edges-across-subprogs"))
 		check_gotox_edges_across_subprogs();
-
 	if (test__start_subtest("gotox-jt-spans-subprogs"))
 		check_gotox_jt_spans_subprogs();
-
 	if (test__start_subtest("gotox-jt-spans-with-own-table"))
 		check_gotox_jt_spans_with_own_table();
-
 	if (test__start_subtest("gotox-target-other-subprog"))
 		check_gotox_target_other_subprog();
-
 	if (test__start_subtest("gotox-jt-per-subprog"))
 		check_gotox_jt_per_subprog();
-
 	if (test__start_subtest("gotox-target-subprog-from-main"))
 		check_gotox_target_subprog_from_main();
-
 	if (test__start_subtest("gotox-index-slice-other-subprog"))
 		check_gotox_index_slice_other_subprog();
-
 	if (test__start_subtest("gotox-target-other-global-subprog"))
 		check_gotox_target_other_global_subprog();
-
 	if (test__start_subtest("gotox-callback-leaves-subprog"))
 		check_gotox_callback_leaves_subprog();
-
 	if (test__start_subtest("gotox-target-nop"))
 		check_gotox_target_nop();
-
 	if (test__start_subtest("insn-array-stale-reuse"))
 		check_insn_array_stale_reuse();
+	if (test__start_subtest("gotox-target-prologue-shift"))
+		check_gotox_target_prologue_shift();
+	if (test__start_subtest("gotox-target-ctx-prologue-shift"))
+		check_gotox_target_ctx_prologue_shift();
 }
 #else
 static void __test_bpf_insn_array(void)
-- 
2.43.0

Re: [PATCH bpf v2 11/11] selftests/bpf: Add tests for gotox targets across prologue prepends

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 7306 bytes --]

> diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> index 9908e538c022b..f102435675c2b 100644
> --- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> +++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
> @@ -1376,77 +1376,161 @@ static void check_insn_array_stale_reuse(void)
>  		close(jt_fd);
> }
>
> +static void check_gotox_target_prologue_shift(void)
> +{
> +	struct bpf_insn insns[] = {
> +		/* insn 0: gotox target and subprog start */
> +		BPF_MOV64_IMM(BPF_REG_0, 0),
> +		/* may_goto +4 -> exit block, bounds the loop */
> +		BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),
> +		/* r1 = &jt[0] (insns 2 and 3) */
> +		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
> +		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */
> +		/* insn 5: gotox r1 -> insn 0 */
> +		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),
> +		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */
> +		BPF_EXIT_INSN(),
> +	};
> +	LIBBPF_OPTS(bpf_prog_load_opts, opts);
> +	struct bpf_insn_array_value val = {};
> +	int prog_fd = -1, map_fd;
> +	__u32 key = 0;
> +
> +	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
> +	if (!ASSERT_GE(map_fd, 0, "map_create"))
> +		return;
> +
> +	val.orig_off = 0;
> +	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
> +		goto cleanup;
> +	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
> +		goto cleanup;
> +
> +	opts.fd_array = &map_fd;
> +	opts.fd_array_cnt = 1;
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
> +	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
> +		goto cleanup;
> +
> +	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
> +		goto cleanup;
> +	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
> +	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
> +	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
> +cleanup:
> +	if (prog_fd >= 0)
> +		close(prog_fd);
> +	close(map_fd);
> +}
> +
> +static void check_gotox_target_ctx_prologue_shift(void)
> +{
> +	struct bpf_insn insns[] = {
> +		/* insn 0: gotox target and subprog start */
> +		BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),
> +		BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */
> +		BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),
> +		BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */
> +		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),
> +		BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),
> +		BPF_MOV64_REG(BPF_REG_4, BPF_REG_2),
> +		BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),
> +		BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),
> +		/* insn 9: direct packet write -> tc unclone prologue prepended at insn 0 */
> +		BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),
> +		/* r5 = &jt[0] (insns 10 and 11) */
> +		BPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),
> +		BPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */
> +		/* insn 13: gotox r5 -> insn 0 */
> +		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),
> +		BPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */
> +		BPF_EXIT_INSN(),
> +	};
> +	LIBBPF_OPTS(bpf_prog_load_opts, opts);
> +	struct bpf_insn_array_value val = {};
> +	int prog_fd = -1, map_fd;
> +	__u32 key = 0;
> +
> +	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
> +	if (!ASSERT_GE(map_fd, 0, "map_create"))
> +		return;
> +
> +	val.orig_off = 0;
> +	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
> +		goto cleanup;
> +	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
> +		goto cleanup;
> +
> +	opts.fd_array = &map_fd;
> +	opts.fd_array_cnt = 1;
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns,
> +				ARRAY_SIZE(insns), &opts);
> +	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
> +		goto cleanup;
> +
> +	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
> +		goto cleanup;
> +	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
> +	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
> +	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
> +cleanup:
> +	if (prog_fd >= 0)
> +		close(prog_fd);
> +	close(map_fd);
> +}

This isn't a bug, but the ~30-line body after the insns[] array in
check_gotox_target_ctx_prologue_shift() is a verbatim copy of the same
block in check_gotox_target_prologue_shift() above. The only differences
are BPF_PROG_TYPE_SCHED_CLS vs BPF_PROG_TYPE_XDP and the wrapping of the
bpf_prog_load() call.

The file's earlier check_gotox_target_nop() test (added in the same
series) is a third near-copy of the same
map-create/update/freeze/load/lookup/assert/cleanup sequence, differing
only in orig_off and the assertion labels. The file otherwise tends to
factor this kind of setup out with helpers like map_create(), prog_load(),
gotox_jt_create(), gotox_prog_load(), and __check_success().

Could these three tests share a small helper - something like
check_gotox_target(prog_type, insns, insn_cnt, orig_off) - instead of
repeating the map create/freeze/load/lookup/cleanup sequence a third time?

>
> static void __test_bpf_insn_array(void)
> {
>  	/* Test if offsets are adjusted properly */
> -
>  	if (test__start_subtest("one2one"))
>  		check_one_to_one_mapping();
> -
>  	if (test__start_subtest("simple"))
>  		check_simple();
> -
>  	if (test__start_subtest("deletions"))
>  		check_deletions();
> -
>  	if (test__start_subtest("deletions-with-functions"))
>  		check_deletions_with_functions();
> -
>  	if (test__start_subtest("blindness"))
>  		check_blindness();
> -
>  	/* Check all kinds of operations and related restrictions */
> -
>  	if (test__start_subtest("incorrect-index"))
>  		check_incorrect_index();

[ ... ]

> -
>  	if (test__start_subtest("insn-array-stale-reuse"))
>  		check_insn_array_stale_reuse();
> +	if (test__start_subtest("gotox-target-prologue-shift"))
> +		check_gotox_target_prologue_shift();
> +	if (test__start_subtest("gotox-target-ctx-prologue-shift"))
> +		check_gotox_target_ctx_prologue_shift();
> }

This isn't a bug, but besides adding the two new subtests, the patch drops
the blank line between every existing subtest in __test_bpf_insn_array(),
which accounts for 44 of the 130 changed lines. It is a pure reflow with
no bearing on the new tests, and it also removes the blank line that used
to set the two grouping comments apart, so '/* Test if offsets are
adjusted properly */' and '/* Check all kinds of operations and related
restrictions */' now abut the surrounding calls and no longer read as
section headers.

Would it be worth splitting the blank-line cleanup in
__test_bpf_insn_array() into its own patch, so the diff for the two new
subtests stays small - and keeping a blank line before the two grouping
comments so they still stand out?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886