[PATCH bpf-next 00/18] bpf: Support stack arguments for BPF functions and kfuncs

[PATCH bpf-next 00/18] bpf: Support stack arguments for BPF functions and kfuncs

[Yonghong Song]

Currently, bpf function calls and kfunc's are limited by 5 reg-level
parameters. For function calls with more than 5 parameters,
developers can use always inlining or pass a struct pointer
after packing more parameters in that struct although it may have
some inconvenience. But there is no workaround for kfunc if more
than 5 parameters is needed.

This patch set lifts the 5-argument limit by introducing stack-based
argument passing for BPF functions and kfunc's, coordinated with
compiler support in LLVM [1]. The compiler emits loads/stores through
a new bpf register r11 (BPF_REG_PARAMS), to pass arguments beyond
the 5th, keeping the stack arg area separate from the r10-based program
stack. The current maximum number of arguments is capped at
MAX_BPF_FUNC_ARGS (12), which is sufficient for the vast majority of
use cases.

All kfunc/bpf-function arguments are caller saved, including stack
arguments. For register arguments (r1-r5), the verifier already marks
them as clobbered after each call. For stack arguments, the verifier
invalidates all outgoing stack arg slots immediately after a call,
requiring the compiler to re-store them before any subsequent call.
This follows the native calling convention where all function
parameters are caller saved.

The x86_64 JIT translates r11-relative accesses to RBP-relative
native instructions. Each function's stack allocation is extended
by 'max_outgoing' bytes to hold the outgoing arg area below the
callee-saved registers. This makes implementation easier as the r10
can be reused for stack argument access. At both BPF-to-BPF and kfunc
calls, outgoing args are pushed onto the expected calling convention
locations directly. The incoming parameters can directly get the value
from caller.

To support kfunc stack arguments, before doing any stack arguments,
existing codes are refactored/modified to use bpf_reg_state as much
as possible instead of using regno, and to pass a non-negative argno,
encoded to support both registers and stack arguments, as a single
variable.

Global subprogs and freplace progs with >5 args are not yet supported.
Only x86_64 and arm64 are supported for now. Same selftests are tested
by both x86_64 and arm64. Please see each individual patch for details.

  [1] https://github.com/llvm/llvm-project/pull/189060

Note:
  - The patch set is on top of the following commit:
    fd30cf86e1194 Merge branch 'bpf-prepare-to-support-stack-arguments'
  - This patch set requires latest llvm23 compiler. It is possible that a build
    failure may appear:
      /home/yhs/work/bpf-next/scripts/mod/modpost.c:59:13: error: variable 'extra_warn' set but not used [-Werror,-Wunused-but-set-global]
             59 | static bool extra_warn;
                |             ^
          1 error generated.
    In this case, the following hack can workaround the build issue:
      --- a/Makefile
      +++ b/Makefile
      @@ -467,7 +467,7 @@ KERNELDOC       = $(srctree)/tools/docs/kernel-doc
       export KERNELDOC

       KBUILD_USERHOSTCFLAGS := -Wall -Wmissing-prototypes -Wstrict-prototypes \
      -                        -O2 -fomit-frame-pointer -std=gnu11
      +                        -O2 -fomit-frame-pointer -std=gnu11 -Wno-unused-but-set-global
       KBUILD_USERCFLAGS  := $(KBUILD_USERHOSTCFLAGS) $(USERCFLAGS)
       KBUILD_USERLDFLAGS := $(USERLDFLAGS)

Puranjay Mohan (3):
  bpf, arm64: Map BPF_REG_0 to x8 instead of x7
  bpf, arm64: Add JIT support for stack arguments
  selftests/bpf: Enable stack argument tests for arm64

Yonghong Song (15):
  bpf: Support stack arguments for bpf functions
  bpf: Add precision marking and backtracking for stack argument slots
  bpf: Refactor record_call_access() to extract per-arg logic
  bpf: Extend liveness analysis to track stack argument slots
  bpf: Reject stack arguments in non-JITed programs
  bpf: Prepare architecture JIT support for stack arguments
  bpf: Enable r11 based insns
  bpf: Support stack arguments for kfunc calls
  bpf: Reject stack arguments if tail call reachable
  bpf,x86: Implement JIT support for stack arguments
  selftests/bpf: Add tests for BPF function stack arguments
  selftests/bpf: Add tests for stack argument validation
  selftests/bpf: Add verifier tests for stack argument validation
  selftests/bpf: Add BTF fixup for __naked subprog parameter names
  selftests/bpf: Add precision backtracking test for stack arguments

 arch/arm64/net/bpf_jit_comp.c                 |  91 +++-
 arch/arm64/net/bpf_timed_may_goto.S           |   8 +-
 arch/x86/net/bpf_jit_comp.c                   | 154 +++++-
 include/linux/bpf.h                           |   2 +
 include/linux/bpf_verifier.h                  |  42 +-
 include/linux/filter.h                        |   1 +
 kernel/bpf/backtrack.c                        |  61 ++-
 kernel/bpf/btf.c                              |  20 +-
 kernel/bpf/const_fold.c                       |  13 +-
 kernel/bpf/core.c                             |   7 +-
 kernel/bpf/fixups.c                           |  28 +-
 kernel/bpf/liveness.c                         | 200 ++++++--
 kernel/bpf/states.c                           |  31 ++
 kernel/bpf/verifier.c                         | 347 +++++++++++--
 .../selftests/bpf/prog_tests/stack_arg.c      | 139 ++++++
 .../selftests/bpf/prog_tests/stack_arg_fail.c |  10 +
 .../bpf/prog_tests/stack_arg_precision.c      |  10 +
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 tools/testing/selftests/bpf/progs/bpf_misc.h  |   1 +
 .../bpf/progs/btf__stack_arg_precision.c      |  24 +
 tools/testing/selftests/bpf/progs/stack_arg.c | 253 ++++++++++
 .../selftests/bpf/progs/stack_arg_fail.c      | 114 +++++
 .../selftests/bpf/progs/stack_arg_kfunc.c     | 164 +++++++
 .../selftests/bpf/progs/stack_arg_precision.c | 122 +++++
 .../selftests/bpf/progs/verifier_jit_inline.c |   2 +-
 .../selftests/bpf/progs/verifier_ldsx.c       |   6 +-
 .../bpf/progs/verifier_private_stack.c        |  10 +-
 .../selftests/bpf/progs/verifier_stack_arg.c  | 457 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  72 +++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  26 +
 tools/testing/selftests/bpf/test_loader.c     | 136 +++++-
 31 files changed, 2448 insertions(+), 105 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/stack_arg.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/stack_arg_fail.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/stack_arg_precision.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg_fail.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg_precision.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_stack_arg.c

-- 
2.52.0

[PATCH bpf-next 01/18] bpf: Support stack arguments for bpf functions

[Yonghong Song]

Currently BPF functions (subprogs) are limited to 5 register arguments.
With [1], the compiler can emit code that passes additional arguments
via a dedicated stack area through bpf register BPF_REG_PARAMS (r11),
introduced in an earlier patch ([2]).

The compiler uses positive r11 offsets for incoming (callee-side) args
and negative r11 offsets for outgoing (caller-side) args, following the
x86_64/arm64 calling convention direction. There is an 8-byte gap at
offset 0 separating two regions:
  Incoming (callee reads):   r11+8 (arg6), r11+16 (arg7), ...
  Outgoing (caller writes):  r11-8 (arg6), r11-16 (arg7), ...

The following is an example to show how stack arguments are saved
and transferred between caller and callee:

  int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) {
    ...
    bar(a1, a2, a3, a4, a5, a6, a7, a8);
    ...
  }

  Caller (foo)                           Callee (bar)
  ============                           ============
  Incoming (positive offsets):           Incoming (positive offsets):

  r11+8:  [incoming arg 6]               r11+8:  [incoming arg 6] <-+
  r11+16: [incoming arg 7]               r11+16: [incoming arg 7] <-|+
                                         r11+24: [incoming arg 8] <-||+
  Outgoing (negative offsets):                                      |||
  r11-8:  [outgoing arg 6 to bar] -------->-------------------------+||
  r11-16: [outgoing arg 7 to bar] -------->--------------------------+|
  r11-24: [outgoing arg 8 to bar] -------->---------------------------+

If the bpf function has more than one call:

  int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) {
    ...
    bar1(a1, a2, a3, a4, a5, a6, a7, a8);
    ...
    bar2(a1, a2, a3, a4, a5, a6, a7, a8, a9);
    ...
  }

  Caller (foo)                             Callee (bar2)
  ============                             ==============
  Incoming (positive offsets):             Incoming (positive offsets):

  r11+8:  [incoming arg 6]                 r11+8:  [incoming arg 6] <+
  r11+16: [incoming arg 7]                 r11+16: [incoming arg 7] <|+
                                           r11+24: [incoming arg 8] <||+
  Outgoing for bar2 (negative offsets):    r11+32: [incoming arg 9] <|||+
  r11-8:  [outgoing arg 6] ---->----------->-------------------------+|||
  r11-16: [outgoing arg 7] ---->----------->--------------------------+||
  r11-24: [outgoing arg 8] ---->----------->---------------------------+|
  r11-32: [outgoing arg 9] ---->----------->----------------------------+

The verifier tracks outgoing stack arguments in stack_arg_regs[] and
out_stack_arg_depth in bpf_func_state, separately from the regular
r10 stack. The callee does not copy incoming args — it reads them
directly from the caller's outgoing slots at positive r11 offsets.
Similar to stacksafe(), introduce stack_arg_safe() to do pruning
check.

Outgoing stack arg slots are invalidated when the callee returns
(in prepare_func_exit), not at call time. This allows the callee to
read incoming args from the caller's outgoing slots during
verification. The following are a few examples.

Example 1:
  *(u64 *)(r11 - 8) = r6;
  *(u64 *)(r11 - 16) = r7;
  call bar1;                // arg6 = r6, arg7 = r7
  call bar2;                // expected with 2 stack arguments, failed

Example 2:
To fix the Example 1:
  *(u64 *)(r11 - 8) = r6;
  *(u64 *)(r11 - 16) = r7;
  call bar1;                // arg6 = r6, arg7 = r7
  *(u64 *)(r11 - 8) = r8;
  *(u64 *)(r11 - 16) = r9;
  call bar2;                // arg6 = r8, arg7 = r9

Example 3:
The compiler can hoist the shared stack arg stores above the branch:
  *(u64 *)(r11 - 16) = r7;
  if cond goto else;
    *(u64 *)(r11 - 8) = r8;
    call bar1;               // arg6 = r8, arg7 = r7
    goto end;
  else:
    *(u64 *)(r11 - 8) = r9;
    call bar2;               // arg6 = r9, arg7 = r7
  end:

Example 4:
Within a loop:
  loop:
    *(u64 *)(r11 - 8) = r6;  // arg6, before loop
    call bar;                // reuses arg6 each iteration
    if ... goto loop;

A separate max_out_stack_arg_depth field in bpf_subprog_info tracks
the deepest outgoing offset actually written. This intends to
reject programs that write to offsets beyond what any callee expects.

Similar to typical compiler generated code, enforce the following
orderings:
  - all stack arg reads must be ahead of any stack arg write
  - all stack arg reads must be before any bpf func, kfunc and helpers
This is needed as jit may emit 'mov' insns for read/write with
the same register.

Callback functions with stack arguments need kernel setup parameter
types (including stack parameters) properly and then callback function
can retrieve such information for verification purpose.

Global subprogs and freplace with >5 args are not yet supported.

  [1] https://github.com/llvm/llvm-project/pull/189060
  [2] https://lore.kernel.org/bpf/20260423033506.2542005-1-yonghong.song@linux.dev/

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 include/linux/bpf.h          |   2 +
 include/linux/bpf_verifier.h |  27 ++++-
 kernel/bpf/btf.c             |  14 ++-
 kernel/bpf/fixups.c          |  22 +++-
 kernel/bpf/states.c          |  31 ++++++
 kernel/bpf/verifier.c        | 198 ++++++++++++++++++++++++++++++++++-
 6 files changed, 282 insertions(+), 12 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 715b6df9c403..831b28a22f4f 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1669,6 +1669,8 @@ struct bpf_prog_aux {
 	u32 max_pkt_offset;
 	u32 max_tp_access;
 	u32 stack_depth;
+	u16 incoming_stack_arg_depth;
+	u16 stack_arg_depth; /* both incoming and max outgoing of stack arguments */
 	u32 id;
 	u32 func_cnt; /* used by non-func prog as the number of func progs */
 	u32 real_func_cnt; /* includes hidden progs, only used for JIT and freeing progs */
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index d5b4303315dd..2cc349d7fc17 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -358,6 +358,7 @@ struct bpf_func_state {
 	 *                           | number of simulations is tracked in frame N
 	 */
 	u32 callback_depth;
+	bool no_stack_arg_load;
 
 	/* The following fields should be last. See copy_func_state() */
 	/* The state of the stack. Each element of the array describes BPF_REG_SIZE
@@ -372,6 +373,9 @@ struct bpf_func_state {
 	 * `stack`. allocated_stack is always a multiple of BPF_REG_SIZE.
 	 */
 	int allocated_stack;
+
+	u16 out_stack_arg_depth; /* Size of max outgoing stack args in bytes. */
+	struct bpf_reg_state *stack_arg_regs; /* Outgoing on-stack arguments */
 };
 
 #define MAX_CALL_FRAMES 8
@@ -508,6 +512,17 @@ struct bpf_verifier_state {
 	     iter < frame->allocated_stack / BPF_REG_SIZE;		\
 	     iter++, reg = bpf_get_spilled_reg(iter, frame, mask))
 
+#define bpf_get_spilled_stack_arg(slot, frame, mask)                   \
+	((((slot) < frame->out_stack_arg_depth / BPF_REG_SIZE) &&           \
+	  (frame->stack_arg_regs[slot].type != NOT_INIT))               \
+	 ? &frame->stack_arg_regs[slot] : NULL)
+
+/* Iterate over 'frame', setting 'reg' to either NULL or a spilled stack arg. */
+#define bpf_for_each_spilled_stack_arg(iter, frame, reg, mask)         \
+	for (iter = 0, reg = bpf_get_spilled_stack_arg(iter, frame, mask); \
+	     iter < frame->out_stack_arg_depth / BPF_REG_SIZE;              \
+	     iter++, reg = bpf_get_spilled_stack_arg(iter, frame, mask))
+
 #define bpf_for_each_reg_in_vstate_mask(__vst, __state, __reg, __mask, __expr)   \
 	({                                                               \
 		struct bpf_verifier_state *___vstate = __vst;            \
@@ -525,6 +540,11 @@ struct bpf_verifier_state {
 					continue;                        \
 				(void)(__expr);                          \
 			}                                                \
+			bpf_for_each_spilled_stack_arg(___j, __state, __reg, __mask) { \
+				if (!__reg)                              \
+					continue;                        \
+				(void)(__expr);                          \
+			}						 \
 		}                                                        \
 	})
 
@@ -739,10 +759,13 @@ struct bpf_subprog_info {
 	bool keep_fastcall_stack: 1;
 	bool changes_pkt_data: 1;
 	bool might_sleep: 1;
-	u8 arg_cnt:3;
+	u8 arg_cnt:4;
 
 	enum priv_stack_mode priv_stack_mode;
-	struct bpf_subprog_arg_info args[MAX_BPF_FUNC_REG_ARGS];
+	struct bpf_subprog_arg_info args[MAX_BPF_FUNC_ARGS];
+	u16 incoming_stack_arg_depth;
+	u16 stack_arg_depth; /* incoming + max outgoing */
+	u16 max_out_stack_arg_depth;
 };
 
 struct bpf_verifier_env;
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 77af44d8a3ad..cfb35a2decf6 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -7880,13 +7880,19 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	}
 	args = (const struct btf_param *)(t + 1);
 	nargs = btf_type_vlen(t);
-	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
-		if (!is_global)
-			return -EINVAL;
-		bpf_log(log, "Global function %s() with %d > %d args. Buggy compiler.\n",
+	if (nargs > MAX_BPF_FUNC_ARGS) {
+		bpf_log(log, "Function %s() with %d > %d args not supported.\n",
+			tname, nargs, MAX_BPF_FUNC_ARGS);
+		return -EINVAL;
+	}
+	if (is_global && nargs > MAX_BPF_FUNC_REG_ARGS) {
+		bpf_log(log, "Global function %s() with %d > %d args not supported.\n",
 			tname, nargs, MAX_BPF_FUNC_REG_ARGS);
 		return -EINVAL;
 	}
+	if (nargs > MAX_BPF_FUNC_REG_ARGS)
+		sub->incoming_stack_arg_depth = (nargs - MAX_BPF_FUNC_REG_ARGS) * BPF_REG_SIZE;
+
 	/* check that function is void or returns int, exception cb also requires this */
 	t = btf_type_by_id(btf, t->type);
 	while (btf_type_is_modifier(t))
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index fba9e8c00878..7d276208f3cc 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -1123,6 +1123,8 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 
 		func[i]->aux->name[0] = 'F';
 		func[i]->aux->stack_depth = env->subprog_info[i].stack_depth;
+		func[i]->aux->incoming_stack_arg_depth = env->subprog_info[i].incoming_stack_arg_depth;
+		func[i]->aux->stack_arg_depth = env->subprog_info[i].stack_arg_depth;
 		if (env->subprog_info[i].priv_stack_mode == PRIV_STACK_ADAPTIVE)
 			func[i]->aux->jits_use_priv_stack = true;
 
@@ -1301,8 +1303,10 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
 	struct bpf_insn_aux_data *orig_insn_aux;
 	u32 *orig_subprog_starts;
 
-	if (env->subprog_cnt <= 1)
+	if (env->subprog_cnt <= 1) {
+		env->prog->aux->stack_arg_depth = env->subprog_info[0].stack_arg_depth;
 		return 0;
+	}
 
 	prog = orig_prog = env->prog;
 	if (bpf_prog_need_blind(prog)) {
@@ -1378,9 +1382,21 @@ int bpf_fixup_call_args(struct bpf_verifier_env *env)
 	struct bpf_prog *prog = env->prog;
 	struct bpf_insn *insn = prog->insnsi;
 	bool has_kfunc_call = bpf_prog_has_kfunc_call(prog);
-	int i, depth;
+	int depth;
 #endif
-	int err = 0;
+	int i, err = 0;
+
+	for (i = 0; i < env->subprog_cnt; i++) {
+		struct bpf_subprog_info *subprog = &env->subprog_info[i];
+		u16 outgoing = subprog->stack_arg_depth - subprog->incoming_stack_arg_depth;
+
+		if (subprog->max_out_stack_arg_depth > outgoing) {
+			verbose(env,
+				"func#%d writes stack arg slot at depth %u, but calls only require %u bytes\n",
+				i, subprog->max_out_stack_arg_depth, outgoing);
+			return -EINVAL;
+		}
+	}
 
 	if (env->prog->jit_requested &&
 	    !bpf_prog_is_offloaded(env->prog->aux)) {
diff --git a/kernel/bpf/states.c b/kernel/bpf/states.c
index 8478d2c6ed5b..3e59d1c3a726 100644
--- a/kernel/bpf/states.c
+++ b/kernel/bpf/states.c
@@ -838,6 +838,34 @@ static bool stacksafe(struct bpf_verifier_env *env, struct bpf_func_state *old,
 	return true;
 }
 
+/*
+ * Compare stack arg slots between old and current states.
+ * Outgoing stack args are path-local state and must agree for pruning.
+ */
+static bool stack_arg_safe(struct bpf_verifier_env *env, struct bpf_func_state *old,
+			   struct bpf_func_state *cur, struct bpf_idmap *idmap,
+			   enum exact_level exact)
+{
+	int i, nslots;
+
+	nslots = min(old->out_stack_arg_depth, cur->out_stack_arg_depth) / BPF_REG_SIZE;
+	for (i = 0; i < nslots; i++) {
+		struct bpf_reg_state *old_arg = &old->stack_arg_regs[i];
+		struct bpf_reg_state *cur_arg = &cur->stack_arg_regs[i];
+
+		if (old_arg->type == NOT_INIT && cur_arg->type == NOT_INIT)
+			continue;
+
+		if (exact == EXACT && old_arg->type != cur_arg->type)
+			return false;
+
+		if (!regsafe(env, old_arg, cur_arg, idmap, exact))
+			return false;
+	}
+
+	return true;
+}
+
 static bool refsafe(struct bpf_verifier_state *old, struct bpf_verifier_state *cur,
 		    struct bpf_idmap *idmap)
 {
@@ -929,6 +957,9 @@ static bool func_states_equal(struct bpf_verifier_env *env, struct bpf_func_stat
 	if (!stacksafe(env, old, cur, &env->idmap_scratch, exact))
 		return false;
 
+	if (!stack_arg_safe(env, old, cur, &env->idmap_scratch, exact))
+		return false;
+
 	return true;
 }
 
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index ff6ff1c27517..bcf81692a22b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -1361,6 +1361,18 @@ static int copy_stack_state(struct bpf_func_state *dst, const struct bpf_func_st
 		return -ENOMEM;
 
 	dst->allocated_stack = src->allocated_stack;
+
+	/* copy stack args state */
+	n = src->out_stack_arg_depth / BPF_REG_SIZE;
+	if (n) {
+		dst->stack_arg_regs = copy_array(dst->stack_arg_regs, src->stack_arg_regs, n,
+						 sizeof(struct bpf_reg_state),
+						 GFP_KERNEL_ACCOUNT);
+		if (!dst->stack_arg_regs)
+			return -ENOMEM;
+	}
+
+	dst->out_stack_arg_depth = src->out_stack_arg_depth;
 	return 0;
 }
 
@@ -1402,6 +1414,22 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state
 	return 0;
 }
 
+static int grow_stack_arg_slots(struct bpf_verifier_env *env,
+				struct bpf_func_state *state, int size)
+{
+	size_t old_n = state->out_stack_arg_depth / BPF_REG_SIZE, n = size / BPF_REG_SIZE;
+	if (old_n >= n)
+		return 0;
+
+	state->stack_arg_regs = realloc_array(state->stack_arg_regs, old_n, n,
+					  sizeof(struct bpf_reg_state));
+	if (!state->stack_arg_regs)
+		return -ENOMEM;
+
+	state->out_stack_arg_depth = size;
+	return 0;
+}
+
 /* Acquire a pointer id from the env and update the state->refs to include
  * this new pointer reference.
  * On success, returns a valid pointer id to associate with the register
@@ -1564,6 +1592,7 @@ static void free_func_state(struct bpf_func_state *state)
 {
 	if (!state)
 		return;
+	kfree(state->stack_arg_regs);
 	kfree(state->stack);
 	kfree(state);
 }
@@ -4417,6 +4446,109 @@ static int check_stack_write(struct bpf_verifier_env *env,
 	return err;
 }
 
+/*
+ * Write a value to the outgoing stack arg area.
+ * off is a negative offset from r11 (e.g. -8 for arg6, -16 for arg7).
+ */
+static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_state *state,
+				 int off, int value_regno)
+{
+	int max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;
+	struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno];
+	int spi = -off / BPF_REG_SIZE - 1;
+	struct bpf_func_state *cur;
+	struct bpf_reg_state *arg;
+	int err;
+
+	if (spi >= max_stack_arg_regs) {
+		verbose(env, "stack arg write offset %d exceeds max %d stack args\n",
+			off, max_stack_arg_regs);
+		return -EINVAL;
+	}
+
+	err = grow_stack_arg_slots(env, state, -off);
+	if (err)
+		return err;
+
+	/* Track the max outgoing stack arg access depth. */
+	if (-off > subprog->max_out_stack_arg_depth)
+		subprog->max_out_stack_arg_depth = -off;
+
+	cur = env->cur_state->frame[env->cur_state->curframe];
+	if (value_regno >= 0) {
+		state->stack_arg_regs[spi] = cur->regs[value_regno];
+	} else {
+		/* BPF_ST: store immediate, treat as scalar */
+		arg = &state->stack_arg_regs[spi];
+		arg->type = SCALAR_VALUE;
+		__mark_reg_known(arg, env->prog->insnsi[env->insn_idx].imm);
+	}
+	state->no_stack_arg_load = true;
+	return 0;
+}
+
+/*
+ * Read a value from the incoming stack arg area.
+ * off is a positive offset from r11 (e.g. +8 for arg6, +16 for arg7).
+ */
+static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_state *state,
+				int off, int dst_regno)
+{
+	struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno];
+	struct bpf_verifier_state *vstate = env->cur_state;
+	int spi = off / BPF_REG_SIZE - 1;
+	struct bpf_func_state *caller, *cur;
+	struct bpf_reg_state *arg;
+
+	if (state->no_stack_arg_load) {
+		verbose(env, "r11 load must be before any r11 store or call insn\n");
+		return -EINVAL;
+	}
+
+	if (off > subprog->incoming_stack_arg_depth) {
+		verbose(env, "invalid read from stack arg off %d depth %d\n",
+			off, subprog->incoming_stack_arg_depth);
+		return -EACCES;
+	}
+
+	caller = vstate->frame[vstate->curframe - 1];
+	arg = &caller->stack_arg_regs[spi];
+	cur = vstate->frame[vstate->curframe];
+
+	if (is_spillable_regtype(arg->type))
+		copy_register_state(&cur->regs[dst_regno], arg);
+	else
+		mark_reg_unknown(env, cur->regs, dst_regno);
+	return 0;
+}
+
+static int check_outgoing_stack_args(struct bpf_verifier_env *env, struct bpf_func_state *caller,
+				     int nargs)
+{
+	int i, spi;
+
+	for (i = MAX_BPF_FUNC_REG_ARGS; i < nargs; i++) {
+		spi = i - MAX_BPF_FUNC_REG_ARGS;
+		if (spi >= (caller->out_stack_arg_depth / BPF_REG_SIZE) ||
+		    caller->stack_arg_regs[spi].type == NOT_INIT) {
+			verbose(env, "stack %s not properly initialized\n",
+				reg_arg_name(env, argno_from_arg(i + 1)));
+			return -EFAULT;
+		}
+	}
+
+	return 0;
+}
+
+static struct bpf_reg_state *get_func_arg_reg(struct bpf_func_state *caller,
+					      struct bpf_reg_state *regs, int arg)
+{
+	if (arg < MAX_BPF_FUNC_REG_ARGS)
+		return &regs[arg + 1];
+
+	return &caller->stack_arg_regs[arg - MAX_BPF_FUNC_REG_ARGS];
+}
+
 static int check_map_access_type(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
 				 int off, int size, enum bpf_access_type type)
 {
@@ -6605,10 +6737,20 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			  bool strict_alignment_once, bool is_ldsx,
 			  bool allow_trust_mismatch, const char *ctx)
 {
+	struct bpf_verifier_state *vstate = env->cur_state;
+	struct bpf_func_state *state = vstate->frame[vstate->curframe];
 	struct bpf_reg_state *regs = cur_regs(env);
 	enum bpf_reg_type src_reg_type;
 	int err;
 
+	/* Handle stack arg read */
+	if (insn->src_reg == BPF_REG_PARAMS) {
+		err = check_reg_arg(env, insn->dst_reg, DST_OP_NO_MARK);
+		if (err)
+			return err;
+		return check_stack_arg_read(env, state, insn->off, insn->dst_reg);
+	}
+
 	/* check src operand */
 	err = check_reg_arg(env, insn->src_reg, SRC_OP);
 	if (err)
@@ -6637,10 +6779,20 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
 static int check_store_reg(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			   bool strict_alignment_once)
 {
+	struct bpf_verifier_state *vstate = env->cur_state;
+	struct bpf_func_state *state = vstate->frame[vstate->curframe];
 	struct bpf_reg_state *regs = cur_regs(env);
 	enum bpf_reg_type dst_reg_type;
 	int err;
 
+	/* Handle stack arg write */
+	if (insn->dst_reg == BPF_REG_PARAMS) {
+		err = check_reg_arg(env, insn->src_reg, SRC_OP);
+		if (err)
+			return err;
+		return check_stack_arg_write(env, state, insn->off, insn->src_reg);
+	}
+
 	/* check src1 operand */
 	err = check_reg_arg(env, insn->src_reg, SRC_OP);
 	if (err)
@@ -9248,6 +9400,14 @@ static void clear_caller_saved_regs(struct bpf_verifier_env *env,
 	}
 }
 
+static void invalidate_outgoing_stack_args(struct bpf_func_state *state)
+{
+	int i, nslots = state->out_stack_arg_depth / BPF_REG_SIZE;
+
+	for (i = 0; i < nslots; i++)
+		state->stack_arg_regs[i].type = NOT_INIT;
+}
+
 typedef int (*set_callee_state_fn)(struct bpf_verifier_env *env,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee,
@@ -9310,6 +9470,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
 				    struct bpf_reg_state *regs)
 {
 	struct bpf_subprog_info *sub = subprog_info(env, subprog);
+	struct bpf_func_state *caller = cur_func(env);
 	struct bpf_verifier_log *log = &env->log;
 	u32 i;
 	int ret;
@@ -9318,13 +9479,16 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
 	if (ret)
 		return ret;
 
+	ret = check_outgoing_stack_args(env, caller, sub->arg_cnt);
+	if (ret)
+		return ret;
+
 	/* check that BTF function arguments match actual types that the
 	 * verifier sees.
 	 */
 	for (i = 0; i < sub->arg_cnt; i++) {
 		argno_t argno = argno_from_arg(i + 1);
-		u32 regno = i + 1;
-		struct bpf_reg_state *reg = &regs[regno];
+		struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
 		struct bpf_subprog_arg_info *arg = &sub->args[i];
 
 		if (arg->arg_type == ARG_ANYTHING) {
@@ -9512,6 +9676,8 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			   int *insn_idx)
 {
 	struct bpf_verifier_state *state = env->cur_state;
+	struct bpf_subprog_info *caller_info;
+	u16 callee_incoming, stack_arg_depth;
 	struct bpf_func_state *caller;
 	int err, subprog, target_insn;
 
@@ -9565,6 +9731,16 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		return 0;
 	}
 
+	/*
+	 * Track caller's total stack arg depth (incoming + max outgoing).
+	 * This is needed so the JIT knows how much stack arg space to allocate.
+	 */
+	caller_info = &env->subprog_info[caller->subprogno];
+	callee_incoming = env->subprog_info[subprog].incoming_stack_arg_depth;
+	stack_arg_depth = caller_info->incoming_stack_arg_depth + callee_incoming;
+	if (stack_arg_depth > caller_info->stack_arg_depth)
+		caller_info->stack_arg_depth = stack_arg_depth;
+
 	/* for regular function entry setup new frame and continue
 	 * from that frame.
 	 */
@@ -9922,6 +10098,7 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
 	 * bpf_throw, this will be done by copy_verifier_state for extra frames. */
 	free_func_state(callee);
 	state->frame[state->curframe--] = NULL;
+	invalidate_outgoing_stack_args(caller);
 
 	/* for callbacks widen imprecise scalars to make programs like below verify:
 	 *
@@ -17627,6 +17804,14 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
 		return check_store_reg(env, insn, false);
 
 	case BPF_ST: {
+		/* Handle stack arg write (store immediate) */
+		if (insn->dst_reg == BPF_REG_PARAMS) {
+			struct bpf_verifier_state *vstate = env->cur_state;
+			struct bpf_func_state *state = vstate->frame[vstate->curframe];
+
+			return check_stack_arg_write(env, state, insn->off, -1);
+		}
+
 		enum bpf_reg_type dst_reg_type;
 
 		err = check_reg_arg(env, insn->dst_reg, SRC_OP);
@@ -17661,6 +17846,7 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
 				}
 			}
 			mark_reg_scratched(env, BPF_REG_0);
+			cur_func(env)->no_stack_arg_load = true;
 			if (insn->src_reg == BPF_PSEUDO_CALL)
 				return check_func_call(env, insn, &env->insn_idx);
 			if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL)
@@ -18776,7 +18962,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
 				goto out;
 			}
 		}
-		for (i = BPF_REG_1; i <= sub->arg_cnt; i++) {
+		for (i = BPF_REG_1; i <= min_t(u32, sub->arg_cnt, MAX_BPF_FUNC_REG_ARGS); i++) {
 			arg = &sub->args[i - BPF_REG_1];
 			reg = &regs[i];
 
@@ -18819,6 +19005,12 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
 				goto out;
 			}
 		}
+		if (env->prog->type == BPF_PROG_TYPE_EXT && sub->arg_cnt > MAX_BPF_FUNC_REG_ARGS) {
+			verbose(env, "freplace programs with >%d args not supported yet\n",
+				MAX_BPF_FUNC_REG_ARGS);
+			ret = -EINVAL;
+			goto out;
+		}
 	} else {
 		/* if main BPF program has associated BTF info, validate that
 		 * it's matching expected signature, and otherwise mark BTF
-- 
2.52.0

[PATCH bpf-next 02/18] bpf: Add precision marking and backtracking for stack argument slots

[Yonghong Song]

Extend the precision marking and backtracking infrastructure to
support stack argument slots (r11-based accesses). Without this,
precision demands for scalar values passed through stack arguments
are silently dropped, which could allow the verifier to incorrectly
prune states with different constant values in stack arg slots.

INSN_F_STACK_ARG_ACCESS is encoded as INSN_F_STACK_ACCESS |
INSN_F_DST_REG_STACK (BIT(9) | BIT(10)). This is safe because
INSN_F_STACK_ACCESS is only used for ST/STX/LDX insns while
INSN_F_DST_REG_STACK is only used for JMP insns — they never appear
on the same instruction. This keeps the total within the 12-bit
jmp_history flags budget.

Three components are added:

1. Jump history recording for stack arg accesses:
   - check_stack_arg_write() records INSN_F_STACK_ARG_ACCESS for
     outgoing stores.
   - check_stack_arg_read() records INSN_F_STACK_ARG_ACCESS for
     incoming loads.

2. backtrack_insn() handling:
   - BPF_LDX: when backtracking through an incoming stack arg load,
     transfer precision demand from the destination register to the
     stack arg slot mask.
   - BPF_STX/BPF_ST: when backtracking through an outgoing stack arg
     store, transfer precision demand from the stack arg slot to the
     source register.
   - Call boundary: when exiting a callee back to the caller,
     propagate the callee's incoming stack arg precision bits to the
     caller's outgoing stack arg slots. The slot index maps directly
     (slot i in callee corresponds to slot i in caller) since the
     caller's stack_arg_regs only contains outgoing slots.

3. bpf_mark_chain_precision() state walking:
   - When iterating parent states, mark stack_arg_regs[spi].precise
     for slots that have pending precision demand.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 include/linux/bpf_verifier.h | 13 ++++++++
 kernel/bpf/backtrack.c       | 61 ++++++++++++++++++++++++++++++++++--
 kernel/bpf/verifier.c        | 30 +++++++++++++++---
 3 files changed, 98 insertions(+), 6 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 2cc349d7fc17..735f33ad3db7 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -393,6 +393,13 @@ enum {
 	INSN_F_SPI_SHIFT = 3, /* shifted 3 bits to the left */
 
 	INSN_F_STACK_ACCESS = BIT(9),
+	/*
+	 * INSN_F_STACK_ARG_ACCESS uses INSN_F_STACK_ACCESS | INSN_F_DST_REG_STACK.
+	 * This is safe because INSN_F_DST_REG_STACK is only used for JMP insns
+	 * while INSN_F_STACK_ACCESS is only used for ST/STX/LDX insns — they
+	 * never appear on the same instruction.
+	 */
+	INSN_F_STACK_ARG_ACCESS = BIT(9) | BIT(10),
 
 	INSN_F_DST_REG_STACK = BIT(10), /* dst_reg is PTR_TO_STACK */
 	INSN_F_SRC_REG_STACK = BIT(11), /* src_reg is PTR_TO_STACK */
@@ -775,6 +782,7 @@ struct backtrack_state {
 	u32 frame;
 	u32 reg_masks[MAX_CALL_FRAMES];
 	u64 stack_masks[MAX_CALL_FRAMES];
+	u8 stack_arg_masks[MAX_CALL_FRAMES];
 };
 
 struct bpf_id_pair {
@@ -1173,6 +1181,11 @@ static inline void bpf_bt_set_frame_slot(struct backtrack_state *bt, u32 frame,
 	bt->stack_masks[frame] |= 1ull << slot;
 }
 
+static inline void bt_set_frame_stack_arg_slot(struct backtrack_state *bt, u32 frame, u32 slot)
+{
+	bt->stack_arg_masks[frame] |= 1 << slot;
+}
+
 static inline bool bt_is_frame_reg_set(struct backtrack_state *bt, u32 frame, u32 reg)
 {
 	return bt->reg_masks[frame] & (1 << reg);
diff --git a/kernel/bpf/backtrack.c b/kernel/bpf/backtrack.c
index 854731dc93fe..73da7eaac47f 100644
--- a/kernel/bpf/backtrack.c
+++ b/kernel/bpf/backtrack.c
@@ -135,11 +135,21 @@ static inline u32 bt_empty(struct backtrack_state *bt)
 	int i;
 
 	for (i = 0; i <= bt->frame; i++)
-		mask |= bt->reg_masks[i] | bt->stack_masks[i];
+		mask |= bt->reg_masks[i] | bt->stack_masks[i] | bt->stack_arg_masks[i];
 
 	return mask == 0;
 }
 
+static inline void bt_clear_frame_stack_arg_slot(struct backtrack_state *bt, u32 frame, u32 slot)
+{
+	bt->stack_arg_masks[frame] &= ~(1 << slot);
+}
+
+static inline bool bt_is_frame_stack_arg_slot_set(struct backtrack_state *bt, u32 frame, u32 slot)
+{
+	return bt->stack_arg_masks[frame] & (1 << slot);
+}
+
 static inline int bt_subprog_enter(struct backtrack_state *bt)
 {
 	if (bt->frame == MAX_CALL_FRAMES - 1) {
@@ -200,6 +210,11 @@ static inline u64 bt_stack_mask(struct backtrack_state *bt)
 	return bt->stack_masks[bt->frame];
 }
 
+static inline u8 bt_stack_arg_mask(struct backtrack_state *bt)
+{
+	return bt->stack_arg_masks[bt->frame];
+}
+
 static inline bool bt_is_reg_set(struct backtrack_state *bt, u32 reg)
 {
 	return bt->reg_masks[bt->frame] & (1 << reg);
@@ -341,6 +356,13 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 			return 0;
 		bt_clear_reg(bt, load_reg);
 
+		if (hist && (hist->flags & INSN_F_STACK_ARG_ACCESS) == INSN_F_STACK_ARG_ACCESS) {
+			spi = insn_stack_access_spi(hist->flags);
+			fr = insn_stack_access_frameno(hist->flags);
+			bt_set_frame_stack_arg_slot(bt, fr, spi);
+			return 0;
+		}
+
 		/* scalars can only be spilled into stack w/o losing precision.
 		 * Load from any other memory can be zero extended.
 		 * The desire to keep that precision is already indicated
@@ -363,6 +385,18 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 			 * encountered a case of pointer subtraction.
 			 */
 			return -ENOTSUPP;
+
+		if (hist && (hist->flags & INSN_F_STACK_ARG_ACCESS) == INSN_F_STACK_ARG_ACCESS) {
+			spi = insn_stack_access_spi(hist->flags);
+			fr = insn_stack_access_frameno(hist->flags);
+			if (!bt_is_frame_stack_arg_slot_set(bt, fr, spi))
+				return 0;
+			bt_clear_frame_stack_arg_slot(bt, fr, spi);
+			if (class == BPF_STX)
+				bt_set_reg(bt, sreg);
+			return 0;
+		}
+
 		/* scalars can only be spilled into stack */
 		if (!hist || !(hist->flags & INSN_F_STACK_ACCESS))
 			return 0;
@@ -431,6 +465,17 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 						bpf_bt_set_frame_reg(bt, bt->frame - 1, i);
 					}
 				}
+				/* propagate callee's incoming stack arg precision
+				 * to caller's outgoing stack arg slots
+				 */
+				if (bt_stack_arg_mask(bt)) {
+					for (i = 0; i < MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS; i++) {
+						if (!bt_is_frame_stack_arg_slot_set(bt, bt->frame, i))
+							continue;
+						bt_clear_frame_stack_arg_slot(bt, bt->frame, i);
+						bt_set_frame_stack_arg_slot(bt, bt->frame - 1, i);
+					}
+				}
 				if (bt_subprog_exit(bt))
 					return -EFAULT;
 				return 0;
@@ -453,9 +498,10 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 					     bt_stack_mask(bt));
 				return -EFAULT;
 			}
-			/* clear r1-r5 in callback subprog's mask */
+			/* clear r1-r5 and stack arg slots in callback subprog's mask */
 			for (i = BPF_REG_1; i <= BPF_REG_5; i++)
 				bt_clear_reg(bt, i);
+			bt->stack_arg_masks[bt->frame] = 0;
 			if (bt_subprog_exit(bt))
 				return -EFAULT;
 			return 0;
@@ -901,6 +947,17 @@ int bpf_mark_chain_precision(struct bpf_verifier_env *env,
 					*changed = true;
 				}
 			}
+			for (i = 0; i < func->out_stack_arg_depth / BPF_REG_SIZE; i++) {
+				if (!bt_is_frame_stack_arg_slot_set(bt, fr, i))
+					continue;
+				reg = &func->stack_arg_regs[i];
+				if (reg->type != SCALAR_VALUE || reg->precise) {
+					bt_clear_frame_stack_arg_slot(bt, fr, i);
+				} else {
+					reg->precise = true;
+					*changed = true;
+				}
+			}
 			if (env->log.level & BPF_LOG_LEVEL2) {
 				fmt_reg_mask(env->tmp_str_buf, TMP_STR_BUF_LEN,
 					     bt_frame_reg_mask(bt, fr));
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index bcf81692a22b..e041c182c614 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3563,6 +3563,11 @@ static int insn_stack_access_flags(int frameno, int spi)
 	return INSN_F_STACK_ACCESS | (spi << INSN_F_SPI_SHIFT) | frameno;
 }
 
+static int insn_stack_arg_access_flags(int frameno, int spi)
+{
+	return INSN_F_STACK_ARG_ACCESS | (spi << INSN_F_SPI_SHIFT) | frameno;
+}
+
 static void mark_indirect_target(struct bpf_verifier_env *env, int idx)
 {
 	env->insn_aux_data[idx].indirect_target = true;
@@ -4484,7 +4489,8 @@ static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_s
 		__mark_reg_known(arg, env->prog->insnsi[env->insn_idx].imm);
 	}
 	state->no_stack_arg_load = true;
-	return 0;
+	return bpf_push_jmp_history(env, env->cur_state,
+				    insn_stack_arg_access_flags(state->frameno, spi), 0);
 }
 
 /*
@@ -4519,7 +4525,17 @@ static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_st
 		copy_register_state(&cur->regs[dst_regno], arg);
 	else
 		mark_reg_unknown(env, cur->regs, dst_regno);
-	return 0;
+	return bpf_push_jmp_history(env, env->cur_state,
+				    insn_stack_arg_access_flags(state->frameno, spi), 0);
+}
+
+static int mark_stack_arg_precision(struct bpf_verifier_env *env, int arg_idx)
+{
+	struct bpf_func_state *caller = cur_func(env);
+	int spi = arg_idx - MAX_BPF_FUNC_REG_ARGS;
+
+	bt_set_frame_stack_arg_slot(&env->bt, caller->frameno, spi);
+	return mark_chain_precision_batch(env, env->cur_state);
 }
 
 static int check_outgoing_stack_args(struct bpf_verifier_env *env, struct bpf_func_state *caller,
@@ -7269,8 +7285,14 @@ static int check_mem_size_reg(struct bpf_verifier_env *env,
 	}
 	err = check_helper_mem_access(env, mem_reg, mem_argno, size_reg->umax_value,
 				      access_type, zero_size_allowed, meta);
-	if (!err)
-		err = mark_chain_precision(env, reg_from_argno(size_argno));
+	if (!err) {
+		int regno = reg_from_argno(size_argno);
+
+		if (regno >= 0)
+			err = mark_chain_precision(env, regno);
+		else
+			err = mark_stack_arg_precision(env, arg_from_argno(size_argno) - 1);
+	}
 	return err;
 }
 
-- 
2.52.0

[PATCH bpf-next 03/18] bpf: Refactor record_call_access() to extract per-arg logic

[Yonghong Song]

Extract the per-argument FP-derived pointer handling from
record_call_access() into a new record_arg_access() helper.

The existing loop body — checking arg_is_fp, querying stack access
bytes, and calling record_stack_access/record_imprecise — will be
reused for stack argument slots in the next patch. Factoring it out
now avoids duplicating the logic.

No functional change.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/liveness.c | 69 +++++++++++++++++++++++++------------------
 1 file changed, 41 insertions(+), 28 deletions(-)

diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 332e6e003f27..1b45621d652c 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -1343,6 +1343,44 @@ static int record_load_store_access(struct bpf_verifier_env *env,
 	return 0;
 }
 
+static int record_arg_access(struct bpf_verifier_env *env,
+			     struct func_instance *instance,
+			     struct bpf_insn *insn,
+			     struct arg_track *at, int arg_idx,
+			     int insn_idx)
+{
+	int depth = instance->depth;
+	int frame = at->frame;
+	s64 bytes;
+	int err;
+
+	if (!arg_is_fp(at))
+		return 0;
+
+	if (bpf_helper_call(insn)) {
+		bytes = bpf_helper_stack_access_bytes(env, insn, arg_idx, insn_idx);
+	} else if (bpf_pseudo_kfunc_call(insn)) {
+		bytes = bpf_kfunc_stack_access_bytes(env, insn, arg_idx, insn_idx);
+	} else {
+		for (int f = 0; f <= depth; f++) {
+			err = mark_stack_read(instance, f, insn_idx, SPIS_ALL);
+			if (err)
+				return err;
+		}
+		return 1;
+	}
+	if (bytes == 0)
+		return 0;
+
+	if (frame >= 0 && frame <= depth)
+		err = record_stack_access(instance, at, bytes, frame, insn_idx);
+	else if (frame == ARG_IMPRECISE)
+		err = record_imprecise(instance, at->mask, insn_idx);
+	else
+		err = 0;
+	return err;
+}
+
 /* Record stack access for a given 'at' state of helper/kfunc 'insn' */
 static int record_call_access(struct bpf_verifier_env *env,
 			      struct func_instance *instance,
@@ -1350,9 +1388,8 @@ static int record_call_access(struct bpf_verifier_env *env,
 			      int insn_idx)
 {
 	struct bpf_insn *insn = &env->prog->insnsi[insn_idx];
-	int depth = instance->depth;
 	struct bpf_call_summary cs;
-	int r, err = 0, num_params = 5;
+	int r, err, num_params = 5;
 
 	if (bpf_pseudo_call(insn))
 		return 0;
@@ -1361,33 +1398,9 @@ static int record_call_access(struct bpf_verifier_env *env,
 		num_params = cs.num_params;
 
 	for (r = BPF_REG_1; r < BPF_REG_1 + num_params; r++) {
-		int frame = at[r].frame;
-		s64 bytes;
-
-		if (!arg_is_fp(&at[r]))
-			continue;
-
-		if (bpf_helper_call(insn)) {
-			bytes = bpf_helper_stack_access_bytes(env, insn, r - 1, insn_idx);
-		} else if (bpf_pseudo_kfunc_call(insn)) {
-			bytes = bpf_kfunc_stack_access_bytes(env, insn, r - 1, insn_idx);
-		} else {
-			for (int f = 0; f <= depth; f++) {
-				err = mark_stack_read(instance, f, insn_idx, SPIS_ALL);
-				if (err)
-					return err;
-			}
-			return 0;
-		}
-		if (bytes == 0)
-			continue;
-
-		if (frame >= 0 && frame <= depth)
-			err = record_stack_access(instance, &at[r], bytes, frame, insn_idx);
-		else if (frame == ARG_IMPRECISE)
-			err = record_imprecise(instance, at[r].mask, insn_idx);
+		err = record_arg_access(env, instance, insn, &at[r], r - 1, insn_idx);
 		if (err)
-			return err;
+			return err > 0 ? 0 : err;
 	}
 	return 0;
 }
-- 
2.52.0

[PATCH bpf-next 04/18] bpf: Extend liveness analysis to track stack argument slots

[Yonghong Song]

BPF_REG_PARAMS (R11) is at index MAX_BPF_REG, which is beyond the
register tracking arrays in const_fold.c and liveness.c. Handle it
explicitly to avoid out-of-bounds accesses.

In liveness.c, extend the arg tracking dataflow to cover stack arg
slots. Otherwise, pointers passed through stack args are invisible
to liveness, causing the pointed-to stack slots to be incorrectly
poisoned.

Add a parallel tracking array (at_stack_arg_out/at_stack_arg_entry)
in arg_track_xfer() to record FP-derived values stored to outgoing
stack arg slots and to restore them on incoming reads. Extend
record_call_access() to check stack arg slots for FP-derived
pointers at kfunc call sites, reusing the record_arg_access() helper
extracted in the previous patch. Pass stack arg state from caller to
callee in analyze_subprog() so that callees can track pointers
received through stack args.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 include/linux/bpf_verifier.h |   2 +
 kernel/bpf/const_fold.c      |  13 +++-
 kernel/bpf/liveness.c        | 133 ++++++++++++++++++++++++++++++++---
 3 files changed, 135 insertions(+), 13 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 735f33ad3db7..de723d27da94 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -905,6 +905,8 @@ struct bpf_verifier_env {
 	struct bpf_jmp_history_entry *cur_hist_ent;
 	/* Per-callsite copy of parent's converged at_stack_in for cross-frame fills. */
 	struct arg_track **callsite_at_stack;
+	/* Per-callsite stack arg state for cross-frame stack arg tracking. */
+	struct arg_track **callsite_at_stack_arg;
 	u32 pass_cnt; /* number of times do_check() was called */
 	u32 subprog_cnt;
 	/* number of instructions analyzed by the verifier */
diff --git a/kernel/bpf/const_fold.c b/kernel/bpf/const_fold.c
index db73c4740b1e..b65285d61efe 100644
--- a/kernel/bpf/const_fold.c
+++ b/kernel/bpf/const_fold.c
@@ -51,13 +51,22 @@ static void const_reg_xfer(struct bpf_verifier_env *env, struct const_arg_info *
 			   struct bpf_insn *insn, struct bpf_insn *insns, int idx)
 {
 	struct const_arg_info unknown = { .state = CONST_ARG_UNKNOWN, .val = 0 };
-	struct const_arg_info *dst = &ci_out[insn->dst_reg];
-	struct const_arg_info *src = &ci_out[insn->src_reg];
+	struct const_arg_info *dst, *src;
 	u8 class = BPF_CLASS(insn->code);
 	u8 mode = BPF_MODE(insn->code);
 	u8 opcode = BPF_OP(insn->code) | BPF_SRC(insn->code);
 	int r;
 
+	/* Stack arguments use BPF_REG_PARAMS which is outside the tracked register set. */
+	if (insn->dst_reg == BPF_REG_PARAMS)
+		return;
+	if (insn->src_reg == BPF_REG_PARAMS) {
+		ci_out[insn->dst_reg] = unknown;
+		return;
+	}
+
+	dst = &ci_out[insn->dst_reg];
+	src = &ci_out[insn->src_reg];
 	switch (class) {
 	case BPF_ALU:
 	case BPF_ALU64:
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 1b45621d652c..b60b5fdae9b9 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -610,6 +610,18 @@ enum arg_track_state {
 /* Track callee stack slots fp-8 through fp-512 (64 slots of 8 bytes each) */
 #define MAX_ARG_SPILL_SLOTS 64
 
+/* Stack arg slots: outgoing at -(i+1)*8, incoming at +(i+1)*8 */
+#define MAX_STACK_ARG_SLOTS (MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS)
+
+static int stack_arg_off_to_slot(s16 off)
+{
+	int aoff = off < 0 ? -off : off;
+
+	if (aoff / 8 > MAX_STACK_ARG_SLOTS)
+		return -1;
+	return aoff / 8 - 1;
+}
+
 static bool arg_is_visited(const struct arg_track *at)
 {
 	return at->frame != ARG_UNVISITED;
@@ -1062,17 +1074,37 @@ static bool can_be_local_fp(int depth, int regno, struct arg_track *at)
 static void arg_track_xfer(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			   int insn_idx,
 			   struct arg_track *at_out, struct arg_track *at_stack_out,
+			   struct arg_track *at_stack_arg_out,
+			   const struct arg_track *at_stack_arg_entry,
 			   struct func_instance *instance,
 			   u32 *callsites)
 {
 	int depth = instance->depth;
 	u8 class = BPF_CLASS(insn->code);
 	u8 code = BPF_OP(insn->code);
-	struct arg_track *dst = &at_out[insn->dst_reg];
-	struct arg_track *src = &at_out[insn->src_reg];
+	struct arg_track *dst, *src;
 	struct arg_track none = { .frame = ARG_NONE };
-	int r;
+	int r, slot;
+
+	/* Handle stack arg stores and loads. */
+	if (insn->dst_reg == BPF_REG_PARAMS) {
+		slot = stack_arg_off_to_slot(insn->off);
+		if (slot >= 0) {
+			if (class == BPF_STX)
+				at_stack_arg_out[slot] = at_out[insn->src_reg];
+			else
+				at_stack_arg_out[slot] = none;
+		}
+		return;
+	}
+	if (insn->src_reg == BPF_REG_PARAMS) {
+		slot = stack_arg_off_to_slot(insn->off);
+		at_out[insn->dst_reg] = (slot >= 0) ? at_stack_arg_entry[slot] : none;
+		return;
+	}
 
+	dst = &at_out[insn->dst_reg];
+	src = &at_out[insn->src_reg];
 	if (class == BPF_ALU64 && BPF_SRC(insn->code) == BPF_K) {
 		if (code == BPF_MOV) {
 			*dst = none;
@@ -1385,6 +1417,7 @@ static int record_arg_access(struct bpf_verifier_env *env,
 static int record_call_access(struct bpf_verifier_env *env,
 			      struct func_instance *instance,
 			      struct arg_track *at,
+			      struct arg_track *at_stack_arg,
 			      int insn_idx)
 {
 	struct bpf_insn *insn = &env->prog->insnsi[insn_idx];
@@ -1402,6 +1435,15 @@ static int record_call_access(struct bpf_verifier_env *env,
 		if (err)
 			return err > 0 ? 0 : err;
 	}
+
+	if (num_params <= MAX_BPF_FUNC_REG_ARGS)
+		return 0;
+	for (r = 0; r < MAX_STACK_ARG_SLOTS && r < num_params - MAX_BPF_FUNC_REG_ARGS; r++) {
+		err = record_arg_access(env, instance, insn, &at_stack_arg[r],
+					r + MAX_BPF_FUNC_REG_ARGS, insn_idx);
+		if (err)
+			return err > 0 ? 0 : err;
+	}
 	return 0;
 }
 
@@ -1545,6 +1587,7 @@ static void print_subprog_arg_access(struct bpf_verifier_env *env,
 static int compute_subprog_args(struct bpf_verifier_env *env,
 				struct subprog_at_info *info,
 				struct arg_track *callee_entry,
+				struct arg_track *callee_stack_arg_entry,
 				struct func_instance *instance,
 				u32 *callsites)
 {
@@ -1560,6 +1603,9 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 	struct arg_track at_out[MAX_BPF_REG];
 	struct arg_track (*at_stack_in)[MAX_ARG_SPILL_SLOTS] = NULL;
 	struct arg_track *at_stack_out = NULL;
+	struct arg_track (*at_stack_arg_in)[MAX_STACK_ARG_SLOTS] = NULL;
+	struct arg_track at_stack_arg_out[MAX_STACK_ARG_SLOTS];
+	struct arg_track at_stack_arg_entry[MAX_STACK_ARG_SLOTS];
 	struct arg_track unvisited = { .frame = ARG_UNVISITED };
 	struct arg_track none = { .frame = ARG_NONE };
 	bool changed;
@@ -1569,6 +1615,10 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 	if (!at_in)
 		goto err_free;
 
+	at_stack_arg_in = kvmalloc_objs(*at_stack_arg_in, len, GFP_KERNEL_ACCOUNT);
+	if (!at_stack_arg_in)
+		goto err_free;
+
 	at_stack_in = kvmalloc_objs(*at_stack_in, len, GFP_KERNEL_ACCOUNT);
 	if (!at_stack_in)
 		goto err_free;
@@ -1582,6 +1632,8 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 			at_in[i][r] = unvisited;
 		for (r = 0; r < MAX_ARG_SPILL_SLOTS; r++)
 			at_stack_in[i][r] = unvisited;
+		for (r = 0; r < MAX_STACK_ARG_SLOTS; r++)
+			at_stack_arg_in[i][r] = unvisited;
 	}
 
 	for (r = 0; r < MAX_BPF_REG; r++)
@@ -1600,6 +1652,12 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 	for (r = 0; r < MAX_ARG_SPILL_SLOTS; r++)
 		at_stack_in[0][r] = none;
 
+	/* Entry: incoming stack args from caller, or ARG_NONE for main */
+	for (r = 0; r < MAX_STACK_ARG_SLOTS; r++) {
+		at_stack_arg_entry[r] = callee_stack_arg_entry ? callee_stack_arg_entry[r] : none;
+		at_stack_arg_in[0][r] = none;
+	}
+
 	if (env->log.level & BPF_LOG_LEVEL2)
 		verbose(env, "subprog#%d: analyzing (depth %d)...\n", subprog, depth);
 
@@ -1617,8 +1675,10 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 
 		memcpy(at_out, at_in[i], sizeof(at_out));
 		memcpy(at_stack_out, at_stack_in[i], MAX_ARG_SPILL_SLOTS * sizeof(*at_stack_out));
+		memcpy(at_stack_arg_out, at_stack_arg_in[i], sizeof(at_stack_arg_out));
 
-		arg_track_xfer(env, insn, idx, at_out, at_stack_out, instance, callsites);
+		arg_track_xfer(env, insn, idx, at_out, at_stack_out,
+			       at_stack_arg_out, at_stack_arg_entry, instance, callsites);
 		arg_track_log(env, insn, idx, at_in[i], at_stack_in[i], at_out, at_stack_out);
 
 		/* Propagate to successors within this subprogram */
@@ -1639,6 +1699,18 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 			for (r = 0; r < MAX_ARG_SPILL_SLOTS; r++)
 				changed |= arg_track_join(env, idx, target, -r - 1,
 							  &at_stack_in[ti][r], at_stack_out[r]);
+
+			/*
+			 * Stack arg slots use indices after spill slots:
+			 * -(r + MAX_ARG_SPILL_SLOTS + 1) so that index * 8 in verbose logging
+			 * yields unique FP-relative offsets that don't * collide with spill slot
+			 * offsets -8..-512.
+			 */
+			for (r = 0; r < MAX_STACK_ARG_SLOTS; r++)
+				changed |= arg_track_join(env, idx, target,
+							  -(r + MAX_ARG_SPILL_SLOTS + 1),
+							  &at_stack_arg_in[ti][r],
+							  at_stack_arg_out[r]);
 		}
 	}
 	if (changed)
@@ -1655,7 +1727,7 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 			goto err_free;
 
 		if (insn->code == (BPF_JMP | BPF_CALL)) {
-			err = record_call_access(env, instance, at_in[i], idx);
+			err = record_call_access(env, instance, at_in[i], at_stack_arg_in[i], idx);
 			if (err)
 				goto err_free;
 		}
@@ -1671,6 +1743,17 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 			}
 			memcpy(env->callsite_at_stack[idx],
 			       at_stack_in[i], sizeof(struct arg_track) * MAX_ARG_SPILL_SLOTS);
+
+			kvfree(env->callsite_at_stack_arg[idx]);
+			env->callsite_at_stack_arg[idx] =
+				kvmalloc_objs(*env->callsite_at_stack_arg[idx],
+					      MAX_STACK_ARG_SLOTS, GFP_KERNEL_ACCOUNT);
+			if (!env->callsite_at_stack_arg[idx]) {
+				err = -ENOMEM;
+				goto err_free;
+			}
+			memcpy(env->callsite_at_stack_arg[idx],
+			       at_stack_arg_in[i], sizeof(struct arg_track) * MAX_STACK_ARG_SLOTS);
 		}
 	}
 
@@ -1683,6 +1766,7 @@ static int compute_subprog_args(struct bpf_verifier_env *env,
 err_free:
 	kvfree(at_stack_out);
 	kvfree(at_stack_in);
+	kvfree(at_stack_arg_in);
 	kvfree(at_in);
 	return err;
 }
@@ -1696,6 +1780,16 @@ static bool has_fp_args(struct arg_track *args)
 	return false;
 }
 
+static bool has_fp_stack_args(struct arg_track *stack_args)
+{
+	if (!stack_args)
+		return false;
+	for (int r = 0; r < MAX_STACK_ARG_SLOTS; r++)
+		if (arg_is_fp(&stack_args[r]))
+			return true;
+	return false;
+}
+
 /*
  * Merge a freshly analyzed instance into the original.
  * may_read: union (any pass might read the slot).
@@ -1768,6 +1862,7 @@ static void free_instance(struct func_instance *instance)
  */
 static int analyze_subprog(struct bpf_verifier_env *env,
 			   struct arg_track *entry_args,
+			   struct arg_track *entry_stack_args,
 			   struct subprog_at_info *info,
 			   struct func_instance *instance,
 			   u32 *callsites)
@@ -1809,7 +1904,8 @@ static int analyze_subprog(struct bpf_verifier_env *env,
 	kvfree(info[subprog].at_in);
 	info[subprog].at_in = NULL;
 
-	err = compute_subprog_args(env, &info[subprog], entry_args, instance, callsites);
+	err = compute_subprog_args(env, &info[subprog], entry_args, entry_stack_args, instance,
+				   callsites);
 	if (err)
 		goto out_free;
 
@@ -1817,6 +1913,7 @@ static int analyze_subprog(struct bpf_verifier_env *env,
 	for (int p = po_start; p < po_end; p++) {
 		int idx = env->cfg.insn_postorder[p];
 		struct arg_track callee_args[BPF_REG_5 + 1];
+		struct arg_track *callee_stack_args = NULL;
 		struct arg_track none = { .frame = ARG_NONE };
 		struct bpf_insn *insn = &insns[idx];
 		struct func_instance *callee_instance;
@@ -1834,6 +1931,7 @@ static int analyze_subprog(struct bpf_verifier_env *env,
 			/* Build entry args: R1-R5 from at_in at call site */
 			for (int r = BPF_REG_1; r <= BPF_REG_5; r++)
 				callee_args[r] = info[subprog].at_in[j][r];
+			callee_stack_args = env->callsite_at_stack_arg[idx];
 		} else if (bpf_calls_callback(env, idx)) {
 			callee = find_callback_subprog(env, insn, idx, &caller_reg, &cb_callee_reg);
 			if (callee == -2) {
@@ -1860,7 +1958,7 @@ static int analyze_subprog(struct bpf_verifier_env *env,
 			continue;
 		}
 
-		if (!has_fp_args(callee_args))
+		if (!has_fp_args(callee_args) && !has_fp_stack_args(callee_stack_args))
 			continue;
 
 		if (depth == MAX_CALL_FRAMES - 1) {
@@ -1874,7 +1972,8 @@ static int analyze_subprog(struct bpf_verifier_env *env,
 			goto out_free;
 		}
 		callsites[depth] = idx;
-		err = analyze_subprog(env, callee_args, info, callee_instance, callsites);
+		err = analyze_subprog(env, callee_args, callee_stack_args, info, callee_instance,
+				      callsites);
 		if (err)
 			goto out_free;
 
@@ -1926,13 +2025,21 @@ int bpf_compute_subprog_arg_access(struct bpf_verifier_env *env)
 		kvfree(info);
 		return -ENOMEM;
 	}
+	env->callsite_at_stack_arg = kvzalloc_objs(*env->callsite_at_stack_arg, insn_cnt,
+					      GFP_KERNEL_ACCOUNT);
+	if (!env->callsite_at_stack_arg) {
+		kvfree(env->callsite_at_stack);
+		env->callsite_at_stack = NULL;
+		kvfree(info);
+		return -ENOMEM;
+	}
 
 	instance = call_instance(env, NULL, 0, 0);
 	if (IS_ERR(instance)) {
 		err = PTR_ERR(instance);
 		goto out;
 	}
-	err = analyze_subprog(env, NULL, info, instance, callsites);
+	err = analyze_subprog(env, NULL, NULL, info, instance, callsites);
 	if (err)
 		goto out;
 
@@ -1958,7 +2065,7 @@ int bpf_compute_subprog_arg_access(struct bpf_verifier_env *env)
 			err = PTR_ERR(instance);
 			goto out;
 		}
-		err = analyze_subprog(env, NULL, info, instance, callsites);
+		err = analyze_subprog(env, NULL, NULL, info, instance, callsites);
 		if (err)
 			goto out;
 	}
@@ -1967,10 +2074,14 @@ int bpf_compute_subprog_arg_access(struct bpf_verifier_env *env)
 		err = print_instances(env);
 
 out:
-	for (k = 0; k < insn_cnt; k++)
+	for (k = 0; k < insn_cnt; k++) {
 		kvfree(env->callsite_at_stack[k]);
+		kvfree(env->callsite_at_stack_arg[k]);
+	}
 	kvfree(env->callsite_at_stack);
 	env->callsite_at_stack = NULL;
+	kvfree(env->callsite_at_stack_arg);
+	env->callsite_at_stack_arg = NULL;
 	for (k = 0; k < env->subprog_cnt; k++)
 		kvfree(info[k].at_in);
 	kvfree(info);
-- 
2.52.0

[PATCH bpf-next 05/18] bpf: Reject stack arguments in non-JITed programs

[Yonghong Song]

The interpreter does not understand the bpf register r11
(BPF_REG_PARAMS) used for stack arguments. So reject interpreter
usage if stack arguments are used either in the main program or
any subprogram.

Acked-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/core.c   | 2 +-
 kernel/bpf/fixups.c | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index ae10b9ca018d..ec8523e6e4eb 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -2599,7 +2599,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
 		goto finalize;
 
 	if (IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON) ||
-	    bpf_prog_has_kfunc_call(fp))
+	    bpf_prog_has_kfunc_call(fp) || fp->aux->stack_arg_depth)
 		jit_needed = true;
 
 	if (!bpf_prog_select_interpreter(fp))
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 7d276208f3cc..86576f5cfe82 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -1411,6 +1411,12 @@ int bpf_fixup_call_args(struct bpf_verifier_env *env)
 		verbose(env, "calling kernel functions are not allowed in non-JITed programs\n");
 		return -EINVAL;
 	}
+	for (i = 1; i < env->subprog_cnt; i++) {
+		if (env->subprog_info[i].incoming_stack_arg_depth) {
+			verbose(env, "stack args are not supported in non-JITed programs\n");
+			return -EINVAL;
+		}
+	}
 	if (env->subprog_cnt > 1 && env->prog->aux->tail_call_reachable) {
 		/* When JIT fails the progs with bpf2bpf calls and tail_calls
 		 * have to be rejected, since interpreter doesn't support them yet.
-- 
2.52.0

[PATCH bpf-next 06/18] bpf: Prepare architecture JIT support for stack arguments

[Yonghong Song]

Add bpf_jit_supports_stack_args() as a weak function defaulting to
false. Architectures that implement JIT support for stack arguments
override it to return true.

Reject BPF functions and kfuncs with more than 5 parameters at
verification time if the architecture does not support stack
arguments.

Acked-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 include/linux/filter.h | 1 +
 kernel/bpf/btf.c       | 8 +++++++-
 kernel/bpf/core.c      | 5 +++++
 kernel/bpf/verifier.c  | 5 +++++
 4 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/include/linux/filter.h b/include/linux/filter.h
index b77d0b06db6e..911205dd670e 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1163,6 +1163,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);
 bool bpf_jit_supports_subprog_tailcalls(void);
 bool bpf_jit_supports_percpu_insn(void);
 bool bpf_jit_supports_kfunc_call(void);
+bool bpf_jit_supports_stack_args(void);
 bool bpf_jit_supports_far_kfunc_call(void);
 bool bpf_jit_supports_exceptions(void);
 bool bpf_jit_supports_ptr_xchg(void);
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index cfb35a2decf6..fa4b971c5289 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -7890,8 +7890,14 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			tname, nargs, MAX_BPF_FUNC_REG_ARGS);
 		return -EINVAL;
 	}
-	if (nargs > MAX_BPF_FUNC_REG_ARGS)
+	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
+		if (!bpf_jit_supports_stack_args()) {
+			bpf_log(log, "JIT does not support function %s() with %d args\n",
+				tname, nargs);
+			return -ENOTSUPP;
+		}
 		sub->incoming_stack_arg_depth = (nargs - MAX_BPF_FUNC_REG_ARGS) * BPF_REG_SIZE;
+	}
 
 	/* check that function is void or returns int, exception cb also requires this */
 	t = btf_type_by_id(btf, t->type);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index ec8523e6e4eb..7522b3d7b267 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3217,6 +3217,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)
 	return false;
 }
 
+bool __weak bpf_jit_supports_stack_args(void)
+{
+	return false;
+}
+
 bool __weak bpf_jit_supports_far_kfunc_call(void)
 {
 	return false;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index e041c182c614..0ba140dabe93 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -12254,6 +12254,11 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 			MAX_BPF_FUNC_REG_ARGS);
 		return -EINVAL;
 	}
+	if (nargs > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
+		verbose(env, "JIT does not support kfunc %s() with %d args\n",
+			func_name, nargs);
+		return -ENOTSUPP;
+	}
 
 	/* Check that BTF function arguments match actual types that the
 	 * verifier sees.
-- 
2.52.0

[PATCH bpf-next 07/18] bpf: Enable r11 based insns

[Yonghong Song]

BPF_REG_PARAMS (r11) is used for stack argument accesses and
the following are the only insns with r11 presence:
    - BPF_LDX | BPF_MEM | BPF_DW (load incoming stack arg)
    - BPF_STX | BPF_MEM | BPF_DW (store register to outgoing stack arg)
    - BPF_ST  | BPF_MEM | BPF_DW (store immediate to outgoing stack arg)

Additionally, validate offsets: loads must use positive 8-byte aligned
offsets (8, 16, ...) since they access incoming args. Stores must use
negative 8-byte aligned offsets (-8, -16, ...) since they write
outgoing args.

The LLVM compiler [1] implemented the above BPF_REG_PARAMS insns.

  [1] https://github.com/llvm/llvm-project/pull/189060

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 0ba140dabe93..6994536b4e04 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18678,13 +18678,34 @@ static int check_and_resolve_insns(struct bpf_verifier_env *env)
 		return err;
 
 	for (i = 0; i < insn_cnt; i++, insn++) {
+		u8 class = BPF_CLASS(insn->code);
+		u8 mode = BPF_MODE(insn->code);
+		u8 size = BPF_SIZE(insn->code);
+
 		if (insn->dst_reg >= MAX_BPF_REG) {
-			verbose(env, "R%d is invalid\n", insn->dst_reg);
-			return -EINVAL;
+			if (insn->dst_reg != BPF_REG_PARAMS ||
+			    (class != BPF_ST  && class != BPF_STX) ||
+			     mode != BPF_MEM || size != BPF_DW) {
+				verbose(env, "R%d is invalid\n", insn->dst_reg);
+				return -EINVAL;
+			}
+			if (insn->off >= 0 || insn->off % BPF_REG_SIZE) {
+				verbose(env, "invalid stack arg store offset %d\n",
+					insn->off);
+				return -EINVAL;
+			}
 		}
 		if (insn->src_reg >= MAX_BPF_REG) {
-			verbose(env, "R%d is invalid\n", insn->src_reg);
-			return -EINVAL;
+			if (insn->src_reg != BPF_REG_PARAMS ||
+			    insn->code != (BPF_LDX | BPF_MEM | BPF_DW)) {
+				verbose(env, "R%d is invalid\n", insn->src_reg);
+				return -EINVAL;
+			}
+			if (insn->off <= 0 || insn->off % BPF_REG_SIZE) {
+				verbose(env, "invalid stack arg load offset %d\n",
+					insn->off);
+				return -EINVAL;
+			}
 		}
 		if (insn[0].code == (BPF_LD | BPF_IMM | BPF_DW)) {
 			struct bpf_insn_aux_data *aux;
-- 
2.52.0

[PATCH bpf-next 08/18] bpf: Support stack arguments for kfunc calls

[Yonghong Song]

Extend the stack argument mechanism to kfunc calls, allowing kfuncs
with more than 5 parameters to receive additional arguments via the
r11-based stack arg area.

For kfuncs, the caller is a BPF program and the callee is a kernel
function. The BPF program writes outgoing args at negative r11
offsets, following the same convention as BPF-to-BPF calls:

  Outgoing: r11 - 8 (arg6), ..., r11 - N*8 (last arg)

The following is an example:

  int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) {
    ...
    kfunc1(a1, a2, a3, a4, a5, a6, a7, a8);
    ...
    kfunc2(a1, a2, a3, a4, a5, a6, a7, a8, a9);
    ...
  }

   Caller (foo), generated by llvm
   ===============================
   Incoming (positive offsets):
     r11+8:  [incoming arg 6]
     r11+16: [incoming arg 7]

   Outgoing for kfunc1 (negative offsets):
     r11-8:  [outgoing arg 6]
     r11-16: [outgoing arg 7]
     r11-24: [outgoing arg 8]

   Outgoing for kfunc2 (negative offsets):
     r11-8:  [outgoing arg 6]
     r11-16: [outgoing arg 7]
     r11-24: [outgoing arg 8]
     r11-32: [outgoing arg 9]

Later JIT will marshal outgoing arguments to the native calling
convention for kfunc1() and kfunc2().

For kfunc calls where stack args are used as constant or size
parameters, a mark_stack_arg_precision() helper is used to propagate
precision within the current frame.

There are two places where meta->release_regno needs to keep
regno for later releasing the reference. Also, 'cur_aux(env)->arg_prog = regno'
is also keeping regno for later fixup. Since stack arguments don't have a valid
register number (regno is set to -1), these three cases are rejected for now
if the argument is on the stack.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/verifier.c | 73 +++++++++++++++++++++++++++++++++----------
 1 file changed, 56 insertions(+), 17 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 6994536b4e04..43aeb04f488a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -11559,14 +11559,12 @@ bool bpf_is_kfunc_pkt_changing(struct bpf_kfunc_call_arg_meta *meta)
 }
 
 static enum kfunc_ptr_arg_type
-get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,
-		       struct bpf_kfunc_call_arg_meta *meta,
+get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, struct bpf_func_state *caller,
+		       struct bpf_reg_state *regs, struct bpf_kfunc_call_arg_meta *meta,
 		       const struct btf_type *t, const struct btf_type *ref_t,
 		       const char *ref_tname, const struct btf_param *args,
 		       int arg, int nargs, argno_t argno, struct bpf_reg_state *reg)
 {
-	u32 regno = arg + 1;
-	struct bpf_reg_state *regs = cur_regs(env);
 	bool arg_mem_size = false;
 
 	if (meta->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx] ||
@@ -11575,8 +11573,8 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,
 		return KF_ARG_PTR_TO_CTX;
 
 	if (arg + 1 < nargs &&
-	    (is_kfunc_arg_mem_size(meta->btf, &args[arg + 1], &regs[regno + 1]) ||
-	     is_kfunc_arg_const_mem_size(meta->btf, &args[arg + 1], &regs[regno + 1])))
+	    (is_kfunc_arg_mem_size(meta->btf, &args[arg + 1], get_func_arg_reg(caller, regs, arg + 1)) ||
+             is_kfunc_arg_const_mem_size(meta->btf, &args[arg + 1], get_func_arg_reg(caller, regs, arg + 1))))
 		arg_mem_size = true;
 
 	/* In this function, we verify the kfunc's BTF as per the argument type,
@@ -12241,6 +12239,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 			    int insn_idx)
 {
 	const char *func_name = meta->func_name, *ref_tname;
+	struct bpf_func_state *caller = cur_func(env);
+	struct bpf_reg_state *regs = cur_regs(env);
 	const struct btf *btf = meta->btf;
 	const struct btf_param *args;
 	struct btf_record *rec;
@@ -12249,9 +12249,9 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 
 	args = (const struct btf_param *)(meta->func_proto + 1);
 	nargs = btf_type_vlen(meta->func_proto);
-	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
+	if (nargs > MAX_BPF_FUNC_ARGS) {
 		verbose(env, "Function %s has %d > %d args\n", func_name, nargs,
-			MAX_BPF_FUNC_REG_ARGS);
+			MAX_BPF_FUNC_ARGS);
 		return -EINVAL;
 	}
 	if (nargs > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
@@ -12260,15 +12260,20 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 		return -ENOTSUPP;
 	}
 
+	ret = check_outgoing_stack_args(env, caller, nargs);
+	if (ret)
+		return ret;
+
 	/* Check that BTF function arguments match actual types that the
 	 * verifier sees.
 	 */
 	for (i = 0; i < nargs; i++) {
-		struct bpf_reg_state *regs = cur_regs(env), *reg = &regs[i + 1];
+		struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
 		const struct btf_type *t, *ref_t, *resolve_ret;
 		enum bpf_arg_type arg_type = ARG_DONTCARE;
 		argno_t argno = argno_from_arg(i + 1);
-		u32 regno = i + 1, ref_id, type_size;
+		int regno = reg_from_argno(argno);
+		u32 ref_id, type_size;
 		bool is_ret_buf_sz = false;
 		int kf_arg_type;
 
@@ -12278,6 +12283,11 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				verifier_bug(env, "Only 1 prog->aux argument supported per-kfunc");
 				return -EFAULT;
 			}
+			if (regno < 0) {
+				verbose(env, "%s prog->aux cannot be a stack argument\n",
+					reg_arg_name(env, argno));
+				return -EINVAL;
+			}
 			meta->arg_prog = true;
 			cur_aux(env)->arg_prog = regno;
 			continue;
@@ -12304,7 +12314,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 						reg_arg_name(env, argno));
 					return -EINVAL;
 				}
-				ret = mark_chain_precision(env, regno);
+				if (regno >= 0)
+					ret = mark_chain_precision(env, regno);
+				else
+					ret = mark_stack_arg_precision(env, i);
 				if (ret < 0)
 					return ret;
 				meta->arg_constant.found = true;
@@ -12329,7 +12342,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				}
 
 				meta->r0_size = reg->var_off.value;
-				ret = mark_chain_precision(env, regno);
+				if (regno >= 0)
+					ret = mark_chain_precision(env, regno);
+				else
+					ret = mark_stack_arg_precision(env, i);
 				if (ret)
 					return ret;
 			}
@@ -12357,14 +12373,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				return -EFAULT;
 			}
 			meta->ref_obj_id = reg->ref_obj_id;
-			if (is_kfunc_release(meta))
+			if (is_kfunc_release(meta)) {
+				if (regno < 0) {
+					verbose(env, "%s release arg cannot be a stack argument\n",
+						reg_arg_name(env, argno));
+					return -EINVAL;
+				}
 				meta->release_regno = regno;
+			}
 		}
 
 		ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id);
 		ref_tname = btf_name_by_offset(btf, ref_t->name_off);
 
-		kf_arg_type = get_kfunc_ptr_arg_type(env, meta, t, ref_t, ref_tname, args, i, nargs, argno, reg);
+		kf_arg_type = get_kfunc_ptr_arg_type(env, caller, regs, meta, t, ref_t, ref_tname,
+						     args, i, nargs, argno, reg);
 		if (kf_arg_type < 0)
 			return kf_arg_type;
 
@@ -12514,6 +12537,11 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				dynptr_arg_type |= DYNPTR_TYPE_FILE;
 			} else if (meta->func_id == special_kfunc_list[KF_bpf_dynptr_file_discard]) {
 				dynptr_arg_type |= DYNPTR_TYPE_FILE | OBJ_RELEASE;
+				if (regno < 0) {
+					verbose(env, "%s release arg cannot be a stack argument\n",
+						reg_arg_name(env, argno));
+					return -EINVAL;
+				}
 				meta->release_regno = regno;
 			} else if (meta->func_id == special_kfunc_list[KF_bpf_dynptr_clone] &&
 				   (dynptr_arg_type & MEM_UNINIT)) {
@@ -12668,9 +12696,9 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 			break;
 		case KF_ARG_PTR_TO_MEM_SIZE:
 		{
-			struct bpf_reg_state *buff_reg = &regs[regno];
+			struct bpf_reg_state *buff_reg = reg;
 			const struct btf_param *buff_arg = &args[i];
-			struct bpf_reg_state *size_reg = &regs[regno + 1];
+			struct bpf_reg_state *size_reg = get_func_arg_reg(caller, regs, i + 1);
 			const struct btf_param *size_arg = &args[i + 1];
 			argno_t next_argno = argno_from_arg(i + 2);
 
@@ -13574,8 +13602,19 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		clear_all_pkt_pointers(env);
 
 	nargs = btf_type_vlen(meta.func_proto);
+	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
+		struct bpf_func_state *caller = cur_func(env);
+		struct bpf_subprog_info *caller_info = &env->subprog_info[caller->subprogno];
+		u16 out_stack_arg_depth = (nargs - MAX_BPF_FUNC_REG_ARGS) * BPF_REG_SIZE;
+		u16 stack_arg_depth = caller_info->incoming_stack_arg_depth + out_stack_arg_depth;
+
+		if (stack_arg_depth > caller_info->stack_arg_depth)
+			caller_info->stack_arg_depth = stack_arg_depth;
+		invalidate_outgoing_stack_args(caller);
+	}
+
 	args = (const struct btf_param *)(meta.func_proto + 1);
-	for (i = 0; i < nargs; i++) {
+	for (i = 0; i < min_t(int, nargs, MAX_BPF_FUNC_REG_ARGS); i++) {
 		u32 regno = i + 1;
 
 		t = btf_type_skip_modifiers(desc_btf, args[i].type, NULL);
-- 
2.52.0

[PATCH bpf-next 09/18] bpf: Reject stack arguments if tail call reachable

[Yonghong Song]

Tailcalls have been deprecated. So reject stack arguments
if tail call is in the way.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/verifier.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 43aeb04f488a..3a15c5c19db0 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5487,6 +5487,11 @@ struct bpf_subprog_call_depth_info {
 	int frame; /* # of consecutive static call stack frames on top of stack */
 };
 
+static bool subprog_has_stack_args(const struct bpf_subprog_info *si)
+{
+	return si->stack_arg_depth;
+}
+
 /* starting from main bpf function walk all instructions of the function
  * and recursively walk all callees that given function can call.
  * Ignore jump and exit insns.
@@ -5640,14 +5645,23 @@ static int check_max_stack_depth_subprog(struct bpf_verifier_env *env, int idx,
 	 * this info will be utilized by JIT so that we will be preserving the
 	 * tail call counter throughout bpf2bpf calls combined with tailcalls
 	 */
-	if (tail_call_reachable)
+	if (tail_call_reachable) {
 		for (tmp = idx; tmp >= 0; tmp = dinfo[tmp].caller) {
 			if (subprog[tmp].is_exception_cb) {
 				verbose(env, "cannot tail call within exception cb\n");
 				return -EINVAL;
 			}
+			if (subprog_has_stack_args(&subprog[tmp])) {
+				verbose(env, "tail_calls are not allowed in programs with stack args\n");
+				return -EINVAL;
+			}
 			subprog[tmp].tail_call_reachable = true;
 		}
+	} else if (!idx && subprog[0].has_tail_call && subprog_has_stack_args(&subprog[0])) {
+		verbose(env, "tail_calls are not allowed in programs with stack args\n");
+		return -EINVAL;
+	}
+
 	if (subprog[0].tail_call_reachable)
 		env->prog->aux->tail_call_reachable = true;
 
-- 
2.52.0

[PATCH bpf-next 10/18] bpf,x86: Implement JIT support for stack arguments

[Yonghong Song]

Add x86_64 JIT support for BPF functions and kfuncs with more than
5 arguments. The extra arguments are passed through a stack area
addressed by register r11 (BPF_REG_PARAMS) in BPF bytecode,
which the JIT translates to native code.

The JIT follows the x86-64 calling convention for both BPF-to-BPF
and kfunc calls:
  - Arg 6 is passed in the R9 register
  - Args 7+ are passed on the stack

Incoming arg 6 (BPF r11+8) is translated to a MOV from R9 rather
than a memory load. Incoming args 7+ (BPF r11+16, r11+24, ...) map
directly to [rbp + 16], [rbp + 24], ..., matching the x86-64 stack
layout after CALL + PUSH RBP, so no offset adjustment is needed.

The verifier guarantees that neither tail_call_reachable nor
priv_stack is set when stack args exist, so R9 is always
available. When BPF bytecode writes to the arg-6 stack slot
(offset -8), the JIT emits a MOV into R9 instead of a memory store.
Outgoing args 7+ are placed at [rsp] in a pre-allocated area below
callee-saved registers, using:
  native_off = outgoing_arg_base - outgoing_rsp - bpf_off - 16

The native x86_64 stack layout with stack arguments:

  high address
  +-------------------------+
  | incoming stack arg N    |  [rbp + 16 + (N-7)*8]  (from caller)
  | ...                     |
  | incoming stack arg 7    |  [rbp + 16]
  +-------------------------+
  | return address          |  [rbp + 8]
  | saved rbp               |  [rbp]
  +-------------------------+
  | BPF program stack       |  (round_up(stack_depth, 8) bytes)
  +-------------------------+
  | callee-saved regs       |  (r12, rbx, r13, r14, r15 as needed)
  +-------------------------+
  | outgoing arg M          |  [rsp + (M-7)*8]
  | ...                     |
  | outgoing arg 7          |  [rsp]
  +-------------------------+  rsp
  low address

Acked-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 arch/x86/net/bpf_jit_comp.c | 154 ++++++++++++++++++++++++++++++++++--
 1 file changed, 148 insertions(+), 6 deletions(-)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index ea9e707e8abf..06f33fef1e8e 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -390,6 +390,34 @@ static void pop_callee_regs(u8 **pprog, bool *callee_regs_used)
 	*pprog = prog;
 }
 
+/* add rsp, depth */
+static void emit_add_rsp(u8 **pprog, u16 depth)
+{
+	u8 *prog = *pprog;
+
+	if (!depth)
+		return;
+	if (is_imm8(depth))
+		EMIT4(0x48, 0x83, 0xC4, depth); /* add rsp, imm8 */
+	else
+		EMIT3_off32(0x48, 0x81, 0xC4, depth); /* add rsp, imm32 */
+	*pprog = prog;
+}
+
+/* sub rsp, depth */
+static void emit_sub_rsp(u8 **pprog, u16 depth)
+{
+	u8 *prog = *pprog;
+
+	if (!depth)
+		return;
+	if (is_imm8(depth))
+		EMIT4(0x48, 0x83, 0xEC, depth); /* sub rsp, imm8 */
+	else
+		EMIT3_off32(0x48, 0x81, 0xEC, depth); /* sub rsp, imm32 */
+	*pprog = prog;
+}
+
 static void emit_nops(u8 **pprog, int len)
 {
 	u8 *prog = *pprog;
@@ -1664,16 +1692,45 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 	int i, excnt = 0;
 	int ilen, proglen = 0;
 	u8 *ip, *prog = temp;
+	u16 stack_arg_depth, incoming_stack_arg_depth, outgoing_stack_arg_depth;
+	u16 outgoing_rsp;
 	u32 stack_depth;
+	int callee_saved_size;
+	s32 outgoing_arg_base;
 	int err;
 
 	stack_depth = bpf_prog->aux->stack_depth;
+	stack_arg_depth = bpf_prog->aux->stack_arg_depth;
+	incoming_stack_arg_depth = bpf_prog->aux->incoming_stack_arg_depth;
+	outgoing_stack_arg_depth = stack_arg_depth - incoming_stack_arg_depth;
 	priv_stack_ptr = bpf_prog->aux->priv_stack_ptr;
 	if (priv_stack_ptr) {
 		priv_frame_ptr = priv_stack_ptr + PRIV_STACK_GUARD_SZ + round_up(stack_depth, 8);
 		stack_depth = 0;
 	}
 
+	/*
+	 * Follow x86-64 calling convention for both BPF-to-BPF and
+	 * kfunc calls:
+	 *   - Arg 6 is passed in R9 register
+	 *   - Args 7+ are passed on the stack at [rsp]
+	 *
+	 * Incoming arg 6 is read from R9 (BPF r11+8 → MOV from R9).
+	 * Incoming args 7+ are read from [rbp + 16], [rbp + 24], ...
+	 * (BPF r11+16, r11+24, ... map directly with no offset change).
+	 *
+	 * The verifier guarantees that neither tail_call_reachable nor
+	 * priv_stack is set when outgoing stack args exist, so R9 is
+	 * always available.
+	 *
+	 * Stack layout (high to low):
+	 *   [rbp + 16 + ...]    incoming stack args 7+ (from caller)
+	 *   [rbp + 8]           return address
+	 *   [rbp]               saved rbp
+	 *   [rbp - prog_stack]  program stack
+	 *   [below]             callee-saved regs
+	 *   [below]             outgoing args 7+ (= rsp)
+	 */
 	arena_vm_start = bpf_arena_get_kern_vm_start(bpf_prog->aux->arena);
 	user_vm_start = bpf_arena_get_user_vm_start(bpf_prog->aux->arena);
 
@@ -1700,6 +1757,42 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 			push_r12(&prog);
 		push_callee_regs(&prog, callee_regs_used);
 	}
+
+	/* Compute callee-saved register area size. */
+	callee_saved_size = 0;
+	if (bpf_prog->aux->exception_boundary || arena_vm_start)
+		callee_saved_size += 8; /* r12 */
+	if (bpf_prog->aux->exception_boundary) {
+		callee_saved_size += 4 * 8; /* rbx, r13, r14, r15 */
+	} else {
+		int j;
+
+		for (j = 0; j < 4; j++)
+			if (callee_regs_used[j])
+				callee_saved_size += 8;
+	}
+	/*
+	 * Base offset from rbp for translating BPF outgoing args 7+
+	 * to native offsets. BPF uses negative offsets from r11
+	 * (r11-8 for arg6, r11-16 for arg7, ...) while x86 uses
+	 * positive offsets from rsp ([rsp+0] for arg7, [rsp+8] for
+	 * arg8, ...). Arg 6 goes to R9 directly.
+	 *
+	 * The translation reverses direction:
+	 *   native_off = outgoing_arg_base - outgoing_rsp - bpf_off - 16
+	 *
+	 * Note that tail_call_reachable is guaranteed to be false when
+	 * stack args exist, so tcc pushes need not be accounted for.
+	 */
+	outgoing_arg_base = -(round_up(stack_depth, 8) + callee_saved_size);
+
+	/*
+	 * Allocate outgoing stack arg area for args 7+ only.
+	 * Arg 6 goes into r9 register, not on stack.
+	 */
+	outgoing_rsp = outgoing_stack_arg_depth > 8 ?  outgoing_stack_arg_depth - 8 : 0;
+	emit_sub_rsp(&prog, outgoing_rsp);
+
 	if (arena_vm_start)
 		emit_mov_imm64(&prog, X86_REG_R12,
 			       arena_vm_start >> 32, (u32) arena_vm_start);
@@ -1721,7 +1814,7 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 		u8 b2 = 0, b3 = 0;
 		u8 *start_of_ldx;
 		s64 jmp_offset;
-		s16 insn_off;
+		s32 insn_off;
 		u8 jmp_cond;
 		u8 *func;
 		int nops;
@@ -2134,12 +2227,26 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 				EMIT1(0xC7);
 			goto st;
 		case BPF_ST | BPF_MEM | BPF_DW:
+			if (dst_reg == BPF_REG_PARAMS && insn->off == -8) {
+				/* Arg 6: store immediate in r9 register */
+				emit_mov_imm64(&prog, X86_REG_R9, imm32 >> 31, (u32)imm32);
+				break;
+			}
 			EMIT2(add_1mod(0x48, dst_reg), 0xC7);
 
-st:			if (is_imm8(insn->off))
-				EMIT2(add_1reg(0x40, dst_reg), insn->off);
+st:			insn_off = insn->off;
+			if (dst_reg == BPF_REG_PARAMS) {
+				/* Args 7+: reverse BPF negative offsets to
+				 * x86 positive rsp offsets.
+				 * BPF off=-16 → [rsp+0], off=-24 → [rsp+8], ...
+				 */
+				insn_off = outgoing_arg_base - outgoing_rsp - insn_off - 16;
+				dst_reg = BPF_REG_FP;
+			}
+			if (is_imm8(insn_off))
+				EMIT2(add_1reg(0x40, dst_reg), insn_off);
 			else
-				EMIT1_off32(add_1reg(0x80, dst_reg), insn->off);
+				EMIT1_off32(add_1reg(0x80, dst_reg), insn_off);
 
 			EMIT(imm32, bpf_size_to_x86_bytes(BPF_SIZE(insn->code)));
 			break;
@@ -2149,7 +2256,17 @@ st:			if (is_imm8(insn->off))
 		case BPF_STX | BPF_MEM | BPF_H:
 		case BPF_STX | BPF_MEM | BPF_W:
 		case BPF_STX | BPF_MEM | BPF_DW:
-			emit_stx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
+			if (dst_reg == BPF_REG_PARAMS && insn->off == -8) {
+				/* Arg 6: store register value in r9 */
+				EMIT_mov(X86_REG_R9, src_reg);
+				break;
+			}
+			insn_off = insn->off;
+			if (dst_reg == BPF_REG_PARAMS) {
+				insn_off = outgoing_arg_base - outgoing_rsp - insn_off - 16;
+				dst_reg = BPF_REG_FP;
+			}
+			emit_stx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
 			break;
 
 		case BPF_ST | BPF_PROBE_MEM32 | BPF_B:
@@ -2248,6 +2365,19 @@ st:			if (is_imm8(insn->off))
 		case BPF_LDX | BPF_PROBE_MEMSX | BPF_H:
 		case BPF_LDX | BPF_PROBE_MEMSX | BPF_W:
 			insn_off = insn->off;
+			if (src_reg == BPF_REG_PARAMS) {
+				if (insn_off == 8) {
+					/* Incoming arg 6: read from r9 */
+					EMIT_mov(dst_reg, X86_REG_R9);
+					break;
+				}
+				src_reg = BPF_REG_FP;
+				/*
+				 * Incoming args 7+: native_off == bpf_off
+				 * (r11+16 → [rbp+16], r11+24 → [rbp+24], ...)
+				 * No offset adjustment needed.
+				 */
+			}
 
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM ||
 			    BPF_MODE(insn->code) == BPF_PROBE_MEMSX) {
@@ -2736,6 +2866,8 @@ st:			if (is_imm8(insn->off))
 				if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
 					return -EINVAL;
 			}
+			/* Deallocate outgoing args 7+ area. */
+			emit_add_rsp(&prog, outgoing_rsp);
 			if (bpf_prog->aux->exception_boundary) {
 				pop_callee_regs(&prog, all_callee_regs_used);
 				pop_r12(&prog);
@@ -3743,7 +3875,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
 		prog->aux->jit_data = jit_data;
 	}
 	priv_stack_ptr = prog->aux->priv_stack_ptr;
-	if (!priv_stack_ptr && prog->aux->jits_use_priv_stack) {
+	/*
+	 * x86-64 uses R9 for both private stack frame pointer and arg 6,
+	 * so disable private stack when stack args are present.
+	 */
+	if (!priv_stack_ptr && prog->aux->jits_use_priv_stack &&
+	    prog->aux->stack_arg_depth == 0) {
 		/* Allocate actual private stack size with verifier-calculated
 		 * stack size plus two memory guards to protect overflow and
 		 * underflow.
@@ -3910,6 +4047,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_stack_args(void)
+{
+	return true;
+}
+
 void *bpf_arch_text_copy(void *dst, void *src, size_t len)
 {
 	if (text_poke_copy(dst, src, len) == NULL)
-- 
2.52.0

[PATCH bpf-next 11/18] selftests/bpf: Add tests for BPF function stack arguments

[Yonghong Song]

Add selftests covering stack argument passing for both BPF-to-BPF
subprog calls and kfunc calls with more than 5 arguments. All tests
are guarded by __BPF_FEATURE_STACK_ARGUMENT and __TARGET_ARCH_x86.

BPF-to-BPF subprog call tests (stack_arg.c):
  - Scalar stack args
  - Pointer stack args
  - Mixed pointer/scalar stack args
  - Nested calls
  - Dynptr stack arg
  - Two callees with different stack arg counts
  - Async callback

Kfunc call tests (stack_arg_kfunc.c, with bpf_testmod kfuncs):
  - Scalar stack args
  - Pointer stack args
  - Mixed pointer/scalar stack args
  - Dynptr stack arg
  - Memory buffer + size pair
  - Iterator
  - Const string pointer
  - Timer pointer

Acked-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 .../selftests/bpf/prog_tests/stack_arg.c      | 139 ++++++++++
 tools/testing/selftests/bpf/progs/stack_arg.c | 252 ++++++++++++++++++
 .../selftests/bpf/progs/stack_arg_kfunc.c     | 163 +++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  65 +++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  20 +-
 5 files changed, 638 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/stack_arg.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg_kfunc.c

diff --git a/tools/testing/selftests/bpf/prog_tests/stack_arg.c b/tools/testing/selftests/bpf/prog_tests/stack_arg.c
new file mode 100644
index 000000000000..d61bac33f809
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/stack_arg.c
@@ -0,0 +1,139 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <test_progs.h>
+#include <network_helpers.h>
+#include "stack_arg.skel.h"
+#include "stack_arg_kfunc.skel.h"
+
+static void run_subtest(struct bpf_program *prog, int expected)
+{
+	int err, prog_fd;
+	LIBBPF_OPTS(bpf_test_run_opts, topts,
+		.data_in = &pkt_v4,
+		.data_size_in = sizeof(pkt_v4),
+		.repeat = 1,
+	);
+
+	prog_fd = bpf_program__fd(prog);
+	err = bpf_prog_test_run_opts(prog_fd, &topts);
+	ASSERT_OK(err, "test_run");
+	ASSERT_EQ(topts.retval, expected, "retval");
+}
+
+static void test_global_many(void)
+{
+	struct stack_arg *skel;
+
+	skel = stack_arg__open();
+	if (!ASSERT_OK_PTR(skel, "open"))
+		return;
+
+	if (!skel->rodata->has_stack_arg) {
+		test__skip();
+		goto out;
+	}
+
+	if (!ASSERT_OK(stack_arg__load(skel), "load"))
+		goto out;
+
+	run_subtest(skel->progs.test_global_many_args, 36);
+
+out:
+	stack_arg__destroy(skel);
+}
+
+static void test_async_cb_many(void)
+{
+	struct stack_arg *skel;
+
+	skel = stack_arg__open();
+	if (!ASSERT_OK_PTR(skel, "open"))
+		return;
+
+	if (!skel->rodata->has_stack_arg) {
+		test__skip();
+		goto out;
+	}
+
+	if (!ASSERT_OK(stack_arg__load(skel), "load"))
+		goto out;
+
+	run_subtest(skel->progs.test_async_cb_many_args, 0);
+
+	/* Wait for the timer callback to fire and verify the result.
+	 * 10+20+30+40+50+60+70+80 = 360
+	 */
+	usleep(50);
+	ASSERT_EQ(skel->bss->timer_result, 360, "timer_result");
+
+out:
+	stack_arg__destroy(skel);
+}
+
+static void test_bpf2bpf(void)
+{
+	struct stack_arg *skel;
+
+	skel = stack_arg__open();
+	if (!ASSERT_OK_PTR(skel, "open"))
+		return;
+
+	if (!skel->rodata->has_stack_arg) {
+		test__skip();
+		goto out;
+	}
+
+	if (!ASSERT_OK(stack_arg__load(skel), "load"))
+		goto out;
+
+	run_subtest(skel->progs.test_bpf2bpf_ptr_stack_arg, 45);
+	run_subtest(skel->progs.test_bpf2bpf_mix_stack_args, 51);
+	run_subtest(skel->progs.test_bpf2bpf_nesting_stack_arg, 50);
+	run_subtest(skel->progs.test_bpf2bpf_dynptr_stack_arg, 69);
+	run_subtest(skel->progs.test_two_callees, 91);
+
+out:
+	stack_arg__destroy(skel);
+}
+
+static void test_kfunc(void)
+{
+	struct stack_arg_kfunc *skel;
+
+	skel = stack_arg_kfunc__open();
+	if (!ASSERT_OK_PTR(skel, "open"))
+		return;
+
+	if (!skel->rodata->has_stack_arg) {
+		test__skip();
+		goto out;
+	}
+
+	if (!ASSERT_OK(stack_arg_kfunc__load(skel), "load"))
+		goto out;
+
+	run_subtest(skel->progs.test_stack_arg_scalar, 36);
+	run_subtest(skel->progs.test_stack_arg_ptr, 45);
+	run_subtest(skel->progs.test_stack_arg_mix, 51);
+	run_subtest(skel->progs.test_stack_arg_dynptr, 69);
+	run_subtest(skel->progs.test_stack_arg_mem, 151);
+	run_subtest(skel->progs.test_stack_arg_iter, 115);
+	run_subtest(skel->progs.test_stack_arg_const_str, 15);
+	run_subtest(skel->progs.test_stack_arg_timer, 15);
+
+out:
+	stack_arg_kfunc__destroy(skel);
+}
+
+void test_stack_arg(void)
+{
+	if (test__start_subtest("global_many_args"))
+		test_global_many();
+	if (test__start_subtest("async_cb_many_args"))
+		test_async_cb_many();
+	if (test__start_subtest("bpf2bpf"))
+		test_bpf2bpf();
+	if (test__start_subtest("kfunc"))
+		test_kfunc();
+}
diff --git a/tools/testing/selftests/bpf/progs/stack_arg.c b/tools/testing/selftests/bpf/progs/stack_arg.c
new file mode 100644
index 000000000000..ab6240b997c5
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/stack_arg.c
@@ -0,0 +1,252 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <vmlinux.h>
+#include <stdbool.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_kfuncs.h"
+
+#define CLOCK_MONOTONIC 1
+
+struct timer_elem {
+	struct bpf_timer timer;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__uint(max_entries, 1);
+	__type(key, int);
+	__type(value, struct timer_elem);
+} timer_map SEC(".maps");
+
+int timer_result;
+
+#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+const volatile bool has_stack_arg = true;
+
+__noinline static int static_func_many_args(int a, int b, int c, int d,
+					    int e, int f, int g, int h)
+{
+	return a + b + c + d + e + f + g + h;
+}
+
+__noinline int global_calls_many_args(int a, int b, int c)
+{
+	return static_func_many_args(a, b, c, 4, 5, 6, 7, 8);
+}
+
+SEC("tc")
+int test_global_many_args(void)
+{
+	return global_calls_many_args(1, 2, 3);
+}
+
+struct test_data {
+	long x;
+	long y;
+};
+
+/* 1 + 2 + 3 + 4 + 5 + 10 + 20 = 45 */
+__noinline static long func_with_ptr_stack_arg(long a, long b, long c, long d,
+					       long e, struct test_data *p)
+{
+	return a + b + c + d + e + p->x + p->y;
+}
+
+__noinline long global_ptr_stack_arg(long a, long b, long c, long d, long e)
+{
+	struct test_data data = { .x = 10, .y = 20 };
+
+	return func_with_ptr_stack_arg(a, b, c, d, e, &data);
+}
+
+SEC("tc")
+int test_bpf2bpf_ptr_stack_arg(void)
+{
+	return global_ptr_stack_arg(1, 2, 3, 4, 5);
+}
+
+/* 1 + 2 + 3 + 4 + 5 + 10 + 6 + 20 = 51 */
+__noinline static long func_with_mix_stack_args(long a, long b, long c, long d,
+						long e, struct test_data *p,
+						long f, struct test_data *q)
+{
+	return a + b + c + d + e + p->x + f + q->y;
+}
+
+__noinline long global_mix_stack_args(long a, long b, long c, long d, long e)
+{
+	struct test_data p = { .x = 10 };
+	struct test_data q = { .y = 20 };
+
+	return func_with_mix_stack_args(a, b, c, d, e, &p, e + 1, &q);
+}
+
+SEC("tc")
+int test_bpf2bpf_mix_stack_args(void)
+{
+	return global_mix_stack_args(1, 2, 3, 4, 5);
+}
+
+/*
+ * Nesting test: func_outer calls func_inner, both with struct pointer
+ * as stack arg.
+ *
+ * func_inner: (a+1) + (b+1) + (c+1) + (d+1) + (e+1) + p->x + p->y
+ *           = 2 + 3 + 4 + 5 + 6 + 10 + 20 = 50
+ */
+__noinline static long func_inner_ptr(long a, long b, long c, long d,
+				      long e, struct test_data *p)
+{
+	return a + b + c + d + e + p->x + p->y;
+}
+
+__noinline static long func_outer_ptr(long a, long b, long c, long d,
+				      long e, struct test_data *p)
+{
+	return func_inner_ptr(a + 1, b + 1, c + 1, d + 1, e + 1, p);
+}
+
+__noinline long global_nesting_ptr(long a, long b, long c, long d, long e)
+{
+	struct test_data data = { .x = 10, .y = 20 };
+
+	return func_outer_ptr(a, b, c, d, e, &data);
+}
+
+SEC("tc")
+int test_bpf2bpf_nesting_stack_arg(void)
+{
+	return global_nesting_ptr(1, 2, 3, 4, 5);
+}
+
+/* 1 + 2 + 3 + 4 + 5 + sizeof(pkt_v4) = 15 + 54 = 69 */
+__noinline static long func_with_dynptr(long a, long b, long c, long d,
+					long e, struct bpf_dynptr *ptr)
+{
+	return a + b + c + d + e + bpf_dynptr_size(ptr);
+}
+
+__noinline long global_dynptr_stack_arg(void *ctx __arg_ctx, long a, long b,
+					long c, long d)
+{
+	struct bpf_dynptr ptr;
+
+	bpf_dynptr_from_skb(ctx, 0, &ptr);
+	return func_with_dynptr(a, b, c, d, d + 1, &ptr);
+}
+
+SEC("tc")
+int test_bpf2bpf_dynptr_stack_arg(struct __sk_buff *skb)
+{
+	return global_dynptr_stack_arg(skb, 1, 2, 3, 4);
+}
+
+/* foo1: a+b+c+d+e+f+g+h */
+__noinline static int foo1(int a, int b, int c, int d,
+			   int e, int f, int g, int h)
+{
+	return a + b + c + d + e + f + g + h;
+}
+
+/* foo2: a+b+c+d+e+f+g+h+i+j */
+__noinline static int foo2(int a, int b, int c, int d, int e,
+			   int f, int g, int h, int i, int j)
+{
+	return a + b + c + d + e + f + g + h + i + j;
+}
+
+/* global_two_callees calls foo1 (3 stack args) and foo2 (5 stack args).
+ * The outgoing stack arg area is sized for foo2 (the larger callee).
+ * Stores for foo1 are a subset of the area used by foo2.
+ * Result: foo1(1,2,3,4,5,6,7,8) + foo2(1,2,3,4,5,6,7,8,9,10) = 36 + 55 = 91
+ *
+ * Pass a-e through so the compiler can't constant-fold the stack args away.
+ */
+__noinline int global_two_callees(int a, int b, int c, int d, int e)
+{
+	int ret;
+
+	ret = foo1(a, b, c, d, e, a + 5, a + 6, a + 7);
+	ret += foo2(a, b, c, d, e, a + 5, a + 6, a + 7, a + 8, a + 9);
+	return ret;
+}
+
+SEC("tc")
+int test_two_callees(void)
+{
+	return global_two_callees(1, 2, 3, 4, 5);
+}
+
+static int timer_cb_many_args(void *map, int *key, struct bpf_timer *timer)
+{
+	timer_result = static_func_many_args(10, 20, 30, 40, 50, 60, 70, 80);
+	return 0;
+}
+
+SEC("tc")
+int test_async_cb_many_args(void)
+{
+	struct timer_elem *elem;
+	int key = 0;
+
+	elem = bpf_map_lookup_elem(&timer_map, &key);
+	if (!elem)
+		return -1;
+
+	bpf_timer_init(&elem->timer, &timer_map, CLOCK_MONOTONIC);
+	bpf_timer_set_callback(&elem->timer, timer_cb_many_args);
+	bpf_timer_start(&elem->timer, 1, 0);
+	return 0;
+}
+
+#else
+
+const volatile bool has_stack_arg = false;
+
+SEC("tc")
+int test_global_many_args(void)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_bpf2bpf_ptr_stack_arg(void)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_bpf2bpf_mix_stack_args(void)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_bpf2bpf_nesting_stack_arg(void)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_bpf2bpf_dynptr_stack_arg(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_two_callees(void)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_async_cb_many_args(void)
+{
+	return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c b/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c
new file mode 100644
index 000000000000..fa9def876ea5
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c
@@ -0,0 +1,163 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_kfuncs.h"
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+const volatile bool has_stack_arg = true;
+
+struct bpf_iter_testmod_seq {
+	u64 :64;
+	u64 :64;
+};
+
+extern int bpf_iter_testmod_seq_new(struct bpf_iter_testmod_seq *it, s64 value, int cnt) __ksym;
+extern void bpf_iter_testmod_seq_destroy(struct bpf_iter_testmod_seq *it) __ksym;
+
+struct timer_map_value {
+	struct bpf_timer timer;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__uint(max_entries, 1);
+	__type(key, int);
+	__type(value, struct timer_map_value);
+} kfunc_timer_map SEC(".maps");
+
+SEC("tc")
+int test_stack_arg_scalar(struct __sk_buff *skb)
+{
+	return bpf_kfunc_call_stack_arg(1, 2, 3, 4, 5, 6, 7, 8);
+}
+
+SEC("tc")
+int test_stack_arg_ptr(struct __sk_buff *skb)
+{
+	struct prog_test_pass1 p = { .x0 = 10, .x1 = 20 };
+
+	return bpf_kfunc_call_stack_arg_ptr(1, 2, 3, 4, 5, &p);
+}
+
+SEC("tc")
+int test_stack_arg_mix(struct __sk_buff *skb)
+{
+	struct prog_test_pass1 p = { .x0 = 10 };
+	struct prog_test_pass1 q = { .x1 = 20 };
+
+	return bpf_kfunc_call_stack_arg_mix(1, 2, 3, 4, 5, &p, 6, &q);
+}
+
+/* 1 + 2 + 3 + 4 + 5 + sizeof(pkt_v4) = 15 + 54 = 69 */
+SEC("tc")
+int test_stack_arg_dynptr(struct __sk_buff *skb)
+{
+	struct bpf_dynptr ptr;
+
+	bpf_dynptr_from_skb(skb, 0, &ptr);
+	return bpf_kfunc_call_stack_arg_dynptr(1, 2, 3, 4, 5, &ptr);
+}
+
+/* 1 + 2 + 3 + 4 + 5 + (1 + 2 + ... + 16) = 15 + 136 = 151 */
+SEC("tc")
+int test_stack_arg_mem(struct __sk_buff *skb)
+{
+	char buf[16] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16};
+
+	return bpf_kfunc_call_stack_arg_mem(1, 2, 3, 4, 5, buf, sizeof(buf));
+}
+
+/* 1 + 2 + 3 + 4 + 5 + 100 = 115 */
+SEC("tc")
+int test_stack_arg_iter(struct __sk_buff *skb)
+{
+	struct bpf_iter_testmod_seq it;
+	u64 ret;
+
+	bpf_iter_testmod_seq_new(&it, 100, 10);
+	ret = bpf_kfunc_call_stack_arg_iter(1, 2, 3, 4, 5, &it);
+	bpf_iter_testmod_seq_destroy(&it);
+	return ret;
+}
+
+const char cstr[] = "hello";
+
+/* 1 + 2 + 3 + 4 + 5 = 15 */
+SEC("tc")
+int test_stack_arg_const_str(struct __sk_buff *skb)
+{
+	return bpf_kfunc_call_stack_arg_const_str(1, 2, 3, 4, 5, cstr);
+}
+
+/* 1 + 2 + 3 + 4 + 5 = 15 */
+SEC("tc")
+int test_stack_arg_timer(struct __sk_buff *skb)
+{
+	struct timer_map_value *val;
+	int key = 0;
+
+	val = bpf_map_lookup_elem(&kfunc_timer_map, &key);
+	if (!val)
+		return 0;
+	return bpf_kfunc_call_stack_arg_timer(1, 2, 3, 4, 5, &val->timer);
+}
+
+#else
+
+const volatile bool has_stack_arg = false;
+
+SEC("tc")
+int test_stack_arg_scalar(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_ptr(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_mix(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_dynptr(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_mem(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_iter(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_const_str(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+SEC("tc")
+int test_stack_arg_timer(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index d876314a4d67..aef2f68b7e83 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -825,6 +825,63 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)
 	return 0;
 }
 
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
+					 u64 e, u64 f, u64 g, u64 h)
+{
+	return a + b + c + d + e + f + g + h;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_ptr(u64 a, u64 b, u64 c, u64 d, u64 e,
+					     struct prog_test_pass1 *p)
+{
+	return a + b + c + d + e + p->x0 + p->x1;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_mix(u64 a, u64 b, u64 c, u64 d, u64 e,
+					     struct prog_test_pass1 *p, u64 f,
+					     struct prog_test_pass1 *q)
+{
+	return a + b + c + d + e + p->x0 + f + q->x1;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_dynptr(u64 a, u64 b, u64 c, u64 d, u64 e,
+					       struct bpf_dynptr *ptr)
+{
+	const struct bpf_dynptr_kern *kern_ptr = (void *)ptr;
+
+	return a + b + c + d + e + (kern_ptr->size & 0xFFFFFF);
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_mem(u64 a, u64 b, u64 c, u64 d, u64 e,
+					     void *mem, int mem__sz)
+{
+	const unsigned char *p = mem;
+	u64 sum = a + b + c + d + e;
+	int i;
+
+	for (i = 0; i < mem__sz; i++)
+		sum += p[i];
+	return sum;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_iter(u64 a, u64 b, u64 c, u64 d, u64 e,
+					      struct bpf_iter_testmod_seq *it__iter)
+{
+	return a + b + c + d + e + it__iter->value;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_const_str(u64 a, u64 b, u64 c, u64 d, u64 e,
+						   const char *str__str)
+{
+	return a + b + c + d + e;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_timer(u64 a, u64 b, u64 c, u64 d, u64 e,
+					       struct bpf_timer *timer)
+{
+	return a + b + c + d + e;
+}
+
 static struct prog_test_ref_kfunc prog_test_struct = {
 	.a = 42,
 	.b = 108,
@@ -1288,6 +1345,14 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test4)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_dynptr)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mem)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_iter)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_const_str)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_timer)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_fail1)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_fail2)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_acquire, KF_ACQUIRE | KF_RET_NULL)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index aa0b8d41e71b..2c1cb118f886 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -26,6 +26,8 @@ struct prog_test_ref_kfunc {
 };
 #endif
 
+struct bpf_iter_testmod_seq;
+
 struct prog_test_pass1 {
 	int x0;
 	struct {
@@ -111,7 +113,23 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
 struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
 long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
 int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
-
+__u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
+			       __u64 e, __u64 f, __u64 g, __u64 h) __ksym;
+__u64 bpf_kfunc_call_stack_arg_ptr(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				   struct prog_test_pass1 *p) __ksym;
+__u64 bpf_kfunc_call_stack_arg_mix(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				   struct prog_test_pass1 *p, __u64 f,
+				   struct prog_test_pass1 *q) __ksym;
+__u64 bpf_kfunc_call_stack_arg_dynptr(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				      struct bpf_dynptr *ptr) __ksym;
+__u64 bpf_kfunc_call_stack_arg_mem(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				   void *mem, int mem__sz) __ksym;
+__u64 bpf_kfunc_call_stack_arg_iter(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				    struct bpf_iter_testmod_seq *it__iter) __ksym;
+__u64 bpf_kfunc_call_stack_arg_const_str(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+					 const char *str__str) __ksym;
+__u64 bpf_kfunc_call_stack_arg_timer(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				     struct bpf_timer *timer) __ksym;
 void bpf_kfunc_call_test_pass_ctx(struct __sk_buff *skb) __ksym;
 void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym;
 void bpf_kfunc_call_test_pass2(struct prog_test_pass2 *p) __ksym;
-- 
2.52.0

[PATCH bpf-next 12/18] selftests/bpf: Add tests for stack argument validation

[Yonghong Song]

Add negative tests that verify the kfunc (rejecting kfunc call
with >8 byte struct as stack argument) and the verifier
(rejecting invalid uses of r11 for stack arguments).

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 .../selftests/bpf/prog_tests/stack_arg_fail.c |  10 ++
 .../selftests/bpf/progs/stack_arg_fail.c      | 114 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |   7 ++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |   8 ++
 4 files changed, 139 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/stack_arg_fail.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg_fail.c

diff --git a/tools/testing/selftests/bpf/prog_tests/stack_arg_fail.c b/tools/testing/selftests/bpf/prog_tests/stack_arg_fail.c
new file mode 100644
index 000000000000..090af1330953
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/stack_arg_fail.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <test_progs.h>
+#include "stack_arg_fail.skel.h"
+
+void test_stack_arg_fail(void)
+{
+	RUN_TESTS(stack_arg_fail);
+}
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_fail.c b/tools/testing/selftests/bpf/progs/stack_arg_fail.c
new file mode 100644
index 000000000000..a36c4d7d870a
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/stack_arg_fail.c
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+#include "bpf_misc.h"
+
+#if defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+SEC("tc")
+__failure __msg("Unrecognized *(R11-8) type STRUCT")
+int test_stack_arg_big(struct __sk_buff *skb)
+{
+	struct prog_test_big_arg s = { .a = 1, .b = 2 };
+
+	return bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s);
+}
+
+#else
+
+SEC("tc")
+__description("stack_arg_fail: not supported, dummy test")
+__success
+int test_stack_arg_big(struct __sk_buff *skb)
+{
+	return 0;
+}
+
+#endif
+
+SEC("socket")
+__description("r11 in ALU instruction")
+__failure __msg("R11 is invalid")
+__naked void r11_alu_reject(void)
+{
+	asm volatile (
+	"r11 += 1;"
+	"r0 = 0;"
+	"exit;"
+	::: __clobber_all);
+}
+
+SEC("socket")
+__description("r11 store with non-DW size")
+__failure __msg("R11 is invalid")
+__naked void r11_store_non_dw(void)
+{
+	asm volatile (
+	"*(u32 *)(r11 - 8) = r1;"
+	"r0 = 0;"
+	"exit;"
+	::: __clobber_all);
+}
+
+SEC("socket")
+__description("r11 store with unaligned offset")
+__failure __msg("invalid stack arg store offset -4")
+__naked void r11_store_unaligned(void)
+{
+	asm volatile (
+	"*(u64 *)(r11 - 4) = r1;"
+	"r0 = 0;"
+	"exit;"
+	::: __clobber_all);
+}
+
+SEC("socket")
+__description("r11 store with positive offset")
+__failure __msg("invalid stack arg store offset 8")
+__naked void r11_store_positive_off(void)
+{
+	asm volatile (
+	"*(u64 *)(r11 + 8) = r1;"
+	"r0 = 0;"
+	"exit;"
+	::: __clobber_all);
+}
+
+SEC("socket")
+__description("r11 load with negative offset")
+__failure __msg("invalid stack arg load offset -8")
+__naked void r11_load_negative_off(void)
+{
+	asm volatile (
+	"r0 = *(u64 *)(r11 - 8);"
+	"exit;"
+	::: __clobber_all);
+}
+
+SEC("socket")
+__description("r11 load with non-DW size")
+__failure __msg("R11 is invalid")
+__naked void r11_load_non_dw(void)
+{
+	asm volatile (
+	"r0 = *(u32 *)(r11 + 8);"
+	"exit;"
+	::: __clobber_all);
+}
+
+SEC("socket")
+__description("r11 store with zero offset")
+__failure __msg("invalid stack arg store offset 0")
+__naked void r11_store_zero_off(void)
+{
+	asm volatile (
+	"*(u64 *)(r11 + 0) = r1;"
+	"r0 = 0;"
+	"exit;"
+	::: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index aef2f68b7e83..0be918fe3021 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -882,6 +882,12 @@ __bpf_kfunc u64 bpf_kfunc_call_stack_arg_timer(u64 a, u64 b, u64 c, u64 d, u64 e
 	return a + b + c + d + e;
 }
 
+__bpf_kfunc u64 bpf_kfunc_call_stack_arg_big(u64 a, u64 b, u64 c, u64 d, u64 e,
+					     struct prog_test_big_arg s)
+{
+	return a + b + c + d + e + s.a + s.b;
+}
+
 static struct prog_test_ref_kfunc prog_test_struct = {
 	.a = 42,
 	.b = 108,
@@ -1353,6 +1359,7 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mem)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_iter)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_const_str)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_timer)
+BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_big)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_fail1)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_fail2)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_acquire, KF_ACQUIRE | KF_RET_NULL)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index 2c1cb118f886..2edc36b66de9 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -50,6 +50,11 @@ struct prog_test_pass2 {
 	} x;
 };
 
+struct prog_test_big_arg {
+	__u64 a;
+	__u64 b;
+};
+
 struct prog_test_fail1 {
 	void *p;
 	int x;
@@ -130,6 +135,9 @@ __u64 bpf_kfunc_call_stack_arg_const_str(__u64 a, __u64 b, __u64 c, __u64 d, __u
 					 const char *str__str) __ksym;
 __u64 bpf_kfunc_call_stack_arg_timer(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
 				     struct bpf_timer *timer) __ksym;
+__u64 bpf_kfunc_call_stack_arg_big(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				   struct prog_test_big_arg s) __ksym;
+
 void bpf_kfunc_call_test_pass_ctx(struct __sk_buff *skb) __ksym;
 void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym;
 void bpf_kfunc_call_test_pass2(struct prog_test_pass2 *p) __ksym;
-- 
2.52.0

[PATCH bpf-next 13/18] selftests/bpf: Add verifier tests for stack argument validation

[Yonghong Song]

Add inline-asm based verifier tests that exercise stack argument
validation logic directly.

Positive tests:
  - subprog call with 6 arg's
  - Two sequential calls to different subprogs (6-arg and 7-arg)
  - Share a r11 store for both branches

Negative tests — verifier rejection:
  - Read from uninitialized incoming stack arg slot
  - Gap in outgoing slots: only r11-16 written, r11-8 missing
  - Write at r11-80, exceeding max 7 stack args
  - Missing store on one branch with a shared store
  - First call has proper stack arguments and the second
    call intends to inherit stack arguments but not working
  - r11 load ordering issue

Negative tests — pointer/ref tracking:
  - Pruning type mismatch: one branch stores PTR_TO_STACK, the
    other stores a scalar, callee dereferences — must not prune
  - Release invalidation: bpf_sk_release invalidates a socket
    pointer stored in a stack arg slot
  - Packet pointer invalidation: bpf_skb_pull_data invalidates
    a packet pointer stored in a stack arg slot
  - Null propagation: PTR_TO_MAP_VALUE_OR_NULL stored in stack
    arg slot, null branch attempts dereference via callee

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_stack_arg.c  | 456 ++++++++++++++++++
 2 files changed, 458 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_stack_arg.c

diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index a96b25ebff23..aef21cf2987b 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -91,6 +91,7 @@
 #include "verifier_sockmap_mutate.skel.h"
 #include "verifier_spill_fill.skel.h"
 #include "verifier_spin_lock.skel.h"
+#include "verifier_stack_arg.skel.h"
 #include "verifier_stack_ptr.skel.h"
 #include "verifier_store_release.skel.h"
 #include "verifier_subprog_precision.skel.h"
@@ -238,6 +239,7 @@ void test_verifier_sock_addr(void)            { RUN(verifier_sock_addr); }
 void test_verifier_sockmap_mutate(void)       { RUN(verifier_sockmap_mutate); }
 void test_verifier_spill_fill(void)           { RUN(verifier_spill_fill); }
 void test_verifier_spin_lock(void)            { RUN(verifier_spin_lock); }
+void test_verifier_stack_arg(void)            { RUN(verifier_stack_arg); }
 void test_verifier_stack_ptr(void)            { RUN(verifier_stack_ptr); }
 void test_verifier_store_release(void)        { RUN(verifier_store_release); }
 void test_verifier_subprog_precision(void)    { RUN(verifier_subprog_precision); }
diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
new file mode 100644
index 000000000000..6b596ad63774
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
@@ -0,0 +1,456 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+struct {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__uint(max_entries, 1);
+	__type(key, long long);
+	__type(value, long long);
+} map_hash_8b SEC(".maps");
+
+#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+__noinline __used
+static int subprog_6args(int a, int b, int c, int d, int e, int f)
+{
+	return a + b + c + d + e + f;
+}
+
+__noinline __used
+static int subprog_7args(int a, int b, int c, int d, int e, int f, int g)
+{
+	return a + b + c + d + e + f + g;
+}
+
+__noinline __used
+static long subprog_deref_arg6(long a, long b, long c, long d, long e, long *f)
+{
+	return *f;
+}
+
+SEC("tc")
+__description("stack_arg: subprog with 6 args")
+__success
+__naked void stack_arg_6args(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 8) = 6;"
+		"call subprog_6args;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: two subprogs with >5 args")
+__success
+__naked void stack_arg_two_subprogs(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 8) = 10;"
+		"call subprog_6args;"
+		"r6 = r0;"
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 16) = 30;"
+		"*(u64 *)(r11 - 8) = 20;"
+		"call subprog_7args;"
+		"r0 += r6;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: read from uninitialized stack arg slot")
+__failure
+__msg("invalid read from stack arg off 8 depth 0")
+__naked void stack_arg_read_uninitialized(void)
+{
+	asm volatile (
+		"r0 = *(u64 *)(r11 + 8);"
+		"r0 = 0;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: gap at offset -8, only wrote -16")
+__failure
+__msg("stack *(R11-8) not properly initialized")
+__naked void stack_arg_gap_at_minus8(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 16) = 30;"
+		"call subprog_7args;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: pruning with different stack arg types")
+__failure
+__flag(BPF_F_TEST_STATE_FREQ)
+__msg("invalid mem access 'scalar'")
+__naked void stack_arg_pruning_type_mismatch(void)
+{
+	asm volatile (
+		"call %[bpf_get_prandom_u32];"
+		"r6 = r0;"
+		/* local = 0 on program stack */
+		"r7 = 0;"
+		"*(u64 *)(r10 - 8) = r7;"
+		/* Branch based on random value */
+		"if r6 s> 3 goto l0_%=;"
+		/* Path 1: store stack pointer to outgoing arg6 */
+		"r1 = r10;"
+		"r1 += -8;"
+		"*(u64 *)(r11 - 8) = r1;"
+		"goto l1_%=;"
+	"l0_%=:"
+		/* Path 2: store scalar to outgoing arg6 */
+		"*(u64 *)(r11 - 8) = 42;"
+	"l1_%=:"
+		/* Call subprog that dereferences arg6 */
+		"r1 = r6;"
+		"r2 = 0;"
+		"r3 = 0;"
+		"r4 = 0;"
+		"r5 = 0;"
+		"call subprog_deref_arg6;"
+		"exit;"
+		:: __imm(bpf_get_prandom_u32)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: release_reference invalidates stack arg slot")
+__failure
+__msg("invalid mem access 'scalar'")
+__naked void stack_arg_release_ref(void)
+{
+	asm volatile (
+		"r6 = r1;"
+		/* struct bpf_sock_tuple tuple = {} */
+		"r2 = 0;"
+		"*(u32 *)(r10 - 8) = r2;"
+		"*(u64 *)(r10 - 16) = r2;"
+		"*(u64 *)(r10 - 24) = r2;"
+		"*(u64 *)(r10 - 32) = r2;"
+		"*(u64 *)(r10 - 40) = r2;"
+		"*(u64 *)(r10 - 48) = r2;"
+		/* sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple), 0, 0) */
+		"r1 = r6;"
+		"r2 = r10;"
+		"r2 += -48;"
+		"r3 = %[sizeof_bpf_sock_tuple];"
+		"r4 = 0;"
+		"r5 = 0;"
+		"call %[bpf_sk_lookup_tcp];"
+		/* r0 = sk (PTR_TO_SOCK_OR_NULL) */
+		"if r0 == 0 goto l0_%=;"
+		/* Store sock ref to outgoing arg6 slot */
+		"*(u64 *)(r11 - 8) = r0;"
+		/* Release the reference — invalidates the stack arg slot */
+		"r1 = r0;"
+		"call %[bpf_sk_release];"
+		/* Call subprog that dereferences arg6 — should fail */
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_deref_arg6;"
+	"l0_%=:"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm(bpf_sk_lookup_tcp),
+		  __imm(bpf_sk_release),
+		  __imm_const(sizeof_bpf_sock_tuple, sizeof(struct bpf_sock_tuple))
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: pkt pointer in stack arg slot invalidated after pull_data")
+__failure
+__msg("invalid mem access 'scalar'")
+__naked void stack_arg_stale_pkt_ptr(void)
+{
+	asm volatile (
+		"r6 = r1;"
+		"r7 = *(u32 *)(r6 + %[__sk_buff_data]);"
+		"r8 = *(u32 *)(r6 + %[__sk_buff_data_end]);"
+		/* check pkt has at least 1 byte */
+		"r0 = r7;"
+		"r0 += 8;"
+		"if r0 > r8 goto l0_%=;"
+		/* Store valid pkt pointer to outgoing arg6 slot */
+		"*(u64 *)(r11 - 8) = r7;"
+		/* bpf_skb_pull_data invalidates all pkt pointers */
+		"r1 = r6;"
+		"r2 = 0;"
+		"call %[bpf_skb_pull_data];"
+		/* Call subprog that dereferences arg6 — should fail */
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_deref_arg6;"
+	"l0_%=:"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm(bpf_skb_pull_data),
+		  __imm_const(__sk_buff_data, offsetof(struct __sk_buff, data)),
+		  __imm_const(__sk_buff_data_end, offsetof(struct __sk_buff, data_end))
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: null propagation rejects deref on null branch")
+__failure
+__msg("invalid mem access 'scalar'")
+__naked void stack_arg_null_propagation_fail(void)
+{
+	asm volatile (
+		"r1 = 0;"
+		"*(u64 *)(r10 - 8) = r1;"
+		/* r0 = bpf_map_lookup_elem(&map_hash_8b, &key) */
+		"r2 = r10;"
+		"r2 += -8;"
+		"r1 = %[map_hash_8b] ll;"
+		"call %[bpf_map_lookup_elem];"
+		/* Store PTR_TO_MAP_VALUE_OR_NULL to outgoing arg6 slot */
+		"*(u64 *)(r11 - 8) = r0;"
+		/* null check on r0 */
+		"if r0 != 0 goto l0_%=;"
+		/*
+		 * On null branch, outgoing slot is SCALAR(0).
+		 * Call subprog that dereferences arg6 — should fail.
+		 */
+		"r1 = 0;"
+		"r2 = 0;"
+		"r3 = 0;"
+		"r4 = 0;"
+		"r5 = 0;"
+		"call subprog_deref_arg6;"
+	"l0_%=:"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm(bpf_map_lookup_elem),
+		  __imm_addr(map_hash_8b)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: missing store on one branch")
+__failure
+__msg("stack *(R11-8) not properly initialized")
+__naked void stack_arg_missing_store_one_branch(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		/* Write arg7 (r11-16) before branch */
+		"*(u64 *)(r11 - 16) = 20;"
+		"call %[bpf_get_prandom_u32];"
+		"if r0 > 0 goto l0_%=;"
+		/* Path 1: write arg6 and call */
+		"*(u64 *)(r11 - 8) = 10;"
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_7args;"
+		"goto l1_%=;"
+	"l0_%=:"
+		/* Path 2: missing arg6 store, call should fail */
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_7args;"
+	"l1_%=:"
+		"r0 = 0;"
+		"exit;"
+		:: __imm(bpf_get_prandom_u32)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: share a store for both branches")
+__success __retval(0)
+__naked void stack_arg_shared_store(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		/* Write arg7 (r11-16) before branch */
+		"*(u64 *)(r11 - 16) = 20;"
+		"call %[bpf_get_prandom_u32];"
+		"if r0 > 0 goto l0_%=;"
+		/* Path 1: write arg6 and call */
+		"*(u64 *)(r11 - 8) = 10;"
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_7args;"
+		"goto l1_%=;"
+	"l0_%=:"
+		/* Path 2: also write arg6 and call */
+		"*(u64 *)(r11 - 8) = 30;"
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_7args;"
+	"l1_%=:"
+		"r0 = 0;"
+		"exit;"
+		:: __imm(bpf_get_prandom_u32)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: write beyond max outgoing depth")
+__failure
+__msg("stack arg write offset -80 exceeds max 7 stack args")
+__naked void stack_arg_write_beyond_max(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		/* Write to offset -80, way beyond any callee's needs */
+		"*(u64 *)(r11 - 80) = 99;"
+		"*(u64 *)(r11 - 16) = 20;"
+		"*(u64 *)(r11 - 8) = 10;"
+		"call subprog_7args;"
+		"r0 = 0;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: sequential calls reuse slots")
+__failure
+__msg("stack *(R11-8) not properly initialized")
+__naked void stack_arg_sequential_calls(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 8) = 6;"
+		"*(u64 *)(r11 - 16) = 7;"
+		"call subprog_7args;"
+		"r6 = r0;"
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call subprog_7args;"
+		"r0 += r6;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: r11 load after r11 store")
+__failure
+__msg("r11 load must be before any r11 store or call insn")
+__naked void stack_arg_load_after_store(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 8) = 6;"
+		"r0 = *(u64 *)(r11 + 8);"
+		"call subprog_6args;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: r11 load after a call")
+__failure
+__msg("r11 load must be before any r11 store or call insn")
+__naked void stack_arg_load_after_call(void)
+{
+	asm volatile (
+		"call %[bpf_get_prandom_u32];"
+		"r0 = *(u64 *)(r11 + 8);"
+		"exit;"
+		:: __imm(bpf_get_prandom_u32)
+		: __clobber_all
+	);
+}
+
+#else
+
+SEC("socket")
+__description("stack_arg is not supported by compiler or jit, use a dummy test")
+__success
+int dummy_test(void)
+{
+	return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
-- 
2.52.0

[PATCH bpf-next 14/18] selftests/bpf: Add BTF fixup for __naked subprog parameter names

[Yonghong Song]

When __naked subprogs are used in verifier tests, clang drops
parameter names from their BTF FUNC_PROTO entries. This prevents
the verifier from resolving stack argument slots by name.

Add a __btf_func_path(path) annotation that points to a separate
BTF file containing properly-named FUNC entries. The test_loader
matches FUNC entries by name, detects anonymous parameters, and
replaces the FUNC_PROTO with a new one that carries parameter
names from the custom file while preserving the original type IDs.

The custom BTF file also serves as btf_custom_path for kfunc
resolution when no separate btf_custom_path is specified.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 tools/testing/selftests/bpf/progs/bpf_misc.h |   1 +
 tools/testing/selftests/bpf/test_loader.c    | 136 ++++++++++++++++++-
 2 files changed, 136 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/bpf_misc.h b/tools/testing/selftests/bpf/progs/bpf_misc.h
index dcd78a3a9052..a9f58900a501 100644
--- a/tools/testing/selftests/bpf/progs/bpf_misc.h
+++ b/tools/testing/selftests/bpf/progs/bpf_misc.h
@@ -152,6 +152,7 @@
 #define __auxiliary		__test_tag("test_auxiliary")
 #define __auxiliary_unpriv	__test_tag("test_auxiliary_unpriv")
 #define __btf_path(path)	__test_tag("test_btf_path=" path)
+#define __btf_func_path(path)	__test_tag("test_btf_func_path=" path)
 #define __arch(arch)		__test_tag("test_arch=" arch)
 #define __arch_x86_64		__arch("X86_64")
 #define __arch_arm64		__arch("ARM64")
diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c
index c4c34cae6102..b9e21696b9ba 100644
--- a/tools/testing/selftests/bpf/test_loader.c
+++ b/tools/testing/selftests/bpf/test_loader.c
@@ -63,6 +63,7 @@ struct test_spec {
 	struct test_subspec priv;
 	struct test_subspec unpriv;
 	const char *btf_custom_path;
+	const char *btf_custom_func_path;
 	int log_level;
 	int prog_flags;
 	int mode_mask;
@@ -590,6 +591,8 @@ static int parse_test_spec(struct test_loader *tester,
 			jit_on_next_line = true;
 		} else if ((val = str_has_pfx(s, "test_btf_path="))) {
 			spec->btf_custom_path = val;
+		} else if ((val = str_has_pfx(s, "test_btf_func_path="))) {
+			spec->btf_custom_func_path = val;
 		} else if ((val = str_has_pfx(s, "test_caps_unpriv="))) {
 			err = parse_caps(val, &spec->unpriv.caps, "test caps");
 			if (err)
@@ -1138,6 +1141,123 @@ static int get_stream(int stream_id, int prog_fd, char *text, size_t text_sz)
 	return ret;
 }
 
+/*
+ * Fix up the program's BTF using BTF from a separate file.
+ *
+ * For __naked subprogs, clang drops parameter names from BTF. Find FUNC
+ * entries with anonymous parameters and replace their FUNC_PROTO with the
+ * properly-named version from the custom file.
+ */
+static int fixup_btf_from_path(struct bpf_object *obj, const char *path)
+{
+	struct btf *prog_btf, *custom_btf;
+	__u32 i, j, cnt, custom_cnt;
+	int err = 0;
+
+	prog_btf = bpf_object__btf(obj);
+	if (!prog_btf)
+		return 0;
+
+	custom_btf = btf__parse(path, NULL);
+	if (!ASSERT_OK_PTR(custom_btf, "parse_custom_btf"))
+		return -EINVAL;
+
+	cnt = btf__type_cnt(prog_btf);
+	custom_cnt = btf__type_cnt(custom_btf);
+
+	/* Fix up FUNC entries with anonymous params.
+	 * Save all data from prog_btf BEFORE calling btf__add_*,
+	 * since those calls may reallocate the BTF data buffer
+	 * and invalidate any pointers obtained from btf__type_by_id.
+	 */
+	for (i = 1; i < cnt; i++) {
+		const struct btf_type *t = btf__type_by_id(prog_btf, i);
+		const struct btf_type *fp, *custom_t, *custom_fp;
+		const struct btf_param *params, *custom_params;
+		__u32 ret_type_id, vlen;
+		__u32 *prog_param_types = NULL;
+		const char *name;
+		int new_proto_id;
+
+		if (!btf_is_func(t))
+			continue;
+
+		fp = btf__type_by_id(prog_btf, t->type);
+		if (!fp || !btf_is_func_proto(fp) || btf_vlen(fp) == 0)
+			continue;
+
+		/* Check if any param is anonymous */
+		params = btf_params(fp);
+		if (params[0].name_off != 0)
+			continue;
+
+		/* Find matching FUNC by name in custom BTF */
+		name = btf__name_by_offset(prog_btf, t->name_off);
+		if (!name)
+			continue;
+
+		for (j = 1; j < custom_cnt; j++) {
+			const char *cname;
+
+			custom_t = btf__type_by_id(custom_btf, j);
+			if (!btf_is_func(custom_t))
+				continue;
+			cname = btf__name_by_offset(custom_btf, custom_t->name_off);
+			if (cname && strcmp(name, cname) == 0)
+				break;
+		}
+		if (j >= custom_cnt)
+			continue;
+
+		custom_fp = btf__type_by_id(custom_btf, custom_t->type);
+		if (!custom_fp || !btf_is_func_proto(custom_fp))
+			continue;
+
+		vlen = btf_vlen(fp);
+		if (vlen != btf_vlen(custom_fp))
+			continue;
+
+		/* Save data before btf__add_* calls invalidate pointers */
+		ret_type_id = fp->type;
+		prog_param_types = malloc(vlen * sizeof(*prog_param_types));
+		if (!prog_param_types) {
+			err = -ENOMEM;
+			break;
+		}
+		for (j = 0; j < vlen; j++)
+			prog_param_types[j] = params[j].type;
+
+		/* Add a new FUNC_PROTO: param names from custom, types from prog */
+		new_proto_id = btf__add_func_proto(prog_btf, ret_type_id);
+		if (new_proto_id < 0) {
+			err = new_proto_id;
+			free(prog_param_types);
+			break;
+		}
+
+		custom_params = btf_params(custom_fp);
+		for (j = 0; j < vlen; j++) {
+			const char *pname;
+
+			pname = btf__name_by_offset(custom_btf, custom_params[j].name_off);
+			err = btf__add_func_param(prog_btf, pname ?: "", prog_param_types[j]);
+			if (err)
+				break;
+		}
+		free(prog_param_types);
+		if (err)
+			break;
+
+		/* Update the FUNC to point to the new FUNC_PROTO (re-fetch
+		 * since btf__add_* may have reallocated the data buffer).
+		 */
+		((struct btf_type *)btf__type_by_id(prog_btf, i))->type = new_proto_id;
+	}
+
+	btf__free(custom_btf);
+	return err;
+}
+
 /* this function is forced noinline and has short generic name to look better
  * in test_progs output (in case of a failure)
  */
@@ -1194,13 +1314,27 @@ void run_subtest(struct test_loader *tester,
 		}
 	}
 
-	/* Implicitly reset to NULL if next test case doesn't specify */
+	/* Implicitly reset to NULL if next test case doesn't specify.
+	 * btf_custom_func_path also serves as btf_custom_path for kfunc resolution.
+	 */
 	open_opts->btf_custom_path = spec->btf_custom_path;
+	if (!open_opts->btf_custom_path)
+		open_opts->btf_custom_path = spec->btf_custom_func_path;
 
 	tobj = bpf_object__open_mem(obj_bytes, obj_byte_cnt, open_opts);
 	if (!ASSERT_OK_PTR(tobj, "obj_open_mem")) /* shouldn't happen */
 		goto subtest_cleanup;
 
+	/* Fix up __naked subprog BTF using a separate file with named params */
+	if (spec->btf_custom_func_path) {
+		err = fixup_btf_from_path(tobj, spec->btf_custom_func_path);
+		if (err) {
+			PRINT_FAIL("failed to fixup BTF from %s: %d\n",
+				   spec->btf_custom_func_path, err);
+			goto subtest_cleanup;
+		}
+	}
+
 	i = 0;
 	bpf_object__for_each_program(tprog_iter, tobj) {
 		spec_iter = &specs[i++];
-- 
2.52.0

[PATCH bpf-next 15/18] selftests/bpf: Add precision backtracking test for stack arguments

[Yonghong Song]

Add a test that verifies precision backtracking works correctly
across BPF-to-BPF calls when stack arguments are involved.

The test passes a size value as incoming stack arg (arg6) to a
subprog, which bounds-checks it and forwards it as the mem__sz
parameter (outgoing arg7) to bpf_kfunc_call_stack_arg_mem. The
expected __msg annotations verify that precision propagates from
the kfunc's mem__sz argument back through the subprog frame to the
caller's outgoing stack arg store.

A companion BTF file (btf__stack_arg_precision.c) provides named
parameter BTF for the __naked subprog via __btf_func_path.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 .../bpf/prog_tests/stack_arg_precision.c      |  10 ++
 .../bpf/progs/btf__stack_arg_precision.c      |  23 ++++
 .../selftests/bpf/progs/stack_arg_precision.c | 121 ++++++++++++++++++
 3 files changed, 154 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/stack_arg_precision.c
 create mode 100644 tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
 create mode 100644 tools/testing/selftests/bpf/progs/stack_arg_precision.c

diff --git a/tools/testing/selftests/bpf/prog_tests/stack_arg_precision.c b/tools/testing/selftests/bpf/prog_tests/stack_arg_precision.c
new file mode 100644
index 000000000000..1ab041d66de3
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/stack_arg_precision.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <test_progs.h>
+#include "stack_arg_precision.skel.h"
+
+void test_stack_arg_precision(void)
+{
+	RUN_TESTS(stack_arg_precision);
+}
diff --git a/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c b/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
new file mode 100644
index 000000000000..296fddfe6804
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+long subprog_call_mem_kfunc(long a, long b, long c, long d, long e, long size)
+{
+	char buf[8] = {};
+
+	return bpf_kfunc_call_stack_arg_mem(a, b, c, d, e, buf, size);
+}
+
+#else
+
+long subprog_call_mem_kfunc(void)
+{
+	return 0;
+}
+
+#endif
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_precision.c b/tools/testing/selftests/bpf/progs/stack_arg_precision.c
new file mode 100644
index 000000000000..29b2f2aea931
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/stack_arg_precision.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+#include "bpf_misc.h"
+
+#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+/* Force kfunc extern BTF generation for inline asm call below.
+ * Uses its own SEC so it's not included as a .text subprog.
+ * The '?' prefix sets autoload=false so libbpf won't load it.
+ */
+SEC("?tc")
+int __btf_kfunc_gen(struct __sk_buff *ctx)
+{
+	char buf[8] = {};
+
+	return bpf_kfunc_call_stack_arg_mem(0, 0, 0, 0, 0, buf, sizeof(buf));
+}
+
+/*
+ * Test precision backtracking across bpf-to-bpf call for kfunc stack arg.
+ * subprog_call_mem_kfunc receives a size as incoming stack arg (arg6),
+ * bounds-checks it, then passes it as mem__sz (arg7) to
+ * bpf_kfunc_call_stack_arg_mem.
+ *
+ * 1+2+3+4+5+(1+2+3+4) = 25
+ */
+__naked __noinline __used
+static long subprog_call_mem_kfunc(long a, long b, long c, long d, long e, long size)
+{
+	asm volatile (
+		"r1 = *(u64 *)(r11 + 8);"	/* r1 = incoming arg6 (size) */
+		"r2 = 0x0807060504030201 ll;"	/* r2 = buf contents */
+		"*(u64 *)(r10 - 8) = r2;"	/* store buf to stack */
+		"r0 = -1;"
+		"if r1 s< 1 goto 1f;"
+		"if r1 s> 8 goto 1f;"
+		"r2 = r10;"
+		"r2 += -8;"			/* r2 = &buf */
+		"*(u64 *)(r11 - 8) = r2;"	/* outgoing arg6 = buf */
+		"*(u64 *)(r11 - 16) = r1;"	/* outgoing arg7 = size */
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"call %[bpf_kfunc_call_stack_arg_mem];"
+		"1: exit;"
+		:
+		: __imm(bpf_kfunc_call_stack_arg_mem)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("stack_arg: precision backtracking across bpf2bpf call for kfunc")
+__success __retval(25)
+__log_level(2)
+__flag(BPF_F_TEST_STATE_FREQ)
+__btf_func_path("btf__stack_arg_precision.bpf.o")
+__msg("mark_precise: frame1: last_idx 24 first_idx 14 subseq_idx -1")
+__msg("mark_precise: frame1: regs= stack= before 23: (b7) r5 = 5")
+__msg("mark_precise: frame1: regs= stack= before 22: (b7) r4 = 4")
+__msg("mark_precise: frame1: regs= stack= before 21: (b7) r3 = 3")
+__msg("mark_precise: frame1: regs= stack= before 20: (b7) r2 = 2")
+__msg("mark_precise: frame1: regs= stack= before 19: (b7) r1 = 1")
+__msg("mark_precise: frame1: regs= stack= before 18: (7b) *(u64 *)(r11 -16) = r1")
+__msg("mark_precise: frame1: regs=r1 stack= before 17: (7b) *(u64 *)(r11 -8) = r2")
+__msg("mark_precise: frame1: regs=r1 stack= before 16: (07) r2 += -8")
+__msg("mark_precise: frame1: regs=r1 stack= before 15: (bf) r2 = r10")
+__msg("mark_precise: frame1: regs=r1 stack= before 14: (65) if r1 s> 0x8 goto pc+10")
+__msg("mark_precise: frame1: parent state regs=r1 stack=:  frame1: R0=-1 R1=Pscalar")
+__msg("mark_precise: frame0: parent state regs= stack=:  R10=fp0")
+__msg("mark_precise: frame1: last_idx 13 first_idx 13 subseq_idx 14")
+__msg("mark_precise: frame1: regs=r1 stack= before 13: (c5) if r1 s< 0x1 goto pc+11")
+__msg("mark_precise: frame1: parent state regs=r1 stack=:  frame1: R0=-1 R1=Pscalar() R10=fp0 fp-8=0x807060504030201")
+__msg("mark_precise: frame0: parent state regs= stack=:  R10=fp0")
+__msg("mark_precise: frame1: last_idx 12 first_idx 8 subseq_idx 1")
+__msg("mark_precise: frame1: regs=r1 stack= before 12: (b7) r0 = -1")
+__msg("mark_precise: frame1: regs=r1 stack= before 11: (7b) *(u64 *)(r10 -8) = r2")
+__msg("mark_precise: frame1: regs=r1 stack= before 9: (18) r2 = 0x807060504030201")
+__msg("mark_precise: frame1: regs=r1 stack= before 8: (79) r1 = *(u64 *)(r11 +8)")
+__msg("mark_precise: frame1: parent state regs= stack=:  frame1: R10=fp0")
+__msg("mark_precise: frame0: parent state regs= stack=:  R10=fp0")
+__msg("mark_precise: frame1: last_idx 6 first_idx 6 subseq_idx 8")
+__msg("mark_precise: frame1: regs= stack= before 6: (85) call pc+1")
+__msg("mark_precise: frame0: parent state regs= stack=:  R1=1 R2=2 R3=3 R4=4 R5=5 R10=fp0")
+__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx 6")
+__msg("mark_precise: frame0: regs= stack= before 5: (7a) *(u64 *)(r11 -8) = 4")
+__msg("mark_precise: frame1: last_idx 24 first_idx 14 subseq_idx -1")
+__naked void stack_arg_precision_bpf2bpf(void)
+{
+	asm volatile (
+		"r1 = 1;"
+		"r2 = 2;"
+		"r3 = 3;"
+		"r4 = 4;"
+		"r5 = 5;"
+		"*(u64 *)(r11 - 8) = 4;"
+		"call subprog_call_mem_kfunc;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+#else
+
+SEC("socket")
+__description("stack_arg_precision: not supported, dummy test")
+__success
+int dummy_test(void)
+{
+	return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
-- 
2.52.0

[PATCH bpf-next 16/18] bpf, arm64: Map BPF_REG_0 to x8 instead of x7

[Yonghong Song]

From: Puranjay Mohan <puranjay@kernel.org>

Move the BPF return value register from x7 to x8, freeing x7 for use
as an argument register. AAPCS64 designates x8 as the indirect result
location register; it is caller-saved and not used for argument
passing, making it a suitable home for BPF_REG_0.

This is a prerequisite for stack argument support, which needs x5-x7
to pass arguments 6-8 to native kfuncs following the AAPCS64 calling
convention.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 arch/arm64/net/bpf_jit_comp.c                          |  4 ++--
 arch/arm64/net/bpf_timed_may_goto.S                    |  8 ++++----
 .../testing/selftests/bpf/progs/verifier_jit_inline.c  |  2 +-
 tools/testing/selftests/bpf/progs/verifier_ldsx.c      |  6 +++---
 .../selftests/bpf/progs/verifier_private_stack.c       | 10 +++++-----
 5 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 0816c40fc7af..085e650662e3 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -47,7 +47,7 @@
 /* Map BPF registers to A64 registers */
 static const int bpf2a64[] = {
 	/* return value from in-kernel function, and exit value from eBPF */
-	[BPF_REG_0] = A64_R(7),
+	[BPF_REG_0] = A64_R(8),
 	/* arguments from eBPF program to in-kernel function */
 	[BPF_REG_1] = A64_R(0),
 	[BPF_REG_2] = A64_R(1),
@@ -1048,7 +1048,7 @@ static void build_epilogue(struct jit_ctx *ctx, bool was_classic)
 	/* Restore FP/LR registers */
 	emit(A64_POP(A64_FP, A64_LR, A64_SP), ctx);
 
-	/* Move the return value from bpf:r0 (aka x7) to x0 */
+	/* Move the return value from bpf:r0 (aka x8) to x0 */
 	emit(A64_MOV(1, A64_R(0), r0), ctx);
 
 	/* Authenticate lr */
diff --git a/arch/arm64/net/bpf_timed_may_goto.S b/arch/arm64/net/bpf_timed_may_goto.S
index 894cfcd7b241..a9a802711a7f 100644
--- a/arch/arm64/net/bpf_timed_may_goto.S
+++ b/arch/arm64/net/bpf_timed_may_goto.S
@@ -8,8 +8,8 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
 	stp     x29, x30, [sp, #-64]!
 	mov     x29, sp
 
-	/* Save BPF registers R0 - R5 (x7, x0-x4)*/
-	stp	x7, x0, [sp, #16]
+	/* Save BPF registers R0 - R5 (x8, x0-x4)*/
+	stp	x8, x0, [sp, #16]
 	stp	x1, x2, [sp, #32]
 	stp	x3, x4, [sp, #48]
 
@@ -28,8 +28,8 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
 	/* BPF_REG_AX(x9) will be stored into count, so move return value to it. */
 	mov	x9, x0
 
-	/* Restore BPF registers R0 - R5 (x7, x0-x4) */
-	ldp	x7, x0, [sp, #16]
+	/* Restore BPF registers R0 - R5 (x8, x0-x4) */
+	ldp	x8, x0, [sp, #16]
 	ldp	x1, x2, [sp, #32]
 	ldp	x3, x4, [sp, #48]
 
diff --git a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
index 4ea254063646..885ff69a3a62 100644
--- a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
+++ b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
@@ -9,7 +9,7 @@ __success __retval(0)
 __arch_x86_64
 __jited("	addq	%gs:{{.*}}, %rax")
 __arch_arm64
-__jited("	mrs	x7, SP_EL0")
+__jited("	mrs	x8, SP_EL0")
 int inline_bpf_get_current_task(void)
 {
 	bpf_get_current_task();
diff --git a/tools/testing/selftests/bpf/progs/verifier_ldsx.c b/tools/testing/selftests/bpf/progs/verifier_ldsx.c
index c8494b682c31..c814e82a7242 100644
--- a/tools/testing/selftests/bpf/progs/verifier_ldsx.c
+++ b/tools/testing/selftests/bpf/progs/verifier_ldsx.c
@@ -274,11 +274,11 @@ __jited("movslq	0x10(%rdi,%r12), %r15")
 __jited("movswq	0x18(%rdi,%r12), %r15")
 __jited("movsbq	0x20(%rdi,%r12), %r15")
 __arch_arm64
-__jited("add	x11, x7, x28")
+__jited("add	x11, x8, x28")
 __jited("ldrsw	x21, [x11, #0x10]")
-__jited("add	x11, x7, x28")
+__jited("add	x11, x8, x28")
 __jited("ldrsh	x21, [x11, #0x18]")
-__jited("add	x11, x7, x28")
+__jited("add	x11, x8, x28")
 __jited("ldrsb	x21, [x11, #0x20]")
 __jited("add	x11, x0, x28")
 __jited("ldrsw	x22, [x11, #0x10]")
diff --git a/tools/testing/selftests/bpf/progs/verifier_private_stack.c b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
index 646e8ef82051..c5078face38d 100644
--- a/tools/testing/selftests/bpf/progs/verifier_private_stack.c
+++ b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
@@ -170,12 +170,12 @@ __jited("	mrs	x10, TPIDR_EL{{[0-1]}}")
 __jited("	add	x27, x27, x10")
 __jited("	add	x25, x27, {{.*}}")
 __jited("	bl	0x{{.*}}")
-__jited("	mov	x7, x0")
+__jited("	mov	x8, x0")
 __jited("	mov	x0, #0x2a")
 __jited("	str	x0, [x27]")
 __jited("	bl	0x{{.*}}")
-__jited("	mov	x7, x0")
-__jited("	mov	x7, #0x0")
+__jited("	mov	x8, x0")
+__jited("	mov	x8, #0x0")
 __jited("	ldp	x25, x27, [sp], {{.*}}")
 __naked void private_stack_callback(void)
 {
@@ -220,7 +220,7 @@ __jited("	mov	x0, #0x2a")
 __jited("	str	x0, [x27]")
 __jited("	mov	x0, #0x0")
 __jited("	bl	0x{{.*}}")
-__jited("	mov	x7, x0")
+__jited("	mov	x8, x0")
 __jited("	ldp	x27, x28, [sp], #0x10")
 int private_stack_exception_main_prog(void)
 {
@@ -258,7 +258,7 @@ __jited("	add	x25, x27, {{.*}}")
 __jited("	mov	x0, #0x2a")
 __jited("	str	x0, [x27]")
 __jited("	bl	0x{{.*}}")
-__jited("	mov	x7, x0")
+__jited("	mov	x8, x0")
 __jited("	ldp	x27, x28, [sp], #0x10")
 int private_stack_exception_sub_prog(void)
 {
-- 
2.52.0

[PATCH bpf-next 17/18] bpf, arm64: Add JIT support for stack arguments

[Yonghong Song]

From: Puranjay Mohan <puranjay@kernel.org>

Implement stack argument passing for BPF-to-BPF and kfunc calls with
more than 5 parameters on arm64, following the AAPCS64 calling
convention.

BPF R1-R5 already map to x0-x4. With BPF_REG_0 moved to x8 by the
previous commit, x5-x7 are free for arguments 6-8. Arguments 9-12
spill onto the stack at [SP+0], [SP+8], ... and the callee reads
them from [FP+16], [FP+24], ... (above the saved FP/LR pair).

BPF convention uses fixed offsets from BPF_REG_PARAMS (r11): off=-8 is
always arg 6, off=-16 arg 7, etc. The verifier invalidates all outgoing
stack arg slots after each call, so the compiler must re-store before
every call. This means x5-x7 don't need to be saved on stack.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 arch/arm64/net/bpf_jit_comp.c | 87 +++++++++++++++++++++++++++++++++--
 1 file changed, 83 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 085e650662e3..7adf2b0f4610 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -86,6 +86,7 @@ struct jit_ctx {
 	__le32 *image;
 	__le32 *ro_image;
 	u32 stack_size;
+	u16 stack_arg_size;
 	u64 user_vm_start;
 	u64 arena_vm_start;
 	bool fp_used;
@@ -533,13 +534,19 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
 	 *                        |     |
 	 *                        +-----+ <= (BPF_FP - prog->aux->stack_depth)
 	 *                        |RSVD | padding
-	 * current A64_SP =>      +-----+ <= (BPF_FP - ctx->stack_size)
+	 *                        +-----+ <= (BPF_FP - ctx->stack_size)
+	 *                        |     |
+	 *                        | ... | outgoing stack args (9+, if any)
+	 *                        |     |
+	 * current A64_SP =>      +-----+
 	 *                        |     |
 	 *                        | ... | Function call stack
 	 *                        |     |
 	 *                        +-----+
 	 *                          low
 	 *
+	 * Stack args 6-8 are passed in x5-x7, args 9+ at [SP].
+	 * Incoming args 9+ are at [FP + 16], [FP + 24], ...
 	 */
 
 	emit_kcfi(is_main_prog ? cfi_bpf_hash : cfi_bpf_subprog_hash, ctx);
@@ -613,6 +620,9 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
 	if (ctx->stack_size && !ctx->priv_sp_used)
 		emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);
 
+	if (ctx->stack_arg_size)
+		emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_arg_size), ctx);
+
 	if (ctx->arena_vm_start)
 		emit_a64_mov_i64(arena_vm_base, ctx->arena_vm_start, ctx);
 
@@ -673,6 +683,9 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx)
 	/* Update tail_call_cnt if the slot is populated. */
 	emit(A64_STR64I(tcc, ptr, 0), ctx);
 
+	if (ctx->stack_arg_size)
+		emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_arg_size), ctx);
+
 	/* restore SP */
 	if (ctx->stack_size && !ctx->priv_sp_used)
 		emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);
@@ -1034,6 +1047,9 @@ static void build_epilogue(struct jit_ctx *ctx, bool was_classic)
 	const u8 r0 = bpf2a64[BPF_REG_0];
 	const u8 ptr = bpf2a64[TCCNT_PTR];
 
+	if (ctx->stack_arg_size)
+		emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_arg_size), ctx);
+
 	/* We're done with BPF stack */
 	if (ctx->stack_size && !ctx->priv_sp_used)
 		emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);
@@ -1191,6 +1207,41 @@ static int add_exception_handler(const struct bpf_insn *insn,
 	return 0;
 }
 
+static const u8 stack_arg_reg[] = { A64_R(5), A64_R(6), A64_R(7) };
+
+#define NR_STACK_ARG_REGS	ARRAY_SIZE(stack_arg_reg)
+
+static void emit_stack_arg_load(u8 dst, s16 bpf_off, struct jit_ctx *ctx)
+{
+	int idx = bpf_off / sizeof(u64) - 1;
+
+	if (idx < NR_STACK_ARG_REGS)
+		emit(A64_MOV(1, dst, stack_arg_reg[idx]), ctx);
+	else
+		emit(A64_LDR64I(dst, A64_FP, (idx - NR_STACK_ARG_REGS) * sizeof(u64) + 16), ctx);
+}
+
+static void emit_stack_arg_store(u8 src_a64, s16 bpf_off, struct jit_ctx *ctx)
+{
+	int idx = -bpf_off / sizeof(u64) - 1;
+
+	if (idx < NR_STACK_ARG_REGS)
+		emit(A64_MOV(1, stack_arg_reg[idx], src_a64), ctx);
+	else
+		emit(A64_STR64I(src_a64, A64_SP, (idx - NR_STACK_ARG_REGS) * sizeof(u64)), ctx);
+}
+
+static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct jit_ctx *ctx)
+{
+	int idx = -bpf_off / sizeof(u64) - 1;
+
+	emit_a64_mov_i(1, tmp, imm, ctx);
+	if (idx < NR_STACK_ARG_REGS)
+		emit(A64_MOV(1, stack_arg_reg[idx], tmp), ctx);
+	else
+		emit(A64_STR64I(tmp, A64_SP, (idx - NR_STACK_ARG_REGS) * sizeof(u64)), ctx);
+}
+
 /* JITs an eBPF instruction.
  * Returns:
  * 0  - successfully JITed an 8-byte eBPF instruction.
@@ -1646,6 +1697,11 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 	case BPF_LDX | BPF_MEM | BPF_H:
 	case BPF_LDX | BPF_MEM | BPF_B:
 	case BPF_LDX | BPF_MEM | BPF_DW:
+		if (insn->src_reg == BPF_REG_PARAMS) {
+			emit_stack_arg_load(dst, off, ctx);
+			break;
+		}
+		fallthrough;
 	case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
 	case BPF_LDX | BPF_PROBE_MEM | BPF_W:
 	case BPF_LDX | BPF_PROBE_MEM | BPF_H:
@@ -1671,7 +1727,7 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 		}
 		if (src == fp) {
 			src_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
-			off_adj = off + ctx->stack_size;
+			off_adj = off + ctx->stack_size + ctx->stack_arg_size;
 		} else {
 			src_adj = src;
 			off_adj = off;
@@ -1752,6 +1808,11 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 	case BPF_ST | BPF_MEM | BPF_H:
 	case BPF_ST | BPF_MEM | BPF_B:
 	case BPF_ST | BPF_MEM | BPF_DW:
+		if (insn->dst_reg == BPF_REG_PARAMS) {
+			emit_stack_arg_store_imm(imm, off, tmp, ctx);
+			break;
+		}
+		fallthrough;
 	case BPF_ST | BPF_PROBE_MEM32 | BPF_B:
 	case BPF_ST | BPF_PROBE_MEM32 | BPF_H:
 	case BPF_ST | BPF_PROBE_MEM32 | BPF_W:
@@ -1762,7 +1823,7 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 		}
 		if (dst == fp) {
 			dst_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
-			off_adj = off + ctx->stack_size;
+			off_adj = off + ctx->stack_size + ctx->stack_arg_size;
 		} else {
 			dst_adj = dst;
 			off_adj = off;
@@ -1814,6 +1875,11 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 	case BPF_STX | BPF_MEM | BPF_H:
 	case BPF_STX | BPF_MEM | BPF_B:
 	case BPF_STX | BPF_MEM | BPF_DW:
+		if (insn->dst_reg == BPF_REG_PARAMS) {
+			emit_stack_arg_store(src, off, ctx);
+			break;
+		}
+		fallthrough;
 	case BPF_STX | BPF_PROBE_MEM32 | BPF_B:
 	case BPF_STX | BPF_PROBE_MEM32 | BPF_H:
 	case BPF_STX | BPF_PROBE_MEM32 | BPF_W:
@@ -1824,7 +1890,7 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 		}
 		if (dst == fp) {
 			dst_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
-			off_adj = off + ctx->stack_size;
+			off_adj = off + ctx->stack_size + ctx->stack_arg_size;
 		} else {
 			dst_adj = dst;
 			off_adj = off;
@@ -2065,6 +2131,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
 	ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
 	ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
 
+	if (prog->aux->stack_arg_depth > prog->aux->incoming_stack_arg_depth) {
+		u16 outgoing = prog->aux->stack_arg_depth - prog->aux->incoming_stack_arg_depth;
+		int nr_on_stack = outgoing / sizeof(u64) - NR_STACK_ARG_REGS;
+
+		if (nr_on_stack > 0)
+			ctx.stack_arg_size = round_up(nr_on_stack * sizeof(u64), 16);
+	}
+
 	if (priv_stack_ptr)
 		ctx.priv_sp_used = true;
 
@@ -2229,6 +2303,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_stack_args(void)
+{
+	return true;
+}
+
 void *bpf_arch_text_copy(void *dst, void *src, size_t len)
 {
 	if (!aarch64_insn_copy(dst, src, len))
-- 
2.52.0

[PATCH bpf-next 18/18] selftests/bpf: Enable stack argument tests for arm64

[Yonghong Song]

From: Puranjay Mohan <puranjay@kernel.org>

Now that arm64 supports stack arguments, enable the existing stack_arg,
stack_arg_kfunc and verifier_stack_arg tests for __TARGET_ARCH_arm64.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c | 3 ++-
 tools/testing/selftests/bpf/progs/stack_arg.c                | 3 ++-
 tools/testing/selftests/bpf/progs/stack_arg_kfunc.c          | 3 ++-
 tools/testing/selftests/bpf/progs/stack_arg_precision.c      | 3 ++-
 tools/testing/selftests/bpf/progs/verifier_stack_arg.c       | 3 ++-
 5 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c b/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
index 296fddfe6804..8d38aafe66a2 100644
--- a/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
+++ b/tools/testing/selftests/bpf/progs/btf__stack_arg_precision.c
@@ -4,7 +4,8 @@
 #include <bpf/bpf_helpers.h>
 #include "../test_kmods/bpf_testmod_kfunc.h"
 
-#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+#if (defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)) && \
+	defined(__BPF_FEATURE_STACK_ARGUMENT)
 
 long subprog_call_mem_kfunc(long a, long b, long c, long d, long e, long size)
 {
diff --git a/tools/testing/selftests/bpf/progs/stack_arg.c b/tools/testing/selftests/bpf/progs/stack_arg.c
index ab6240b997c5..b5e9929a4d63 100644
--- a/tools/testing/selftests/bpf/progs/stack_arg.c
+++ b/tools/testing/selftests/bpf/progs/stack_arg.c
@@ -21,7 +21,8 @@ struct {
 
 int timer_result;
 
-#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+#if (defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)) && \
+	defined(__BPF_FEATURE_STACK_ARGUMENT)
 
 const volatile bool has_stack_arg = true;
 
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c b/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c
index fa9def876ea5..da0d4f91d273 100644
--- a/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c
+++ b/tools/testing/selftests/bpf/progs/stack_arg_kfunc.c
@@ -6,7 +6,8 @@
 #include "bpf_kfuncs.h"
 #include "../test_kmods/bpf_testmod_kfunc.h"
 
-#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+#if (defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)) && \
+	defined(__BPF_FEATURE_STACK_ARGUMENT)
 
 const volatile bool has_stack_arg = true;
 
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_precision.c b/tools/testing/selftests/bpf/progs/stack_arg_precision.c
index 29b2f2aea931..460d1872a84c 100644
--- a/tools/testing/selftests/bpf/progs/stack_arg_precision.c
+++ b/tools/testing/selftests/bpf/progs/stack_arg_precision.c
@@ -6,7 +6,8 @@
 #include "../test_kmods/bpf_testmod_kfunc.h"
 #include "bpf_misc.h"
 
-#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+#if (defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)) && \
+	defined(__BPF_FEATURE_STACK_ARGUMENT)
 
 /* Force kfunc extern BTF generation for inline asm call below.
  * Uses its own SEC so it's not included as a .text subprog.
diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
index 6b596ad63774..b412c311b757 100644
--- a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
+++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
@@ -12,7 +12,8 @@ struct {
 	__type(value, long long);
 } map_hash_8b SEC(".maps");
 
-#if defined(__TARGET_ARCH_x86) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+#if (defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)) && \
+	defined(__BPF_FEATURE_STACK_ARGUMENT)
 
 __noinline __used
 static int subprog_6args(int a, int b, int c, int d, int e, int f)
-- 
2.52.0

Re: [PATCH bpf-next 13/18] selftests/bpf: Add verifier tests for stack argument validation

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2453 bytes --]

> diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
> new file mode 100644
> index 000000000000..6b596ad63774
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
> @@ -0,0 +1,456 @@

[ ... ]

> +SEC("tc")
> +__description("stack_arg: pruning with different stack arg types")
> +__failure
> +__flag(BPF_F_TEST_STATE_FREQ)
> +__msg("invalid mem access 'scalar'")
> +__naked void stack_arg_pruning_type_mismatch(void)
> +{
> +	asm volatile (
> +		"call %[bpf_get_prandom_u32];"
> +		"r6 = r0;"
> +		/* local = 0 on program stack */
> +		"r7 = 0;"
> +		"*(u64 *)(r10 - 8) = r7;"
> +		/* Branch based on random value */
> +		"if r6 s> 3 goto l0_%=;"
> +		/* Path 1: store stack pointer to outgoing arg6 */
> +		"r1 = r10;"
> +		"r1 += -8;"
> +		"*(u64 *)(r11 - 8) = r1;"
> +		"goto l1_%=;"
> +	"l0_%=:"
> +		/* Path 2: store scalar to outgoing arg6 */
> +		"*(u64 *)(r11 - 8) = 42;"
> +	"l1_%=:"
> +		/* Call subprog that dereferences arg6 */
> +		"r1 = r6;"
> +		"r2 = 0;"
> +		"r3 = 0;"
> +		"r4 = 0;"
> +		"r5 = 0;"
> +		"call subprog_deref_arg6;"
> +		"exit;"
> +		:: __imm(bpf_get_prandom_u32)
> +		: __clobber_all
> +	);
> +}

Does the error message string match the expected register output?

During v6 review, sashiko-bot raised a concern about test fragility:
since subprog_deref_arg6 is standard C (not __naked), LLVM determines
which register is used to dereference f. If a future LLVM version
chooses a different register, the verifier error will mention that
register instead of R1, causing the test to fail.

You acknowledged the issue and said you would use a regex like
__msg("R{{[0-9]}} invalid mem access 'scalar'") to match the register
dynamically. However, the current code uses __msg("invalid mem access
'scalar'") without any register prefix.

The same pattern appears in stack_arg_release_ref,
stack_arg_stale_pkt_ptr, and stack_arg_null_propagation_fail. Should
these tests use the regex pattern to avoid compiler-dependent failures,
or be rewritten to make subprog_deref_arg6 a __naked function?

Reference:
https://lore.kernel.org/bpf/20260419172100.A89FEC2BCAF@smtp.kernel.org/

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 06/18] bpf: Prepare architecture JIT support for stack arguments

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2400 bytes --]

> diff --git a/include/linux/filter.h b/include/linux/filter.h
> index b77d0b06db6e..911205dd670e 100644
> --- a/include/linux/filter.h
> +++ b/include/linux/filter.h

[ ... ]

> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index cfb35a2decf6..fa4b971c5289 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c

[ ... ]

> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> index ec8523e6e4eb..7522b3d7b267 100644
> --- a/kernel/bpf/core.c
> +++ b/kernel/bpf/core.c

[ ... ]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index e041c182c614..0ba140dabe93 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -12254,6 +12254,11 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>  			MAX_BPF_FUNC_REG_ARGS);
>  		return -EINVAL;
>  	}
> +	if (nargs > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
> +		verbose(env, "JIT does not support kfunc %s() with %d args\n",
> +			func_name, nargs);
> +		return -ENOTSUPP;
> +	}

Can this new check ever execute? The existing check immediately above already
rejects all kfuncs with nargs > MAX_BPF_FUNC_REG_ARGS by returning -EINVAL.

Since control only reaches the new if statement when the previous check
passes, nargs must be <= MAX_BPF_FUNC_REG_ARGS. The condition
'nargs > MAX_BPF_FUNC_REG_ARGS' in the new check is therefore always false
at this point.

Looking at the commit message's statement "Reject BPF functions and kfuncs
with more than 5 parameters at verification time if the architecture does not
support stack arguments," it appears the intention was to conditionally reject
based on JIT support. However, for kfuncs, the unconditional rejection
happens first.

The btf_prepare_func_args() change does work as intended because its earlier
guards allow nargs up to MAX_BPF_FUNC_ARGS for certain subprogs before
reaching the stack-args check.

A later commit in this series (73a990fcc429) raises the first check in
check_kfunc_args() from MAX_BPF_FUNC_REG_ARGS to MAX_BPF_FUNC_ARGS, which
would make the new branch reachable. Should this hunk be combined with that
change to avoid introducing unreachable code?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 05/18] bpf: Reject stack arguments in non-JITed programs

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1505 bytes --]

> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> index ae10b9ca018d..ec8523e6e4eb 100644
> --- a/kernel/bpf/core.c
> +++ b/kernel/bpf/core.c
> @@ -2599,7 +2599,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
>  		goto finalize;
>
>  	if (IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON) ||
> -	    bpf_prog_has_kfunc_call(fp))
> +	    bpf_prog_has_kfunc_call(fp) || fp->aux->stack_arg_depth)
>  		jit_needed = true;
>
>  	if (!bpf_prog_select_interpreter(fp))
> diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
> index 7d276208f3cc..86576f5cfe82 100644
> --- a/kernel/bpf/fixups.c
> +++ b/kernel/bpf/fixups.c
> @@ -1411,6 +1411,12 @@ int bpf_fixup_call_args(struct bpf_verifier_env *env)
>  		verbose(env, "calling kernel functions are not allowed in non-JITed programs\n");
>  		return -EINVAL;
>  	}
> +	for (i = 1; i < env->subprog_cnt; i++) {
> +		if (env->subprog_info[i].incoming_stack_arg_depth) {
> +			verbose(env, "stack args are not supported in non-JITed programs\n");
> +			return -EINVAL;
> +		}
> +	}
>  	if (env->subprog_cnt > 1 && env->prog->aux->tail_call_reachable) {
>  		/* When JIT fails the progs with bpf2bpf calls and tail_calls
>  		 * have to be rejected, since interpreter doesn't support them yet.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 17/18] bpf, arm64: Add JIT support for stack arguments

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4171 bytes --]

> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c

[ ... ]

> @@ -1671,7 +1727,7 @@
>  	}
>  	if (src == fp) {
>  		src_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
> -		off_adj = off + ctx->stack_size;
> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>  	} else {
>  		src_adj = src;
>  		off_adj = off;

When priv_sp_used is true, does this offset calculation produce correct
addresses for BPF fp-relative accesses?

Looking at build_prologue:

arch/arm64/net/bpf_jit_comp.c:build_prologue() {
    ...
    emit(A64_MOV(1, fp, priv_sp), ctx);
    emit(A64_ADD_I(1, fp, fp, ctx->stack_size), ctx);  /* line 612 */

    if (ctx->stack_size && !ctx->priv_sp_used)
        emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);  /* line 620 */

    if (ctx->stack_arg_size)
        emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_arg_size), ctx);  /* line 623 */
    ...
}

When priv_sp_used is true:
- fp = priv_sp + stack_size (line 612)
- main SP is not decremented by stack_size (skipped at line 620)
- main SP is decremented by stack_arg_size (line 623-624)

So the BPF program data lives on the private stack, while outgoing stack
args live on the real SP. These are separate memory regions.

In build_insn, the computed address for fp-relative loads/stores becomes:
    addr = priv_sp + off + stack_size + stack_arg_size

But the intended BPF address is:
    addr = fp + off = priv_sp + stack_size + off

The computed address is stack_arg_size bytes too high, accessing memory
above the intended location on the private stack.

This would read/write into the private-stack guard region (16 bytes) or
past it when stack_arg_size > 16. Can this occur?  The verifier only
forbids tail_calls + stack_args (verifier.c:5657), not priv_stack +
stack_args. A kprobe BPF program that uses BPF_REG_FP and calls a kfunc
with 9+ arguments would trigger wrong memory accesses.

Note that x86's bpf_int_jit_compile prevents priv_stack allocation when
stack_arg_depth != 0 (x86/bpf_jit_comp.c:3883). The arm64 version has no
equivalent guard.

The same issue affects the ST case:

> @@ -1762,7 +1823,7 @@
>  	}
>  	if (dst == fp) {
>  		dst_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
> -		off_adj = off + ctx->stack_size;
> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>  	} else {
>  		dst_adj = dst;
>  		off_adj = off;

and the STX case:

> @@ -1824,7 +1890,7 @@
>  	}
>  	if (dst == fp) {
>  		dst_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
> -		off_adj = off + ctx->stack_size;
> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>  	} else {
>  		dst_adj = dst;
>  		off_adj = off;

[ ... ]

> @@ -2065,6 +2131,14 @@
>  	ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
>  	ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
>
> +	if (prog->aux->stack_arg_depth > prog->aux->incoming_stack_arg_depth) {
> +		u16 outgoing = prog->aux->stack_arg_depth - prog->aux->incoming_stack_arg_depth;
> +		int nr_on_stack = outgoing / sizeof(u64) - NR_STACK_ARG_REGS;
> +
> +		if (nr_on_stack > 0)
> +			ctx.stack_arg_size = round_up(nr_on_stack * sizeof(u64), 16);
> +	}
> +

Alexei Starovoitov raised concerns about this logic during v1 review (Apr
21, 2026). He found the comparison between stack_arg_depth and
incoming_stack_arg_depth confusing and suggested simplifying it:

"I think stack_arg_depth should mean outgoing and
incoming_stack_arg_depth should mean incoming only and it shouldn't be
even used by JIT. That memory was allocated by caller, so to JIT this
callee the conversion of r11+const is straightforward and no checks
necessary. [...] I think it should really be one check based on
stack_arg_depth."

From https://lore.kernel.org/bpf/CAADnVQ+uNGK_XKwb09d+xHZk_5-RWk=4_ZQTaQP24ej4+Gc8HA@mail.gmail.com/

Should the logic be simplified as Alexei suggested?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 09/18] bpf: Reject stack arguments if tail call reachable

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1674 bytes --]

> bpf: Reject stack arguments if tail call reachable
>
> Tailcalls have been deprecated. So reject stack arguments
> if tail call is in the way.

Multiple reviewers (bot+bpf-ci@kernel.org and sashiko-bot@kernel.org) raised
concerns across v4, v5, and v6 that the commit message inaccurately states
"Tailcalls have been deprecated" when the actual technical reason is that tail
calls and stack arguments require incompatible stack frame handling.

The author acknowledged this feedback in v5 but chose to keep the commit
message as-is, stating "Stack arguments and tail calls CAN work together.
However, since tail calls are in deprecation stage, it is not worthwhile to
involve stack arguments and tail calls together."

Could the commit message be more precise about the technical reason for this
restriction? Something along the lines of "tail calls and stack arguments
require incompatible stack frame handling" would better explain to future
readers why the combination is rejected, rather than relying on the
deprecation rationale.

Reference: https://lore.kernel.org/bpf/f163dcbf46e9511eba14b22ed10467984bda3bf07a3d1bd6cd27ae14eadc2056@mail.kernel.org/

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

This commit fixes a bug but lacks a Fixes: tag. The suggested tag is:

Fixes: 8ade62ae1543 ("bpf: Support stack arguments for bpf functions")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 08/18] bpf: Support stack arguments for kfunc calls

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2582 bytes --]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 6994536b4e04..43aeb04f488a 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> @@ -12304,7 +12314,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>  					reg_arg_name(env, argno));
>  				return -EINVAL;
>  			}
> -			ret = mark_chain_precision(env, regno);
> +			if (regno >= 0)
> +				ret = mark_chain_precision(env, regno);
> +			else
> +				ret = mark_stack_arg_precision(env, i);
>  			if (ret < 0)
>  				return ret;
>  			meta->arg_constant.found = true;

[ ... ]

> @@ -12329,7 +12342,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>  			}
>
>  			meta->r0_size = reg->var_off.value;
> -			ret = mark_chain_precision(env, regno);
> +			if (regno >= 0)
> +				ret = mark_chain_precision(env, regno);
> +			else
> +				ret = mark_stack_arg_precision(env, i);
>  			if (ret)
>  				return ret;
>  		}

In the v6 review, Sashiko AI pointed out that setting reg->precise = true
without calling mark_chain_precision() breaks data-flow backtracking. The
concern was that if only the current state's flag is set, parent states and
source registers that populated the stack argument won't be marked precise,
potentially allowing incorrect state pruning when execution paths merge before
the stack store.

The author acknowledged this issue in the v6 discussion with "You are right.
Let me think how to resolve this." However, the current v7 code still calls
mark_stack_arg_precision() for stack arguments in check_kfunc_args().

Does mark_stack_arg_precision() properly handle backtracking through parent
states? Looking at the two locations where stack argument precision is tracked:

check_kfunc_args() for is_kfunc_arg_const_mem_size():
    if (regno >= 0)
        ret = mark_chain_precision(env, regno);
    else
        ret = mark_stack_arg_precision(env, i);

check_kfunc_args() for KF_ARG_PTR_TO_ALLOC_BTF_ID with is_ret_buf_sz:
    if (regno >= 0)
        ret = mark_chain_precision(env, regno);
    else
        ret = mark_stack_arg_precision(env, i);

Is mark_stack_arg_precision() correctly marking parent states and the source
registers that stored values into the stack argument slots, or does it only
mark the current state as the v6 review suggested?



---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 10/18] bpf,x86: Implement JIT support for stack arguments

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 3988 bytes --]

> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index ea9e707e8abf..06f33fef1e8e 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c

[ ... ]

> @@ -1664,16 +1692,45 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
>  	int i, excnt = 0;
>  	int ilen, proglen = 0;
>  	u8 *ip, *prog = temp;
> +	u16 stack_arg_depth, incoming_stack_arg_depth, outgoing_stack_arg_depth;
> +	u16 outgoing_rsp;
>  	u32 stack_depth;
> +	int callee_saved_size;
> +	s32 outgoing_arg_base;
>  	int err;
>
>  	stack_depth = bpf_prog->aux->stack_depth;
> +	stack_arg_depth = bpf_prog->aux->stack_arg_depth;
> +	incoming_stack_arg_depth = bpf_prog->aux->incoming_stack_arg_depth;
> +	outgoing_stack_arg_depth = stack_arg_depth - incoming_stack_arg_depth;
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Can this underflow? All three variables are u16, so the subtraction
wraps around modulo 65536.

Looking at btf_prepare_func_args() in kernel/bpf/btf.c, it sets
'sub->incoming_stack_arg_depth = (nargs - MAX_BPF_FUNC_REG_ARGS) *
BPF_REG_SIZE' for any static subprog with more than 5 arguments. It does
not initialize sub->stack_arg_depth.

The 'stack_arg_depth' field is only written by check_func_call() in
kernel/bpf/verifier.c and check_kfunc_call(), which update the *caller's*
subprog entry when the caller makes a call with more than 5 arguments.

Since env->subprog_info[] is zero-initialized (kvcalloc), leaf subprograms
that receive stack arguments but never call other functions with stack
arguments will have:
  incoming_stack_arg_depth = (nargs - 5) * 8
  stack_arg_depth = 0

For example:
  static int leaf(int a1, int a2, int a3, int a4, int a5, int a6, int a7)
  {
      return a1 + a7;  /* r11+16 load, then BPF_EXIT - no calls */
  }

  /* some other subprog does: ret = leaf(1,2,3,4,5,6,7); */

For 'leaf', the verifier sets:
  incoming_stack_arg_depth = (7 - 5) * 8 = 16
  stack_arg_depth = 0    (never updated, no calls made)

When do_jit() computes:
  outgoing_stack_arg_depth = 0 - 16 = 0xFFF0 (u16 underflow)

Then at the new code below:

> +	/*
> +	 * Allocate outgoing stack arg area for args 7+ only.
> +	 * Arg 6 goes into r9 register, not on stack.
> +	 */
> +	outgoing_rsp = outgoing_stack_arg_depth > 8 ?  outgoing_stack_arg_depth - 8 : 0;
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +	emit_sub_rsp(&prog, outgoing_rsp);

This emits 'sub rsp, 0xFFE8' (about 65 KB), which overflows the kernel
stack (THREAD_SIZE is only 16 KB on x86_64). The epilogue then emits a
matching 'add rsp, 0xFFE8', but the damage is already done at function
entry.

The commit message states "The verifier guarantees that neither
tail_call_reachable nor priv_stack is set when stack args exist, so R9 is
always available." It does not claim the verifier maintains
'stack_arg_depth >= incoming_stack_arg_depth', and that invariant is not
maintained for leaf subprograms.

The sanity check in bpf_fixup_call_args() at kernel/bpf/fixups.c does not
catch this because the same underflow happens there:
  u16 outgoing = subprog->stack_arg_depth - subprog->incoming_stack_arg_depth;
but since leaf subprograms have max_out_stack_arg_depth == 0, the
comparison 'outgoing > max_out_stack_arg_depth' is false (0xFFF0 wraps
around in the comparison).

Should the JIT compute outgoing_stack_arg_depth safely to prevent the
underflow? For example:
  outgoing_stack_arg_depth = stack_arg_depth > incoming_stack_arg_depth ?
                             stack_arg_depth - incoming_stack_arg_depth : 0;

Or should the verifier ensure the invariant 'stack_arg_depth >=
incoming_stack_arg_depth' after verification completes?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 04/18] bpf: Extend liveness analysis to track stack argument slots

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1603 bytes --]

> bpf: Extend liveness analysis to track stack argument slots
>
> BPF_REG_PARAMS (R11) is at index MAX_BPF_REG, which is beyond the
> register tracking arrays in const_fold.c and liveness.c. Handle it
> explicitly to avoid out-of-bounds accesses.
>
> In liveness.c, extend the arg tracking dataflow to cover stack arg
> slots. Otherwise, pointers passed through stack args are invisible
> to liveness, causing the pointed-to stack slots to be incorrectly
> poisoned.
>
> Add a parallel tracking array (at_stack_arg_out/at_stack_arg_entry)
> in arg_track_xfer() to record FP-derived values stored to outgoing
> stack arg slots and to restore them on incoming reads. Extend
> record_call_access() to check stack arg slots for FP-derived
> pointers at kfunc call sites, reusing the record_arg_access() helper
> extracted in the previous patch. Pass stack arg state from caller to
> callee in analyze_subprog() so that callees can track pointers
> received through stack args.
>
> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>

The commit message describes fixing out-of-bounds accesses and extending
liveness analysis, which suggests this fixes a bug introduced in a prior
commit. Should this include a Fixes: tag? Looking at the commit history, it
appears this should reference:

    Fixes: 8ade62ae1543 ("bpf: Support stack arguments for bpf functions")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 02/18] bpf: Add precision marking and backtracking for stack argument slots

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

> bpf: Add precision marking and backtracking for stack argument slots
>
> Extend the precision marking and backtracking infrastructure to
> support stack argument slots (r11-based accesses). Without this,
> precision demands for scalar values passed through stack arguments
> are silently dropped, which could allow the verifier to incorrectly
> prune states with different constant values in stack arg slots.

This commit appears to be fixing a bug introduced by commit 8ade62ae1543
("bpf: Support stack arguments for bpf functions"). Should this commit include
a Fixes: tag?

Suggested: Fixes: 8ade62ae1543 ("bpf: Support stack arguments for bpf functions")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 01/18] bpf: Support stack arguments for bpf functions

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 8162 bytes --]

> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> index d5b4303315dd..2cc349d7fc17 100644
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h

[ ... ]

> @@ -739,10 +759,13 @@ struct bpf_subprog_info {
>  	bool keep_fastcall_stack: 1;
>  	bool changes_pkt_data: 1;
>  	bool might_sleep: 1;
> -	u8 arg_cnt:3;
> +	u8 arg_cnt:4;
>
>  	enum priv_stack_mode priv_stack_mode;
> -	struct bpf_subprog_arg_info args[MAX_BPF_FUNC_REG_ARGS];
> +	struct bpf_subprog_arg_info args[MAX_BPF_FUNC_ARGS];
> +	u16 incoming_stack_arg_depth;
> +	u16 stack_arg_depth; /* incoming + max outgoing */
> +	u16 max_out_stack_arg_depth;
>  };

Alexei Starovoitov raised a concern in the v6 review about duplicated
fields. The incoming_stack_arg_depth, stack_arg_depth, and
max_out_stack_arg_depth fields are present in both bpf_subprog_info and
bpf_prog_aux structures. His comment was:

  "but you already have them in prog_aux?! another copy in
  bpf_subprog_info?! Remove one of them. JIT only need one set."

Looking at include/linux/bpf.h, these fields appear in bpf_prog_aux:

  struct bpf_prog_aux {
      ...
      u16 incoming_stack_arg_depth;
      u16 stack_arg_depth;
      ...
  };

Does the verifier actually need both copies? The concern was that having
duplicate state creates maintenance burden.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index ff6ff1c27517..bcf81692a22b 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> +/*
> + * Write a value to the outgoing stack arg area.
> + * off is a negative offset from r11 (e.g. -8 for arg6, -16 for arg7).
> + */
> +static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_state *state,
> +				 int off, int value_regno)
> +{
> +	int max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;
> +	struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno];
> +	int spi = -off / BPF_REG_SIZE - 1;
> +	struct bpf_func_state *cur;
> +	struct bpf_reg_state *arg;
> +	int err;
> +
> +	if (spi >= max_stack_arg_regs) {
> +		verbose(env, "stack arg write offset %d exceeds max %d stack args\n",
> +			off, max_stack_arg_regs);
> +		return -EINVAL;
> +	}
> +
> +	err = grow_stack_arg_slots(env, state, -off);
> +	if (err)
> +		return err;
> +
> +	/* Track the max outgoing stack arg access depth. */
> +	if (-off > subprog->max_out_stack_arg_depth)
> +		subprog->max_out_stack_arg_depth = -off;
> +
> +	cur = env->cur_state->frame[env->cur_state->curframe];
> +	if (value_regno >= 0) {
> +		state->stack_arg_regs[spi] = cur->regs[value_regno];

Can this code handle offset=0 correctly? For an instruction like
'*(u64 *)(r11 + 0) = r6', the computation spi = -0 / 8 - 1 = -1. The
bounds check 'spi >= max_stack_arg_regs' would be false since -1 < 7,
so execution proceeds to 'state->stack_arg_regs[-1]' which is an
out-of-bounds array access.

This also affects misaligned negative offsets. For off=-4, the result is
spi = 4/8 - 1 = -1, causing the same OOB write.

Additionally, there's no validation of the access size. A 4-byte write
'*(u32 *)(r11 - 8) = r6' would be accepted and recorded as an 8-byte
register spill in stack_arg_regs[0], but the JIT would emit a 4-byte
store. The verifier's tracked state would not match the machine state.

Looking at the referenced commit 5a36f6a2934e ("bpf: Enable r11 based
insns") in the patch series, it adds validation in
check_and_resolve_insns() that rejects r11 stores with off >= 0, non
8-byte aligned offsets, and non-BPF_DW sizes. Should this validation be
in the current commit instead?

> +/*
> + * Read a value from the incoming stack arg area.
> + * off is a positive offset from r11 (e.g. +8 for arg6, +16 for arg7).
> + */
> +static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_state *state,
> +				int off, int dst_regno)
> +{
> +	struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno];
> +	struct bpf_verifier_state *vstate = env->cur_state;
> +	int spi = off / BPF_REG_SIZE - 1;
> +	struct bpf_func_state *caller, *cur;
> +	struct bpf_reg_state *arg;
> +
> +	if (state->no_stack_arg_load) {
> +		verbose(env, "r11 load must be before any r11 store or call insn\n");
> +		return -EINVAL;
> +	}
> +
> +	if (off > subprog->incoming_stack_arg_depth) {
> +		verbose(env, "invalid read from stack arg off %d depth %d\n",
> +			off, subprog->incoming_stack_arg_depth);
> +		return -EACCES;
> +	}
> +
> +	caller = vstate->frame[vstate->curframe - 1];
> +	arg = &caller->stack_arg_regs[spi];

Can this handle off=0 safely? For a load '*(u64 *)(r11 + 0)', spi would
be 0/8 - 1 = -1, and the check 'off > incoming_stack_arg_depth' (0 >
non-negative) would be false. Execution would continue to
'caller->stack_arg_regs[-1]', which is out-of-bounds.

What about when this runs in the main program where curframe==0? The
expression 'vstate->frame[vstate->curframe - 1]' would compute
frame[(u32)-1], accessing frame[0xFFFFFFFF] which is far beyond the
bounds of the frame[] array (MAX_CALL_FRAMES entries).

Additionally, the code does not validate access size. A 4-byte load
'r0 = *(u32 *)(r11 + 8)' would be accepted and the verifier would copy
the full 8-byte register from caller->stack_arg_regs[0], but the JIT
would emit a 4-byte read.

The later commit 5a36f6a2934e ("bpf: Enable r11 based insns") adds
validation requiring loads to be BPF_LDX|BPF_MEM|BPF_DW with off > 0 and
8-byte aligned. Should that be part of this commit?

> @@ -6605,10 +6737,20 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  			  bool strict_alignment_once, bool is_ldsx,
>  			  bool allow_trust_mismatch, const char *ctx)
>  {
> +	struct bpf_verifier_state *vstate = env->cur_state;
> +	struct bpf_func_state *state = vstate->frame[vstate->curframe];
>  	struct bpf_reg_state *regs = cur_regs(env);
>  	enum bpf_reg_type src_reg_type;
>  	int err;
>
> +	/* Handle stack arg read */
> +	if (insn->src_reg == BPF_REG_PARAMS) {
> +		err = check_reg_arg(env, insn->dst_reg, DST_OP_NO_MARK);
> +		if (err)
> +			return err;
> +		return check_stack_arg_read(env, state, insn->off, insn->dst_reg);
> +	}

Does this dispatch check the BPF_SIZE? A non-BPF_DW load through r11
(e.g., 'r0 = *(u32 *)(r11 + 8)' which is BPF_W) goes to
check_stack_arg_read, which treats it as a full 8-byte register copy
from the caller. The verifier state would not match what the JIT emits.

> @@ -6637,10 +6779,20 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  static int check_store_reg(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  			   bool strict_alignment_once)
>  {
> +	struct bpf_verifier_state *vstate = env->cur_state;
> +	struct bpf_func_state *state = vstate->frame[vstate->curframe];
>  	struct bpf_reg_state *regs = cur_regs(env);
>  	enum bpf_reg_type dst_reg_type;
>  	int err;
>
> +	/* Handle stack arg write */
> +	if (insn->dst_reg == BPF_REG_PARAMS) {
> +		err = check_reg_arg(env, insn->src_reg, SRC_OP);
> +		if (err)
> +			return err;
> +		return check_stack_arg_write(env, state, insn->off, insn->src_reg);
> +	}

Similarly for stores, does this gate on BPF_SIZE? A 4-byte store
'*(u32 *)(r11 - 8) = r6' (BPF_W) would be accepted here and recorded as
an 8-byte spill, creating the same verifier/JIT mismatch.

> @@ -17627,6 +17804,14 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
>  		return check_store_reg(env, insn, false);
>
>  	case BPF_ST: {
> +		/* Handle stack arg write (store immediate) */
> +		if (insn->dst_reg == BPF_REG_PARAMS) {
> +			struct bpf_verifier_state *vstate = env->cur_state;
> +			struct bpf_func_state *state = vstate->frame[vstate->curframe];
> +
> +			return check_stack_arg_write(env, state, insn->off, -1);
> +		}

Does this check BPF_SIZE as well? A BPF_ST with dst_reg=BPF_REG_PARAMS
and size != BPF_DW would have the same issue.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 01/18] bpf: Support stack arguments for bpf functions

[Yonghong Song]

On 4/24/26 11:13 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
>> index d5b4303315dd..2cc349d7fc17 100644
>> --- a/include/linux/bpf_verifier.h
>> +++ b/include/linux/bpf_verifier.h
> [ ... ]
>
>> @@ -739,10 +759,13 @@ struct bpf_subprog_info {
>>   	bool keep_fastcall_stack: 1;
>>   	bool changes_pkt_data: 1;
>>   	bool might_sleep: 1;
>> -	u8 arg_cnt:3;
>> +	u8 arg_cnt:4;
>>
>>   	enum priv_stack_mode priv_stack_mode;
>> -	struct bpf_subprog_arg_info args[MAX_BPF_FUNC_REG_ARGS];
>> +	struct bpf_subprog_arg_info args[MAX_BPF_FUNC_ARGS];
>> +	u16 incoming_stack_arg_depth;
>> +	u16 stack_arg_depth; /* incoming + max outgoing */
>> +	u16 max_out_stack_arg_depth;
>>   };
> Alexei Starovoitov raised a concern in the v6 review about duplicated
> fields. The incoming_stack_arg_depth, stack_arg_depth, and
> max_out_stack_arg_depth fields are present in both bpf_subprog_info and
> bpf_prog_aux structures. His comment was:
>
>    "but you already have them in prog_aux?! another copy in
>    bpf_subprog_info?! Remove one of them. JIT only need one set."
>
> Looking at include/linux/bpf.h, these fields appear in bpf_prog_aux:
>
>    struct bpf_prog_aux {
>        ...
>        u16 incoming_stack_arg_depth;
>        u16 stack_arg_depth;
>        ...
>    };
>
> Does the verifier actually need both copies? The concern was that having
> duplicate state creates maintenance burden.

Before jit_subprogs(), bpf_subprog_info stores the information for each subprog.
At this point, the subprog has not been allocated and only main prog is available.
Therefore, it is not possible to copy bpf_subprog_info to each subprog.

So I think it might be necessary to have both fields in bpf_prog_aux and
bpf_subprog_info.

This version made some changes so max_out_stack_arg_depth is not needed
any more. Will remove it in the next revision.

>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index ff6ff1c27517..bcf81692a22b 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
> [ ... ]
>
>> +/*
>> + * Write a value to the outgoing stack arg area.
>> + * off is a negative offset from r11 (e.g. -8 for arg6, -16 for arg7).
>> + */
>> +static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_state *state,
>> +				 int off, int value_regno)
>> +{
>> +	int max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;
>> +	struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno];
>> +	int spi = -off / BPF_REG_SIZE - 1;
>> +	struct bpf_func_state *cur;
>> +	struct bpf_reg_state *arg;
>> +	int err;
>> +
>> +	if (spi >= max_stack_arg_regs) {
>> +		verbose(env, "stack arg write offset %d exceeds max %d stack args\n",
>> +			off, max_stack_arg_regs);
>> +		return -EINVAL;
>> +	}
>> +
>> +	err = grow_stack_arg_slots(env, state, -off);
>> +	if (err)
>> +		return err;
>> +
>> +	/* Track the max outgoing stack arg access depth. */
>> +	if (-off > subprog->max_out_stack_arg_depth)
>> +		subprog->max_out_stack_arg_depth = -off;
>> +
>> +	cur = env->cur_state->frame[env->cur_state->curframe];
>> +	if (value_regno >= 0) {
>> +		state->stack_arg_regs[spi] = cur->regs[value_regno];
> Can this code handle offset=0 correctly? For an instruction like

This is not possible for this one and below other questions.
See commit 'bpf: Enable r11 based insns'. For load, the offset
must be 8, 16, .... For store, the offset must be -8, -16, etc.
And the size must be 8 (BPF_DW).

Also, at this point, r11 usage is not enabled and it will fail
at check_and_resolve_insns() way before any actual verification
(do_check_main()). So we should be okay.

> '*(u64 *)(r11 + 0) = r6', the computation spi = -0 / 8 - 1 = -1. The
> bounds check 'spi >= max_stack_arg_regs' would be false since -1 < 7,
> so execution proceeds to 'state->stack_arg_regs[-1]' which is an
> out-of-bounds array access.
>
> This also affects misaligned negative offsets. For off=-4, the result is
> spi = 4/8 - 1 = -1, causing the same OOB write.
>
> Additionally, there's no validation of the access size. A 4-byte write
> '*(u32 *)(r11 - 8) = r6' would be accepted and recorded as an 8-byte
> register spill in stack_arg_regs[0], but the JIT would emit a 4-byte
> store. The verifier's tracked state would not match the machine state.
>
> Looking at the referenced commit 5a36f6a2934e ("bpf: Enable r11 based
> insns") in the patch series, it adds validation in
> check_and_resolve_insns() that rejects r11 stores with off >= 0, non
> 8-byte aligned offsets, and non-BPF_DW sizes. Should this validation be
> in the current commit instead?
>
>> +/*
>> + * Read a value from the incoming stack arg area.
>> + * off is a positive offset from r11 (e.g. +8 for arg6, +16 for arg7).
>> + */
>> +static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_state *state,
>> +				int off, int dst_regno)
>> +{
>> +	struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno];
>> +	struct bpf_verifier_state *vstate = env->cur_state;
>> +	int spi = off / BPF_REG_SIZE - 1;
>> +	struct bpf_func_state *caller, *cur;
>> +	struct bpf_reg_state *arg;
>> +
>> +	if (state->no_stack_arg_load) {
>> +		verbose(env, "r11 load must be before any r11 store or call insn\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	if (off > subprog->incoming_stack_arg_depth) {
>> +		verbose(env, "invalid read from stack arg off %d depth %d\n",
>> +			off, subprog->incoming_stack_arg_depth);
>> +		return -EACCES;
>> +	}
>> +
>> +	caller = vstate->frame[vstate->curframe - 1];
>> +	arg = &caller->stack_arg_regs[spi];
> Can this handle off=0 safely? For a load '*(u64 *)(r11 + 0)', spi would
> be 0/8 - 1 = -1, and the check 'off > incoming_stack_arg_depth' (0 >
> non-negative) would be false. Execution would continue to
> 'caller->stack_arg_regs[-1]', which is out-of-bounds.
>
> What about when this runs in the main program where curframe==0? The
> expression 'vstate->frame[vstate->curframe - 1]' would compute
> frame[(u32)-1], accessing frame[0xFFFFFFFF] which is far beyond the
> bounds of the frame[] array (MAX_CALL_FRAMES entries).
>
> Additionally, the code does not validate access size. A 4-byte load
> 'r0 = *(u32 *)(r11 + 8)' would be accepted and the verifier would copy
> the full 8-byte register from caller->stack_arg_regs[0], but the JIT
> would emit a 4-byte read.
>
> The later commit 5a36f6a2934e ("bpf: Enable r11 based insns") adds
> validation requiring loads to be BPF_LDX|BPF_MEM|BPF_DW with off > 0 and
> 8-byte aligned. Should that be part of this commit?
>
>> @@ -6605,10 +6737,20 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
>>   			  bool strict_alignment_once, bool is_ldsx,
>>   			  bool allow_trust_mismatch, const char *ctx)
>>   {
>> +	struct bpf_verifier_state *vstate = env->cur_state;
>> +	struct bpf_func_state *state = vstate->frame[vstate->curframe];
>>   	struct bpf_reg_state *regs = cur_regs(env);
>>   	enum bpf_reg_type src_reg_type;
>>   	int err;
>>
>> +	/* Handle stack arg read */
>> +	if (insn->src_reg == BPF_REG_PARAMS) {
>> +		err = check_reg_arg(env, insn->dst_reg, DST_OP_NO_MARK);
>> +		if (err)
>> +			return err;
>> +		return check_stack_arg_read(env, state, insn->off, insn->dst_reg);
>> +	}
> Does this dispatch check the BPF_SIZE? A non-BPF_DW load through r11
> (e.g., 'r0 = *(u32 *)(r11 + 8)' which is BPF_W) goes to
> check_stack_arg_read, which treats it as a full 8-byte register copy
> from the caller. The verifier state would not match what the JIT emits.
>
>> @@ -6637,10 +6779,20 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
>>   static int check_store_reg(struct bpf_verifier_env *env, struct bpf_insn *insn,
>>   			   bool strict_alignment_once)
>>   {
>> +	struct bpf_verifier_state *vstate = env->cur_state;
>> +	struct bpf_func_state *state = vstate->frame[vstate->curframe];
>>   	struct bpf_reg_state *regs = cur_regs(env);
>>   	enum bpf_reg_type dst_reg_type;
>>   	int err;
>>
>> +	/* Handle stack arg write */
>> +	if (insn->dst_reg == BPF_REG_PARAMS) {
>> +		err = check_reg_arg(env, insn->src_reg, SRC_OP);
>> +		if (err)
>> +			return err;
>> +		return check_stack_arg_write(env, state, insn->off, insn->src_reg);
>> +	}
> Similarly for stores, does this gate on BPF_SIZE? A 4-byte store
> '*(u32 *)(r11 - 8) = r6' (BPF_W) would be accepted here and recorded as
> an 8-byte spill, creating the same verifier/JIT mismatch.
>
>> @@ -17627,6 +17804,14 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
>>   		return check_store_reg(env, insn, false);
>>
>>   	case BPF_ST: {
>> +		/* Handle stack arg write (store immediate) */
>> +		if (insn->dst_reg == BPF_REG_PARAMS) {
>> +			struct bpf_verifier_state *vstate = env->cur_state;
>> +			struct bpf_func_state *state = vstate->frame[vstate->curframe];
>> +
>> +			return check_stack_arg_write(env, state, insn->off, -1);
>> +		}
> Does this check BPF_SIZE as well? A BPF_ST with dst_reg=BPF_REG_PARAMS
> and size != BPF_DW would have the same issue.
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 02/18] bpf: Add precision marking and backtracking for stack argument slots

[Yonghong Song]

On 4/24/26 11:00 AM, bot+bpf-ci@kernel.org wrote:
>> bpf: Add precision marking and backtracking for stack argument slots
>>
>> Extend the precision marking and backtracking infrastructure to
>> support stack argument slots (r11-based accesses). Without this,
>> precision demands for scalar values passed through stack arguments
>> are silently dropped, which could allow the verifier to incorrectly
>> prune states with different constant values in stack arg slots.
> This commit appears to be fixing a bug introduced by commit 8ade62ae1543
> ("bpf: Support stack arguments for bpf functions"). Should this commit include
> a Fixes: tag?
>
> Suggested: Fixes: 8ade62ae1543 ("bpf: Support stack arguments for bpf functions")

Not really. See "bpf: Enable r11 based insns". So you can consider this
still a preparation patch.

>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 04/18] bpf: Extend liveness analysis to track stack argument slots

[Yonghong Song]

On 4/24/26 11:00 AM, bot+bpf-ci@kernel.org wrote:
>> bpf: Extend liveness analysis to track stack argument slots
>>
>> BPF_REG_PARAMS (R11) is at index MAX_BPF_REG, which is beyond the
>> register tracking arrays in const_fold.c and liveness.c. Handle it
>> explicitly to avoid out-of-bounds accesses.
>>
>> In liveness.c, extend the arg tracking dataflow to cover stack arg
>> slots. Otherwise, pointers passed through stack args are invisible
>> to liveness, causing the pointed-to stack slots to be incorrectly
>> poisoned.
>>
>> Add a parallel tracking array (at_stack_arg_out/at_stack_arg_entry)
>> in arg_track_xfer() to record FP-derived values stored to outgoing
>> stack arg slots and to restore them on incoming reads. Extend
>> record_call_access() to check stack arg slots for FP-derived
>> pointers at kfunc call sites, reusing the record_arg_access() helper
>> extracted in the previous patch. Pass stack arg state from caller to
>> callee in analyze_subprog() so that callees can track pointers
>> received through stack args.
>>
>> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
> The commit message describes fixing out-of-bounds accesses and extending
> liveness analysis, which suggests this fixes a bug introduced in a prior
> commit. Should this include a Fixes: tag? Looking at the commit history, it
> appears this should reference:
>
>      Fixes: 8ade62ae1543 ("bpf: Support stack arguments for bpf functions")

Not really. See "bpf: Enable r11 based insns". So you can consider this
still a preparation patch.

>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 06/18] bpf: Prepare architecture JIT support for stack arguments

[Yonghong Song]

On 4/24/26 10:48 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/include/linux/filter.h b/include/linux/filter.h
>> index b77d0b06db6e..911205dd670e 100644
>> --- a/include/linux/filter.h
>> +++ b/include/linux/filter.h
> [ ... ]
>
>> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
>> index cfb35a2decf6..fa4b971c5289 100644
>> --- a/kernel/bpf/btf.c
>> +++ b/kernel/bpf/btf.c
> [ ... ]
>
>> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
>> index ec8523e6e4eb..7522b3d7b267 100644
>> --- a/kernel/bpf/core.c
>> +++ b/kernel/bpf/core.c
> [ ... ]
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index e041c182c614..0ba140dabe93 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
>> @@ -12254,6 +12254,11 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>>   			MAX_BPF_FUNC_REG_ARGS);
>>   		return -EINVAL;
>>   	}
>> +	if (nargs > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
>> +		verbose(env, "JIT does not support kfunc %s() with %d args\n",
>> +			func_name, nargs);
>> +		return -ENOTSUPP;
>> +	}
> Can this new check ever execute? The existing check immediately above already
> rejects all kfuncs with nargs > MAX_BPF_FUNC_REG_ARGS by returning -EINVAL.
>
> Since control only reaches the new if statement when the previous check
> passes, nargs must be <= MAX_BPF_FUNC_REG_ARGS. The condition
> 'nargs > MAX_BPF_FUNC_REG_ARGS' in the new check is therefore always false
> at this point.
>
> Looking at the commit message's statement "Reject BPF functions and kfuncs
> with more than 5 parameters at verification time if the architecture does not
> support stack arguments," it appears the intention was to conditionally reject
> based on JIT support. However, for kfuncs, the unconditional rejection
> happens first.
>
> The btf_prepare_func_args() change does work as intended because its earlier
> guards allow nargs up to MAX_BPF_FUNC_ARGS for certain subprogs before
> reaching the stack-args check.
>
> A later commit in this series (73a990fcc429) raises the first check in
> check_kfunc_args() from MAX_BPF_FUNC_REG_ARGS to MAX_BPF_FUNC_ARGS, which
> would make the new branch reachable. Should this hunk be combined with that
> change to avoid introducing unreachable code?

You are right. The above check does not really work as it is a noop.
And it really works until "bpf: Support stack arguments for kfunc calls".
Therefore, as you mentioned, it should be combined with
"bpf: Support stack arguments for kfunc calls".

Will do with next revision.

>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 08/18] bpf: Support stack arguments for kfunc calls

[Yonghong Song]

On 4/24/26 11:00 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index 6994536b4e04..43aeb04f488a 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
> [ ... ]
>
>> @@ -12304,7 +12314,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>>   					reg_arg_name(env, argno));
>>   				return -EINVAL;
>>   			}
>> -			ret = mark_chain_precision(env, regno);
>> +			if (regno >= 0)
>> +				ret = mark_chain_precision(env, regno);
>> +			else
>> +				ret = mark_stack_arg_precision(env, i);
>>   			if (ret < 0)
>>   				return ret;
>>   			meta->arg_constant.found = true;
> [ ... ]
>
>> @@ -12329,7 +12342,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>>   			}
>>
>>   			meta->r0_size = reg->var_off.value;
>> -			ret = mark_chain_precision(env, regno);
>> +			if (regno >= 0)
>> +				ret = mark_chain_precision(env, regno);
>> +			else
>> +				ret = mark_stack_arg_precision(env, i);
>>   			if (ret)
>>   				return ret;
>>   		}
> In the v6 review, Sashiko AI pointed out that setting reg->precise = true
> without calling mark_chain_precision() breaks data-flow backtracking. The
> concern was that if only the current state's flag is set, parent states and
> source registers that populated the stack argument won't be marked precise,
> potentially allowing incorrect state pruning when execution paths merge before
> the stack store.
>
> The author acknowledged this issue in the v6 discussion with "You are right.
> Let me think how to resolve this." However, the current v7 code still calls
> mark_stack_arg_precision() for stack arguments in check_kfunc_args().
>
> Does mark_stack_arg_precision() properly handle backtracking through parent
> states? Looking at the two locations where stack argument precision is tracked:
>
> check_kfunc_args() for is_kfunc_arg_const_mem_size():
>      if (regno >= 0)
>          ret = mark_chain_precision(env, regno);
>      else
>          ret = mark_stack_arg_precision(env, i);
>
> check_kfunc_args() for KF_ARG_PTR_TO_ALLOC_BTF_ID with is_ret_buf_sz:
>      if (regno >= 0)
>          ret = mark_chain_precision(env, regno);
>      else
>          ret = mark_stack_arg_precision(env, i);
>
> Is mark_stack_arg_precision() correctly marking parent states and the source
> registers that stored values into the stack argument slots, or does it only
> mark the current state as the v6 review suggested?

I think it does proper precision marking and backtracking. There is a test
for this, "selftests/bpf: Add precision backtracking test for stack arguments".
Please take a look.

>
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 10/18] bpf,x86: Implement JIT support for stack arguments

[Yonghong Song]

On 4/24/26 11:00 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
>> index ea9e707e8abf..06f33fef1e8e 100644
>> --- a/arch/x86/net/bpf_jit_comp.c
>> +++ b/arch/x86/net/bpf_jit_comp.c
> [ ... ]
>
>> @@ -1664,16 +1692,45 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
>>   	int i, excnt = 0;
>>   	int ilen, proglen = 0;
>>   	u8 *ip, *prog = temp;
>> +	u16 stack_arg_depth, incoming_stack_arg_depth, outgoing_stack_arg_depth;
>> +	u16 outgoing_rsp;
>>   	u32 stack_depth;
>> +	int callee_saved_size;
>> +	s32 outgoing_arg_base;
>>   	int err;
>>
>>   	stack_depth = bpf_prog->aux->stack_depth;
>> +	stack_arg_depth = bpf_prog->aux->stack_arg_depth;
>> +	incoming_stack_arg_depth = bpf_prog->aux->incoming_stack_arg_depth;
>> +	outgoing_stack_arg_depth = stack_arg_depth - incoming_stack_arg_depth;
>                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Can this underflow? All three variables are u16, so the subtraction
> wraps around modulo 65536.
>
> Looking at btf_prepare_func_args() in kernel/bpf/btf.c, it sets
> 'sub->incoming_stack_arg_depth = (nargs - MAX_BPF_FUNC_REG_ARGS) *
> BPF_REG_SIZE' for any static subprog with more than 5 arguments. It does
> not initialize sub->stack_arg_depth.
>
> The 'stack_arg_depth' field is only written by check_func_call() in
> kernel/bpf/verifier.c and check_kfunc_call(), which update the *caller's*
> subprog entry when the caller makes a call with more than 5 arguments.
>
> Since env->subprog_info[] is zero-initialized (kvcalloc), leaf subprograms
> that receive stack arguments but never call other functions with stack
> arguments will have:
>    incoming_stack_arg_depth = (nargs - 5) * 8
>    stack_arg_depth = 0
>
> For example:
>    static int leaf(int a1, int a2, int a3, int a4, int a5, int a6, int a7)
>    {
>        return a1 + a7;  /* r11+16 load, then BPF_EXIT - no calls */
>    }
>
>    /* some other subprog does: ret = leaf(1,2,3,4,5,6,7); */
>
> For 'leaf', the verifier sets:
>    incoming_stack_arg_depth = (7 - 5) * 8 = 16
>    stack_arg_depth = 0    (never updated, no calls made)
>
> When do_jit() computes:
>    outgoing_stack_arg_depth = 0 - 16 = 0xFFF0 (u16 underflow)
>
> Then at the new code below:
>
>> +	/*
>> +	 * Allocate outgoing stack arg area for args 7+ only.
>> +	 * Arg 6 goes into r9 register, not on stack.
>> +	 */
>> +	outgoing_rsp = outgoing_stack_arg_depth > 8 ?  outgoing_stack_arg_depth - 8 : 0;
>                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +	emit_sub_rsp(&prog, outgoing_rsp);
> This emits 'sub rsp, 0xFFE8' (about 65 KB), which overflows the kernel
> stack (THREAD_SIZE is only 16 KB on x86_64). The epilogue then emits a
> matching 'add rsp, 0xFFE8', but the damage is already done at function
> entry.
>
> The commit message states "The verifier guarantees that neither
> tail_call_reachable nor priv_stack is set when stack args exist, so R9 is
> always available." It does not claim the verifier maintains
> 'stack_arg_depth >= incoming_stack_arg_depth', and that invariant is not
> maintained for leaf subprograms.
>
> The sanity check in bpf_fixup_call_args() at kernel/bpf/fixups.c does not
> catch this because the same underflow happens there:
>    u16 outgoing = subprog->stack_arg_depth - subprog->incoming_stack_arg_depth;
> but since leaf subprograms have max_out_stack_arg_depth == 0, the
> comparison 'outgoing > max_out_stack_arg_depth' is false (0xFFF0 wraps
> around in the comparison).
>
> Should the JIT compute outgoing_stack_arg_depth safely to prevent the
> underflow? For example:
>    outgoing_stack_arg_depth = stack_arg_depth > incoming_stack_arg_depth ?
>                               stack_arg_depth - incoming_stack_arg_depth : 0;
>
> Or should the verifier ensure the invariant 'stack_arg_depth >=
> incoming_stack_arg_depth' after verification completes?

You are right! I missed this case where the caller signature has more
than 5 arguments and the caller itself does not have any callee's.
This caused the problem as you mentioned above. I will fix the issue
and added a selftest for this in the next revision.

>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 13/18] selftests/bpf: Add verifier tests for stack argument validation

[Yonghong Song]

On 4/24/26 10:48 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
>> new file mode 100644
>> index 000000000000..6b596ad63774
>> --- /dev/null
>> +++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
>> @@ -0,0 +1,456 @@
> [ ... ]
>
>> +SEC("tc")
>> +__description("stack_arg: pruning with different stack arg types")
>> +__failure
>> +__flag(BPF_F_TEST_STATE_FREQ)
>> +__msg("invalid mem access 'scalar'")
>> +__naked void stack_arg_pruning_type_mismatch(void)
>> +{
>> +	asm volatile (
>> +		"call %[bpf_get_prandom_u32];"
>> +		"r6 = r0;"
>> +		/* local = 0 on program stack */
>> +		"r7 = 0;"
>> +		"*(u64 *)(r10 - 8) = r7;"
>> +		/* Branch based on random value */
>> +		"if r6 s> 3 goto l0_%=;"
>> +		/* Path 1: store stack pointer to outgoing arg6 */
>> +		"r1 = r10;"
>> +		"r1 += -8;"
>> +		"*(u64 *)(r11 - 8) = r1;"
>> +		"goto l1_%=;"
>> +	"l0_%=:"
>> +		/* Path 2: store scalar to outgoing arg6 */
>> +		"*(u64 *)(r11 - 8) = 42;"
>> +	"l1_%=:"
>> +		/* Call subprog that dereferences arg6 */
>> +		"r1 = r6;"
>> +		"r2 = 0;"
>> +		"r3 = 0;"
>> +		"r4 = 0;"
>> +		"r5 = 0;"
>> +		"call subprog_deref_arg6;"
>> +		"exit;"
>> +		:: __imm(bpf_get_prandom_u32)
>> +		: __clobber_all
>> +	);
>> +}
> Does the error message string match the expected register output?
>
> During v6 review, sashiko-bot raised a concern about test fragility:
> since subprog_deref_arg6 is standard C (not __naked), LLVM determines
> which register is used to dereference f. If a future LLVM version
> chooses a different register, the verifier error will mention that
> register instead of R1, causing the test to fail.
>
> You acknowledged the issue and said you would use a regex like
> __msg("R{{[0-9]}} invalid mem access 'scalar'") to match the register
> dynamically. However, the current code uses __msg("invalid mem access
> 'scalar'") without any register prefix.
>
> The same pattern appears in stack_arg_release_ref,
> stack_arg_stale_pkt_ptr, and stack_arg_null_propagation_fail. Should
> these tests use the regex pattern to avoid compiler-dependent failures,
> or be rewritten to make subprog_deref_arg6 a __naked function?

Yes, I removed the "R{{[0-9]}}' stuff since I thought "R{{[0-9]}}"
will be equivalent to ignore it since we do not have a definite register
any more.

There is an option to use __naked function for subprog_deref_arg6
(see patch "selftests/bpf: Add precision backtracking test for stack arguments").
It becomes more complicated so I prefer the current result.

>
> Reference:
> https://lore.kernel.org/bpf/20260419172100.A89FEC2BCAF@smtp.kernel.org/
>
> [ ... ]
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 17/18] bpf, arm64: Add JIT support for stack arguments

[Puranjay Mohan]

bot+bpf-ci@kernel.org writes:

>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>> --- a/arch/arm64/net/bpf_jit_comp.c
>> +++ b/arch/arm64/net/bpf_jit_comp.c
>
> [ ... ]
>
>> @@ -1671,7 +1727,7 @@
>>  	}
>>  	if (src == fp) {
>>  		src_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
>> -		off_adj = off + ctx->stack_size;
>> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>>  	} else {
>>  		src_adj = src;
>>  		off_adj = off;
>
> When priv_sp_used is true, does this offset calculation produce correct
> addresses for BPF fp-relative accesses?
>
> Looking at build_prologue:
>
> arch/arm64/net/bpf_jit_comp.c:build_prologue() {
>     ...
>     emit(A64_MOV(1, fp, priv_sp), ctx);
>     emit(A64_ADD_I(1, fp, fp, ctx->stack_size), ctx);  /* line 612 */
>
>     if (ctx->stack_size && !ctx->priv_sp_used)
>         emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);  /* line 620 */
>
>     if (ctx->stack_arg_size)
>         emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_arg_size), ctx);  /* line 623 */
>     ...
> }
>
> When priv_sp_used is true:
> - fp = priv_sp + stack_size (line 612)
> - main SP is not decremented by stack_size (skipped at line 620)
> - main SP is decremented by stack_arg_size (line 623-624)
>
> So the BPF program data lives on the private stack, while outgoing stack
> args live on the real SP. These are separate memory regions.
>
> In build_insn, the computed address for fp-relative loads/stores becomes:
>     addr = priv_sp + off + stack_size + stack_arg_size
>
> But the intended BPF address is:
>     addr = fp + off = priv_sp + stack_size + off
>
> The computed address is stack_arg_size bytes too high, accessing memory
> above the intended location on the private stack.

> This would read/write into the private-stack guard region (16 bytes) or
> past it when stack_arg_size > 16. Can this occur?  The verifier only
> forbids tail_calls + stack_args (verifier.c:5657), not priv_stack +
> stack_args. A kprobe BPF program that uses BPF_REG_FP and calls a kfunc
> with 9+ arguments would trigger wrong memory accesses.

This analysis is correct, will fix in the next version.

> Note that x86's bpf_int_jit_compile prevents priv_stack allocation when
> stack_arg_depth != 0 (x86/bpf_jit_comp.c:3883). The arm64 version has no
> equivalent guard.
>
> The same issue affects the ST case:
>
>> @@ -1762,7 +1823,7 @@
>>  	}
>>  	if (dst == fp) {
>>  		dst_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
>> -		off_adj = off + ctx->stack_size;
>> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>>  	} else {
>>  		dst_adj = dst;
>>  		off_adj = off;
>
> and the STX case:
>
>> @@ -1824,7 +1890,7 @@
>>  	}
>>  	if (dst == fp) {
>>  		dst_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
>> -		off_adj = off + ctx->stack_size;
>> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>>  	} else {
>>  		dst_adj = dst;
>>  		off_adj = off;
>
> [ ... ]
>
>> @@ -2065,6 +2131,14 @@
>>  	ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
>>  	ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
>>
>> +	if (prog->aux->stack_arg_depth > prog->aux->incoming_stack_arg_depth) {
>> +		u16 outgoing = prog->aux->stack_arg_depth - prog->aux->incoming_stack_arg_depth;
>> +		int nr_on_stack = outgoing / sizeof(u64) - NR_STACK_ARG_REGS;
>> +
>> +		if (nr_on_stack > 0)
>> +			ctx.stack_arg_size = round_up(nr_on_stack * sizeof(u64), 16);
>> +	}
>> +
> Alexei Starovoitov raised concerns about this logic during v1 review (Apr
> 21, 2026). He found the comparison between stack_arg_depth and
> incoming_stack_arg_depth confusing and suggested simplifying it:
>
> "I think stack_arg_depth should mean outgoing and
> incoming_stack_arg_depth should mean incoming only and it shouldn't be
> even used by JIT. That memory was allocated by caller, so to JIT this
> callee the conversion of r11+const is straightforward and no checks
> necessary. [...] I think it should really be one check based on
> stack_arg_depth."


But if you see where these feilds are defined:

+	u16 incoming_stack_arg_depth;
+	u16 stack_arg_depth; /* incoming + max outgoing */

So, if stack_arg_depth is not the outgoing stack depth, rather it is the
sum of both outgoing and incoming. We need to do stack_arg_depth - incoming_stack_arg_depth 
to find out the amount of space to reserve to passing stack arguments.

Only if stack_arg_depth meant outgoing stack arg depth only, then we
could simplify it.

>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240

Re: [PATCH bpf-next 01/18] bpf: Support stack arguments for bpf functions

[Yonghong Song]

On 4/24/26 10:09 PM, Yonghong Song wrote:
>
>
> On 4/24/26 11:13 AM, bot+bpf-ci@kernel.org wrote:
>>> diff --git a/include/linux/bpf_verifier.h 
>>> b/include/linux/bpf_verifier.h
>>> index d5b4303315dd..2cc349d7fc17 100644
>>> --- a/include/linux/bpf_verifier.h
>>> +++ b/include/linux/bpf_verifier.h
>> [ ... ]
>>
>>> @@ -739,10 +759,13 @@ struct bpf_subprog_info {
>>>       bool keep_fastcall_stack: 1;
>>>       bool changes_pkt_data: 1;
>>>       bool might_sleep: 1;
>>> -    u8 arg_cnt:3;
>>> +    u8 arg_cnt:4;
>>>
>>>       enum priv_stack_mode priv_stack_mode;
>>> -    struct bpf_subprog_arg_info args[MAX_BPF_FUNC_REG_ARGS];
>>> +    struct bpf_subprog_arg_info args[MAX_BPF_FUNC_ARGS];
>>> +    u16 incoming_stack_arg_depth;
>>> +    u16 stack_arg_depth; /* incoming + max outgoing */
>>> +    u16 max_out_stack_arg_depth;
>>>   };
>> Alexei Starovoitov raised a concern in the v6 review about duplicated
>> fields. The incoming_stack_arg_depth, stack_arg_depth, and
>> max_out_stack_arg_depth fields are present in both bpf_subprog_info and
>> bpf_prog_aux structures. His comment was:
>>
>>    "but you already have them in prog_aux?! another copy in
>>    bpf_subprog_info?! Remove one of them. JIT only need one set."
>>
>> Looking at include/linux/bpf.h, these fields appear in bpf_prog_aux:
>>
>>    struct bpf_prog_aux {
>>        ...
>>        u16 incoming_stack_arg_depth;
>>        u16 stack_arg_depth;
>>        ...
>>    };
>>
>> Does the verifier actually need both copies? The concern was that having
>> duplicate state creates maintenance burden.
>
> Before jit_subprogs(), bpf_subprog_info stores the information for 
> each subprog.
> At this point, the subprog has not been allocated and only main prog 
> is available.
> Therefore, it is not possible to copy bpf_subprog_info to each subprog.
>
> So I think it might be necessary to have both fields in bpf_prog_aux and
> bpf_subprog_info.
>
> This version made some changes so max_out_stack_arg_depth is not needed
> any more. Will remove it in the next revision.

I did some futher checking and found that max_out_stack_arg_depth is still
needed. Otherwise, they could silently corrupting stack. For example,
for
     bar(int a) {
       ...
       foo(a1, a2, a3, a4, a5, a6);
       ...
     }

let us say before foo(), we have
     *(u64 *)(r11 - 8) = ...
     *(u64 *)(r11 - 16) = ...
     call foo() ...

In JIT, the outgoing stack will be 8 bytes (for *(u64 *)(r11 - 8) = ...).
But due to '*(u64 *)(r11 - 16) = ...', it may have an outgoing stack writing
with offset -16, which may cause an issue since other applicaiton, e.g.,
nmi/irq etc. So in such ases, we should reject in verifier.

>
>>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>>> index ff6ff1c27517..bcf81692a22b 100644
>>> --- a/kernel/bpf/verifier.c
>>> +++ b/kernel/bpf/verifier.c 

[...]

Re: [PATCH bpf-next 17/18] bpf, arm64: Add JIT support for stack arguments

[Yonghong Song]

On 4/27/26 2:06 AM, Puranjay Mohan wrote:
> bot+bpf-ci@kernel.org writes:
>
>>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>>> --- a/arch/arm64/net/bpf_jit_comp.c
>>> +++ b/arch/arm64/net/bpf_jit_comp.c
>> [ ... ]
>>
>>> @@ -1671,7 +1727,7 @@
>>>   	}
>>>   	if (src == fp) {
>>>   		src_adj = ctx->priv_sp_used ? priv_sp : A64_SP;
>>> -		off_adj = off + ctx->stack_size;
>>> +		off_adj = off + ctx->stack_size + ctx->stack_arg_size;
>>>   	} else {
>>>   		src_adj = src;
>>>   		off_adj = off;
>> When priv_sp_used is true, does this offset calculation produce correct
>> addresses for BPF fp-relative accesses?
>>
>> Looking at build_prologue:
>>
>> arch/arm64/net/bpf_jit_comp.c:build_prologue() {
>>      ...
>>      emit(A64_MOV(1, fp, priv_sp), ctx);
>>      emit(A64_ADD_I(1, fp, fp, ctx->stack_size), ctx);  /* line 612 */
>>
>>      if (ctx->stack_size && !ctx->priv_sp_used)
>>          emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);  /* line 620 */
>>
>>      if (ctx->stack_arg_size)
>>          emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_arg_size), ctx);  /* line 623 */
>>      ...
>> }
>>
>> When priv_sp_used is true:
>> - fp = priv_sp + stack_size (line 612)
>> - main SP is not decremented by stack_size (skipped at line 620)
>> - main SP is decremented by stack_arg_size (line 623-624)
>>
>> So the BPF program data lives on the private stack, while outgoing stack
>> args live on the real SP. These are separate memory regions.
>>
>> In build_insn, the computed address for fp-relative loads/stores becomes:
>>      addr = priv_sp + off + stack_size + stack_arg_size
>>
>> But the intended BPF address is:
>>      addr = fp + off = priv_sp + stack_size + off
>>
>> The computed address is stack_arg_size bytes too high, accessing memory
>> above the intended location on the private stack.
>> This would read/write into the private-stack guard region (16 bytes) or
>> past it when stack_arg_size > 16. Can this occur?  The verifier only
>> forbids tail_calls + stack_args (verifier.c:5657), not priv_stack +
>> stack_args. A kprobe BPF program that uses BPF_REG_FP and calls a kfunc
>> with 9+ arguments would trigger wrong memory accesses.
> This analysis is correct, will fix in the next version.

Thanks Puranjay, once you fixed this issue, I can submit next revision
with your and my fixes.

>
>> Note that x86's bpf_int_jit_compile prevents priv_stack allocation when
>> stack_arg_depth != 0 (x86/bpf_jit_comp.c:3883). The arm64 version has no
>> equivalent guard.
>>
>> The same issue affects the ST case:
>>
>>
[...]

>> ---
>> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
>> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>>
>> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24902767240