* [PATCH bpf-next v1 1/9] bpf: Support __arena and __arena_nullable kfunc argument suffixes
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 23:05 ` bot+bpf-ci
2026-07-15 22:00 ` [PATCH bpf-next v1 2/9] bpf: Support __arena and __arena_nullable on struct_ops stub arguments Kumar Kartikeya Dwivedi
` (7 subsequent siblings)
8 siblings, 1 reply; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Passing an arena pointer to a kfunc takes two steps today. There is no
arena pointer argument type, so the pointer crosses the boundary as a
bare scalar, and the kfunc then offsets it by the arena base and casts
it before it can touch the memory. Every such kfunc open-codes the same
translation.
Add the __arena and __arena_nullable argument suffixes to make this more
convenient. The kfunc declares the parameter by its real pointer type
and dereferences it directly, with the JIT rebasing the value at the
call site, rN = kern_vm_start + (u32)rN. No bounds check is needed: the
u32 offset stays within the guard-padded arena kernel mapping, and a
fault on an unpopulated page recovers through the per-arena scratch
page. A suffixed argument accepts a PTR_TO_ARENA or scalar register,
matching global subprog arena arguments.
__arena rebases unconditionally, so the kfunc never sees NULL and a
value with zero in the low 32 bits arrives as the arena base.
__arena_nullable preserves NULL for optional arguments by skipping the
rebase when the truncated value, arena offset 0, is zero. Keeping the
plain form NULL-free saves the NULL test on every call.
This patch adds the verifier side: the suffixes are recognized in
check_kfunc_args() and recorded as two per-call register bitmasks in
insn_aux_data, which JITs retrieve through bpf_kfunc_arena_args() while
emitting the call.
JITs declare support with bpf_jit_supports_arena_args() and verification
fails with -ENOTSUPP elsewhere.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
Documentation/bpf/kfuncs.rst | 29 +++++++++++++++++++++++++
include/linux/bpf.h | 9 ++++++++
include/linux/bpf_verifier.h | 4 ++++
include/linux/filter.h | 1 +
kernel/bpf/core.c | 18 ++++++++++++++++
kernel/bpf/verifier.c | 42 ++++++++++++++++++++++++++++++++++++
6 files changed, 103 insertions(+)
diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
index c801a330aece..a980266ec788 100644
--- a/Documentation/bpf/kfuncs.rst
+++ b/Documentation/bpf/kfuncs.rst
@@ -250,6 +250,33 @@ Or::
...
}
+2.3.7 __arena and __arena_nullable Annotations
+----------------------------------------------
+
+Both annotations indicate that the pointer argument points into the
+calling program's arena. The JIT rebases the value at the call site so
+the kfunc receives a directly dereferenceable kernel address, subject to
+the access rules described in :ref:`BPF_kfunc_arena_access` (at most
+``GUARD_SZ / 2``, 32 KiB, past the pointer in a single unchecked access).
+
+With ``__arena`` the rebase is unconditional and the argument is never
+NULL: a value whose lower 32 bits are zero arrives as the arena base
+address (arena offset 0). The kfunc must not check the argument for NULL.
+With ``__arena_nullable`` such a value arrives as NULL instead and the
+kfunc must check before dereferencing.
+
+An example is given below::
+
+ __bpf_kfunc int bpf_process_item(struct item *item__arena)
+ {
+ ...
+ }
+
+Calling such a kfunc requires the program to use an arena map and a JIT
+with arena argument support (currently x86-64 and arm64); verification
+fails otherwise. The program can pass any value without compromising the
+kernel. A value that does not point into the arena is a program bug.
+
.. _BPF_kfunc_nodef:
2.4 Using an existing kernel function
@@ -487,6 +514,8 @@ In order to accommodate such requirements, the verifier will enforce strict
PTR_TO_BTF_ID type matching if two types have the exact same name, with one
being suffixed with ``___init``.
+.. _BPF_kfunc_arena_access:
+
2.8 Accessing arena memory through kfunc arguments
--------------------------------------------------
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 31181e0c2b80..08d1aec8e30e 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1601,6 +1601,15 @@ void bpf_jit_uncharge_modmem(u32 size);
bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
int insn_idx);
+
+/* which R1-R5 args of a kfunc call carry arena pointers to be rebased by the JIT */
+struct bpf_jit_arena_args {
+ u8 regs;
+ u8 nullable_regs; /* subset of @regs where NULL is preserved */
+};
+
+struct bpf_jit_arena_args bpf_kfunc_arena_args(const struct bpf_verifier_env *env,
+ const struct bpf_prog *prog, int insn_idx);
u16 bpf_out_stack_arg_cnt(const struct bpf_verifier_env *env, const struct bpf_prog *prog);
#else
static inline int bpf_trampoline_link_prog(struct bpf_tramp_node *node,
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 682c2cd3b844..8ed1ea315581 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -690,6 +690,10 @@ struct bpf_insn_aux_data {
*/
u8 fastcall_spills_num:3;
u8 arg_prog:4;
+ /* bitmask of R1-R5 kfunc args to rebase to arena kernel addresses */
+ u8 arg_arena_regs;
+ /* subset of arg_arena_regs where NULL is preserved across the rebase */
+ u8 arg_arena_nullable_regs;
/* below fields are initialized once */
unsigned int orig_idx; /* original instruction index */
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 14acb2455746..7b673333f846 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1183,6 +1183,7 @@ bool bpf_jit_supports_subprog_tailcalls(void);
bool bpf_jit_supports_percpu_insn(void);
bool bpf_jit_supports_kfunc_call(void);
bool bpf_jit_supports_stack_args(void);
+bool bpf_jit_supports_arena_args(void);
bool bpf_jit_supports_far_kfunc_call(void);
bool bpf_jit_supports_exceptions(void);
bool bpf_jit_supports_ptr_xchg(void);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 47fe047ad30b..db99e01f11fc 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1640,6 +1640,19 @@ bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struc
return env->insn_aux_data[insn_idx].indirect_target;
}
+struct bpf_jit_arena_args bpf_kfunc_arena_args(const struct bpf_verifier_env *env,
+ const struct bpf_prog *prog, int insn_idx)
+{
+ struct bpf_jit_arena_args args = {};
+
+ if (!env)
+ return args;
+ insn_idx += prog->aux->subprog_start;
+ args.regs = env->insn_aux_data[insn_idx].arg_arena_regs;
+ args.nullable_regs = env->insn_aux_data[insn_idx].arg_arena_nullable_regs;
+ return args;
+}
+
u16 bpf_out_stack_arg_cnt(const struct bpf_verifier_env *env, const struct bpf_prog *prog)
{
const struct bpf_subprog_info *sub;
@@ -3296,6 +3309,11 @@ bool __weak bpf_jit_supports_stack_args(void)
return false;
}
+bool __weak bpf_jit_supports_arena_args(void)
+{
+ return false;
+}
+
bool __weak bpf_jit_supports_far_kfunc_call(void)
{
return false;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index de816063ae63..2affe2e1a6a1 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -10833,6 +10833,16 @@ static bool is_kfunc_arg_irq_flag(const struct btf *btf, const struct btf_param
return btf_param_match_suffix(btf, arg, "__irq_flag");
}
+static bool is_kfunc_arg_arena(const struct btf *btf, const struct btf_param *arg)
+{
+ return btf_param_match_suffix(btf, arg, "__arena");
+}
+
+static bool is_kfunc_arg_arena_nullable(const struct btf *btf, const struct btf_param *arg)
+{
+ return btf_param_match_suffix(btf, arg, "__arena_nullable");
+}
+
static bool is_kfunc_arg_scalar_with_name(const struct btf *btf,
const struct btf_param *arg,
const char *name)
@@ -12042,6 +12052,38 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+ if (is_kfunc_arg_arena(btf, &args[i]) || is_kfunc_arg_arena_nullable(btf, &args[i])) {
+ if (verifier_bug_if(!btf_type_is_ptr(t), env,
+ "kfunc %s arg#%d has __arena tag on non-pointer",
+ func_name, i))
+ return -EFAULT;
+ if (!bpf_jit_supports_arena_args()) {
+ verbose(env, "JIT does not support kfunc %s() with arena pointer arguments\n",
+ func_name);
+ return -ENOTSUPP;
+ }
+ if (!env->prog->aux->arena) {
+ verbose(env,
+ "%s arena pointer requires a program with an associated arena\n",
+ reg_arg_name(env, argno));
+ return -EINVAL;
+ }
+ if (regno < 0) {
+ verbose(env, "%s arena pointer cannot be a stack argument\n",
+ reg_arg_name(env, argno));
+ return -EINVAL;
+ }
+ if (reg->type != PTR_TO_ARENA && reg->type != SCALAR_VALUE) {
+ verbose(env, "%s is not a pointer to arena or scalar\n",
+ reg_arg_name(env, argno));
+ return -EINVAL;
+ }
+ cur_aux(env)->arg_arena_regs |= BIT(regno - BPF_REG_1);
+ if (is_kfunc_arg_arena_nullable(btf, &args[i]))
+ cur_aux(env)->arg_arena_nullable_regs |= BIT(regno - BPF_REG_1);
+ continue;
+ }
+
if (btf_type_is_scalar(t)) {
if (reg->type != SCALAR_VALUE) {
verbose(env, "%s is not a scalar\n", reg_arg_name(env, argno));
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH bpf-next v1 1/9] bpf: Support __arena and __arena_nullable kfunc argument suffixes
2026-07-15 22:00 ` [PATCH bpf-next v1 1/9] bpf: Support __arena and __arena_nullable kfunc argument suffixes Kumar Kartikeya Dwivedi
@ 2026-07-15 23:05 ` bot+bpf-ci
0 siblings, 0 replies; 14+ messages in thread
From: bot+bpf-ci @ 2026-07-15 23:05 UTC (permalink / raw)
To: memxor, bpf
Cc: tj, ast, andrii, daniel, eddyz87, emil, kkd, kernel-team, ast,
andrii, daniel, martin.lau, eddyz87, yonghong.song, clm,
ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2413 bytes --]
> diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
> index c801a330aece..a980266ec788 100644
> --- a/Documentation/bpf/kfuncs.rst
> +++ b/Documentation/bpf/kfuncs.rst
> @@ -250,6 +250,33 @@ Or::
> ...
> }
>
> +2.3.7 __arena and __arena_nullable Annotations
> +----------------------------------------------
> +
> +Both annotations indicate that the pointer argument points into the
> +calling program's arena. The JIT rebases the value at the call site so
> +the kfunc receives a directly dereferenceable kernel address, subject to
> +the access rules described in :ref:`BPF_kfunc_arena_access` (at most
> +``GUARD_SZ / 2``, 32 KiB, past the pointer in a single unchecked access).
> +
> +With ``__arena`` the rebase is unconditional and the argument is never
> +NULL: a value whose lower 32 bits are zero arrives as the arena base
> +address (arena offset 0). The kfunc must not check the argument for NULL.
> +With ``__arena_nullable`` such a value arrives as NULL instead and the
> +kfunc must check before dereferencing.
> +
> +An example is given below::
> +
> + __bpf_kfunc int bpf_process_item(struct item *item__arena)
> + {
> + ...
> + }
> +
> +Calling such a kfunc requires the program to use an arena map and a JIT
> +with arena argument support (currently x86-64 and arm64); verification
> +fails otherwise. The program can pass any value without compromising the
> +kernel. A value that does not point into the arena is a program bug.
The documentation says arena argument support is available on "currently
x86-64 and arm64", but does arm64 actually support it here?
bpf_jit_supports_arena_args() is only defined to return true in the
x86-64 JIT. The arm64 JIT is not touched, so on arm64
bpf_jit_supports_arena_args() falls through to the __weak stub added in
this commit:
kernel/bpf/core.c:
bool __weak bpf_jit_supports_arena_args(void)
which returns false.
On arm64, check_kfunc_args() would then reject any __arena or
__arena_nullable argument with -ENOTSUPP. Should the documentation drop
arm64 from the list of supported architectures until the arm64 JIT gains
support?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29454748905
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH bpf-next v1 2/9] bpf: Support __arena and __arena_nullable on struct_ops stub arguments
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 1/9] bpf: Support __arena and __arena_nullable kfunc argument suffixes Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:22 ` sashiko-bot
2026-07-15 22:00 ` [PATCH bpf-next v1 3/9] bpf, x86: JIT __arena kfunc argument rebasing Kumar Kartikeya Dwivedi
` (6 subsequent siblings)
8 siblings, 1 reply; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
A struct_ops callback cannot receive an arena pointer directly, so
passing one takes two steps. The pointer arrives as a bare u64 that the
callback casts, and because the two sides address the arena through
different bases it also has to be rebased by hand on the way in.
Add the __arena and __arena_nullable stub argument suffixes to make this
convenient. The callback declares the parameter as an arena pointer,
receives it as a PTR_TO_ARENA register, and dereferences it directly,
while the kernel caller just passes the natural kernel arena address
(kaddr). The trampoline converts the value while saving the arguments
into the BPF ctx, ctx[slot] = (u32)(kaddr - kern_vm_start), so the
program never sees a kernel address and nothing rewrites the ctx after
the fact. The converted value keeps the upper 32 bits clear as the JITs
require of arena pointer registers and behaves like any cast_kern'ed
arena pointer, so cast_user recovers the full user-visible address.
__arena converts unconditionally and the kernel caller must not pass
NULL. __arena_nullable preserves NULL, tested on the full 64-bit kernel
pointer, and surfaces to the verifier as PTR_TO_ARENA (but not as a
PTR_TO_ARENA | PTR_MAYBE_NULL). The reason is that PTR_TO_ARENA in
program's type state already encompasses NULL-ness, so it is not
meaningful to force a NULL check for the program.
This patch adds the generic side. bpf_tramp_collect_arena_args() derives
the conversion map from the prog's ctx_arg_info, keyed by the flattened
ctx byte offset since preceding 16-byte arguments occupy two slots. Only
the struct_ops indirect trampoline converts: it dispatches to a single
prog whose arena is fixed at generation time. Generic trampolines can
mix progs with different arenas and reject arena ctx args defensively,
which is unreachable today as only struct_ops progs carry them. Arch
trampolines that do not implement the conversion are gated out at
verification time with bpf_jit_supports_arena_args().
Signed-off-by: Tejun Heo <tj@kernel.org>
Co-developed-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
Documentation/bpf/kfuncs.rst | 10 ++++++
include/linux/bpf.h | 19 +++++++++++
kernel/bpf/bpf_struct_ops.c | 34 ++++++++++++++-----
kernel/bpf/btf.c | 10 ++++--
kernel/bpf/trampoline.c | 64 ++++++++++++++++++++++++++++++++++++
kernel/bpf/verifier.c | 23 ++++++++++---
6 files changed, 143 insertions(+), 17 deletions(-)
diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
index a980266ec788..bcec48788ad4 100644
--- a/Documentation/bpf/kfuncs.rst
+++ b/Documentation/bpf/kfuncs.rst
@@ -277,6 +277,16 @@ with arena argument support (currently x86-64 and arm64); verification
fails otherwise. The program can pass any value without compromising the
kernel. A value that does not point into the arena is a program bug.
+The suffixes have the same meaning on the arguments of struct_ops stub
+functions, with the conversion running in the opposite direction. The
+kernel caller passes the kernel arena address and the trampoline converts
+it while saving the arguments, so the callback receives an arena pointer
+it can dereference directly. With ``__arena`` the kernel caller must not
+pass NULL. With ``__arena_nullable`` a NULL kernel pointer arrives as NULL.
+However, there is no obligation to prove to the verifier that such a pointer is
+non-NULL before use, in-line with existing semantics of arena pointers used in
+a program (or obtained from any other source).
+
.. _BPF_kfunc_nodef:
2.4 Using an existing kernel function
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 08d1aec8e30e..82b1e2db80e2 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1286,6 +1286,20 @@ struct bpf_tramp_nodes {
int nr_nodes;
};
+/*
+ * Which 8-byte ctx slots of a struct_ops trampoline hold arena kernel
+ * pointers that save_args() converts to the arena pointer form,
+ * ctx[slot] = (u32)(kaddr - kern_vm_start).
+ */
+struct bpf_tramp_arena_args {
+ u32 slots;
+ u32 nullable_slots; /* subset of @slots where NULL is preserved */
+ u64 kern_vm_start;
+};
+
+bool bpf_tramp_collect_arena_args(struct bpf_tramp_nodes *tnodes, u32 flags,
+ struct bpf_tramp_arena_args *aargs);
+
struct bpf_tramp_run_ctx;
/* Different use cases for BPF trampoline:
@@ -1694,6 +1708,11 @@ struct bpf_ctx_arg_aux {
u32 btf_id;
u32 ref_id;
bool refcounted;
+ /*
+ * We don't encode NULL-ness in the type for the program, but still need
+ * to distinguish it for the purposes of telling JITs what sequence to emit.
+ */
+ bool arena_nullable;
};
struct btf_mod_pair {
diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c
index 51b16e5f5534..2757380194f8 100644
--- a/kernel/bpf/bpf_struct_ops.c
+++ b/kernel/bpf/bpf_struct_ops.c
@@ -147,6 +147,8 @@ void bpf_struct_ops_image_free(void *image)
#define MAYBE_NULL_SUFFIX "__nullable"
#define REFCOUNTED_SUFFIX "__ref"
+#define ARENA_SUFFIX "__arena"
+#define ARENA_MAYBE_NULL_SUFFIX "__arena_nullable"
/* Prepare argument info for every nullable argument of a member of a
* struct_ops type.
@@ -159,7 +161,7 @@ void bpf_struct_ops_image_free(void *image)
* to provide an array of struct bpf_ctx_arg_aux, which in turn provides
* the information that used by the verifier to check the arguments of the
* BPF struct_ops program assigned to the member. Here, we only care about
- * the arguments that are marked as __nullable.
+ * the arguments that are marked as __nullable, __ref or __arena.
*
* The array of struct bpf_ctx_arg_aux is eventually assigned to
* prog->aux->ctx_arg_info of BPF struct_ops programs and passed to the
@@ -175,7 +177,8 @@ static int prepare_arg_info(struct btf *btf,
struct bpf_struct_ops_arg_info *arg_info)
{
const struct btf_type *stub_func_proto, *pointed_type;
- bool is_nullable = false, is_refcounted = false;
+ bool is_nullable = false, is_refcounted = false, is_arena = false;
+ bool is_arena_nullable = false;
const struct btf_param *stub_args, *args;
struct bpf_ctx_arg_aux *info, *info_buf;
u32 nargs, arg_no, info_cnt = 0;
@@ -226,26 +229,30 @@ static int prepare_arg_info(struct btf *btf,
info = info_buf;
for (arg_no = 0; arg_no < nargs; arg_no++) {
/* Skip arguments that is not suffixed with
- * "__nullable or __ref".
+ * "__nullable", "__ref", "__arena" or "__arena_nullable".
*/
is_nullable = btf_param_match_suffix(btf, &stub_args[arg_no],
MAYBE_NULL_SUFFIX);
is_refcounted = btf_param_match_suffix(btf, &stub_args[arg_no],
REFCOUNTED_SUFFIX);
+ is_arena_nullable = btf_param_match_suffix(btf, &stub_args[arg_no],
+ ARENA_MAYBE_NULL_SUFFIX);
+ is_arena = btf_param_match_suffix(btf, &stub_args[arg_no], ARENA_SUFFIX);
if (is_nullable)
suffix = MAYBE_NULL_SUFFIX;
else if (is_refcounted)
suffix = REFCOUNTED_SUFFIX;
+ else if (is_arena_nullable)
+ suffix = ARENA_MAYBE_NULL_SUFFIX;
+ else if (is_arena)
+ suffix = ARENA_SUFFIX;
else
continue;
- /* Should be a pointer to struct */
- pointed_type = btf_type_resolve_ptr(btf,
- args[arg_no].type,
- &arg_btf_id);
- if (!pointed_type ||
- !btf_type_is_struct(pointed_type)) {
+ /* Should be a pointer to struct, or any pointer for __arena/__arena_nullable */
+ pointed_type = btf_type_resolve_ptr(btf, args[arg_no].type, &arg_btf_id);
+ if (!pointed_type || (!is_arena && !is_arena_nullable && !btf_type_is_struct(pointed_type))) {
pr_warn("stub function %s has %s tagging to an unsupported type\n",
stub_fname, suffix);
goto err_out;
@@ -273,6 +280,15 @@ static int prepare_arg_info(struct btf *btf,
} else if (is_refcounted) {
info->reg_type = PTR_TRUSTED | PTR_TO_BTF_ID;
info->refcounted = true;
+ } else if (is_arena || is_arena_nullable) {
+ /*
+ * Both types get PTR_TO_ARENA. In verifier state,
+ * PTR_TO_ARENA encompasses potential NULL values, but
+ * we do not force the program to check it, or maintain
+ * precision around it, since it has no safety implication.
+ */
+ info->reg_type = PTR_TO_ARENA;
+ info->arena_nullable = is_arena_nullable;
}
info++;
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 8c04c340f499..d959ae07d4be 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6961,15 +6961,19 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,
return false;
}
- /* check for PTR_TO_RDONLY_BUF_OR_NULL or PTR_TO_RDWR_BUF_OR_NULL */
+ /*
+ * Check for PTR_TO_RDONLY_BUF_OR_NULL, PTR_TO_RDWR_BUF_OR_NULL or
+ * PTR_TO_ARENA (both nullable and non-nullable cases).
+ */
for (i = 0; i < prog->aux->ctx_arg_info_size; i++) {
const struct bpf_ctx_arg_aux *ctx_arg_info = &prog->aux->ctx_arg_info[i];
u32 type, flag;
type = base_type(ctx_arg_info->reg_type);
flag = type_flag(ctx_arg_info->reg_type);
- if (ctx_arg_info->offset == off && type == PTR_TO_BUF &&
- (flag & PTR_MAYBE_NULL)) {
+ if (ctx_arg_info->offset == off &&
+ (type == PTR_TO_ARENA ||
+ (type == PTR_TO_BUF && (flag & PTR_MAYBE_NULL)))) {
info->reg_type = ctx_arg_info->reg_type;
return true;
}
diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 6eadf64f7ec9..56759891eab7 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -529,6 +529,53 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_a
return tnodes;
}
+static bool bpf_prog_has_arena_ctx_arg(const struct bpf_prog *prog)
+{
+ int i;
+
+ for (i = 0; i < prog->aux->ctx_arg_info_size; i++)
+ if (base_type(prog->aux->ctx_arg_info[i].reg_type) == PTR_TO_ARENA)
+ return true;
+ return false;
+}
+
+/*
+ * Collect which ctx slots of a struct_ops trampoline hold arena kernel
+ * pointers that save_args() must convert to the arena pointer form. Only
+ * the struct_ops indirect trampoline converts: it dispatches to a single
+ * prog whose arena is known at generation time. Return false when there
+ * is nothing to convert.
+ */
+bool bpf_tramp_collect_arena_args(struct bpf_tramp_nodes *tnodes, u32 flags,
+ struct bpf_tramp_arena_args *aargs)
+{
+ const struct bpf_prog *prog;
+ int i;
+
+ memset(aargs, 0, sizeof(*aargs));
+
+ if (!(flags & BPF_TRAMP_F_INDIRECT) ||
+ tnodes[BPF_TRAMP_FENTRY].nr_nodes != 1)
+ return false;
+
+ prog = tnodes[BPF_TRAMP_FENTRY].nodes[0]->link->prog;
+ for (i = 0; i < prog->aux->ctx_arg_info_size; i++) {
+ const struct bpf_ctx_arg_aux *info = &prog->aux->ctx_arg_info[i];
+
+ if (base_type(info->reg_type) != PTR_TO_ARENA)
+ continue;
+ aargs->slots |= BIT(info->offset / 8);
+ if (info->arena_nullable)
+ aargs->nullable_slots |= BIT(info->offset / 8);
+ }
+ if (!aargs->slots)
+ return false;
+ if (WARN_ON_ONCE(!prog->aux->arena))
+ return false;
+ aargs->kern_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
+ return true;
+}
+
static void bpf_tramp_image_free(struct bpf_tramp_image *im)
{
bpf_image_ksym_del(&im->ksym);
@@ -678,6 +725,7 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
u32 orig_flags = tr->flags;
bool ip_arg = false;
int err, total, size;
+ int kind, i;
tnodes = bpf_trampoline_get_progs(tr, &total, &ip_arg);
if (IS_ERR(tnodes))
@@ -688,6 +736,22 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
goto out;
}
+ /*
+ * Arena ctx args are converted only by the struct_ops indirect
+ * trampoline, which dispatches to a single known prog. Generic
+ * trampolines can mix progs with different arenas, so no conversion
+ * is possible here. Not reachable today: only struct_ops progs get
+ * arena ctx args and they never ride generic trampolines.
+ */
+ for (kind = 0; kind < BPF_TRAMP_MAX; kind++) {
+ for (i = 0; i < tnodes[kind].nr_nodes; i++) {
+ if (bpf_prog_has_arena_ctx_arg(tnodes[kind].nodes[i]->link->prog)) {
+ err = -ENOTSUPP;
+ goto out;
+ }
+ }
+ }
+
/* clear all bits except SHARE_IPMODIFY and TAIL_CALL_CTX */
tr->flags &= (BPF_TRAMP_F_SHARE_IPMODIFY | BPF_TRAMP_F_TAIL_CALL_CTX);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 2affe2e1a6a1..2405e48aa5bb 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18756,6 +18756,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
{
const struct btf_type *t, *func_proto;
const struct bpf_struct_ops_desc *st_ops_desc;
+ const struct bpf_struct_ops_arg_info *arg_info;
const struct bpf_struct_ops *st_ops;
const struct btf_member *member;
struct bpf_prog *prog = env->prog;
@@ -18834,10 +18835,23 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
return -EACCES;
}
- for (i = 0; i < st_ops_desc->arg_info[member_idx].cnt; i++) {
- if (st_ops_desc->arg_info[member_idx].info[i].refcounted) {
+ arg_info = &st_ops_desc->arg_info[member_idx];
+ for (i = 0; i < arg_info->cnt; i++) {
+ const struct bpf_ctx_arg_aux *info = &arg_info->info[i];
+
+ if (info->refcounted)
has_refcounted_arg = true;
- break;
+ if (base_type(info->reg_type) == PTR_TO_ARENA) {
+ if (!bpf_jit_supports_arena_args()) {
+ verbose(env, "JIT does not support arena arguments\n");
+ return -ENOTSUPP;
+ }
+ if (!prog->aux->arena) {
+ verbose(env,
+ "arena argument of %s requires a program with an associated arena\n",
+ mname);
+ return -EINVAL;
+ }
}
}
@@ -18858,8 +18872,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
prog->aux->attach_func_name = mname;
env->ops = st_ops->verifier_ops;
- return bpf_prog_ctx_arg_info_init(prog, st_ops_desc->arg_info[member_idx].info,
- st_ops_desc->arg_info[member_idx].cnt);
+ return bpf_prog_ctx_arg_info_init(prog, arg_info->info, arg_info->cnt);
}
#define SECURITY_PREFIX "security_"
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH bpf-next v1 2/9] bpf: Support __arena and __arena_nullable on struct_ops stub arguments
2026-07-15 22:00 ` [PATCH bpf-next v1 2/9] bpf: Support __arena and __arena_nullable on struct_ops stub arguments Kumar Kartikeya Dwivedi
@ 2026-07-15 22:22 ` sashiko-bot
0 siblings, 0 replies; 14+ messages in thread
From: sashiko-bot @ 2026-07-15 22:22 UTC (permalink / raw)
To: Kumar Kartikeya Dwivedi; +Cc: bpf
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] EXT (freplace) and TRACING (fentry/fexit) programs attached to struct_ops programs bypass defensive checks and `ctx_arg_info` inheritance, causing them to incorrectly verify arena pointers as trusted kernel pointers (`PTR_TO_BTF_ID`).
--
--- Patch [2]: [PATCH bpf-next v1 2/9] bpf: Support __arena and __arena_nullable on struct_ops stub arguments ---
commit 592a1abca0acd2ddc98e47619cf772faa71d7c0d
Author: Tejun Heo <tj@kernel.org>
bpf: Support __arena and __arena_nullable on struct_ops stub arguments
This patch adds support for the __arena and __arena_nullable suffixes on
struct_ops callback arguments. The trampoline automatically converts the
kernel pointer to an arena pointer when saving arguments, allowing the BPF
program to dereference it directly.
> diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
> index 6eadf64f7ec90..56759891eab79 100644
> --- a/kernel/bpf/trampoline.c
> +++ b/kernel/bpf/trampoline.c
[ ... ]
> @@ -688,6 +736,22 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
> goto out;
> }
>
> + /*
> + * Arena ctx args are converted only by the struct_ops indirect
> + * trampoline, which dispatches to a single known prog. Generic
> + * trampolines can mix progs with different arenas, so no conversion
> + * is possible here. Not reachable today: only struct_ops progs get
> + * arena ctx args and they never ride generic trampolines.
> + */
> + for (kind = 0; kind < BPF_TRAMP_MAX; kind++) {
> + for (i = 0; i < tnodes[kind].nr_nodes; i++) {
> + if (bpf_prog_has_arena_ctx_arg(tnodes[kind].nodes[i]->link->prog)) {
> + err = -ENOTSUPP;
> + goto out;
> + }
> + }
> + }
[Severity: Critical]
Will this defensive check be bypassed for FENTRY/EXT programs attached to
struct_ops programs?
When an FENTRY or EXT program attaches to a struct_ops program, it does not
inherit ctx_arg_info from its target in bpf_check_attach_target(). Because
the attaching program lacks ctx_arg_info, bpf_prog_has_arena_ctx_arg()
would return false here, allowing the attachment to proceed instead of
returning -ENOTSUPP.
If this check is bypassed, could the attaching FENTRY/EXT program evaluate
the argument as a standard PTR_TO_BTF_ID? Since the generic FENTRY
trampoline receives the already-converted, user-controlled arena offset from
the struct_ops caller, this might allow an attaching program to treat an
untrusted offset as a trusted kernel pointer.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260715220052.1590783-1-memxor@gmail.com?part=2
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH bpf-next v1 3/9] bpf, x86: JIT __arena kfunc argument rebasing
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 1/9] bpf: Support __arena and __arena_nullable kfunc argument suffixes Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 2/9] bpf: Support __arena and __arena_nullable on struct_ops stub arguments Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 4/9] bpf, x86: Convert struct_ops arena arguments in the trampoline Kumar Kartikeya Dwivedi
` (5 subsequent siblings)
8 siblings, 0 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Implement arena argument rebasing for kfunc calls on x86. R12 already
holds kern_vm_start whenever the prog has an arena, so each tagged
argument costs two instructions emitted right before the call:
movl %eN, %eN /* truncate, clear the upper 32 bits */
addq %r12, %rN
A nullable argument tests the truncated value and jumps over the add:
movl %eN, %eN
testl %eN, %eN
jz 1f
addq %r12, %rN
1:
addq carries a REX prefix for every argument register and is always
three bytes, so the jz displacement is constant. The sequence is native
code generated after constant blinding has run on the BPF instruction
stream, so blinding never sees the rebase and needs no special handling.
bpf_jit_supports_arena_args() is not flipped yet; that happens when the
struct_ops trampoline side is in place as well.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
arch/x86/net/bpf_jit_comp.c | 49 +++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index de7515ea1bea..dd9009fcdc39 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1678,6 +1678,49 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
+/*
+ * Rebase the __arena args of a kfunc call to arena kernel addresses,
+ * rN = kern_vm_start + (u32)rN, with R12 holding kern_vm_start. A nullable
+ * arg preserves NULL by skipping the add, tested on the truncated value as
+ * arena NULL is offset 0. Return the number of emitted bytes.
+ */
+static int emit_kfunc_arena_args(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog,
+ int insn_idx, u8 **pprog)
+{
+ struct bpf_jit_arena_args args;
+ unsigned long regs;
+ u8 *prog = *pprog;
+ u8 *start = prog;
+ int bit;
+
+ args = bpf_kfunc_arena_args(env, bpf_prog, insn_idx);
+ if (!args.regs)
+ return 0;
+ if (WARN_ON_ONCE(args.nullable_regs & ~args.regs) ||
+ WARN_ON_ONCE(!bpf_prog->aux->arena))
+ return -EINVAL;
+
+ regs = args.regs;
+ for_each_set_bit(bit, ®s, MAX_BPF_FUNC_REG_ARGS) {
+ u32 reg = BPF_REG_1 + bit;
+
+ /* mov eN, eN: truncate and clear the upper 32 bits */
+ emit_mov_reg(&prog, false, reg, reg);
+ if (args.nullable_regs & BIT(bit)) {
+ /* test eN, eN; jz over the 3-byte add */
+ maybe_emit_mod(&prog, reg, reg, false);
+ EMIT2(0x85, add_2reg(0xC0, reg, reg));
+ EMIT2(X86_JE, 3);
+ }
+ /* add rN, r12 */
+ maybe_emit_mod(&prog, reg, X86_REG_R12, true);
+ EMIT2(0x01, add_2reg(0xC0, reg, X86_REG_R12));
+ }
+
+ *pprog = prog;
+ return prog - start;
+}
+
static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
{
@@ -2583,6 +2626,12 @@ st: insn_off = insn->off;
}
if (!imm32)
return -EINVAL;
+ if (src_reg == BPF_PSEUDO_KFUNC_CALL) {
+ err = emit_kfunc_arena_args(env, bpf_prog, i - 1, &prog);
+ if (err < 0)
+ return err;
+ ip += err;
+ }
if (priv_frame_ptr) {
push_r9(&prog);
ip += 2;
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH bpf-next v1 4/9] bpf, x86: Convert struct_ops arena arguments in the trampoline
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
` (2 preceding siblings ...)
2026-07-15 22:00 ` [PATCH bpf-next v1 3/9] bpf, x86: JIT __arena kfunc argument rebasing Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 5/9] selftests/bpf: Add kfunc __arena and __arena_nullable argument tests Kumar Kartikeya Dwivedi
` (4 subsequent siblings)
8 siblings, 0 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Implement the struct_ops arena argument conversion on x86. save_args()
gains the conversion map from bpf_tramp_collect_arena_args() and, as it
copies each native argument into the BPF ctx, routes a marked slot
through RAX:
movl %esrc, %eax /* truncate and clear the upper 32 bits */
subl $base_lo, %eax
movq %rax, ctx_slot
A nullable slot tests the full 64-bit kernel pointer first:
movq %rsrc, %rax
testq %rax, %rax
jz 1f
subl $base_lo, %eax
1:
movq %rax, ctx_slot
The 32-bit subtraction is sufficient since (u32)(kaddr - base) ==
(u32)kaddr - (u32)base, and it clears the upper half as the JITs require
of arena pointer registers. Stack-passed arguments already reload
through RAX, so only the subtraction (and the NULL test) is inserted
there. The marked slots are tracked with a running slot counter shared
by the register and stack branches, matching the flattened ctx offsets
in ctx_arg_info. The size probe reruns the same emission with the same
tnodes, so the image size matches by construction.
With both the kfunc and struct_ops directions implemented, flip
bpf_jit_supports_arena_args() on for x86.
v3: New patch, moving the conversion from a BPF entry prologue into the
trampoline, per review from Kumar.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
arch/x86/net/bpf_jit_comp.c | 64 +++++++++++++++++++++++++++++++++----
1 file changed, 57 insertions(+), 7 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index dd9009fcdc39..2f25353f135b 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3042,12 +3042,39 @@ static int get_nr_used_regs(const struct btf_func_model *m)
return nr_used_regs;
}
+/*
+ * Convert an arena kernel address into the arena pointer form on its way
+ * into the BPF ctx, rax = (u32)(src - kern_vm_start). A nullable arg
+ * preserves NULL, tested on the full 64-bit kernel pointer. The 32-bit
+ * subtraction both truncates and clears the upper half, so the stored
+ * value satisfies the JIT invariant for arena pointer registers.
+ */
+static void emit_arena_arg_conv(u8 **pprog, u32 src_reg, bool nullable, u32 base_lo)
+{
+ u8 *prog = *pprog;
+
+ if (nullable) {
+ if (src_reg != BPF_REG_0)
+ emit_mov_reg(&prog, true, BPF_REG_0, src_reg);
+ /* test rax, rax; jz over the 5-byte sub */
+ EMIT3(0x48, 0x85, 0xC0);
+ EMIT2(X86_JE, 5);
+ } else if (src_reg != BPF_REG_0) {
+ emit_mov_reg(&prog, false, BPF_REG_0, src_reg);
+ }
+ /* sub eax, base_lo */
+ EMIT1_off32(0x2D, base_lo);
+
+ *pprog = prog;
+}
+
static void save_args(const struct btf_func_model *m, u8 **prog,
- int stack_size, bool for_call_origin, u32 flags)
+ int stack_size, bool for_call_origin, u32 flags,
+ const struct bpf_tramp_arena_args *aargs)
{
int arg_regs, first_off = 0, nr_regs = 0, nr_stack_slots = 0;
bool use_jmp = bpf_trampoline_use_jmp(flags);
- int i, j;
+ int i, j, slot = 0;
/* Store function arguments to stack.
* For a function that accepts two pointers the sequence will be:
@@ -3088,6 +3115,10 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
for (j = 0; j < arg_regs; j++) {
emit_ldx(prog, BPF_DW, BPF_REG_0, BPF_REG_FP,
nr_stack_slots * 8 + 16 + (!use_jmp) * 8);
+ if (aargs && (aargs->slots & BIT(slot)))
+ emit_arena_arg_conv(prog, BPF_REG_0,
+ aargs->nullable_slots & BIT(slot),
+ (u32)aargs->kern_vm_start);
emit_stx(prog, BPF_DW, BPF_REG_FP, BPF_REG_0,
-stack_size);
@@ -3095,6 +3126,7 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
first_off = stack_size;
stack_size -= 8;
nr_stack_slots++;
+ slot++;
}
} else {
/* Only copy the arguments on-stack to current
@@ -3103,16 +3135,24 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
*/
if (for_call_origin) {
nr_regs += arg_regs;
+ slot += arg_regs;
continue;
}
/* copy the arguments from regs into stack */
for (j = 0; j < arg_regs; j++) {
- emit_stx(prog, BPF_DW, BPF_REG_FP,
- nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs,
- -stack_size);
+ u32 src = nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs;
+
+ if (aargs && (aargs->slots & BIT(slot))) {
+ emit_arena_arg_conv(prog, src,
+ aargs->nullable_slots & BIT(slot),
+ (u32)aargs->kern_vm_start);
+ src = BPF_REG_0;
+ }
+ emit_stx(prog, BPF_DW, BPF_REG_FP, src, -stack_size);
stack_size -= 8;
nr_regs++;
+ slot++;
}
}
}
@@ -3403,11 +3443,13 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
struct bpf_tramp_nodes *fentry = &tnodes[BPF_TRAMP_FENTRY];
struct bpf_tramp_nodes *fexit = &tnodes[BPF_TRAMP_FEXIT];
struct bpf_tramp_nodes *fmod_ret = &tnodes[BPF_TRAMP_MODIFY_RETURN];
+ struct bpf_tramp_arena_args aargs;
void *orig_call = func_addr;
int cookie_off, cookie_cnt;
u8 **branches = NULL;
u64 func_meta;
u8 *prog;
+ bool has_aargs;
bool save_ret;
/*
@@ -3418,6 +3460,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
WARN_ON_ONCE((flags & BPF_TRAMP_F_INDIRECT) &&
(flags & ~(BPF_TRAMP_F_INDIRECT | BPF_TRAMP_F_RET_FENTRY_RET)));
+ has_aargs = bpf_tramp_collect_arena_args(tnodes, flags, &aargs);
+
/* extra registers for struct arguments */
for (i = 0; i < m->nr_args; i++) {
if (m->arg_flags[i] & BTF_FMODEL_STRUCT_ARG)
@@ -3555,7 +3599,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
emit_store_stack_imm64(&prog, BPF_REG_0, -ip_off, (long)func_addr);
}
- save_args(m, &prog, regs_off, false, flags);
+ save_args(m, &prog, regs_off, false, flags,
+ has_aargs ? &aargs : NULL);
if (flags & BPF_TRAMP_F_CALL_ORIG) {
/* arg1: mov rdi, im */
@@ -3597,7 +3642,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
if (flags & BPF_TRAMP_F_CALL_ORIG) {
restore_regs(m, &prog, regs_off);
- save_args(m, &prog, arg_stack_off, true, flags);
+ save_args(m, &prog, arg_stack_off, true, flags, NULL);
if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) {
/* Before calling the original function, load the
@@ -4099,6 +4144,11 @@ bool bpf_jit_supports_stack_args(void)
return true;
}
+bool bpf_jit_supports_arena_args(void)
+{
+ return true;
+}
+
void *bpf_arch_text_copy(void *dst, void *src, size_t len)
{
if (text_poke_copy(dst, src, len) == NULL)
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH bpf-next v1 5/9] selftests/bpf: Add kfunc __arena and __arena_nullable argument tests
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
` (3 preceding siblings ...)
2026-07-15 22:00 ` [PATCH bpf-next v1 4/9] bpf, x86: Convert struct_ops arena arguments in the trampoline Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 6/9] selftests/bpf: Add JIT-sequence tests for __arena kfunc arguments Kumar Kartikeya Dwivedi
` (3 subsequent siblings)
8 siblings, 0 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Add arena-argument kfuncs to bpf_testmod, which also exercises the
argument rebasing on module kfuncs, and tests covering the accepted
argument forms (arena pointer, low 32 bits as a scalar, full user
address as a scalar), the exact rebase semantics via capture kfuncs
returning the raw argument (zero low 32 bits arrive as the arena kernel
base under __arena and as NULL under __arena_nullable), a NULL round
trip through a nullable deref kfunc, five arena arguments in one call, a
mixed __arena plus __arena_nullable call exercising both bitmasks on one
call site, a kernel-side dereference of an unpopulated page recovering
through the scratch page, and the rejections (no arena in the program,
incompatible register type).
The tests run on x86-64 and skip elsewhere, as programs with
arena-tagged kfunc args fail verification where the JIT lacks support.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
.../selftests/bpf/prog_tests/arena_kfunc.c | 15 ++
.../testing/selftests/bpf/progs/arena_kfunc.c | 250 ++++++++++++++++++
.../selftests/bpf/test_kmods/bpf_testmod.c | 57 ++++
.../bpf/test_kmods/bpf_testmod_kfunc.h | 10 +
4 files changed, 332 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_kfunc.c
create mode 100644 tools/testing/selftests/bpf/progs/arena_kfunc.c
diff --git a/tools/testing/selftests/bpf/prog_tests/arena_kfunc.c b/tools/testing/selftests/bpf/prog_tests/arena_kfunc.c
new file mode 100644
index 000000000000..9be19565f0d5
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/arena_kfunc.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+
+#include "arena_kfunc.skel.h"
+
+/*
+ * The test kfuncs live in bpf_testmod. Resolving kfuncs against module
+ * BTFs needs CAP_SYS_ADMIN, so run with full capabilities instead of
+ * through the verifier tests' capability-restricted runner.
+ */
+void test_arena_kfunc(void)
+{
+ RUN_TESTS(arena_kfunc);
+}
diff --git a/tools/testing/selftests/bpf/progs/arena_kfunc.c b/tools/testing/selftests/bpf/progs/arena_kfunc.c
new file mode 100644
index 000000000000..e7250c5197ab
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/arena_kfunc.c
@@ -0,0 +1,250 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#define BPF_NO_KFUNC_PROTOTYPES
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "bpf_experimental.h"
+#include <bpf_arena_common.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARENA);
+ __uint(map_flags, BPF_F_MMAPABLE);
+ /* page 0 hosts the arena global, page 1 is for allocations */
+ __uint(max_entries, 2);
+} arena SEC(".maps");
+
+/*
+ * Occupies page 0 so no allocation lands at arena offset 0, which the
+ * nullable tests below must be able to tell apart from NULL.
+ */
+u64 __arena arena_pad;
+
+/* volatile to force the scalar reloads below */
+volatile u64 stash;
+
+SEC("syscall")
+__arch_x86_64
+__success __retval(0)
+int arena_arg_forms(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+ u64 ret;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ /* PTR_TO_ARENA argument */
+ *val = 41;
+ ret = bpf_kfunc_arena_arg_test((u64 *)val);
+ if (ret != 41 || *val != 42)
+ return 2;
+
+ /* the low 32 bits as a scalar */
+ stash = (u32)(u64)val;
+ ret = bpf_kfunc_arena_arg_test((u64 *)stash);
+ if (ret != 42 || *val != 43)
+ return 3;
+
+ /* the full user address as a scalar */
+ stash = (u64)val;
+ bpf_addr_space_cast(stash, 1, 0);
+ ret = bpf_kfunc_arena_arg_test((u64 *)stash);
+ if (ret != 43 || *val != 44)
+ return 4;
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
+
+/*
+ * Pin the rebase semantics using the capture kfuncs, which return the raw
+ * argument value: __arena rebases unconditionally, so zero low 32 bits
+ * arrive as the arena kernel base, while __arena_nullable turns them into
+ * NULL.
+ */
+SEC("syscall")
+__arch_x86_64
+__success __retval(0)
+int arena_arg_rebase(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+ u64 base, off;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ base = bpf_kfunc_arena_cap_test(NULL);
+ if (!base)
+ return 2;
+
+ /* only the low 32 bits contribute */
+ stash = 0xbadc0ffe00000000;
+ if (bpf_kfunc_arena_cap_test((u64 *)stash) != base)
+ return 3;
+
+ off = (u32)(u64)val;
+ if (bpf_kfunc_arena_cap_test((u64 *)val) != base + off)
+ return 4;
+
+ if (bpf_kfunc_arena_cap_nullable_test(NULL) != 0)
+ return 5;
+
+ stash = 0xbadc0ffe00000000;
+ if (bpf_kfunc_arena_cap_nullable_test((u64 *)stash) != 0)
+ return 6;
+
+ if (bpf_kfunc_arena_cap_nullable_test((u64 *)val) != base + off)
+ return 7;
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__success __retval(0)
+int arena_arg_nullable(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+ u64 ret;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ *val = 41;
+ ret = bpf_kfunc_arena_nullable_arg_test((u64 *)val);
+ if (ret != 41 || *val != 42)
+ return 2;
+
+ if (bpf_kfunc_arena_nullable_arg_test(NULL) != 0xdeadbeef)
+ return 3;
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__success __retval(0)
+int arena_args5(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ val[0] = 1;
+ val[1] = 2;
+ val[2] = 4;
+ val[3] = 8;
+ val[4] = 16;
+
+ if (bpf_kfunc_arena_args5_test((u64 *)&val[0], (u64 *)&val[1],
+ (u64 *)&val[2], (u64 *)&val[3],
+ (u64 *)&val[4]) != 31)
+ return 2;
+ if (bpf_kfunc_arena_args5_test((u64 *)&val[0], (u64 *)&val[1],
+ (u64 *)&val[2], (u64 *)&val[3], NULL) != 15)
+ return 3;
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__success __retval(0)
+int arena_arg_mixed(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ val[0] = 7;
+ val[1] = 5;
+
+ if (bpf_kfunc_arena_mixed_test((u64 *)&val[0], NULL) != 7)
+ return 2;
+
+ if (bpf_kfunc_arena_mixed_test((u64 *)&val[0], (u64 *)&val[1]) != 12)
+ return 3;
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
+
+/* kernel-side faults on unpopulated pages recover via the scratch page */
+SEC("syscall")
+__arch_x86_64
+__success __retval(0)
+int arena_arg_unpopulated(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ stash = (u64)val + PAGE_SIZE;
+ bpf_kfunc_arena_arg_test((u64 *)stash);
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__failure __msg("arena pointer requires a program with an associated arena")
+int arena_arg_no_arena(void *ctx)
+{
+ bpf_kfunc_arena_arg_test((u64 *)1);
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__failure __msg("is not a pointer to arena or scalar")
+int arena_arg_bad_reg(void *ctx)
+{
+ u64 buf = 0;
+
+ /* use the arena so the program passes the arena presence check */
+ bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ bpf_kfunc_arena_arg_test(&buf);
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__failure __msg("arena pointer cannot be a stack argument")
+int arena_arg_stack(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ bpf_kfunc_arena_stack_arg_test(1, 2, 3, 4, 5, (u64 *)1);
+#endif
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index 30f1cd23093c..fad106b1549c 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -210,6 +210,56 @@ __bpf_kfunc void bpf_kfunc_common_test(void)
{
}
+__bpf_kfunc u64 bpf_kfunc_arena_arg_test(u64 *val__arena)
+{
+ u64 old;
+
+ old = *val__arena;
+ *val__arena = old + 1;
+ return old;
+}
+
+__bpf_kfunc u64 bpf_kfunc_arena_nullable_arg_test(u64 *val__arena_nullable)
+{
+ u64 old;
+
+ if (!val__arena_nullable)
+ return 0xdeadbeef;
+
+ old = *val__arena_nullable;
+ *val__arena_nullable = old + 1;
+ return old;
+}
+
+__bpf_kfunc u64 bpf_kfunc_arena_cap_test(u64 *val__arena)
+{
+ return (u64)val__arena;
+}
+
+__bpf_kfunc u64 bpf_kfunc_arena_cap_nullable_test(u64 *val__arena_nullable)
+{
+ return (u64)val__arena_nullable;
+}
+
+__bpf_kfunc u64 bpf_kfunc_arena_args5_test(u64 *a__arena, u64 *b__arena,
+ u64 *c__arena, u64 *d__arena,
+ u64 *e__arena_nullable)
+{
+ return *a__arena + *b__arena + *c__arena + *d__arena +
+ (e__arena_nullable ? *e__arena_nullable : 0);
+}
+
+__bpf_kfunc u64 bpf_kfunc_arena_stack_arg_test(u64 a, u64 b, u64 c, u64 d, u64 e,
+ u64 *f__arena)
+{
+ return a + b + c + d + e + *f__arena;
+}
+
+__bpf_kfunc u64 bpf_kfunc_arena_mixed_test(u64 *a__arena, u64 *b__arena_nullable)
+{
+ return *a__arena + (b__arena_nullable ? *b__arena_nullable : 0);
+}
+
__bpf_kfunc void bpf_kfunc_dynptr_test(struct bpf_dynptr *ptr,
struct bpf_dynptr *ptr__nullable)
{
@@ -723,6 +773,13 @@ BTF_ID_FLAGS(func, bpf_iter_testmod_seq_next, KF_ITER_NEXT | KF_RET_NULL)
BTF_ID_FLAGS(func, bpf_iter_testmod_seq_destroy, KF_ITER_DESTROY)
BTF_ID_FLAGS(func, bpf_iter_testmod_seq_value)
BTF_ID_FLAGS(func, bpf_kfunc_common_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_arg_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_nullable_arg_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_cap_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_cap_nullable_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_args5_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_stack_arg_test)
+BTF_ID_FLAGS(func, bpf_kfunc_arena_mixed_test)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_pass1)
BTF_ID_FLAGS(func, bpf_kfunc_dynptr_test)
BTF_ID_FLAGS(func, bpf_kfunc_nested_acquire_nonzero_offset_test, KF_ACQUIRE)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index c36bb911defa..42d3fccc6ff9 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -98,6 +98,16 @@ void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p) __ksym;
void bpf_kfunc_call_test_ref(struct prog_test_ref_kfunc *p) __ksym;
void bpf_kfunc_call_test_mem_len_pass1(void *mem, int len) __ksym;
+__u64 bpf_kfunc_arena_arg_test(__u64 *val__arena) __ksym;
+__u64 bpf_kfunc_arena_nullable_arg_test(__u64 *val__arena_nullable) __ksym;
+__u64 bpf_kfunc_arena_cap_test(__u64 *val__arena) __ksym;
+__u64 bpf_kfunc_arena_cap_nullable_test(__u64 *val__arena_nullable) __ksym;
+__u64 bpf_kfunc_arena_args5_test(__u64 *a__arena, __u64 *b__arena,
+ __u64 *c__arena, __u64 *d__arena,
+ __u64 *e__arena_nullable) __ksym;
+__u64 bpf_kfunc_arena_stack_arg_test(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+ __u64 *f__arena) __ksym;
+__u64 bpf_kfunc_arena_mixed_test(__u64 *a__arena, __u64 *b__arena_nullable) __ksym;
int *bpf_kfunc_call_test_get_rdwr_mem(struct prog_test_ref_kfunc *p, const int rdwr_buf_size) __ksym;
int *bpf_kfunc_call_test_get_rdonly_mem(struct prog_test_ref_kfunc *p, const int rdonly_buf_size) __ksym;
int *bpf_kfunc_call_test_acq_rdonly_mem(struct prog_test_ref_kfunc *p, const int rdonly_buf_size) __ksym;
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH bpf-next v1 6/9] selftests/bpf: Add JIT-sequence tests for __arena kfunc arguments
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
` (4 preceding siblings ...)
2026-07-15 22:00 ` [PATCH bpf-next v1 5/9] selftests/bpf: Add kfunc __arena and __arena_nullable argument tests Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 7/9] selftests/bpf: Add struct_ops __arena and __arena_nullable argument tests Kumar Kartikeya Dwivedi
` (2 subsequent siblings)
8 siblings, 0 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Pin the exact rebase sequences the JITs emit for __arena and
__arena_nullable kfunc arguments with __jited assertions on x86-64: the
unconditional truncate-and-add, the nullable test-and-skip variant, and
all five argument registers in one call, which also covers the
REX-prefixed encoding of r8 on x86. The capture kfuncs take the argument
without dereferencing, so only the emitted code is under test. The
tests skip without LLVM disassembler support.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
.../bpf/prog_tests/arena_kfunc_jit.c | 13 +++
.../selftests/bpf/progs/arena_kfunc_jit.c | 96 +++++++++++++++++++
2 files changed, 109 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_kfunc_jit.c
create mode 100644 tools/testing/selftests/bpf/progs/arena_kfunc_jit.c
diff --git a/tools/testing/selftests/bpf/prog_tests/arena_kfunc_jit.c b/tools/testing/selftests/bpf/prog_tests/arena_kfunc_jit.c
new file mode 100644
index 000000000000..2359cde24c45
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/arena_kfunc_jit.c
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "arena_kfunc_jit.skel.h"
+
+/*
+ * Runs with full capabilities: resolving module kfunc ksyms requires
+ * CAP_SYS_ADMIN, which rules out the capability-restricted runner.
+ */
+void test_arena_kfunc_jit(void)
+{
+ RUN_TESTS(arena_kfunc_jit);
+}
diff --git a/tools/testing/selftests/bpf/progs/arena_kfunc_jit.c b/tools/testing/selftests/bpf/progs/arena_kfunc_jit.c
new file mode 100644
index 000000000000..84747611725f
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/arena_kfunc_jit.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+/*
+ * Verify the JIT-emitted rebase sequences for __arena and __arena_nullable
+ * kfunc arguments. The capture kfuncs take the argument without
+ * dereferencing it, so these tests pin only the emitted code.
+ */
+#define BPF_NO_KFUNC_PROTOTYPES
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "bpf_experimental.h"
+#include <bpf_arena_common.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARENA);
+ __uint(map_flags, BPF_F_MMAPABLE);
+ __uint(max_entries, 1);
+} arena SEC(".maps");
+
+/* volatile to force the scalar reloads below */
+volatile u64 stash;
+
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+
+SEC("syscall")
+__arch_x86_64
+__jited("...")
+__jited(" movl %edi, %edi")
+__jited(" addq %r12, %rdi")
+__jited("...")
+__jited(" callq {{.*}}")
+__success
+int arena_arg_jit_rebase(void *ctx)
+{
+ stash = (u64)bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ bpf_kfunc_arena_cap_test((u64 *)stash);
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__jited("...")
+__jited(" movl %edi, %edi")
+__jited(" testl %edi, %edi")
+__jited(" je {{.*}}")
+__jited(" addq %r12, %rdi")
+__success
+int arena_arg_jit_nullable(void *ctx)
+{
+ stash = (u64)bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ bpf_kfunc_arena_cap_nullable_test((u64 *)stash);
+ return 0;
+}
+
+SEC("syscall")
+__arch_x86_64
+__jited("...")
+__jited(" movl %edi, %edi")
+__jited(" addq %r12, %rdi")
+__jited(" movl %esi, %esi")
+__jited(" addq %r12, %rsi")
+__jited(" movl %edx, %edx")
+__jited(" addq %r12, %rdx")
+__jited(" movl %ecx, %ecx")
+__jited(" addq %r12, %rcx")
+__jited(" movl %r8d, %r8d")
+__jited(" testl %r8d, %r8d")
+__jited(" je {{.*}}")
+__jited(" addq %r12, %r8")
+__success
+int arena_arg_jit_args5(void *ctx)
+{
+ u64 __arena *val;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ val[0] = 1;
+ val[1] = 2;
+ val[2] = 4;
+ val[3] = 8;
+ val[4] = 16;
+
+ bpf_kfunc_arena_args5_test((u64 *)&val[0], (u64 *)&val[1],
+ (u64 *)&val[2], (u64 *)&val[3],
+ (u64 *)&val[4]);
+ return 0;
+}
+
+#endif /* __BPF_FEATURE_ADDR_SPACE_CAST */
+
+char _license[] SEC("license") = "GPL";
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH bpf-next v1 7/9] selftests/bpf: Add struct_ops __arena and __arena_nullable argument tests
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
` (5 preceding siblings ...)
2026-07-15 22:00 ` [PATCH bpf-next v1 6/9] selftests/bpf: Add JIT-sequence tests for __arena kfunc arguments Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines Kumar Kartikeya Dwivedi
2026-07-15 22:00 ` [PATCH bpf-next v1 9/9] selftests/bpf: Test stack-passed struct_ops arena arguments Kumar Kartikeya Dwivedi
8 siblings, 0 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Add test_arena and test_arena_nullable members to bpf_testmod_ops3 with
arena-tagged stub arguments and kfuncs that forward a caller-provided
pointer to them. The kfuncs take arena-tagged arguments, so each round
trip exercises both conversion directions end to end: the kfunc receives
a kernel arena address and the trampoline converts it back to an arena
pointer for the callback.
The non-nullable callback dereferences its argument with no NULL branch
and captures the raw ctx value, which the trigger program compares
against the arena offset of the passed object, pinning the exact
(u32)(kaddr - kern_vm_start) conversion. The nullable callback verifies
that only a true kernel NULL arrives as NULL. Failure coverage: a
program with no arena is rejected when it loads. The tests run on x86-64
and skip elsewhere, as the programs fail verification where the JIT
lacks arena argument support.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
.../bpf/prog_tests/test_struct_ops_arena.c | 72 ++++++++++++++
.../selftests/bpf/progs/struct_ops_arena.c | 94 +++++++++++++++++++
.../bpf/progs/struct_ops_arena_fail.c | 20 ++++
.../selftests/bpf/test_kmods/bpf_testmod.c | 24 +++++
.../selftests/bpf/test_kmods/bpf_testmod.h | 3 +
.../bpf/test_kmods/bpf_testmod_kfunc.h | 2 +
6 files changed, 215 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_arena.c
create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_arena.c
create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_arena_fail.c
diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_arena.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_arena.c
new file mode 100644
index 000000000000..fbf1a29d9af6
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_arena.c
@@ -0,0 +1,72 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+
+#include "struct_ops_arena.skel.h"
+#include "struct_ops_arena_fail.skel.h"
+
+/*
+ * Attach callbacks with __arena and __arena_nullable arguments and drive
+ * them through the bpf_testmod_ops3_call_test_arena*() kfuncs.
+ */
+static void arena_arg(void)
+{
+ LIBBPF_OPTS(bpf_test_run_opts, topts);
+ struct struct_ops_arena *skel;
+ struct bpf_link *link = NULL;
+ int err;
+
+ skel = struct_ops_arena__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "struct_ops_arena__open_and_load"))
+ return;
+
+ link = bpf_map__attach_struct_ops(skel->maps.testmod_arena);
+ if (!ASSERT_OK_PTR(link, "attach_struct_ops"))
+ goto out;
+
+ err = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.trigger),
+ &topts);
+ ASSERT_OK(err, "test_run");
+ ASSERT_EQ(topts.retval, 0, "trigger_retval");
+
+out:
+ bpf_link__destroy(link);
+ struct_ops_arena__destroy(skel);
+}
+
+/*
+ * A program with no arena cannot attach to a member with an __arena
+ * argument.
+ */
+static void arena_arg_fail(void)
+{
+ struct struct_ops_arena_fail *skel;
+
+ skel = struct_ops_arena_fail__open_and_load();
+ if (ASSERT_ERR_PTR(skel, "struct_ops_arena_fail__open_and_load"))
+ return;
+
+ struct_ops_arena_fail__destroy(skel);
+}
+
+/*
+ * Serialized because it attaches the singleton bpf_testmod_ops3, which
+ * test_struct_ops_private_stack also attaches; registering it twice fails
+ * with -EEXIST.
+ */
+void serial_test_struct_ops_arena(void)
+{
+ /*
+ * Arena struct_ops arguments need JIT support, currently x86-64 only.
+ * Elsewhere verification fails with "JIT does not support arena
+ * arguments", so the programs cannot even load.
+ */
+#if defined(__x86_64__)
+ if (test__start_subtest("arena_arg"))
+ arena_arg();
+ if (test__start_subtest("arena_arg_fail"))
+ arena_arg_fail();
+#else
+ test__skip();
+#endif
+}
diff --git a/tools/testing/selftests/bpf/progs/struct_ops_arena.c b/tools/testing/selftests/bpf/progs/struct_ops_arena.c
new file mode 100644
index 000000000000..40c856a748d2
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/struct_ops_arena.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#define BPF_NO_KFUNC_PROTOTYPES
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_experimental.h"
+#include <bpf_arena_common.h>
+#include "../test_kmods/bpf_testmod.h"
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARENA);
+ __uint(map_flags, BPF_F_MMAPABLE);
+ /* page 0 hosts the arena globals, page 1 is for allocations */
+ __uint(max_entries, 2);
+} arena SEC(".maps");
+
+/* also associates the callbacks with the arena */
+u64 __arena arena_touch;
+/* raw value of the last __arena ctx argument, captured by test_arena_cb */
+u64 __arena cb_ptr_val;
+
+SEC("struct_ops/test_arena")
+int test_arena_cb(unsigned long long *ctx)
+{
+ u64 __arena *ptr = (u64 __arena *)ctx[0];
+
+ arena_touch++;
+ cb_ptr_val = ctx[0];
+ *ptr += 1;
+ return 0;
+}
+
+SEC("struct_ops/test_arena_nullable")
+int test_arena_nullable_cb(unsigned long long *ctx)
+{
+ u64 __arena *ptr = (u64 __arena *)ctx[0];
+
+ arena_touch++;
+ if (!ptr)
+ return 0xbee;
+ *ptr += 1;
+ return 0;
+}
+
+SEC(".struct_ops.link")
+struct bpf_testmod_ops3 testmod_arena = {
+ .test_arena = (void *)test_arena_cb,
+ .test_arena_nullable = (void *)test_arena_nullable_cb,
+};
+
+SEC("syscall")
+int trigger(void *ctx)
+{
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+ u64 __arena *val;
+ int ret;
+
+ val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ if (!val)
+ return 1;
+
+ *val = 41;
+ ret = bpf_testmod_ops3_call_test_arena((u64 *)val);
+ if (ret)
+ return 2;
+ if (*val != 42)
+ return 3;
+
+ /*
+ * The callback must have seen exactly (u32)(kaddr - kern_vm_start),
+ * which is the arena offset of val with the upper 32 bits clear.
+ */
+ if (cb_ptr_val != (u32)(u64)val)
+ return 4;
+
+ ret = bpf_testmod_ops3_call_test_arena_nullable((u64 *)val);
+ if (ret)
+ return 5;
+ if (*val != 43)
+ return 6;
+
+ /* NULL survives the nullable kfunc and the trampoline as NULL */
+ ret = bpf_testmod_ops3_call_test_arena_nullable(NULL);
+ if (ret != 0xbee)
+ return 7;
+
+ bpf_arena_free_pages(&arena, (void __arena *)val, 1);
+#endif
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/struct_ops_arena_fail.c b/tools/testing/selftests/bpf/progs/struct_ops_arena_fail.c
new file mode 100644
index 000000000000..1c0ec727d637
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/struct_ops_arena_fail.c
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod.h"
+
+char _license[] SEC("license") = "GPL";
+
+/* No arena in the program: attaching to test_arena must be rejected. */
+SEC("struct_ops/test_arena")
+int test_arena_no_arena(unsigned long long *ctx)
+{
+ return 0;
+}
+
+SEC(".struct_ops.link")
+struct bpf_testmod_ops3 testmod_arena_fail = {
+ .test_arena = (void *)test_arena_no_arena,
+};
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index fad106b1549c..48ce36a6b82f 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -370,9 +370,21 @@ static int bpf_testmod_test_4(void)
return 0;
}
+static int bpf_testmod_ops3__test_arena(u64 *ptr__arena)
+{
+ return 0;
+}
+
+static int bpf_testmod_ops3__test_arena_nullable(u64 *ptr__arena_nullable)
+{
+ return 0;
+}
+
static struct bpf_testmod_ops3 __bpf_testmod_ops3 = {
.test_1 = bpf_testmod_test_3,
.test_2 = bpf_testmod_test_4,
+ .test_arena = bpf_testmod_ops3__test_arena,
+ .test_arena_nullable = bpf_testmod_ops3__test_arena_nullable,
};
static void bpf_testmod_test_struct_ops3(void)
@@ -391,6 +403,16 @@ __bpf_kfunc void bpf_testmod_ops3_call_test_2(void)
st_ops3->test_2();
}
+__bpf_kfunc int bpf_testmod_ops3_call_test_arena(u64 *ptr__arena)
+{
+ return st_ops3->test_arena(ptr__arena);
+}
+
+__bpf_kfunc int bpf_testmod_ops3_call_test_arena_nullable(u64 *ptr__arena_nullable)
+{
+ return st_ops3->test_arena_nullable(ptr__arena_nullable);
+}
+
struct bpf_testmod_btf_type_tag_1 {
int a;
};
@@ -795,6 +817,8 @@ BTF_ID_FLAGS(func, bpf_testmod_ctx_create, KF_ACQUIRE | KF_RET_NULL)
BTF_ID_FLAGS(func, bpf_testmod_ctx_release, KF_RELEASE)
BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_1)
BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_2)
+BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_arena)
+BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_arena_nullable)
BTF_ID_FLAGS(func, bpf_kfunc_get_default_trusted_ptr_test);
BTF_ID_FLAGS(func, bpf_kfunc_put_default_trusted_ptr_test);
BTF_KFUNCS_END(bpf_testmod_common_kfunc_ids)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
index 863fd10f1619..c367ec856776 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
@@ -106,6 +106,9 @@ struct bpf_testmod_ops2 {
struct bpf_testmod_ops3 {
int (*test_1)(void);
int (*test_2)(void);
+ /* Used to test arena pointer arguments. */
+ int (*test_arena)(u64 *ptr);
+ int (*test_arena_nullable)(u64 *ptr);
};
struct st_ops_args {
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index 42d3fccc6ff9..ff0d3894d7af 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -121,6 +121,8 @@ u32 bpf_kfunc_call_test_static_unused_arg(u32 arg, u32 unused) __ksym;
#endif
void bpf_testmod_test_mod_kfunc(int i) __ksym;
+int bpf_testmod_ops3_call_test_arena(__u64 *ptr__arena) __ksym;
+int bpf_testmod_ops3_call_test_arena_nullable(__u64 *ptr__arena_nullable) __ksym;
__u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b,
__u32 c, __u64 d) __ksym;
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
` (6 preceding siblings ...)
2026-07-15 22:00 ` [PATCH bpf-next v1 7/9] selftests/bpf: Add struct_ops __arena and __arena_nullable argument tests Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
2026-07-15 22:14 ` sashiko-bot
2026-07-15 22:51 ` bot+bpf-ci
2026-07-15 22:00 ` [PATCH bpf-next v1 9/9] selftests/bpf: Test stack-passed struct_ops arena arguments Kumar Kartikeya Dwivedi
8 siblings, 2 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
save_args() reads stack-passed arguments relative to rbp assuming two
return addresses sit between the saved rbp and the arguments, which
holds when the trampoline is entered through the fentry call from a
traced function. An indirect trampoline is called through a function
pointer, so only the caller's return address is on the stack and the
arguments start at rbp + 16, not rbp + 24. Every stack-passed argument
of a struct_ops callback with more than six argument slots is read one
slot off.
This has gone unnoticed because no in-tree struct_ops member passes
arguments on the stack. The jmp-entry form already accounts for having
a single return address; treat BPF_TRAMP_F_INDIRECT the same way.
Fixes: 473e3150e30a ("bpf, x86: allow function arguments up to 12 for TRACING")
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
arch/x86/net/bpf_jit_comp.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 2f25353f135b..672fe54dfeb8 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3074,6 +3074,7 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
{
int arg_regs, first_off = 0, nr_regs = 0, nr_stack_slots = 0;
bool use_jmp = bpf_trampoline_use_jmp(flags);
+ int stack_args_off = (use_jmp || (flags & BPF_TRAMP_F_INDIRECT)) ? 16 : 24;
int i, j, slot = 0;
/* Store function arguments to stack.
@@ -3105,16 +3106,16 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
/* copy function arguments from origin stack frame
* into current stack frame.
*
- * The starting address of the arguments on-stack
- * is:
- * rbp + 8(push rbp) +
- * 8(return addr of origin call) +
- * 8(return addr of the caller)
- * which means: rbp + 24
+ * The arguments on-stack start above the saved rbp
+ * and the return addresses: two return addresses
+ * (origin call and caller) when the trampoline is
+ * entered through the fentry call, so rbp + 24, and
+ * a single one when it is entered with a jmp or
+ * called indirectly, so rbp + 16.
*/
for (j = 0; j < arg_regs; j++) {
emit_ldx(prog, BPF_DW, BPF_REG_0, BPF_REG_FP,
- nr_stack_slots * 8 + 16 + (!use_jmp) * 8);
+ nr_stack_slots * 8 + stack_args_off);
if (aargs && (aargs->slots & BIT(slot)))
emit_arena_arg_conv(prog, BPF_REG_0,
aargs->nullable_slots & BIT(slot),
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines
2026-07-15 22:00 ` [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines Kumar Kartikeya Dwivedi
@ 2026-07-15 22:14 ` sashiko-bot
2026-07-15 22:51 ` bot+bpf-ci
1 sibling, 0 replies; 14+ messages in thread
From: sashiko-bot @ 2026-07-15 22:14 UTC (permalink / raw)
To: Kumar Kartikeya Dwivedi; +Cc: bpf
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Medium] The patch correctly fixes the struct_ops stack argument offset bug for x86, but leaves the exact same bug unfixed in the ARM64, RISC-V, and LoongArch BPF JIT implementations.
--
--- Patch [8]: [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines ---
commit 804aec167442bbd8ddbbc13495bc188618bbaeaf
Author: Tejun Heo <tj@kernel.org>
bpf, x86: Fix stack-passed arguments for indirect trampolines
This commit fixes a bug in the x86 BPF JIT where stack-passed arguments
for indirect trampolines (such as struct_ops) were read from the wrong
offset. It correctly adjusts the offset to account for only the caller's
return address being on the stack, which fixes callbacks that have more
than six arguments.
> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index 2f25353f135bd..672fe54dfeb8d 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c
> @@ -3074,6 +3074,7 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
> {
> int arg_regs, first_off = 0, nr_regs = 0, nr_stack_slots = 0;
> bool use_jmp = bpf_trampoline_use_jmp(flags);
> + int stack_args_off = (use_jmp || (flags & BPF_TRAMP_F_INDIRECT)) ? 16 : 24;
[Severity: Medium]
This is a pre-existing issue, but does this same bug also need to be fixed
in the ARM64, RISC-V, and LoongArch BPF JIT implementations?
On ARM64, save_args() unconditionally hardcodes a 32-byte offset:
arch/arm64/net/bpf_jit_comp.c:save_args() {
...
soff = 32; /* on stack arguments start from FP + 32 */
...
}
This seems to assume two 16-byte frames (FP+LR and FP+R9), but struct_ops
only pushes one, meaning stack arguments would start at FP + 16.
Similarly, on RISC-V and LoongArch, store_args() unconditionally skips 16
bytes for the traced function's T0 and FP frame:
arch/riscv/net/bpf_jit_comp64.c:store_args() {
...
/* skip slots for T0 and FP of traced function */
emit_ld(RV_REG_T1, 16 + (i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
...
}
For BPF_TRAMP_F_INDIRECT (is_struct_ops), the prologue aligns FP directly
to the caller's SP, so arguments start exactly at FP. Will these hardcoded
offsets cause the JIT to read 16 bytes past the actual arguments on these
architectures as well?
> int i, j, slot = 0;
>
> /* Store function arguments to stack.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260715220052.1590783-1-memxor@gmail.com?part=8
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines
2026-07-15 22:00 ` [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines Kumar Kartikeya Dwivedi
2026-07-15 22:14 ` sashiko-bot
@ 2026-07-15 22:51 ` bot+bpf-ci
1 sibling, 0 replies; 14+ messages in thread
From: bot+bpf-ci @ 2026-07-15 22:51 UTC (permalink / raw)
To: memxor, bpf
Cc: tj, ast, andrii, daniel, eddyz87, emil, kkd, kernel-team, ast,
andrii, daniel, martin.lau, eddyz87, yonghong.song, clm,
ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2055 bytes --]
> bpf, x86: Fix stack-passed arguments for indirect trampolines
>
> save_args() reads stack-passed arguments relative to rbp assuming two
> return addresses sit between the saved rbp and the arguments, which
> holds when the trampoline is entered through the fentry call from a
> traced function. An indirect trampoline is called through a function
> pointer, so only the caller's return address is on the stack and the
> arguments start at rbp + 16, not rbp + 24. Every stack-passed argument
> of a struct_ops callback with more than six argument slots is read one
> slot off.
>
> This has gone unnoticed because no in-tree struct_ops member passes
> arguments on the stack. The jmp-entry form already accounts for having
> a single return address; treat BPF_TRAMP_F_INDIRECT the same way.
>
> Fixes: 473e3150e30a ("bpf, x86: allow function arguments up to 12 for TRACING")
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Is the Fixes: tag pointing at the right commit here?
473e3150e30a introduced save_args() and its rbp + 24 stack read, but at
that point trampolines were only entered through the fentry call, where
both the origin-call and caller return addresses are on the stack, so
rbp + 24 was correct and the described bug did not yet exist.
The indirect entry path that leaves a single return address on the stack
came in with BPF_TRAMP_F_INDIRECT, added by 2cd3e3772e41 ("x86/cfi,bpf:
Fix bpf_struct_ops CFI"), which reused save_args() unchanged. That looks
like the point where a struct_ops callback with more than six argument
slots starts reading each stack-passed argument one slot off.
Would this be more accurate?
Fixes: 2cd3e3772e41 ("x86/cfi,bpf: Fix bpf_struct_ops CFI")
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29454748905
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH bpf-next v1 9/9] selftests/bpf: Test stack-passed struct_ops arena arguments
2026-07-15 22:00 [PATCH bpf-next v1 0/9] Add arena argument support to kfuncs and struct_ops Kumar Kartikeya Dwivedi
` (7 preceding siblings ...)
2026-07-15 22:00 ` [PATCH bpf-next v1 8/9] bpf, x86: Fix stack-passed arguments for indirect trampolines Kumar Kartikeya Dwivedi
@ 2026-07-15 22:00 ` Kumar Kartikeya Dwivedi
8 siblings, 0 replies; 14+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2026-07-15 22:00 UTC (permalink / raw)
To: bpf
Cc: Tejun Heo, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Emil Tsalapatis, kkd, kernel-team
From: Tejun Heo <tj@kernel.org>
Add a test_arena_stack member with eight leading scalar arguments so the
arena pointer is passed on the stack.
The callback validates the first and last scalar ctx slots before
dereferencing the pointer in ctx[8]. This exercises the indirect
trampoline stack layout and arena conversion together, and prevents a
regression where stack arguments are read one slot late.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
.../selftests/bpf/progs/struct_ops_arena.c | 21 +++++++++++++++++++
.../selftests/bpf/test_kmods/bpf_testmod.c | 14 +++++++++++++
.../selftests/bpf/test_kmods/bpf_testmod.h | 3 +++
.../bpf/test_kmods/bpf_testmod_kfunc.h | 1 +
4 files changed, 39 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/struct_ops_arena.c b/tools/testing/selftests/bpf/progs/struct_ops_arena.c
index 40c856a748d2..ba04c73d8d96 100644
--- a/tools/testing/selftests/bpf/progs/struct_ops_arena.c
+++ b/tools/testing/selftests/bpf/progs/struct_ops_arena.c
@@ -46,10 +46,24 @@ int test_arena_nullable_cb(unsigned long long *ctx)
return 0;
}
+SEC("struct_ops/test_arena_stack")
+int test_arena_stack_cb(unsigned long long *ctx)
+{
+ u64 __arena *ptr = (u64 __arena *)ctx[8];
+
+ arena_touch++;
+ /* pin the slot layout: the leading args fill ctx[0]..ctx[7] */
+ if (ctx[0] != 1 || ctx[7] != 8)
+ return 0xbad;
+ *ptr += 1;
+ return 0;
+}
+
SEC(".struct_ops.link")
struct bpf_testmod_ops3 testmod_arena = {
.test_arena = (void *)test_arena_cb,
.test_arena_nullable = (void *)test_arena_nullable_cb,
+ .test_arena_stack = (void *)test_arena_stack_cb,
};
SEC("syscall")
@@ -88,6 +102,13 @@ int trigger(void *ctx)
if (ret != 0xbee)
return 7;
+ /* the arena pointer is stack-passed into the trampoline here */
+ ret = bpf_testmod_ops3_call_test_arena_stack((u64 *)val);
+ if (ret)
+ return 8;
+ if (*val != 44)
+ return 9;
+
bpf_arena_free_pages(&arena, (void __arena *)val, 1);
#endif
return 0;
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index 48ce36a6b82f..9e268fd69a93 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -380,11 +380,19 @@ static int bpf_testmod_ops3__test_arena_nullable(u64 *ptr__arena_nullable)
return 0;
}
+static int bpf_testmod_ops3__test_arena_stack(u64 a, u64 b, u64 c, u64 d,
+ u64 e, u64 f, u64 g, u64 h,
+ u64 *ptr__arena)
+{
+ return 0;
+}
+
static struct bpf_testmod_ops3 __bpf_testmod_ops3 = {
.test_1 = bpf_testmod_test_3,
.test_2 = bpf_testmod_test_4,
.test_arena = bpf_testmod_ops3__test_arena,
.test_arena_nullable = bpf_testmod_ops3__test_arena_nullable,
+ .test_arena_stack = bpf_testmod_ops3__test_arena_stack,
};
static void bpf_testmod_test_struct_ops3(void)
@@ -413,6 +421,11 @@ __bpf_kfunc int bpf_testmod_ops3_call_test_arena_nullable(u64 *ptr__arena_nullab
return st_ops3->test_arena_nullable(ptr__arena_nullable);
}
+__bpf_kfunc int bpf_testmod_ops3_call_test_arena_stack(u64 *ptr__arena)
+{
+ return st_ops3->test_arena_stack(1, 2, 3, 4, 5, 6, 7, 8, ptr__arena);
+}
+
struct bpf_testmod_btf_type_tag_1 {
int a;
};
@@ -819,6 +832,7 @@ BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_1)
BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_2)
BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_arena)
BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_arena_nullable)
+BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_arena_stack)
BTF_ID_FLAGS(func, bpf_kfunc_get_default_trusted_ptr_test);
BTF_ID_FLAGS(func, bpf_kfunc_put_default_trusted_ptr_test);
BTF_KFUNCS_END(bpf_testmod_common_kfunc_ids)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
index c367ec856776..33f2af5b7085 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
@@ -109,6 +109,9 @@ struct bpf_testmod_ops3 {
/* Used to test arena pointer arguments. */
int (*test_arena)(u64 *ptr);
int (*test_arena_nullable)(u64 *ptr);
+ /* enough leading args to force @ptr onto the stack on x86 and arm64 */
+ int (*test_arena_stack)(u64 a, u64 b, u64 c, u64 d, u64 e, u64 f,
+ u64 g, u64 h, u64 *ptr);
};
struct st_ops_args {
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index ff0d3894d7af..1a72d0fda53c 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -123,6 +123,7 @@ u32 bpf_kfunc_call_test_static_unused_arg(u32 arg, u32 unused) __ksym;
void bpf_testmod_test_mod_kfunc(int i) __ksym;
int bpf_testmod_ops3_call_test_arena(__u64 *ptr__arena) __ksym;
int bpf_testmod_ops3_call_test_arena_nullable(__u64 *ptr__arena_nullable) __ksym;
+int bpf_testmod_ops3_call_test_arena_stack(__u64 *ptr__arena) __ksym;
__u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b,
__u32 c, __u64 d) __ksym;
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread