* [PATCH bpf-next v3 01/15] bpf: Read a kfunc's __sz argument only when it is in a register
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 02/15] selftests/bpf: Add a test for an __int128 by-value argument Yonghong Song
` (13 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
Commit e0b7b91c72db ("bpf: Support stack arguments for kfunc calls")
supported stack arguments for kfunc's. In bpf_kfunc_stack_access_bytes(),
the size of a ptr + __sz pair is read from const_reg_vals[] at index
'BPF_REG_1 + arg + 1'. Past the fifth argument that index leaves the
argument registers and reaches 6 through 9, which are the callee saved
registers R6 through R9. The verifier does record constants for those,
so a __sz argument passed on the stack can take the value of an
unrelated register as its size.
Fix it by guard size_reg which has to be less than or equal to
MAX_BPF_FUNC_REG_ARGS.
Fixes: e0b7b91c72db ("bpf: Support stack arguments for kfunc calls")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
kernel/bpf/verifier.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9e79750e2480..fed576b8f7fe 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -13771,13 +13771,14 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
goto out;
}
- /* ptr + __sz/__szk pair: size is in the next register */
+ /* ptr + __sz/__szk pair: the size follows the pointer */
if (arg + 1 < nargs &&
(btf_param_match_suffix(btf, &args[arg + 1], "__sz") ||
btf_param_match_suffix(btf, &args[arg + 1], "__szk"))) {
int size_reg = BPF_REG_1 + arg + 1;
- if (aux->const_reg_mask & BIT(size_reg)) {
+ if (size_reg <= MAX_BPF_FUNC_REG_ARGS &&
+ (aux->const_reg_mask & BIT(size_reg))) {
size = (s64)aux->const_reg_vals[size_reg];
goto out;
}
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 02/15] selftests/bpf: Add a test for an __int128 by-value argument
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 01/15] bpf: Read a kfunc's __sz argument only when it is in a register Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
2026-09-11 15:49 ` [PATCH bpf-next v3 03/15] bpf: Rename bpf_subprog_info::arg_cnt to arg_slot_cnt Yonghong Song
` (12 subsequent siblings)
14 siblings, 1 reply; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
A 128-bit integer is passed in two consecutive argument registers, but
the verifier counts one argument register per parameter whatever its
size. For
__u64 take_i128_global(int a, u128 v, int c)
the compiler passes a in R1, v in R2:R3 and c in R4, while the verifier
marks only R1 through R3 at the entry of the global function. The callee
then reads its own third parameter out of a register the verifier
considers uninitialized, and the program is rejected for a register the
source never names:
Validating take_i128_global() func#1...
20: R1=scalar() R2=scalar() R3=scalar() R10=fp0
; __noinline __u64 take_i128_global(int a, u128 v, int c) @ verifier_aggregate_arg.c:12
20: (bf) r0 = r2 ; R0=scalar(id=4) R2=scalar(id=4)
; return (__u64)a + (__u64)(v >> 64) + (__u64)v + c; @ verifier_aggregate_arg.c:14
21: (bc) w1 = w1 ; R1=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
22: (67) r1 <<= 32 ; R1=scalar(smax=0x7fffffff00000000,smin32=0,smax32=umax32=0,var_off=(0x0; 0xffffffff00000000))
23: (c7) r1 s>>= 32 ; R1=scalar(smin=0xffffffff80000000,smax=0x7fffffff)
24: (0f) r0 += r1 ; R0=scalar() R1=scalar(smin=0xffffffff80000000,smax=0x7fffffff)
25: (0f) r0 += r3 ; R0=scalar() R3=scalar()
26: (bc) w1 = w4
R4 !read_ok
The log is from clang 23. LLVM 21/22 place the argument in the same
registers.
Add the test with the failure it produces now. A later patch, "bpf:
Support __int128 as a by-value function argument", places the two slots
and flips this test to __success.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
.../selftests/bpf/prog_tests/verifier.c | 2 +
.../bpf/progs/verifier_aggregate_arg.c | 40 +++++++++++++++++++
2 files changed, 42 insertions(+)
create mode 100644 tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index f7f94ccebce2..3c7ded537314 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -5,6 +5,7 @@
#include "arena_kfunc.skel.h"
#include "arena_kfunc_jit.skel.h"
#include "cap_helpers.h"
+#include "verifier_aggregate_arg.skel.h"
#include "verifier_aggregate_ret.skel.h"
#include "verifier_align.skel.h"
#include "verifier_and.skel.h"
@@ -171,6 +172,7 @@ void test_arena_kfunc(void) { RUN_TESTS(arena_kfunc); }
void test_arena_kfunc_jit(void) { RUN_TESTS(arena_kfunc_jit); }
+void test_verifier_aggregate_arg(void) { RUN_TESTS(verifier_aggregate_arg); }
void test_verifier_aggregate_ret(void) { RUN_TESTS(verifier_aggregate_ret); }
void test_verifier_align(void) { RUN(verifier_align); }
void test_verifier_and(void) { RUN(verifier_and); }
diff --git a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
new file mode 100644
index 000000000000..d90f754396d0
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+#ifdef __SIZEOF_INT128__
+
+typedef unsigned __int128 u128;
+
+__noinline __u64 take_i128_global(int a, u128 v, int c)
+{
+ return (__u64)a + (__u64)(v >> 64) + (__u64)v + c;
+}
+
+SEC("tc")
+/*
+ * The verifier counts one argument register for the __int128 and marks only
+ * R1 through R3 at the entry of take_i128_global(), while the compiler passed
+ * a in R1, v in R2:R3 and c in R4.
+ */
+__failure __msg("R4 !read_ok")
+int aggregate_arg_int128_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v = ((u128)a << 64) | b;
+
+ if (take_i128_global(1, v, 2) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+#endif /* __SIZEOF_INT128__ */
+
+char _license[] SEC("license") = "GPL";
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 02/15] selftests/bpf: Add a test for an __int128 by-value argument
2026-09-11 15:49 ` [PATCH bpf-next v3 02/15] selftests/bpf: Add a test for an __int128 by-value argument Yonghong Song
@ 2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 17:07 ` Yonghong Song
0 siblings, 1 reply; 32+ messages in thread
From: bot+bpf-ci @ 2026-09-11 16:47 UTC (permalink / raw)
To: yonghong.song, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 4265 bytes --]
> diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
> index f7f94ccebce2..3c7ded537314 100644
> --- a/tools/testing/selftests/bpf/prog_tests/verifier.c
> +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
> @@ -5,6 +5,7 @@
> #include "arena_kfunc.skel.h"
> #include "arena_kfunc_jit.skel.h"
> #include "cap_helpers.h"
> +#include "verifier_aggregate_arg.skel.h"
> #include "verifier_aggregate_ret.skel.h"
> #include "verifier_align.skel.h"
> #include "verifier_and.skel.h"
> @@ -171,6 +172,7 @@ void test_arena_kfunc(void) { RUN_TESTS(arena_kfunc); }
>
> void test_arena_kfunc_jit(void) { RUN_TESTS(arena_kfunc_jit); }
>
> +void test_verifier_aggregate_arg(void) { RUN_TESTS(verifier_aggregate_arg); }
> void test_verifier_aggregate_ret(void) { RUN_TESTS(verifier_aggregate_ret); }
> void test_verifier_align(void) { RUN(verifier_align); }
> void test_verifier_and(void) { RUN(verifier_and); }
> diff --git a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
> new file mode 100644
> index 000000000000..d90f754396d0
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
> @@ -0,0 +1,40 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#include <vmlinux.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_misc.h"
> +
> +#define MIX_A 0xdeadbeefcafef00dULL
> +#define MIX_B 0x0123456789abcdefULL
> +
> +#ifdef __SIZEOF_INT128__
> +
> +typedef unsigned __int128 u128;
> +
> +__noinline __u64 take_i128_global(int a, u128 v, int c)
> +{
> + return (__u64)a + (__u64)(v >> 64) + (__u64)v + c;
> +}
> +
> +SEC("tc")
> +/*
> + * The verifier counts one argument register for the __int128 and marks only
> + * R1 through R3 at the entry of take_i128_global(), while the compiler passed
> + * a in R1, v in R2:R3 and c in R4.
> + */
> +__failure __msg("R4 !read_ok")
> +int aggregate_arg_int128_c_test(struct __sk_buff *skb)
> +{
> + __u64 a = skb->len ^ MIX_A;
> + __u64 b = skb->len ^ MIX_B;
> + u128 v = ((u128)a << 64) | b;
> +
> + if (take_i128_global(1, v, 2) != a + b + 3)
> + return 1;
> +
> + return 0;
> +}
> +
> +#endif /* __SIZEOF_INT128__ */
> +
> +char _license[] SEC("license") = "GPL";
A subsystem pattern flags this as potentially concerning: every program
in the new file is wrapped in '#ifdef __SIZEOF_INT128__' with no '#else'
branch.
When a BPF compiler does not define __SIZEOF_INT128__ (for example, when
building with bpf-gcc via BPF_GCC=1), the generated verifier_aggregate_arg.bpf.o
contains no programs at all. The skeleton still compiles because bpftool
guards maps/progs/links members on their counts, but test_loader__run_subtests()
then iterates zero programs and test_progs.c reports the test as passing with
no indication that the __int128 coverage was compiled out.
Every sibling prog file in progs/ that wraps all of its tests in a feature
#ifdef adds an '#else' dummy carrying __description()/__skip() so the result
is a visible SKIP instead. For instance, verifier_aggregate_ret.c (from the
same series) includes such a dummy at lines 167-175 with the message "needs
LLVM 23, dummy test". The same pattern appears in verifier_gotol.c,
verifier_bswap.c, verifier_ldsx.c, verifier_movsx.c, verifier_sdiv.c,
verifier_private_stack.c, verifier_stack_arg_order.c, verifier_load_acquire.c,
verifier_store_release.c, arena_kfunc.c, stack_arg_precision.c, and
stack_arg_fail.c.
Should this file include an '#else' dummy with __skip() to match the
pattern used by its sibling tests, so that builds without __SIZEOF_INT128__
report SKIP rather than silently passing?
Note: __SIZEOF_INT128__ is defined by every clang for the bpf target, so
the normal CI matrix (clang test_progs, no_alu32, cpuv4) does compile and
run the test. Only the optional bpf-gcc flavor would be affected.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 02/15] selftests/bpf: Add a test for an __int128 by-value argument
2026-09-11 16:47 ` bot+bpf-ci
@ 2026-09-12 17:07 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:07 UTC (permalink / raw)
To: bot+bpf-ci, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, martin.lau, mason,
ihor.solodrai
On 9/11/26 9:47 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
>> index f7f94ccebce2..3c7ded537314 100644
>> --- a/tools/testing/selftests/bpf/prog_tests/verifier.c
>> +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
>> @@ -5,6 +5,7 @@
>> #include "arena_kfunc.skel.h"
>> #include "arena_kfunc_jit.skel.h"
>> #include "cap_helpers.h"
>> +#include "verifier_aggregate_arg.skel.h"
>> #include "verifier_aggregate_ret.skel.h"
>> #include "verifier_align.skel.h"
>> #include "verifier_and.skel.h"
>> @@ -171,6 +172,7 @@ void test_arena_kfunc(void) { RUN_TESTS(arena_kfunc); }
>>
>> void test_arena_kfunc_jit(void) { RUN_TESTS(arena_kfunc_jit); }
>>
>> +void test_verifier_aggregate_arg(void) { RUN_TESTS(verifier_aggregate_arg); }
>> void test_verifier_aggregate_ret(void) { RUN_TESTS(verifier_aggregate_ret); }
>> void test_verifier_align(void) { RUN(verifier_align); }
>> void test_verifier_and(void) { RUN(verifier_and); }
>> diff --git a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
>> new file mode 100644
>> index 000000000000..d90f754396d0
>> --- /dev/null
>> +++ b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
>> @@ -0,0 +1,40 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
>> +#include <vmlinux.h>
>> +#include <bpf/bpf_helpers.h>
>> +#include "bpf_misc.h"
>> +
>> +#define MIX_A 0xdeadbeefcafef00dULL
>> +#define MIX_B 0x0123456789abcdefULL
>> +
>> +#ifdef __SIZEOF_INT128__
>> +
>> +typedef unsigned __int128 u128;
>> +
>> +__noinline __u64 take_i128_global(int a, u128 v, int c)
>> +{
>> + return (__u64)a + (__u64)(v >> 64) + (__u64)v + c;
>> +}
>> +
>> +SEC("tc")
>> +/*
>> + * The verifier counts one argument register for the __int128 and marks only
>> + * R1 through R3 at the entry of take_i128_global(), while the compiler passed
>> + * a in R1, v in R2:R3 and c in R4.
>> + */
>> +__failure __msg("R4 !read_ok")
>> +int aggregate_arg_int128_c_test(struct __sk_buff *skb)
>> +{
>> + __u64 a = skb->len ^ MIX_A;
>> + __u64 b = skb->len ^ MIX_B;
>> + u128 v = ((u128)a << 64) | b;
>> +
>> + if (take_i128_global(1, v, 2) != a + b + 3)
>> + return 1;
>> +
>> + return 0;
>> +}
>> +
>> +#endif /* __SIZEOF_INT128__ */
>> +
>> +char _license[] SEC("license") = "GPL";
> A subsystem pattern flags this as potentially concerning: every program
> in the new file is wrapped in '#ifdef __SIZEOF_INT128__' with no '#else'
> branch.
>
> When a BPF compiler does not define __SIZEOF_INT128__ (for example, when
> building with bpf-gcc via BPF_GCC=1), the generated verifier_aggregate_arg.bpf.o
> contains no programs at all. The skeleton still compiles because bpftool
> guards maps/progs/links members on their counts, but test_loader__run_subtests()
> then iterates zero programs and test_progs.c reports the test as passing with
> no indication that the __int128 coverage was compiled out.
>
> Every sibling prog file in progs/ that wraps all of its tests in a feature
> #ifdef adds an '#else' dummy carrying __description()/__skip() so the result
> is a visible SKIP instead. For instance, verifier_aggregate_ret.c (from the
> same series) includes such a dummy at lines 167-175 with the message "needs
> LLVM 23, dummy test". The same pattern appears in verifier_gotol.c,
> verifier_bswap.c, verifier_ldsx.c, verifier_movsx.c, verifier_sdiv.c,
> verifier_private_stack.c, verifier_stack_arg_order.c, verifier_load_acquire.c,
> verifier_store_release.c, arena_kfunc.c, stack_arg_precision.c, and
> stack_arg_fail.c.
>
> Should this file include an '#else' dummy with __skip() to match the
> pattern used by its sibling tests, so that builds without __SIZEOF_INT128__
> report SKIP rather than silently passing?
>
> Note: __SIZEOF_INT128__ is defined by every clang for the bpf target, so
> the normal CI matrix (clang test_progs, no_alu32, cpuv4) does compile and
> run the test. Only the optional bpf-gcc flavor would be affected.
This should be okay. Later selftest will have more contents in file verifier_aggregate_arg.c
which has some not-guarded bpf programs.
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH bpf-next v3 03/15] bpf: Rename bpf_subprog_info::arg_cnt to arg_slot_cnt
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 01/15] bpf: Read a kfunc's __sz argument only when it is in a register Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 02/15] selftests/bpf: Add a test for an __int128 by-value argument Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 04/15] bpf: Index global function arguments by argument slot Yonghong Song
` (11 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
Rename arg_cnt to arg_slot_cnt, as a later patch gives a parameter that
takes two argument registers, an __int128 or a 16-byte aggregate, two
slots. No functional change.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
include/linux/bpf_verifier.h | 6 +++---
kernel/bpf/btf.c | 2 +-
kernel/bpf/verifier.c | 19 +++++++++++--------
.../bpf/progs/verifier_stack_arg_order.c | 4 ++--
4 files changed, 17 insertions(+), 14 deletions(-)
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 9727df5af83a..edb904424aba 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -828,7 +828,7 @@ struct bpf_subprog_info {
bool keep_fastcall_stack: 1;
bool changes_pkt_data: 1;
bool might_sleep: 1;
- u8 arg_cnt:4;
+ u8 arg_slot_cnt:4;
enum priv_stack_mode priv_stack_mode;
struct bpf_subprog_arg_info args[MAX_BPF_FUNC_ARGS];
@@ -838,8 +838,8 @@ struct bpf_subprog_info {
static inline u16 bpf_in_stack_arg_cnt(const struct bpf_subprog_info *sub)
{
- if (sub->arg_cnt > MAX_BPF_FUNC_REG_ARGS)
- return sub->arg_cnt - MAX_BPF_FUNC_REG_ARGS;
+ if (sub->arg_slot_cnt > MAX_BPF_FUNC_REG_ARGS)
+ return sub->arg_slot_cnt - MAX_BPF_FUNC_REG_ARGS;
return 0;
}
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 31057c8f3a7c..01ec2b40f376 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -8081,7 +8081,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
}
args = (const struct btf_param *)(t + 1);
nargs = btf_type_vlen(t);
- sub->arg_cnt = nargs;
+ sub->arg_slot_cnt = nargs;
if (nargs > MAX_BPF_FUNC_ARGS) {
bpf_log(log, "kernel supports at most %d parameters, function %s has %d\n",
MAX_BPF_FUNC_ARGS, tname, nargs);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index fed576b8f7fe..e8f4c17fb27d 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -9772,7 +9772,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
ret = btf_prepare_func_args(env, subprog);
if (ret) {
if (bpf_in_stack_arg_cnt(sub) > 0) {
- err = check_outgoing_stack_args(env, caller, sub->arg_cnt,
+ err = check_outgoing_stack_args(env, caller, sub->arg_slot_cnt,
bpf_subprog_name(env, subprog),
NULL, NULL);
if (err)
@@ -9784,7 +9784,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
func = btf_type_by_id(btf, env->prog->aux->func_info[subprog].type_id);
func_proto = btf_type_by_id(btf, func->type);
args = btf_params(func_proto);
- ret = check_outgoing_stack_args(env, caller, sub->arg_cnt,
+ ret = check_outgoing_stack_args(env, caller, sub->arg_slot_cnt,
bpf_subprog_name(env, subprog), btf, args);
if (ret)
return ret;
@@ -9792,7 +9792,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
/* check that BTF function arguments match actual types that the
* verifier sees.
*/
- for (i = 0; i < sub->arg_cnt; i++) {
+ for (i = 0; i < sub->arg_slot_cnt; i++) {
argno_t argno = argno_from_arg(i + 1);
struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
struct bpf_subprog_arg_info *arg = &sub->args[i];
@@ -19777,13 +19777,14 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
}
/* Also ensure the callback only has a single scalar argument. */
- if (sub->arg_cnt != 1 || sub->args[0].arg_type != ARG_ANYTHING) {
+ if (sub->arg_slot_cnt != 1 || sub->args[0].arg_type != ARG_ANYTHING) {
verbose(env, "exception cb only supports single integer argument\n");
ret = -EINVAL;
goto out;
}
}
- for (i = BPF_REG_1; i <= min_t(u32, sub->arg_cnt, MAX_BPF_FUNC_REG_ARGS); i++) {
+ for (i = BPF_REG_1;
+ i <= min_t(u32, sub->arg_slot_cnt, MAX_BPF_FUNC_REG_ARGS); i++) {
arg = &sub->args[i - BPF_REG_1];
reg = ®s[i];
@@ -19826,7 +19827,8 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
goto out;
}
}
- if (env->prog->type == BPF_PROG_TYPE_EXT && sub->arg_cnt > MAX_BPF_FUNC_REG_ARGS) {
+ if (env->prog->type == BPF_PROG_TYPE_EXT &&
+ sub->arg_slot_cnt > MAX_BPF_FUNC_REG_ARGS) {
verbose(env, "freplace programs with >%d args not supported yet\n",
MAX_BPF_FUNC_REG_ARGS);
ret = -EINVAL;
@@ -19839,9 +19841,10 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
*/
if (env->prog->aux->func_info_aux) {
ret = btf_prepare_func_args(env, 0);
- if (ret || sub->arg_cnt != 1 || sub->args[0].arg_type != ARG_PTR_TO_CTX) {
+ if (ret || sub->arg_slot_cnt != 1 ||
+ sub->args[0].arg_type != ARG_PTR_TO_CTX) {
env->prog->aux->func_info_aux[0].unreliable = true;
- sub->arg_cnt = 1;
+ sub->arg_slot_cnt = 1;
sub->stack_arg_cnt = 0;
}
}
diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c
index 57f22691744a..ab1955852233 100644
--- a/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c
+++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c
@@ -116,8 +116,8 @@ __naked void stack_arg_pruning_load_after_call(void)
/*
* "bad_ptr": the first arg is 'long *', which is not a recognized pointer
* type for static subprogs (not ctx, dynptr, or tagged). btf_prepare_func_args()
- * sets arg_cnt = 7 / stack_arg_cnt = 2, then fails with -EINVAL. The subprog
- * is marked unreliable but the call still proceeds for static subprogs.
+ * sets arg_slot_cnt = 7 / stack_arg_cnt = 2, then fails with -EINVAL. The
+ * subprog is marked unreliable but the call still proceeds for static subprogs.
*/
__noinline __used __naked
static void subprog_bad_ptr_7args(long *a, int b, int c, int d, int e, int f, int g)
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 04/15] bpf: Index global function arguments by argument slot
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (2 preceding siblings ...)
2026-09-11 15:49 ` [PATCH bpf-next v3 03/15] bpf: Rename bpf_subprog_info::arg_cnt to arg_slot_cnt Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 05/15] bpf: Support by-value struct arguments up to 16 bytes Yonghong Song
` (10 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
btf_prepare_func_args() indexes sub->args[] by BTF parameter, so a
parameter can only ever stand for a single argument register. Give the
loop a second index: 'i' keeps walking the BTF parameters while
'slots_used' walks the argument slots, and sub->arg_slot_cnt becomes the
number of slots, so that a later patch can give a parameter two of them.
No functional change: every parameter still takes exactly one slot.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
kernel/bpf/btf.c | 40 ++++++++++++++++++++++++----------------
1 file changed, 24 insertions(+), 16 deletions(-)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 01ec2b40f376..35eaa28c85b5 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -8037,7 +8037,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
const struct btf_param *args;
const struct btf_type *t, *ref_t, *fn_t;
int err;
- u32 i, nargs, btf_id;
+ u32 i, slots_used, nargs, btf_id;
const char *tname;
if (sub->args_cached)
@@ -8122,8 +8122,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
/* Convert BTF function arguments into verifier types.
* Only PTR_TO_CTX and SCALAR are supported atm.
*/
- for (i = 0; i < nargs; i++) {
+ for (i = 0, slots_used = 0; i < nargs; i++) {
u32 tags = 0;
+
err = btf_scan_decl_tags(env, btf, fn_t, i, is_global, &tags);
if (err)
return err;
@@ -8145,7 +8146,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
btf_validate_prog_ctx_type(log, btf, t, i, prog_type,
prog->expected_attach_type))
return -EINVAL;
- sub->args[i].arg_type = ARG_PTR_TO_CTX;
+ sub->args[slots_used++].arg_type = ARG_PTR_TO_CTX;
continue;
}
if (btf_is_dynptr_ptr(btf, t)) {
@@ -8153,7 +8154,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
bpf_log(log, "arg#%d has invalid combination of tags\n", i);
return -EINVAL;
}
- sub->args[i].arg_type = ARG_PTR_TO_DYNPTR;
+ sub->args[slots_used++].arg_type = ARG_PTR_TO_DYNPTR;
continue;
}
if (tags & ARG_TAG_TRUSTED) {
@@ -8168,10 +8169,11 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
if (kern_type_id < 0)
return kern_type_id;
- sub->args[i].arg_type = ARG_PTR_TO_BTF_ID | PTR_TRUSTED;
+ sub->args[slots_used].arg_type = ARG_PTR_TO_BTF_ID | PTR_TRUSTED;
if (tags & ARG_TAG_NULLABLE)
- sub->args[i].arg_type |= PTR_MAYBE_NULL;
- sub->args[i].btf_id = kern_type_id;
+ sub->args[slots_used].arg_type |= PTR_MAYBE_NULL;
+ sub->args[slots_used].btf_id = kern_type_id;
+ slots_used++;
continue;
}
if (tags & ARG_TAG_UNTRUSTED) {
@@ -8185,8 +8187,10 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
ref_t = btf_type_skip_modifiers(btf, t->type, NULL);
if (btf_type_is_void(ref_t) || btf_type_is_primitive(ref_t)) {
- sub->args[i].arg_type = ARG_PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED;
- sub->args[i].mem_size = 0;
+ sub->args[slots_used].arg_type = ARG_PTR_TO_MEM | MEM_RDONLY |
+ PTR_UNTRUSTED;
+ sub->args[slots_used].mem_size = 0;
+ slots_used++;
continue;
}
@@ -8202,8 +8206,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
i, btf_type_str(ref_t), tname);
return -EINVAL;
}
- sub->args[i].arg_type = ARG_PTR_TO_BTF_ID | PTR_UNTRUSTED;
- sub->args[i].btf_id = kern_type_id;
+ sub->args[slots_used].arg_type = ARG_PTR_TO_BTF_ID | PTR_UNTRUSTED;
+ sub->args[slots_used].btf_id = kern_type_id;
+ slots_used++;
continue;
}
if (tags & ARG_TAG_ARENA) {
@@ -8211,7 +8216,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
bpf_log(log, "arg#%d arena cannot be combined with any other tags\n", i);
return -EINVAL;
}
- sub->args[i].arg_type = ARG_PTR_TO_ARENA;
+ sub->args[slots_used++].arg_type = ARG_PTR_TO_ARENA;
continue;
}
if (is_global) { /* generic user data pointer */
@@ -8231,10 +8236,11 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return -EINVAL;
}
- sub->args[i].arg_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL;
+ sub->args[slots_used].arg_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL;
if (tags & ARG_TAG_NONNULL)
- sub->args[i].arg_type &= ~PTR_MAYBE_NULL;
- sub->args[i].mem_size = mem_size;
+ sub->args[slots_used].arg_type &= ~PTR_MAYBE_NULL;
+ sub->args[slots_used].mem_size = mem_size;
+ slots_used++;
continue;
}
@@ -8244,7 +8250,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return -EINVAL;
}
if (btf_type_is_int(t) || btf_is_any_enum(t)) {
- sub->args[i].arg_type = ARG_ANYTHING;
+ sub->args[slots_used++].arg_type = ARG_ANYTHING;
continue;
}
if (!is_global)
@@ -8254,6 +8260,8 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return -EINVAL;
}
+ sub->arg_slot_cnt = slots_used;
+
sub->args_cached = true;
return 0;
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 05/15] bpf: Support by-value struct arguments up to 16 bytes
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (3 preceding siblings ...)
2026-09-11 15:49 ` [PATCH bpf-next v3 04/15] bpf: Index global function arguments by argument slot Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 06/15] bpf: Support __int128 as a by-value function argument Yonghong Song
` (9 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
A global function taking a struct or union by value is rejected today:
Arg#1 type STRUCT in tar() is not supported yet.
Accept one of at most 16 bytes, which arrives in one or two consecutive
argument registers. Only structs composed entirely of scalars are taken
for now; btf_struct_is_composed_of() enforces that.
The slots of a value are independent of each other, so the compiler may
split one across the last argument register and the stack, as in
static void f(int a, int b, int c, int d, struct pair p);
or place it wholly past the registers, and the verifier describes either
the same way. What has to follow the slots is stack_arg_cnt, recomputed
from the slots consumed, together with the "no stack args in global
functions" and JIT support checks, and the MAX_BPF_FUNC_ARGS bound, which
now has to account for a parameter that takes two slots at once.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
kernel/bpf/btf.c | 80 +++++++++++++++++++++++++++++++++++--------
kernel/bpf/verifier.c | 2 ++
2 files changed, 68 insertions(+), 14 deletions(-)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 35eaa28c85b5..5bbc1ba00ae2 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -8018,6 +8018,29 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
return -EOPNOTSUPP;
}
+static int btf_check_arg_slots(struct bpf_verifier_log *log, const char *tname,
+ bool is_global, u32 slot_cnt,
+ struct bpf_subprog_info *sub)
+{
+ if (slot_cnt <= MAX_BPF_FUNC_REG_ARGS)
+ return 0;
+
+ if (is_global) {
+ bpf_log(log,
+ "global function %s() needs %d > %d argument slots, "
+ "stack args not supported\n",
+ tname, slot_cnt, MAX_BPF_FUNC_REG_ARGS);
+ return -EINVAL;
+ }
+ if (!bpf_jit_supports_stack_args()) {
+ bpf_log(log, "JIT does not support function %s() with %d argument slots\n",
+ tname, slot_cnt);
+ return -EFAULT;
+ }
+ sub->stack_arg_cnt = slot_cnt - MAX_BPF_FUNC_REG_ARGS;
+ return 0;
+}
+
/* Process BTF of a function to produce high-level expectation of function
* arguments (like ARG_PTR_TO_CTX, or ARG_PTR_TO_MEM, etc). This information
* is cached in subprog info for reuse.
@@ -8087,20 +8110,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
MAX_BPF_FUNC_ARGS, tname, nargs);
return -EFAULT;
}
- if (nargs > MAX_BPF_FUNC_REG_ARGS) {
- if (!bpf_jit_supports_stack_args()) {
- bpf_log(log, "JIT does not support function %s() with %d args\n",
- tname, nargs);
- return -EFAULT;
- }
- sub->stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS;
- }
-
- if (is_global && nargs > MAX_BPF_FUNC_REG_ARGS) {
- bpf_log(log, "global function %s has %d > %d args, stack args not supported\n",
- tname, nargs, MAX_BPF_FUNC_REG_ARGS);
- return -EINVAL;
- }
+ err = btf_check_arg_slots(log, tname, is_global, nargs, sub);
+ if (err)
+ return err;
err = btf_validate_return_type(env, btf, t, subprog, is_global);
if (err) {
@@ -8125,6 +8137,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
for (i = 0, slots_used = 0; i < nargs; i++) {
u32 tags = 0;
+ if (slots_used >= MAX_BPF_FUNC_ARGS)
+ goto too_many_slots;
+
err = btf_scan_decl_tags(env, btf, fn_t, i, is_global, &tags);
if (err)
return err;
@@ -8253,6 +8268,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
sub->args[slots_used++].arg_type = ARG_ANYTHING;
continue;
}
+ if (btf_type_is_struct(t)) {
+ u32 nslots;
+
+ if (!t->size || t->size > 2 * BPF_REG_SIZE) {
+ if (!is_global)
+ return -EINVAL;
+ bpf_log(log,
+ "Arg#%d type %s in %s() has size %u, only 1 to %d bytes "
+ "can be passed by value\n",
+ i, btf_type_str(t), tname, t->size, 2 * BPF_REG_SIZE);
+ return -EINVAL;
+ }
+ if (!btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
+ if (!is_global)
+ return -EINVAL;
+ bpf_log(log, "Arg#%d type %s in %s() is not composed of scalars\n",
+ i, btf_type_str(t), tname);
+ return -EINVAL;
+ }
+
+ nslots = (t->size + BPF_REG_SIZE - 1) / BPF_REG_SIZE;
+ if (slots_used + nslots > MAX_BPF_FUNC_ARGS)
+ goto too_many_slots;
+ while (nslots--)
+ sub->args[slots_used++].arg_type = ARG_ANYTHING;
+ continue;
+ }
if (!is_global)
return -EINVAL;
bpf_log(log, "Arg#%d type %s in %s() is not supported yet.\n",
@@ -8260,11 +8302,21 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return -EINVAL;
}
+ err = btf_check_arg_slots(log, tname, is_global, slots_used, sub);
+ if (err)
+ return err;
sub->arg_slot_cnt = slots_used;
sub->args_cached = true;
return 0;
+
+too_many_slots:
+ if (!is_global)
+ return -EINVAL;
+ bpf_log(log, "Arguments of %s() need more than %d argument slots\n",
+ tname, MAX_BPF_FUNC_ARGS);
+ return -EINVAL;
}
static void btf_type_show(const struct btf *btf, u32 type_id, void *obj,
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index e8f4c17fb27d..500092a2c924 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -9784,6 +9784,8 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
func = btf_type_by_id(btf, env->prog->aux->func_info[subprog].type_id);
func_proto = btf_type_by_id(btf, func->type);
args = btf_params(func_proto);
+ if (sub->arg_slot_cnt != btf_type_vlen(func_proto))
+ args = NULL;
ret = check_outgoing_stack_args(env, caller, sub->arg_slot_cnt,
bpf_subprog_name(env, subprog), btf, args);
if (ret)
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 06/15] bpf: Support __int128 as a by-value function argument
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (4 preceding siblings ...)
2026-09-11 15:49 ` [PATCH bpf-next v3 05/15] bpf: Support by-value struct arguments up to 16 bytes Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 07/15] bpf: Rename bpf_call_summary::num_params to arg_slot_cnt Yonghong Song
` (8 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
A 128-bit integer follows the same calling convention as a 16-byte
by-value struct: LLVM emits it as a 16-byte BTF_KIND_INT and passes it in
two consecutive argument registers. So a __int128 gets its two slots.
The test added at the start of the series, which recorded the wrong
register map as an R4 !read_ok rejection, now passes and is flipped to
__success.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
kernel/bpf/btf.c | 9 +++------
.../testing/selftests/bpf/progs/verifier_aggregate_arg.c | 7 +------
2 files changed, 4 insertions(+), 12 deletions(-)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 5bbc1ba00ae2..088788e29b03 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -8264,11 +8264,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
bpf_log(log, "arg#%d has pointer tag, but is not a pointer type\n", i);
return -EINVAL;
}
- if (btf_type_is_int(t) || btf_is_any_enum(t)) {
- sub->args[slots_used++].arg_type = ARG_ANYTHING;
- continue;
- }
- if (btf_type_is_struct(t)) {
+ if (btf_type_is_int(t) || btf_is_any_enum(t) || btf_type_is_struct(t)) {
u32 nslots;
if (!t->size || t->size > 2 * BPF_REG_SIZE) {
@@ -8280,7 +8276,8 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
i, btf_type_str(t), tname, t->size, 2 * BPF_REG_SIZE);
return -EINVAL;
}
- if (!btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
+ if (btf_type_is_struct(t) &&
+ !btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
if (!is_global)
return -EINVAL;
bpf_log(log, "Arg#%d type %s in %s() is not composed of scalars\n",
diff --git a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
index d90f754396d0..fc7c1b18bc40 100644
--- a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
+++ b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
@@ -17,12 +17,7 @@ __noinline __u64 take_i128_global(int a, u128 v, int c)
}
SEC("tc")
-/*
- * The verifier counts one argument register for the __int128 and marks only
- * R1 through R3 at the entry of take_i128_global(), while the compiler passed
- * a in R1, v in R2:R3 and c in R4.
- */
-__failure __msg("R4 !read_ok")
+__success __retval(0)
int aggregate_arg_int128_c_test(struct __sk_buff *skb)
{
__u64 a = skb->len ^ MIX_A;
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 07/15] bpf: Rename bpf_call_summary::num_params to arg_slot_cnt
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (5 preceding siblings ...)
2026-09-11 15:49 ` [PATCH bpf-next v3 06/15] bpf: Support __int128 as a by-value function argument Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 15:49 ` [PATCH bpf-next v3 08/15] bpf: Recognize by-value struct and __int128 kfunc arguments Yonghong Song
` (7 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
The field 'num_params' counts the argument registers and outgoing
stack slots a helper or kfunc call takes. The next patch gives a
16-byte parameter two slots, so the name stops describing what the
field holds.
Rename 'num_params' to 'arg_slot_cnt'. No functional change.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
include/linux/bpf_verifier.h | 2 +-
kernel/bpf/liveness.c | 10 +++++-----
kernel/bpf/verifier.c | 8 ++++----
3 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index edb904424aba..64dfa29f6b45 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -1066,7 +1066,7 @@ static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
}
struct bpf_call_summary {
- u8 num_params;
+ u8 arg_slot_cnt;
bool is_void;
bool fastcall;
};
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 7165ea325961..44ecdc5b4ec2 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -1434,21 +1434,21 @@ static int record_call_access(struct bpf_verifier_env *env,
{
struct bpf_insn *insn = &env->prog->insnsi[insn_idx];
struct bpf_call_summary cs;
- int r, err, num_params = 5;
+ int r, err, arg_slot_cnt = 5;
if (bpf_pseudo_call(insn))
return 0;
if (bpf_get_call_summary(env, insn, &cs))
- num_params = cs.num_params;
+ arg_slot_cnt = cs.arg_slot_cnt;
- for (r = BPF_REG_1; r < BPF_REG_1 + min(num_params, MAX_BPF_FUNC_REG_ARGS); r++) {
+ for (r = BPF_REG_1; r < BPF_REG_1 + min(arg_slot_cnt, MAX_BPF_FUNC_REG_ARGS); r++) {
err = record_arg_access(env, instance, insn, &at[r], r - 1, insn_idx);
if (err)
return err;
}
- for (r = 0; r < MAX_STACK_ARG_SLOTS && r < num_params - MAX_BPF_FUNC_REG_ARGS; r++) {
+ for (r = 0; r < MAX_STACK_ARG_SLOTS && r < arg_slot_cnt - MAX_BPF_FUNC_REG_ARGS; r++) {
err = record_arg_access(env, instance, insn, &at[MAX_BPF_REG + r],
r + MAX_BPF_FUNC_REG_ARGS, insn_idx);
if (err)
@@ -2199,7 +2199,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
def = ALL_CALLER_SAVED_REGS;
use = def & ~BIT(BPF_REG_0);
if (bpf_get_call_summary(env, insn, &cs))
- use = GENMASK(min_t(u8, cs.num_params, MAX_BPF_FUNC_REG_ARGS), 1);
+ use = GENMASK(min_t(u8, cs.arg_slot_cnt, MAX_BPF_FUNC_REG_ARGS), 1);
def = mask_widen(def);
use = mask_widen(use);
break;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 500092a2c924..c520c37bb3c9 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -17973,11 +17973,11 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
(bpf_verifier_inlines_helper_call(env, call->imm) ||
bpf_jit_inlines_helper_call(call->imm));
cs->is_void = fn->ret_type == RET_VOID;
- cs->num_params = 0;
+ cs->arg_slot_cnt = 0;
for (i = 0; i < ARRAY_SIZE(fn->arg_type); ++i) {
if (fn->arg_type[i] == ARG_DONTCARE)
break;
- cs->num_params++;
+ cs->arg_slot_cnt++;
}
return true;
}
@@ -17989,7 +17989,7 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
if (err < 0)
/* error would be reported later */
return false;
- cs->num_params = btf_type_vlen(meta.func_proto);
+ cs->arg_slot_cnt = btf_type_vlen(meta.func_proto);
cs->fastcall = meta.kfunc_flags & KF_FASTCALL;
cs->is_void = btf_type_is_void(btf_type_by_id(meta.btf, meta.func_proto->type));
return true;
@@ -18098,7 +18098,7 @@ static void mark_fastcall_pattern_for_call(struct bpf_verifier_env *env,
* - includes R1-R5 if corresponding parameter has is described
* in the function prototype.
*/
- clobbered_regs_mask = GENMASK(cs.num_params, cs.is_void ? 1 : 0);
+ clobbered_regs_mask = GENMASK(cs.arg_slot_cnt, cs.is_void ? 1 : 0);
/* e.g. if helper call clobbers r{0,1}, expect r{2,3,4,5} in the pattern */
expected_regs_mask = ~clobbered_regs_mask & ALL_CALLER_SAVED_REGS;
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 08/15] bpf: Recognize by-value struct and __int128 kfunc arguments
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (6 preceding siblings ...)
2026-09-11 15:49 ` [PATCH bpf-next v3 07/15] bpf: Rename bpf_call_summary::num_params to arg_slot_cnt Yonghong Song
@ 2026-09-11 15:49 ` Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
2026-09-11 15:50 ` [PATCH bpf-next v3 09/15] bpf: Prepare kfunc arguments for the JIT from an ABI description Yonghong Song
` (6 subsequent siblings)
14 siblings, 1 reply; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:49 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
A kfunc taking a struct or union by value is rejected today, and one
taking an __int128 is accepted but mis-counted:
Unrecognized R2 type STRUCT
The kfunc arguments walk the same slot as a BPF-to-BPF call:
one argument register per eightbyte, and a 16-byte value takes two.
The outgoing stack argument count at the call site follows the slots for
the same reason. Similar to BPF-to-BPF aggregate handling, a kfunc
aggregate argument is only supported when it is composed of scalars.
Everything that maps a kfunc argument to a register has to follow the
slots too.
An argument of a single eightbyte lands in the same place under every
calling convention, so those are taken. The conventions the JIT has to
reconcile do not agree on where a larger one goes, so refuse it for now
with
Function f arg#1 type INT cannot be passed at argument slot 1 on this
architecture
which the next patch turns into an answer from the JIT. The paths that
handle a two-slot argument are therefore unreachable until then.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
kernel/bpf/verifier.c | 177 +++++++++++++++++++++++++++++++++---------
1 file changed, 141 insertions(+), 36 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c520c37bb3c9..9e70995c763f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -12149,12 +12149,36 @@ bool bpf_is_kfunc_pkt_changing(struct bpf_call_arg_meta *meta)
return meta->func_id == special_kfunc_list[KF_bpf_xdp_pull_data];
}
+/*
+ * The slots a parameter takes, from its BTF type. This has to agree with
+ * btf_func_model_arg_slots(), which answers the same from the size the
+ * func model recorded, or the verifier would check an argument at a slot
+ * the JIT does not place it at.
+ */
+static u32 kfunc_arg_slots(const struct btf_type *t)
+{
+ if (btf_type_is_int(t) || btf_type_is_struct(t))
+ return (t->size + BPF_REG_SIZE - 1) / BPF_REG_SIZE;
+ return 1;
+}
+
+static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_proto)
+{
+ const struct btf_param *args = btf_params(func_proto);
+ u32 i, nargs = btf_type_vlen(func_proto), slots_used = 0;
+
+ for (i = 0; i < nargs; i++)
+ slots_used += kfunc_arg_slots(btf_type_skip_modifiers(btf, args[i].type, NULL));
+
+ return slots_used;
+}
+
static int
get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
- const struct btf_param *args, int arg, int nargs)
+ const struct btf_param *args, int arg, int nargs, u32 slot)
{
const struct btf_type *t, *ref_t = NULL;
- argno_t argno = argno_from_arg(arg + 1);
+ argno_t argno = argno_from_arg(slot + 1);
const char *ref_tname = NULL;
int arg_type;
@@ -12174,6 +12198,23 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
return KF_ARG_ANYTHING;
}
+ if (btf_type_is_struct(t)) {
+ if (!t->size || t->size > 2 * BPF_REG_SIZE) {
+ verbose(env,
+ "%s type %s has size %u, only 1 to %d bytes "
+ "can be passed by value\n",
+ reg_arg_name(env, argno), btf_type_str(t), t->size,
+ 2 * BPF_REG_SIZE);
+ return -EINVAL;
+ }
+ if (!btf_type_is_scalar_struct(env, meta->btf, t)) {
+ verbose(env, "%s type %s is not composed of scalars\n",
+ reg_arg_name(env, argno), btf_type_str(t));
+ return -EINVAL;
+ }
+ return KF_ARG_ANYTHING;
+ }
+
if (!btf_type_is_ptr(t)) {
verbose(env, "Unrecognized %s type %s\n",
reg_arg_name(env, argno), btf_type_str(t));
@@ -12289,7 +12330,7 @@ static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg
{
const struct btf *btf = meta->btf;
const struct btf_param *args;
- u32 i, nargs;
+ u32 i, nargs, slots_used;
int arg_type;
args = (const struct btf_param *)(meta->func_proto + 1);
@@ -12305,19 +12346,44 @@ static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg
return -ENOTSUPP;
}
- for (i = 0; i < nargs; i++) {
+ for (i = 0, slots_used = 0; i < nargs; i++) {
+ const struct btf_type *t;
+ u32 nslots;
+
+ t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+ nslots = kfunc_arg_slots(t);
+ /*
+ * The calling conventions the JIT has to reconcile do not
+ * agree on where an argument of more than one eightbyte goes,
+ * so refuse one until the JIT can say where this arch puts it.
+ */
+ if (nslots > 1) {
+ verbose(env,
+ "Function %s arg#%d type %s cannot be passed at "
+ "argument slot %d on this architecture\n",
+ meta->func_name, i, btf_type_str(t), slots_used);
+ return -EINVAL;
+ }
+ slots_used += nslots;
+
if (is_kfunc_arg_prog_aux(btf, &args[i]) ||
is_kfunc_arg_ignore(btf, &args[i]) ||
is_kfunc_arg_implicit(meta, i))
continue;
- arg_type = get_kfunc_arg_type(env, meta, args, i, nargs);
+ arg_type = get_kfunc_arg_type(env, meta, args, i, nargs, slots_used - nslots);
if (arg_type < 0)
return arg_type;
proto->arg_type[i] = arg_type;
}
+ if (slots_used > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
+ verbose(env, "JIT does not support kfunc %s() with %d argument slots\n",
+ meta->func_name, slots_used);
+ return -ENOTSUPP;
+ }
+
return 0;
}
@@ -12883,6 +12949,26 @@ static bool check_css_task_iter_allowlist(struct bpf_verifier_env *env)
}
}
+/* The argument slot @slot of the call has to hold a scalar. */
+static int check_kfunc_scalar_arg(struct bpf_verifier_env *env, struct bpf_func_state *caller,
+ struct bpf_reg_state *regs, u32 slot, int insn_idx,
+ const char *func_name)
+{
+ struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, slot);
+ argno_t argno = argno_from_arg(slot + 1);
+
+ if (reg->type == SCALAR_VALUE)
+ return 0;
+
+ verbose(env, "%s is not a scalar\n", reg_arg_name(env, argno));
+ bpf_diag_call_arg_fmt(env, insn_idx, argno, func_name,
+ "Pass an integer scalar value for this argument, not a pointer or resource object.",
+ "the kfunc expects an integer scalar, but %s is %s",
+ reg_arg_name(env, argno),
+ bpf_diag_reg_type_plain(env, reg->type));
+ return -EINVAL;
+}
+
static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
int insn_idx)
{
@@ -12892,29 +12978,35 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
const struct btf *btf = meta->btf;
const struct btf_param *args;
struct btf_record *rec;
- u32 i, nargs;
+ u32 i, k, nargs, proto_slots, slots_used, prev_slot = 0, nslots = 0;
int ret;
args = (const struct btf_param *)(meta->func_proto + 1);
nargs = btf_type_vlen(meta->func_proto);
+ proto_slots = kfunc_proto_slots(btf, meta->func_proto);
- ret = check_outgoing_stack_args(env, caller, nargs, func_name, btf, args);
+ ret = check_outgoing_stack_args(env, caller, proto_slots, func_name, btf,
+ proto_slots == nargs ? args : NULL);
if (ret)
return ret;
/* Check that BTF function arguments match actual types that the
* verifier sees.
*/
- for (i = 0; i < nargs; i++) {
- struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
+ for (i = 0, slots_used = 0; i < nargs;
+ i++, prev_slot = slots_used, slots_used += nslots) {
+ struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, slots_used);
const struct btf_type *t, *ref_t, *resolve_ret;
enum bpf_arg_type arg_type = ARG_DONTCARE;
- argno_t argno = argno_from_arg(i + 1);
+ argno_t argno = argno_from_arg(slots_used + 1);
int regno = reg_from_argno(argno);
bool btf_id_fixed_off_ok = true;
u32 ref_id = args[i].type, type_size;
int kf_arg_type = meta->fn->arg_type[i];
+ t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+ nslots = kfunc_arg_slots(t);
+
if (is_kfunc_arg_prog_aux(btf, &args[i])) {
/* Reject repeated use bpf_prog_aux */
if (meta->arg_prog) {
@@ -12934,8 +13026,6 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
if (is_kfunc_arg_ignore(btf, &args[i]) || is_kfunc_arg_implicit(meta, i))
continue;
- t = btf_type_skip_modifiers(btf, args[i].type, NULL);
-
if (btf_type_is_ptr(t)) {
ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id);
ref_tname = btf_name_by_offset(btf, ref_t->name_off);
@@ -12986,6 +13076,17 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
ref_tname = btf_name_by_offset(btf, ref_t->name_off);
}
+ /*
+ * The first slot is checked in below KF_ARG_ANYTHING.
+ * The rest of it has to be a scalar.
+ */
+ for (k = 1; k < nslots; k++) {
+ ret = check_kfunc_scalar_arg(env, caller, regs, slots_used + k,
+ insn_idx, func_name);
+ if (ret)
+ return ret;
+ }
+
switch (base_type(kf_arg_type)) {
case KF_ARG_CONST:
case KF_ARG_CONST_MEM_SIZE:
@@ -13055,15 +13156,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
}
break;
case KF_ARG_ANYTHING:
- if (reg->type != SCALAR_VALUE) {
- verbose(env, "%s is not a scalar\n", reg_arg_name(env, argno));
- bpf_diag_call_arg_fmt(env, insn_idx, argno, func_name,
- "Pass an integer scalar value for this argument, not a pointer or resource object.",
- "the kfunc expects an integer scalar, but %s is %s",
- reg_arg_name(env, argno),
- bpf_diag_reg_type_plain(env, reg->type));
- return -EINVAL;
- }
+ ret = check_kfunc_scalar_arg(env, caller, regs, slots_used,
+ insn_idx, func_name);
+ if (ret)
+ return ret;
break;
case KF_ARG_CONST_ALLOC_SIZE_OR_ZERO:
if (reg->type != SCALAR_VALUE) {
@@ -13403,9 +13499,9 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
fallthrough;
case KF_ARG_MEM_SIZE:
{
- struct bpf_reg_state *buff_reg = get_func_arg_reg(caller, regs, i - 1);
+ struct bpf_reg_state *buff_reg = get_func_arg_reg(caller, regs, prev_slot);
struct bpf_reg_state *size_reg = reg;
- argno_t buff_argno = argno_from_arg(i);
+ argno_t buff_argno = argno_from_arg(prev_slot + 1);
enum bpf_mem_size_failure failure;
if (reg->type != SCALAR_VALUE) {
@@ -13751,7 +13847,7 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
const struct btf_param *args;
const struct btf_type *t, *ref_t;
const struct btf *btf;
- u32 nargs, type_size;
+ u32 i, slot, nargs, type_size;
s64 size;
if (bpf_fetch_kfunc_arg_meta(env, insn->imm, insn->off, &meta) < 0)
@@ -13760,23 +13856,32 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
btf = meta.btf;
args = btf_params(meta.func_proto);
nargs = btf_type_vlen(meta.func_proto);
- if (arg >= nargs)
+
+ /*
+ * @arg is an argument slot and a 16-byte parameter takes two of them,
+ * so walk the parameters to find the one that starts at this slot. A
+ * slot holding the upper eightbyte of such a parameter belongs to no
+ * pointer, and neither does a slot past the last parameter.
+ */
+ for (i = 0, slot = 0; i < nargs && slot < arg; i++)
+ slot += kfunc_arg_slots(btf_type_skip_modifiers(btf, args[i].type, NULL));
+ if (i >= nargs || slot != arg)
return 0;
- t = btf_type_skip_modifiers(btf, args[arg].type, NULL);
+ t = btf_type_skip_modifiers(btf, args[i].type, NULL);
if (!btf_type_is_ptr(t))
return 0;
/* dynptr: fixed 16-byte on-stack representation */
- if (is_kfunc_arg_dynptr(btf, &args[arg])) {
+ if (is_kfunc_arg_dynptr(btf, &args[i])) {
size = BPF_DYNPTR_SIZE;
goto out;
}
/* ptr + __sz/__szk pair: the size follows the pointer */
- if (arg + 1 < nargs &&
- (btf_param_match_suffix(btf, &args[arg + 1], "__sz") ||
- btf_param_match_suffix(btf, &args[arg + 1], "__szk"))) {
+ if (i + 1 < nargs &&
+ (btf_param_match_suffix(btf, &args[i + 1], "__sz") ||
+ btf_param_match_suffix(btf, &args[i + 1], "__szk"))) {
int size_reg = BPF_REG_1 + arg + 1;
if (size_reg <= MAX_BPF_FUNC_REG_ARGS &&
@@ -13799,7 +13904,7 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
/* KF_ITER_NEW kfuncs initialize the iterator state at arg 0 */
if (arg == 0 && meta.kfunc_flags & KF_ITER_NEW)
return -size;
- if (is_kfunc_arg_uninit(btf, &args[arg]))
+ if (is_kfunc_arg_uninit(btf, &args[i]))
return -size;
return size;
}
@@ -13992,7 +14097,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
struct bpf_insn_aux_data *insn_aux;
const char *operation;
int err, insn_idx = *insn_idx_p;
- u32 i, nargs, ptr_type_id, ret_nregs = 1;
+ u32 i, proto_slots, ptr_type_id, ret_nregs = 1;
struct bpf_kfunc_desc *desc;
struct btf *desc_btf;
int id;
@@ -14418,11 +14523,11 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
if (bpf_is_kfunc_pkt_changing(&meta))
clear_all_pkt_pointers(env);
- nargs = btf_type_vlen(meta.func_proto);
- if (nargs > MAX_BPF_FUNC_REG_ARGS) {
+ proto_slots = kfunc_proto_slots(desc_btf, meta.func_proto);
+ if (proto_slots > MAX_BPF_FUNC_REG_ARGS) {
struct bpf_func_state *caller = cur_func(env);
struct bpf_subprog_info *caller_info = &env->subprog_info[caller->subprogno];
- u16 out_stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS;
+ u16 out_stack_arg_cnt = proto_slots - MAX_BPF_FUNC_REG_ARGS;
u16 stack_arg_cnt = bpf_in_stack_arg_cnt(caller_info) + out_stack_arg_cnt;
if (stack_arg_cnt > caller_info->stack_arg_cnt)
@@ -17989,7 +18094,7 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
if (err < 0)
/* error would be reported later */
return false;
- cs->arg_slot_cnt = btf_type_vlen(meta.func_proto);
+ cs->arg_slot_cnt = kfunc_proto_slots(meta.btf, meta.func_proto);
cs->fastcall = meta.kfunc_flags & KF_FASTCALL;
cs->is_void = btf_type_is_void(btf_type_by_id(meta.btf, meta.func_proto->type));
return true;
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 08/15] bpf: Recognize by-value struct and __int128 kfunc arguments
2026-09-11 15:49 ` [PATCH bpf-next v3 08/15] bpf: Recognize by-value struct and __int128 kfunc arguments Yonghong Song
@ 2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 17:13 ` Yonghong Song
0 siblings, 1 reply; 32+ messages in thread
From: bot+bpf-ci @ 2026-09-11 16:47 UTC (permalink / raw)
To: yonghong.song, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 4239 bytes --]
> bpf: Recognize by-value struct and __int128 kfunc arguments
>
> A kfunc taking a struct or union by value is rejected today, and one
> taking an __int128 is accepted but mis-counted:
>
> Unrecognized R2 type STRUCT
>
> The kfunc arguments walk the same slot as a BPF-to-BPF call:
> one argument register per eightbyte, and a 16-byte value takes two.
> The outgoing stack argument count at the call site follows the slots for
> the same reason. Similar to BPF-to-BPF aggregate handling, a kfunc
> aggregate argument is only supported when it is composed of scalars.
> Everything that maps a kfunc argument to a register has to follow the
> slots too.
>
> An argument of a single eightbyte lands in the same place under every
> calling convention, so those are taken. The conventions the JIT has to
> reconcile do not agree on where a larger one goes, so refuse it for now
> with
>
> Function f arg#1 type INT cannot be passed at argument slot 1 on this
> architecture
>
> which the next patch turns into an answer from the JIT. The paths that
> handle a two-slot argument are therefore unreachable until then.
>
> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
This looks like a bug fix for kfunc struct argument support. Should this
include:
Fixes: 0ab8eb34abe4 ("bpf: Support by-value struct arguments up to 16 bytes")
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index c520c37bb3c9..9e70995c763f 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
[ ... ]
> @@ -12892,29 +12978,35 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
> const struct btf *btf = meta->btf;
> const struct btf_param *args;
> struct btf_record *rec;
> - u32 i, nargs;
> + u32 i, k, nargs, proto_slots, slots_used, prev_slot = 0, nslots = 0;
> int ret;
>
> args = (const struct btf_param *)(meta->func_proto + 1);
> nargs = btf_type_vlen(meta->func_proto);
> + proto_slots = kfunc_proto_slots(btf, meta->func_proto);
>
> - ret = check_outgoing_stack_args(env, caller, nargs, func_name, btf, args);
> + ret = check_outgoing_stack_args(env, caller, proto_slots, func_name, btf,
> + proto_slots == nargs ? args : NULL);
> if (ret)
> return ret;
>
> /* Check that BTF function arguments match actual types that the
> * verifier sees.
> */
> - for (i = 0; i < nargs; i++) {
> - struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
> + for (i = 0, slots_used = 0; i < nargs;
> + i++, prev_slot = slots_used, slots_used += nslots) {
> + struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, slots_used);
> const struct btf_type *t, *ref_t, *resolve_ret;
> enum bpf_arg_type arg_type = ARG_DONTCARE;
> - argno_t argno = argno_from_arg(i + 1);
> + argno_t argno = argno_from_arg(slots_used + 1);
> int regno = reg_from_argno(argno);
> bool btf_id_fixed_off_ok = true;
> u32 ref_id = args[i].type, type_size;
> int kf_arg_type = meta->fn->arg_type[i];
>
> + t = btf_type_skip_modifiers(btf, args[i].type, NULL);
> + nslots = kfunc_arg_slots(t);
> +
> if (is_kfunc_arg_prog_aux(btf, &args[i])) {
> /* Reject repeated use bpf_prog_aux */
> if (meta->arg_prog) {
[ ... ]
> @@ -12986,6 +13076,17 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
> ref_tname = btf_name_by_offset(btf, ref_t->name_off);
> }
>
> + /*
> + * The first slot is checked in below KF_ARG_ANYTHING.
> + * The rest of it has to be a scalar.
> + */
> + for (k = 1; k < nslots; k++) {
> + ret = check_kfunc_scalar_arg(env, caller, regs, slots_used + k,
> + insn_idx, func_name);
> + if (ret)
> + return ret;
> + }
> +
This isn't a bug, but would it be worth rewording this comment to name
the KF_ARG_ANYTHING case explicitly and speak of the remaining slots in
the plural? The phrase "checked in below KF_ARG_ANYTHING" is a bit hard
to parse, and "the rest of it" refers to slots while the sentence subject
is singular.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 08/15] bpf: Recognize by-value struct and __int128 kfunc arguments
2026-09-11 16:47 ` bot+bpf-ci
@ 2026-09-12 17:13 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:13 UTC (permalink / raw)
To: bot+bpf-ci, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, martin.lau, mason,
ihor.solodrai
On 9/11/26 9:47 AM, bot+bpf-ci@kernel.org wrote:
>> bpf: Recognize by-value struct and __int128 kfunc arguments
>>
>> A kfunc taking a struct or union by value is rejected today, and one
>> taking an __int128 is accepted but mis-counted:
>>
>> Unrecognized R2 type STRUCT
>>
>> The kfunc arguments walk the same slot as a BPF-to-BPF call:
>> one argument register per eightbyte, and a 16-byte value takes two.
>> The outgoing stack argument count at the call site follows the slots for
>> the same reason. Similar to BPF-to-BPF aggregate handling, a kfunc
>> aggregate argument is only supported when it is composed of scalars.
>> Everything that maps a kfunc argument to a register has to follow the
>> slots too.
>>
>> An argument of a single eightbyte lands in the same place under every
>> calling convention, so those are taken. The conventions the JIT has to
>> reconcile do not agree on where a larger one goes, so refuse it for now
>> with
>>
>> Function f arg#1 type INT cannot be passed at argument slot 1 on this
>> architecture
>>
>> which the next patch turns into an answer from the JIT. The paths that
>> handle a two-slot argument are therefore unreachable until then.
>>
>> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
> This looks like a bug fix for kfunc struct argument support. Should this
> include:
>
> Fixes: 0ab8eb34abe4 ("bpf: Support by-value struct arguments up to 16 bytes")
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index c520c37bb3c9..9e70995c763f 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
> [ ... ]
>
>> @@ -12892,29 +12978,35 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
>> const struct btf *btf = meta->btf;
>> const struct btf_param *args;
>> struct btf_record *rec;
>> - u32 i, nargs;
>> + u32 i, k, nargs, proto_slots, slots_used, prev_slot = 0, nslots = 0;
>> int ret;
>>
>> args = (const struct btf_param *)(meta->func_proto + 1);
>> nargs = btf_type_vlen(meta->func_proto);
>> + proto_slots = kfunc_proto_slots(btf, meta->func_proto);
>>
>> - ret = check_outgoing_stack_args(env, caller, nargs, func_name, btf, args);
>> + ret = check_outgoing_stack_args(env, caller, proto_slots, func_name, btf,
>> + proto_slots == nargs ? args : NULL);
>> if (ret)
>> return ret;
>>
>> /* Check that BTF function arguments match actual types that the
>> * verifier sees.
>> */
>> - for (i = 0; i < nargs; i++) {
>> - struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
>> + for (i = 0, slots_used = 0; i < nargs;
>> + i++, prev_slot = slots_used, slots_used += nslots) {
>> + struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, slots_used);
>> const struct btf_type *t, *ref_t, *resolve_ret;
>> enum bpf_arg_type arg_type = ARG_DONTCARE;
>> - argno_t argno = argno_from_arg(i + 1);
>> + argno_t argno = argno_from_arg(slots_used + 1);
>> int regno = reg_from_argno(argno);
>> bool btf_id_fixed_off_ok = true;
>> u32 ref_id = args[i].type, type_size;
>> int kf_arg_type = meta->fn->arg_type[i];
>>
>> + t = btf_type_skip_modifiers(btf, args[i].type, NULL);
>> + nslots = kfunc_arg_slots(t);
>> +
>> if (is_kfunc_arg_prog_aux(btf, &args[i])) {
>> /* Reject repeated use bpf_prog_aux */
>> if (meta->arg_prog) {
> [ ... ]
>
>> @@ -12986,6 +13076,17 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
>> ref_tname = btf_name_by_offset(btf, ref_t->name_off);
>> }
>>
>> + /*
>> + * The first slot is checked in below KF_ARG_ANYTHING.
>> + * The rest of it has to be a scalar.
>> + */
>> + for (k = 1; k < nslots; k++) {
>> + ret = check_kfunc_scalar_arg(env, caller, regs, slots_used + k,
>> + insn_idx, func_name);
>> + if (ret)
>> + return ret;
>> + }
>> +
> This isn't a bug, but would it be worth rewording this comment to name
> the KF_ARG_ANYTHING case explicitly and speak of the remaining slots in
> the plural? The phrase "checked in below KF_ARG_ANYTHING" is a bit hard
> to parse, and "the rest of it" refers to slots while the sentence subject
> is singular.
I will update above comments to be clear about 'below KF_ARG_ANYTHING'.
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH bpf-next v3 09/15] bpf: Prepare kfunc arguments for the JIT from an ABI description
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (7 preceding siblings ...)
2026-09-11 15:49 ` [PATCH bpf-next v3 08/15] bpf: Recognize by-value struct and __int128 kfunc arguments Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 15:50 ` [PATCH bpf-next v3 10/15] bpf, x86: Move kfunc arguments into the x86-64 calling convention Yonghong Song
` (5 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
The previous patch refuses a kfunc argument of more than one eightbyte.
This patch allows up to 16 byte kfunc arguments.
But different architectures have different ways to map the BPF calling
convention (no gap, no backfill) to the native one. Rather than have each
arch open-code where it wants an argument, describe the convention with a
register count and four booleans, and let each arch set what applies to
it:
struct bpf_jit_arg_abi {
u8 nr_arg_regs;
bool even_reg_align;
bool even_stack_align;
bool split_at_boundary;
bool backfill_after_stack;
};
The four booleans are meant to cover x86-64, arm64, RISC-V LP64 and
PowerPC64 ELFv2, although only x86-64 and arm64 fill the struct in here.
bpf_jit_place_args() and bpf_jit_plan_arg_moves() use that description to
work out where each argument belongs and which slots the JIT then has to
move, to be used in the JIT later on.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
include/linux/bpf.h | 15 ++++++
include/linux/bpf_verifier.h | 1 +
include/linux/filter.h | 32 ++++++++++++
kernel/bpf/btf.c | 3 ++
kernel/bpf/core.c | 94 ++++++++++++++++++++++++++++++++++
kernel/bpf/verifier.c | 98 +++++++++++++++++++++++++++++++-----
6 files changed, 230 insertions(+), 13 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e80963971f68..e939fb448197 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -977,6 +977,13 @@ static_assert(__BPF_RET_TYPE_MAX <= BPF_BASE_TYPE_LIMIT);
*/
#define MAX_BPF_FUNC_REG_ARGS 5
+/* A by-value argument takes two eightbytes at most, so the maximum number of
+ * argument slots of any function is 2 * MAX_BPF_FUNC_ARGS. A local array may
+ * need that size for processing, although eventually the maximum slots will
+ * be capped at MAX_BPF_FUNC_ARGS.
+ */
+#define MAX_BPF_FUNC_ARG_SLOTS (2 * MAX_BPF_FUNC_ARGS)
+
/* eBPF function prototype used by verifier to allow BPF_CALLs from eBPF programs
* to in-kernel helper functions and for adjusting imm32 field in BPF_CALL
* instructions after verifying
@@ -1194,6 +1201,9 @@ struct bpf_prog_offload {
u32 jited_len;
};
+/* The argument is aligned to 16 bytes. */
+#define BTF_FMODEL_ALIGN16_ARG BIT(0)
+
/* The argument is signed. */
#define BTF_FMODEL_SIGNED_ARG BIT(1)
@@ -1211,6 +1221,11 @@ struct btf_func_model {
u8 arg_flags[MAX_BPF_FUNC_ARGS];
};
+static inline u32 btf_func_model_arg_slots(const struct btf_func_model *m, u32 arg)
+{
+ return (m->arg_size[arg] + sizeof(u64) - 1) / sizeof(u64);
+}
+
/* Restore arguments before returning from trampoline to let original function
* continue executing. This flag is used for fentry progs when there are no
* fexit progs.
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 64dfa29f6b45..65d6d5444dc4 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -1512,6 +1512,7 @@ enum btf_member_kind {
bool btf_struct_is_composed_of(struct bpf_verifier_env *env, const struct btf *btf,
const struct btf_type *t, u32 member_kinds);
+u32 btf_func_arg_align(const struct btf *btf, const struct btf_type *t);
int bpf_find_subprog(struct bpf_verifier_env *env, int off);
bool bpf_is_throw_kfunc(struct bpf_insn *insn);
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 00ad8b63aa47..b17222db2efc 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1248,6 +1248,38 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena);
bool bpf_jit_supports_private_stack(void);
bool bpf_jit_supports_timed_may_goto(void);
bool bpf_jit_supports_fsession(void);
+
+struct bpf_jit_arg_abi {
+ /* Argument registers of the kernel convention. */
+ u8 nr_arg_regs;
+ /* Round the register number up to an even one for 16-byte alignment. */
+ bool even_reg_align;
+ /* Round the stack slot up to an even one for 16-byte alignment. */
+ bool even_stack_align;
+ /* An argument may straddle the last register and the stack. */
+ bool split_at_boundary;
+ /* A later argument may reuse a register a stack-passed one skipped. */
+ bool backfill_after_stack;
+};
+
+const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void);
+u32 bpf_jit_place_args(const struct bpf_jit_arg_abi *abi,
+ const struct btf_func_model *fm, u8 *pos_of_slot);
+
+/* The JIT's scratch register, in place of an argument slot. */
+#define BPF_JIT_ARG_TMP 0xff
+
+/* Every argument slot moves at most once, and the scratch goes out and back. */
+#define BPF_JIT_MAX_ARG_MOVES (MAX_BPF_FUNC_ARG_SLOTS + 2)
+
+struct bpf_jit_arg_move {
+ u8 dst;
+ u8 src;
+};
+
+u32 bpf_jit_plan_arg_moves(const struct bpf_jit_arg_abi *abi,
+ const struct btf_func_model *fm,
+ struct bpf_jit_arg_move *moves);
u64 bpf_arch_uaddress_limit(void);
void arch_bpf_stack_walk(bool (*consume_fn)(void *cookie, u64 ip, u64 sp, u64 bp), void *cookie);
u64 arch_bpf_timed_may_goto(void);
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 088788e29b03..12f216515447 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -7579,6 +7579,9 @@ static u8 __get_arg_fmodel_flags(const struct btf *btf,
{
u8 flags = __get_type_fmodel_flags(t);
+ if (btf_func_arg_align(btf, t) > sizeof(u64))
+ flags |= BTF_FMODEL_ALIGN16_ARG;
+
if (btf_param_match_suffix(btf, arg, "__arena__nullable"))
flags |= BTF_FMODEL_ARENA_ARG | BTF_FMODEL_NULLABLE_ARG;
else if (btf_param_match_suffix(btf, arg, "__arena"))
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index c673b02d55a6..4e208cc94752 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3287,6 +3287,100 @@ bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
return false;
}
+/*
+ * How this arch places a by-value kfunc argument, or NULL for one that has
+ * not opted in and so only takes an argument of a single eightbyte, which
+ * every convention places in slot order.
+ */
+const struct bpf_jit_arg_abi * __weak bpf_jit_arg_abi(void)
+{
+ return NULL;
+}
+
+u32 bpf_jit_place_args(const struct bpf_jit_arg_abi *abi,
+ const struct btf_func_model *fm, u8 *pos_of_slot)
+{
+ u32 i, k, nslots, slot = 0, nregs_used = 0, stack_off = 0;
+ bool on_stack = false;
+
+ for (i = 0; i < fm->nr_args; i++) {
+ bool align16 = fm->arg_flags[i] & BTF_FMODEL_ALIGN16_ARG;
+ u32 pos;
+
+ nslots = btf_func_model_arg_slots(fm, i);
+
+ if (align16 && abi->even_reg_align)
+ nregs_used = round_up(nregs_used, 2);
+
+ if (!on_stack && nregs_used + nslots <= abi->nr_arg_regs) {
+ /* wholly in registers */
+ pos = nregs_used;
+ nregs_used += nslots;
+ } else if (!on_stack && abi->split_at_boundary) {
+ /* the last registers hold what fits, the stack the rest */
+ pos = nregs_used;
+ stack_off = (nregs_used + nslots - abi->nr_arg_regs) * BPF_REG_SIZE;
+ nregs_used = abi->nr_arg_regs;
+ on_stack = true;
+ } else {
+ /* wholly on the stack */
+ if (align16 && abi->even_stack_align)
+ stack_off = round_up(stack_off, 2 * BPF_REG_SIZE);
+ pos = abi->nr_arg_regs + stack_off / BPF_REG_SIZE;
+ stack_off += nslots * BPF_REG_SIZE;
+ if (!abi->backfill_after_stack)
+ on_stack = true;
+ }
+
+ for (k = 0; k < nslots; k++)
+ pos_of_slot[slot + k] = pos + k;
+ slot += nslots;
+ }
+
+ return slot;
+}
+
+u32 bpf_jit_plan_arg_moves(const struct bpf_jit_arg_abi *abi,
+ const struct btf_func_model *fm,
+ struct bpf_jit_arg_move *moves)
+{
+ u8 pos_of_slot[MAX_BPF_FUNC_ARG_SLOTS];
+ u32 nslots, n = 0, s, back;
+
+ nslots = bpf_jit_place_args(abi, fm, pos_of_slot);
+ back = nslots;
+
+ /*
+ * An argument is two eightbytes at most, so it frees one register at
+ * most and only one argument ever moves down. Its destination is
+ * still in use, so carry it in the scratch. Only a lower slot can
+ * take the one it leaves, so the walk reaches it first.
+ */
+ for (s = nslots; s > 0; s--) {
+ u8 slot = s - 1, pos = pos_of_slot[slot];
+
+ if (pos == slot)
+ continue;
+
+ if (pos < slot) {
+ moves[n].dst = BPF_JIT_ARG_TMP;
+ back = slot;
+ } else {
+ moves[n].dst = pos;
+ }
+ moves[n].src = slot;
+ n++;
+ }
+
+ if (back < nslots) {
+ moves[n].dst = pos_of_slot[back];
+ moves[n].src = BPF_JIT_ARG_TMP;
+ n++;
+ }
+
+ return n;
+}
+
bool __weak bpf_jit_supports_stack_args(void)
{
return false;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9e70995c763f..723b695d7ded 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2840,7 +2840,7 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env,
}
static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
- struct bpf_func_proto *proto);
+ const struct btf_func_model *fm, struct bpf_func_proto *proto);
int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
{
@@ -2957,7 +2957,7 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
desc = &tab->descs[tab->nr_descs];
memset(desc, 0, sizeof(*desc));
- err = gen_kfunc_arg_proto(env, &meta, &desc->proto);
+ err = gen_kfunc_arg_proto(env, &meta, &func_model, &desc->proto);
if (err)
return err;
@@ -12173,6 +12173,58 @@ static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_
return slots_used;
}
+static u32 kfunc_abi_slots(const struct btf_func_model *fm)
+{
+ const struct bpf_jit_arg_abi *abi = bpf_jit_arg_abi();
+ u8 pos_of_slot[MAX_BPF_FUNC_ARG_SLOTS];
+ u32 i, nslots, slots = 0;
+
+ for (i = 0; i < fm->nr_args; i++)
+ slots += btf_func_model_arg_slots(fm, i);
+
+ if (!abi)
+ return slots;
+
+ nslots = bpf_jit_place_args(abi, fm, pos_of_slot);
+ for (i = 0; i < nslots; i++)
+ if (pos_of_slot[i] + 1 > slots)
+ slots = pos_of_slot[i] + 1;
+
+ return slots;
+}
+
+static u32 __btf_func_arg_align(const struct btf *btf, const struct btf_type *t, int rec)
+{
+ const struct btf_member *member;
+ const struct btf_type *mt;
+ u32 align, i;
+
+ while (btf_type_is_array(t))
+ t = btf_type_skip_modifiers(btf, btf_array(t)->type, NULL);
+
+ if (btf_type_is_int(t))
+ return t->size > BPF_REG_SIZE ? t->size : BPF_REG_SIZE;
+ if (!btf_type_is_struct(t))
+ return BPF_REG_SIZE;
+ if (rec >= BTF_MEMBER_MAX_DEPTH)
+ return 0;
+
+ for_each_member(i, t, member) {
+ mt = btf_type_skip_modifiers(btf, member->type, NULL);
+ align = __btf_func_arg_align(btf, mt, rec + 1);
+ if (!align)
+ return 0;
+ if (align > BPF_REG_SIZE)
+ return 2 * BPF_REG_SIZE;
+ }
+ return BPF_REG_SIZE;
+}
+
+u32 btf_func_arg_align(const struct btf *btf, const struct btf_type *t)
+{
+ return __btf_func_arg_align(btf, t, 0);
+}
+
static int
get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
const struct btf_param *args, int arg, int nargs, u32 slot)
@@ -12326,10 +12378,12 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
}
static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
- struct bpf_func_proto *proto)
+ const struct btf_func_model *fm, struct bpf_func_proto *proto)
{
+ const struct bpf_jit_arg_abi *abi;
const struct btf *btf = meta->btf;
const struct btf_param *args;
+ const struct btf_type *t;
u32 i, nargs, slots_used;
int arg_type;
@@ -12347,17 +12401,35 @@ static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg
}
for (i = 0, slots_used = 0; i < nargs; i++) {
- const struct btf_type *t;
- u32 nslots;
+ u32 nslots = btf_func_model_arg_slots(fm, i);
- t = btf_type_skip_modifiers(btf, args[i].type, NULL);
- nslots = kfunc_arg_slots(t);
- /*
- * The calling conventions the JIT has to reconcile do not
- * agree on where an argument of more than one eightbyte goes,
- * so refuse one until the JIT can say where this arch puts it.
- */
if (nslots > 1) {
+ t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+ if (!btf_func_arg_align(btf, t)) {
+ verbose(env,
+ "Function %s arg#%d type %s nests structs more than "
+ "%d levels deep\n",
+ meta->func_name, i, btf_type_str(t),
+ BTF_MEMBER_MAX_DEPTH);
+ return -EINVAL;
+ }
+ }
+ slots_used += nslots;
+ }
+
+ if (slots_used > MAX_BPF_FUNC_ARGS) {
+ verbose(env, "Function %s needs %d > %d argument slots\n", meta->func_name,
+ slots_used, MAX_BPF_FUNC_ARGS);
+ return -EINVAL;
+ }
+
+ abi = bpf_jit_arg_abi();
+
+ for (i = 0, slots_used = 0; i < nargs; i++) {
+ u32 nslots = btf_func_model_arg_slots(fm, i);
+
+ if (!abi && nslots > 1) {
+ t = btf_type_skip_modifiers(btf, args[i].type, NULL);
verbose(env,
"Function %s arg#%d type %s cannot be passed at "
"argument slot %d on this architecture\n",
@@ -14523,7 +14595,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
if (bpf_is_kfunc_pkt_changing(&meta))
clear_all_pkt_pointers(env);
- proto_slots = kfunc_proto_slots(desc_btf, meta.func_proto);
+ proto_slots = kfunc_abi_slots(&desc->func_model);
if (proto_slots > MAX_BPF_FUNC_REG_ARGS) {
struct bpf_func_state *caller = cur_func(env);
struct bpf_subprog_info *caller_info = &env->subprog_info[caller->subprogno];
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 10/15] bpf, x86: Move kfunc arguments into the x86-64 calling convention
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (8 preceding siblings ...)
2026-09-11 15:50 ` [PATCH bpf-next v3 09/15] bpf: Prepare kfunc arguments for the JIT from an ABI description Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
2026-09-11 15:50 ` [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 " Yonghong Song
` (4 subsequent siblings)
14 siblings, 1 reply; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
Do the proper move from the BPF calling convention to the x86-64 calling
convention to satisfy the native requirement.
In addition, the arena argument walk counts eightbytes rather than
parameters, as an argument may take two registers.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
arch/x86/net/bpf_jit_comp.c | 73 +++++++++++++++++++++++++++++++++++--
1 file changed, 70 insertions(+), 3 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index bba351944202..0496607a7003 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1839,6 +1839,60 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
+static const struct bpf_jit_arg_abi x86_arg_abi = {
+ .nr_arg_regs = 6,
+ .backfill_after_stack = true,
+ .even_stack_align = true,
+};
+
+static const u8 x86_arg_reg[] = {
+ BPF_REG_1, BPF_REG_2, BPF_REG_3, BPF_REG_4, BPF_REG_5, X86_REG_R9,
+};
+
+/*
+ * Move the arguments the x86-64 ABI places somewhere other than the argument
+ * slot the BPF calling convention gave them. @stack_base addresses the
+ * outgoing stack argument area from RBP. Return the number of emitted bytes.
+ */
+static int emit_kfunc_arg_moves(const struct btf_func_model *fm, s32 stack_base, u8 **pprog)
+{
+ struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
+ const u8 nreg = x86_arg_abi.nr_arg_regs;
+ u8 *prog = *pprog, *start = prog;
+ u32 i, n;
+
+ n = bpf_jit_plan_arg_moves(&x86_arg_abi, fm, moves);
+
+ for (i = 0; i < n; i++) {
+ u8 dst = moves[i].dst, src = moves[i].src, reg;
+ bool dst_mem = dst != BPF_JIT_ARG_TMP && dst >= nreg;
+ bool src_mem = src != BPF_JIT_ARG_TMP && src >= nreg;
+
+ /* Take the value into a register. */
+ if (src == BPF_JIT_ARG_TMP) {
+ reg = AUX_REG;
+ } else if (src_mem) {
+ reg = dst_mem || dst == BPF_JIT_ARG_TMP ? BPF_REG_AX : x86_arg_reg[dst];
+ emit_ldx(&prog, BPF_DW, reg, BPF_REG_FP,
+ stack_base + (src - nreg) * 8);
+ } else {
+ reg = x86_arg_reg[src];
+ }
+
+ /* And leave it where the argument belongs. */
+ if (dst == BPF_JIT_ARG_TMP)
+ emit_mov_reg(&prog, true, AUX_REG, reg);
+ else if (dst_mem)
+ emit_stx(&prog, BPF_DW, BPF_REG_FP, reg,
+ stack_base + (dst - nreg) * 8);
+ else if (reg != x86_arg_reg[dst])
+ emit_mov_reg(&prog, true, x86_arg_reg[dst], reg);
+ }
+
+ *pprog = prog;
+ return prog - start;
+}
+
/*
* Rebase the __arena args of a kfunc call to arena kernel addresses,
* rN = kern_vm_start + (u32)rN, with R12 holding kern_vm_start. A nullable
@@ -1850,11 +1904,17 @@ static int emit_kfunc_arena_args(struct bpf_prog *bpf_prog,
{
u8 *prog = *pprog;
u8 *start = prog;
- int i;
+ int i, slot;
- for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
+ for (i = 0, slot = 0; i < fm->nr_args; i++) {
+ u32 arg_regs = (fm->arg_size[i] + 7) / 8;
u8 flags = fm->arg_flags[i];
- u32 reg = BPF_REG_1 + i;
+ u32 reg;
+
+ if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
+ break;
+ reg = BPF_REG_1 + slot;
+ slot += arg_regs;
if (!(flags & BTF_FMODEL_ARENA_ARG))
continue;
@@ -2837,6 +2897,8 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
if (err < 0)
return err;
ip += err;
+ ip += emit_kfunc_arg_moves(fm, outgoing_arg_base -
+ outgoing_rsp, &prog);
}
if (priv_frame_ptr) {
push_r9(&prog);
@@ -4351,6 +4413,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
return true;
}
+const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
+{
+ return &x86_arg_abi;
+}
+
bool bpf_jit_supports_stack_args(void)
{
return true;
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 10/15] bpf, x86: Move kfunc arguments into the x86-64 calling convention
2026-09-11 15:50 ` [PATCH bpf-next v3 10/15] bpf, x86: Move kfunc arguments into the x86-64 calling convention Yonghong Song
@ 2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 17:14 ` Yonghong Song
0 siblings, 1 reply; 32+ messages in thread
From: bot+bpf-ci @ 2026-09-11 16:47 UTC (permalink / raw)
To: yonghong.song, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 4125 bytes --]
> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index bba351944202..0496607a7003 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c
> @@ -1839,6 +1839,60 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
> return 0;
> }
>
> +static const struct bpf_jit_arg_abi x86_arg_abi = {
> + .nr_arg_regs = 6,
> + .backfill_after_stack = true,
> + .even_stack_align = true,
> +};
> +
> +static const u8 x86_arg_reg[] = {
> + BPF_REG_1, BPF_REG_2, BPF_REG_3, BPF_REG_4, BPF_REG_5, X86_REG_R9,
> +};
> +
> +/*
> + * Move the arguments the x86-64 ABI places somewhere other than the argument
> + * slot the BPF calling convention gave them. @stack_base addresses the
> + * outgoing stack argument area from RBP. Return the number of emitted bytes.
> + */
> +static int emit_kfunc_arg_moves(const struct btf_func_model *fm, s32 stack_base, u8 **pprog)
> +{
> + struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
> + const u8 nreg = x86_arg_abi.nr_arg_regs;
> + u8 *prog = *pprog, *start = prog;
> + u32 i, n;
> +
> + n = bpf_jit_plan_arg_moves(&x86_arg_abi, fm, moves);
> +
> + for (i = 0; i < n; i++) {
> + u8 dst = moves[i].dst, src = moves[i].src, reg;
> + bool dst_mem = dst != BPF_JIT_ARG_TMP && dst >= nreg;
> + bool src_mem = src != BPF_JIT_ARG_TMP && src >= nreg;
> +
> + /* Take the value into a register. */
> + if (src == BPF_JIT_ARG_TMP) {
> + reg = AUX_REG;
> + } else if (src_mem) {
> + reg = dst_mem || dst == BPF_JIT_ARG_TMP ? BPF_REG_AX : x86_arg_reg[dst];
> + emit_ldx(&prog, BPF_DW, reg, BPF_REG_FP,
> + stack_base + (src - nreg) * 8);
> + } else {
> + reg = x86_arg_reg[src];
> + }
> +
> + /* And leave it where the argument belongs. */
> + if (dst == BPF_JIT_ARG_TMP)
> + emit_mov_reg(&prog, true, AUX_REG, reg);
This isn't a bug, but for the memory-source case, could reg be AUX_REG when
dst == BPF_JIT_ARG_TMP so the load lands directly in the scratch instead of
going through BPF_REG_AX?
The case is reachable: a prototype such as (u64 a..e, struct {u64; u64;}
s, u64 f) backfills f into R9 and hands the planner a down-move whose
source is a stack slot. With the current code, emit_kfunc_arg_moves()
generates:
mov r10, [rbp+off]
mov r11, r10
when this would do:
mov r11, [rbp+off]
Picking AUX_REG when dst == BPF_JIT_ARG_TMP (and BPF_REG_AX only for the
memory-to-memory case) would drop the extra instruction from every such
call site.
> + else if (dst_mem)
> + emit_stx(&prog, BPF_DW, BPF_REG_FP, reg,
> + stack_base + (dst - nreg) * 8);
> + else if (reg != x86_arg_reg[dst])
> + emit_mov_reg(&prog, true, x86_arg_reg[dst], reg);
> + }
> +
> + *pprog = prog;
> + return prog - start;
> +}
[ ... ]
> @@ -2837,6 +2897,8 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
> if (err < 0)
> return err;
> ip += err;
> + ip += emit_kfunc_arg_moves(fm, outgoing_arg_base -
> + outgoing_rsp, &prog);
> }
[ ... ]
> bpf, x86: Move kfunc arguments into the x86-64 calling convention
>
> Do the proper move from the BPF calling convention to the x86-64 calling
> convention to satisfy the native requirement.
This isn't a bug, but could the changelog name the case that actually
needs a move (a by-value argument spilling to the stack followed by a
backfilled register argument) and note that at most one argument moves
down, so the scratch register carry is safe?
The current wording says what the patch does without saying which
arguments actually end up somewhere other than their BPF slot, or why a
reviewer has to reconstruct that from bpf_jit_place_args() and
bpf_jit_plan_arg_moves().
> In addition, the arena argument walk counts eightbytes rather than
> parameters, as an argument may take two registers.
(This second paragraph reads well and is concrete.)
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 10/15] bpf, x86: Move kfunc arguments into the x86-64 calling convention
2026-09-11 16:47 ` bot+bpf-ci
@ 2026-09-12 17:14 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:14 UTC (permalink / raw)
To: bot+bpf-ci, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, martin.lau, mason,
ihor.solodrai
On 9/11/26 9:47 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
>> index bba351944202..0496607a7003 100644
>> --- a/arch/x86/net/bpf_jit_comp.c
>> +++ b/arch/x86/net/bpf_jit_comp.c
>> @@ -1839,6 +1839,60 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
>> return 0;
>> }
>>
>> +static const struct bpf_jit_arg_abi x86_arg_abi = {
>> + .nr_arg_regs = 6,
>> + .backfill_after_stack = true,
>> + .even_stack_align = true,
>> +};
>> +
>> +static const u8 x86_arg_reg[] = {
>> + BPF_REG_1, BPF_REG_2, BPF_REG_3, BPF_REG_4, BPF_REG_5, X86_REG_R9,
>> +};
>> +
>> +/*
>> + * Move the arguments the x86-64 ABI places somewhere other than the argument
>> + * slot the BPF calling convention gave them. @stack_base addresses the
>> + * outgoing stack argument area from RBP. Return the number of emitted bytes.
>> + */
>> +static int emit_kfunc_arg_moves(const struct btf_func_model *fm, s32 stack_base, u8 **pprog)
>> +{
>> + struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
>> + const u8 nreg = x86_arg_abi.nr_arg_regs;
>> + u8 *prog = *pprog, *start = prog;
>> + u32 i, n;
>> +
>> + n = bpf_jit_plan_arg_moves(&x86_arg_abi, fm, moves);
>> +
>> + for (i = 0; i < n; i++) {
>> + u8 dst = moves[i].dst, src = moves[i].src, reg;
>> + bool dst_mem = dst != BPF_JIT_ARG_TMP && dst >= nreg;
>> + bool src_mem = src != BPF_JIT_ARG_TMP && src >= nreg;
>> +
>> + /* Take the value into a register. */
>> + if (src == BPF_JIT_ARG_TMP) {
>> + reg = AUX_REG;
>> + } else if (src_mem) {
>> + reg = dst_mem || dst == BPF_JIT_ARG_TMP ? BPF_REG_AX : x86_arg_reg[dst];
>> + emit_ldx(&prog, BPF_DW, reg, BPF_REG_FP,
>> + stack_base + (src - nreg) * 8);
>> + } else {
>> + reg = x86_arg_reg[src];
>> + }
>> +
>> + /* And leave it where the argument belongs. */
>> + if (dst == BPF_JIT_ARG_TMP)
>> + emit_mov_reg(&prog, true, AUX_REG, reg);
> This isn't a bug, but for the memory-source case, could reg be AUX_REG when
> dst == BPF_JIT_ARG_TMP so the load lands directly in the scratch instead of
> going through BPF_REG_AX?
>
> The case is reachable: a prototype such as (u64 a..e, struct {u64; u64;}
> s, u64 f) backfills f into R9 and hands the planner a down-move whose
> source is a stack slot. With the current code, emit_kfunc_arg_moves()
> generates:
>
> mov r10, [rbp+off]
> mov r11, r10
>
> when this would do:
>
> mov r11, [rbp+off]
>
> Picking AUX_REG when dst == BPF_JIT_ARG_TMP (and BPF_REG_AX only for the
> memory-to-memory case) would drop the extra instruction from every such
> call site.
Sounds good. A little bit optimization. Will do.
>> + else if (dst_mem)
>> + emit_stx(&prog, BPF_DW, BPF_REG_FP, reg,
>> + stack_base + (dst - nreg) * 8);
>> + else if (reg != x86_arg_reg[dst])
>> + emit_mov_reg(&prog, true, x86_arg_reg[dst], reg);
>> + }
>> +
>> + *pprog = prog;
>> + return prog - start;
>> +}
> [ ... ]
>
>> @@ -2837,6 +2897,8 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
>> if (err < 0)
>> return err;
>> ip += err;
>> + ip += emit_kfunc_arg_moves(fm, outgoing_arg_base -
>> + outgoing_rsp, &prog);
>> }
> [ ... ]
>
>> bpf, x86: Move kfunc arguments into the x86-64 calling convention
>>
>> Do the proper move from the BPF calling convention to the x86-64 calling
>> convention to satisfy the native requirement.
> This isn't a bug, but could the changelog name the case that actually
> needs a move (a by-value argument spilling to the stack followed by a
> backfilled register argument) and note that at most one argument moves
> down, so the scratch register carry is safe?
>
> The current wording says what the patch does without saying which
> arguments actually end up somewhere other than their BPF slot, or why a
> reviewer has to reconstruct that from bpf_jit_place_args() and
> bpf_jit_plan_arg_moves().
Indeed, the above description is not enough. I will add more things like you
mentioned above.
>
>> In addition, the arena argument walk counts eightbytes rather than
>> parameters, as an argument may take two registers.
> (This second paragraph reads well and is concrete.)
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 calling convention
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (9 preceding siblings ...)
2026-09-11 15:50 ` [PATCH bpf-next v3 10/15] bpf, x86: Move kfunc arguments into the x86-64 calling convention Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 16:19 ` sashiko-bot
2026-09-11 16:47 ` bot+bpf-ci
2026-09-11 15:50 ` [PATCH bpf-next v3 12/15] selftests/bpf: Add C tests for by-value arguments up to 16 bytes Yonghong Song
` (3 subsequent siblings)
14 siblings, 2 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
Do the proper move from the BPF calling convention to the arm64 calling
convention to satisfy the native requirement. AAPCS64 only ever moves an
argument to a higher slot, so the moves need one scratch register to carry
an eightbyte from one stack slot to another, and never the one a
convention moving an argument down would need.
In addition, the arena argument walk counts eightbytes rather than
parameters, as an argument may take two registers. The walk takes the
func model from the caller now, as the moves need it too, and runs first
so that they carry the rebased value.
btf_distill_func_proto() only bounds the argument count although a
by-value argument could take two slots. Similar to x86-64, let us support
up to MAX_BPF_FUNC_ARGS argument slots.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
arch/arm64/net/bpf_jit_comp.c | 92 ++++++++++++++++++++++++++++++-----
1 file changed, 81 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 3aa3ea0bc30b..bbde7c0836ae 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1220,6 +1220,12 @@ static int add_exception_handler(const struct bpf_insn *insn,
return 0;
}
+static const struct bpf_jit_arg_abi arm64_arg_abi = {
+ .nr_arg_regs = 8,
+ .even_reg_align = true,
+ .even_stack_align = true,
+};
+
static const u8 stack_arg_reg[] = { A64_R(5), A64_R(6), A64_R(7) };
#define NR_STACK_ARG_REGS ARRAY_SIZE(stack_arg_reg)
@@ -1262,19 +1268,20 @@ static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct
* kern_vm_start. A nullable arg preserves NULL by skipping the add, tested
* on the truncated value as arena NULL is offset 0.
*/
-static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *insn)
+static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct btf_func_model *fm)
{
const u8 arena_vm_base = bpf2a64[ARENA_VM_START];
- const struct btf_func_model *fm;
- int i;
-
- fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
- if (!fm)
- return -EINVAL;
+ int i, slot;
- for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
- const u8 reg = bpf2a64[BPF_REG_1 + i];
+ for (i = 0, slot = 0; i < fm->nr_args; i++) {
+ u32 arg_regs = (fm->arg_size[i] + 7) / 8;
u8 flags = fm->arg_flags[i];
+ u8 reg;
+
+ if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
+ break;
+ reg = bpf2a64[BPF_REG_1 + slot];
+ slot += arg_regs;
if (!(flags & BTF_FMODEL_ARENA_ARG))
continue;
@@ -1293,6 +1300,52 @@ static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins
return 0;
}
+static bool a64_arg_on_stack(u8 slot)
+{
+ return slot >= arm64_arg_abi.nr_arg_regs;
+}
+
+static s32 a64_arg_stack_off(u8 slot)
+{
+ return (slot - arm64_arg_abi.nr_arg_regs) * sizeof(u64);
+}
+
+/*
+ * Move the arguments AAPCS64 places somewhere other than the argument slot the
+ * BPF calling convention gave them. Slot N is X(N) up to the eighth, and the
+ * outgoing stack argument area from SP beyond it, both for the slot an
+ * argument comes from and for the one it goes to.
+ *
+ * AAPCS64 only ever moves an argument to a higher slot, so no move here ever
+ * takes BPF_JIT_ARG_TMP: bpf_jit_plan_arg_moves() hands out the scratch only
+ * for a convention that moves one down, which needs a register to carry the
+ * value past its own destination.
+ */
+static void emit_kfunc_arg_moves(struct jit_ctx *ctx, const struct btf_func_model *fm)
+{
+ struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
+ const u8 tmp = bpf2a64[TMP_REG_1];
+ u32 i, n;
+
+ n = bpf_jit_plan_arg_moves(&arm64_arg_abi, fm, moves);
+
+ for (i = 0; i < n; i++) {
+ u8 dst = moves[i].dst, src = moves[i].src, reg;
+
+ if (a64_arg_on_stack(src)) {
+ reg = tmp;
+ emit(A64_LDR64I(reg, A64_SP, a64_arg_stack_off(src)), ctx);
+ } else {
+ reg = src;
+ }
+
+ if (a64_arg_on_stack(dst))
+ emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
+ else if (reg != dst)
+ emit(A64_MOV(1, dst, reg), ctx);
+ }
+}
+
/* JITs an eBPF instruction.
* Returns:
* 0 - successfully JITed an 8-byte eBPF instruction.
@@ -1716,9 +1769,15 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
if (ret < 0)
return ret;
if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
- ret = emit_kfunc_arena_args(ctx, insn);
+ const struct btf_func_model *fm;
+
+ fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
+ if (!fm)
+ return -EINVAL;
+ ret = emit_kfunc_arena_args(ctx, fm);
if (ret < 0)
return ret;
+ emit_kfunc_arg_moves(ctx, fm);
}
emit_call(func_addr, ctx);
/*
@@ -2393,6 +2452,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
return true;
}
+const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
+{
+ return &arm64_arg_abi;
+}
+
bool bpf_jit_supports_stack_args(void)
{
return true;
@@ -2534,7 +2598,13 @@ struct arg_aux {
static int calc_arg_aux(const struct btf_func_model *m,
struct arg_aux *a)
{
- int stack_slots, nregs, slots, i;
+ int stack_slots, nregs, slots, i, total;
+
+ /* arm64 supports up to MAX_BPF_FUNC_ARGS argument slots */
+ for (i = 0, total = 0; i < m->nr_args; i++)
+ total += (m->arg_size[i] + 7) / 8;
+ if (total > MAX_BPF_FUNC_ARGS)
+ return -ENOTSUPP;
/* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */
for (i = 0, nregs = 0; i < m->nr_args; i++) {
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 calling convention
2026-09-11 15:50 ` [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 " Yonghong Song
@ 2026-09-11 16:19 ` sashiko-bot
2026-09-12 17:16 ` Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
1 sibling, 1 reply; 32+ messages in thread
From: sashiko-bot @ 2026-09-11 16:19 UTC (permalink / raw)
To: Yonghong Song; +Cc: bpf
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The ARM64 BPF trampoline incorrectly saves and restores 16-byte aligned arguments, violating AAPCS64 alignment rules.
--
commit cd40aa31f4bf1158416008f7fcb9c0b7717dda3f
Author: Yonghong Song <yonghong.song@linux.dev>
bpf, arm64: Move kfunc arguments into the arm64 calling convention
Do the proper move from the BPF calling convention to the arm64 calling
convention to satisfy the native requirement. AAPCS64 only ever moves an
argument to a higher slot, so the moves need one scratch register to carry
an eightbyte from one stack slot to another, and never the one a
convention moving an argument down would need.
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index 3aa3ea0bc30bb..bbde7c0836ae5 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
[ ... ]
> @@ -2534,7 +2598,13 @@ struct arg_aux {
> static int calc_arg_aux(const struct btf_func_model *m,
> struct arg_aux *a)
> {
> - int stack_slots, nregs, slots, i;
> + int stack_slots, nregs, slots, i, total;
> +
> + /* arm64 supports up to MAX_BPF_FUNC_ARGS argument slots */
> + for (i = 0, total = 0; i < m->nr_args; i++)
> + total += (m->arg_size[i] + 7) / 8;
> + if (total > MAX_BPF_FUNC_ARGS)
> + return -ENOTSUPP;
>
> /* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */
> for (i = 0, nregs = 0; i < m->nr_args; i++) {
[Severity: High]
This isn't a bug introduced by this patch, but since the series introduces
support for by-value 16-byte arguments, this previously dormant ABI violation
in the BPF trampoline appears to now be actively reachable.
Does calc_arg_aux() fail to account for skipped registers due to alignment
requirements?
If a user attaches a BPF program to a kernel function (or struct_ops
callback) that takes a 16-byte aligned argument (e.g., __int128 or
by-value struct) preceded by an odd number of register-sized arguments,
the native AAPCS64 caller places the 16-byte argument in an even/odd
register pair (like x2 and x3, skipping x1).
However, the nregs calculation here simply increments by slots without
checking for skipped odd registers:
arch/arm64/net/bpf_jit_comp.c:calc_arg_aux() {
...
for (i = 0, nregs = 0; i < m->nr_args; i++) {
slots = (m->arg_size[i] + 7) / 8;
if (nregs + slots <= 8) /* passed through register ? */
nregs += slots;
...
}
This strictly sequential register handling seems to propagate to both
save_args() and restore_args() in the trampoline, where arguments are
read and written sequentially without skipping odd registers:
arch/arm64/net/bpf_jit_comp.c:save_args() {
...
slots = (m->arg_size[i] + 7) / 8;
while (slots-- > 0) {
...
emit(A64_STR64I(reg, A64_SP, bargs_off), ctx);
reg++;
bargs_off += 8;
}
...
}
arch/arm64/net/bpf_jit_comp.c:restore_args() {
...
for (reg = 0; reg < nregs; reg++) {
emit(A64_LDR64I(reg, A64_SP, bargs_off), ctx);
bargs_off += 8;
}
...
}
Could this cause data corruption for arguments passed to BPF programs
attached via fentry, fexit, or struct_ops when 16-byte arguments are
involved?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260911154914.2004336-1-yonghong.song@linux.dev?part=11
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 calling convention
2026-09-11 16:19 ` sashiko-bot
@ 2026-09-12 17:16 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:16 UTC (permalink / raw)
To: sashiko-reviews; +Cc: bpf
On 9/11/26 9:19 AM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
>
> Pre-existing issues:
> - [High] The ARM64 BPF trampoline incorrectly saves and restores 16-byte aligned arguments, violating AAPCS64 alignment rules.
> --
>
> commit cd40aa31f4bf1158416008f7fcb9c0b7717dda3f
> Author: Yonghong Song <yonghong.song@linux.dev>
>
> bpf, arm64: Move kfunc arguments into the arm64 calling convention
>
> Do the proper move from the BPF calling convention to the arm64 calling
> convention to satisfy the native requirement. AAPCS64 only ever moves an
> argument to a higher slot, so the moves need one scratch register to carry
> an eightbyte from one stack slot to another, and never the one a
> convention moving an argument down would need.
>
>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>> index 3aa3ea0bc30bb..bbde7c0836ae5 100644
>> --- a/arch/arm64/net/bpf_jit_comp.c
>> +++ b/arch/arm64/net/bpf_jit_comp.c
> [ ... ]
>
>> @@ -2534,7 +2598,13 @@ struct arg_aux {
>> static int calc_arg_aux(const struct btf_func_model *m,
>> struct arg_aux *a)
>> {
>> - int stack_slots, nregs, slots, i;
>> + int stack_slots, nregs, slots, i, total;
>> +
>> + /* arm64 supports up to MAX_BPF_FUNC_ARGS argument slots */
>> + for (i = 0, total = 0; i < m->nr_args; i++)
>> + total += (m->arg_size[i] + 7) / 8;
>> + if (total > MAX_BPF_FUNC_ARGS)
>> + return -ENOTSUPP;
>>
>> /* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */
>> for (i = 0, nregs = 0; i < m->nr_args; i++) {
> [Severity: High]
> This isn't a bug introduced by this patch, but since the series introduces
> support for by-value 16-byte arguments, this previously dormant ABI violation
> in the BPF trampoline appears to now be actively reachable.
>
> Does calc_arg_aux() fail to account for skipped registers due to alignment
> requirements?
>
> If a user attaches a BPF program to a kernel function (or struct_ops
> callback) that takes a 16-byte aligned argument (e.g., __int128 or
> by-value struct) preceded by an odd number of register-sized arguments,
> the native AAPCS64 caller places the 16-byte argument in an even/odd
> register pair (like x2 and x3, skipping x1).
>
> However, the nregs calculation here simply increments by slots without
> checking for skipped odd registers:
>
> arch/arm64/net/bpf_jit_comp.c:calc_arg_aux() {
> ...
> for (i = 0, nregs = 0; i < m->nr_args; i++) {
> slots = (m->arg_size[i] + 7) / 8;
> if (nregs + slots <= 8) /* passed through register ? */
> nregs += slots;
> ...
> }
>
> This strictly sequential register handling seems to propagate to both
> save_args() and restore_args() in the trampoline, where arguments are
> read and written sequentially without skipping odd registers:
>
> arch/arm64/net/bpf_jit_comp.c:save_args() {
> ...
> slots = (m->arg_size[i] + 7) / 8;
> while (slots-- > 0) {
> ...
> emit(A64_STR64I(reg, A64_SP, bargs_off), ctx);
> reg++;
> bargs_off += 8;
> }
> ...
> }
>
> arch/arm64/net/bpf_jit_comp.c:restore_args() {
> ...
> for (reg = 0; reg < nregs; reg++) {
> emit(A64_LDR64I(reg, A64_SP, bargs_off), ctx);
> bargs_off += 8;
> }
> ...
> }
>
> Could this cause data corruption for arguments passed to BPF programs
> attached via fentry, fexit, or struct_ops when 16-byte arguments are
> involved?
>
Yes, this is a bug. I will need to consider 16-byte alignment.
Will fix.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 calling convention
2026-09-11 15:50 ` [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 " Yonghong Song
2026-09-11 16:19 ` sashiko-bot
@ 2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 17:19 ` Yonghong Song
1 sibling, 1 reply; 32+ messages in thread
From: bot+bpf-ci @ 2026-09-11 16:47 UTC (permalink / raw)
To: yonghong.song, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 5131 bytes --]
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index 3aa3ea0bc30b..bbde7c0836ae 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
[ ... ]
> @@ -1262,19 +1268,20 @@ static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct
> * kern_vm_start. A nullable arg preserves NULL by skipping the add, tested
> * on the truncated value as arena NULL is offset 0.
> */
> -static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *insn)
> +static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct btf_func_model *fm)
> {
> const u8 arena_vm_base = bpf2a64[ARENA_VM_START];
> - const struct btf_func_model *fm;
> - int i;
> -
> - fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
> - if (!fm)
> - return -EINVAL;
> + int i, slot;
>
> - for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
> - const u8 reg = bpf2a64[BPF_REG_1 + i];
> + for (i = 0, slot = 0; i < fm->nr_args; i++) {
> + u32 arg_regs = (fm->arg_size[i] + 7) / 8;
> u8 flags = fm->arg_flags[i];
> + u8 reg;
> +
> + if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
> + break;
> + reg = bpf2a64[BPF_REG_1 + slot];
> + slot += arg_regs;
>
> if (!(flags & BTF_FMODEL_ARENA_ARG))
> continue;
> @@ -1293,6 +1300,52 @@ static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins
> return 0;
> }
>
> +static bool a64_arg_on_stack(u8 slot)
> +{
> + return slot >= arm64_arg_abi.nr_arg_regs;
> +}
> +
> +static s32 a64_arg_stack_off(u8 slot)
> +{
> + return (slot - arm64_arg_abi.nr_arg_regs) * sizeof(u64);
> +}
This isn't a bug, but would it be worth expressing
emit_stack_arg_load()/emit_stack_arg_store() in terms of
a64_arg_on_stack()/a64_arg_stack_off() (or noting in a comment that the
two forms describe the same slot-to-location map), so the layout lives
in one place? The file now carries two independent encodings of the same
physical outgoing-argument layout: the pre-existing helpers express it
as 'idx = bpf slot - 5', register when idx < NR_STACK_ARG_REGS
(x5/x6/x7) and memory at SP + (idx - NR_STACK_ARG_REGS) * 8 otherwise;
the new helpers express it as 'slot', register when slot <
arm64_arg_abi.nr_arg_regs and memory at SP + (slot - nr_arg_regs) * 8.
> +
> +/*
> + * Move the arguments AAPCS64 places somewhere other than the argument slot the
> + * BPF calling convention gave them. Slot N is X(N) up to the eighth, and the
> + * outgoing stack argument area from SP beyond it, both for the slot an
> + * argument comes from and for the one it goes to.
> + *
> + * AAPCS64 only ever moves an argument to a higher slot, so no move here ever
> + * takes BPF_JIT_ARG_TMP: bpf_jit_plan_arg_moves() hands out the scratch only
> + * for a convention that moves one down, which needs a register to carry the
> + * value past its own destination.
> + */
> +static void emit_kfunc_arg_moves(struct jit_ctx *ctx, const struct btf_func_model *fm)
> +{
> + struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
> + const u8 tmp = bpf2a64[TMP_REG_1];
> + u32 i, n;
> +
> + n = bpf_jit_plan_arg_moves(&arm64_arg_abi, fm, moves);
> +
> + for (i = 0; i < n; i++) {
> + u8 dst = moves[i].dst, src = moves[i].src, reg;
> +
> + if (a64_arg_on_stack(src)) {
> + reg = tmp;
> + emit(A64_LDR64I(reg, A64_SP, a64_arg_stack_off(src)), ctx);
> + } else {
> + reg = src;
> + }
> +
> + if (a64_arg_on_stack(dst))
> + emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
> + else if (reg != dst)
> + emit(A64_MOV(1, dst, reg), ctx);
> + }
> +}
[ ... ]
> @@ -2534,7 +2598,13 @@ struct arg_aux {
> static int calc_arg_aux(const struct btf_func_model *m,
> struct arg_aux *a)
> {
> - int stack_slots, nregs, slots, i;
> + int stack_slots, nregs, slots, i, total;
> +
> + /* arm64 supports up to MAX_BPF_FUNC_ARGS argument slots */
> + for (i = 0, total = 0; i < m->nr_args; i++)
> + total += (m->arg_size[i] + 7) / 8;
> + if (total > MAX_BPF_FUNC_ARGS)
> + return -ENOTSUPP;
>
> /* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */
> for (i = 0; i < m->nr_args; i++) {
This isn't a bug, but could the changelog say explicitly that arm64's
trampoline path now rejects models whose arguments exceed
MAX_BPF_FUNC_ARGS eightbytes (previously it accepted up to twice that),
so the narrowing is visible to anyone bisecting an fentry attach
failure? The third changelog paragraph describes the calc_arg_aux() hunk
as adding support ('let us support up to MAX_BPF_FUNC_ARGS argument
slots'), while on the trampoline path the hunk is a new restriction:
calc_arg_aux() previously accepted any model btf_distill_func_proto()
admitted (up to 12 arguments of up to 16 bytes, i.e. up to 24
eightbytes), and now returns -ENOTSUPP for anything above 12 eightbytes,
so arch_prepare_bpf_trampoline()/arch_bpf_trampoline_size() fail for
those targets.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 calling convention
2026-09-11 16:47 ` bot+bpf-ci
@ 2026-09-12 17:19 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:19 UTC (permalink / raw)
To: bot+bpf-ci, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, martin.lau, mason,
ihor.solodrai
On 9/11/26 9:47 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>> index 3aa3ea0bc30b..bbde7c0836ae 100644
>> --- a/arch/arm64/net/bpf_jit_comp.c
>> +++ b/arch/arm64/net/bpf_jit_comp.c
> [ ... ]
>
>> @@ -1262,19 +1268,20 @@ static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct
>> * kern_vm_start. A nullable arg preserves NULL by skipping the add, tested
>> * on the truncated value as arena NULL is offset 0.
>> */
>> -static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *insn)
>> +static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct btf_func_model *fm)
>> {
>> const u8 arena_vm_base = bpf2a64[ARENA_VM_START];
>> - const struct btf_func_model *fm;
>> - int i;
>> -
>> - fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
>> - if (!fm)
>> - return -EINVAL;
>> + int i, slot;
>>
>> - for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
>> - const u8 reg = bpf2a64[BPF_REG_1 + i];
>> + for (i = 0, slot = 0; i < fm->nr_args; i++) {
>> + u32 arg_regs = (fm->arg_size[i] + 7) / 8;
>> u8 flags = fm->arg_flags[i];
>> + u8 reg;
>> +
>> + if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
>> + break;
>> + reg = bpf2a64[BPF_REG_1 + slot];
>> + slot += arg_regs;
>>
>> if (!(flags & BTF_FMODEL_ARENA_ARG))
>> continue;
>> @@ -1293,6 +1300,52 @@ static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins
>> return 0;
>> }
>>
>> +static bool a64_arg_on_stack(u8 slot)
>> +{
>> + return slot >= arm64_arg_abi.nr_arg_regs;
>> +}
>> +
>> +static s32 a64_arg_stack_off(u8 slot)
>> +{
>> + return (slot - arm64_arg_abi.nr_arg_regs) * sizeof(u64);
>> +}
> This isn't a bug, but would it be worth expressing
> emit_stack_arg_load()/emit_stack_arg_store() in terms of
> a64_arg_on_stack()/a64_arg_stack_off() (or noting in a comment that the
> two forms describe the same slot-to-location map), so the layout lives
> in one place? The file now carries two independent encodings of the same
> physical outgoing-argument layout: the pre-existing helpers express it
> as 'idx = bpf slot - 5', register when idx < NR_STACK_ARG_REGS
> (x5/x6/x7) and memory at SP + (idx - NR_STACK_ARG_REGS) * 8 otherwise;
> the new helpers express it as 'slot', register when slot <
> arm64_arg_abi.nr_arg_regs and memory at SP + (slot - nr_arg_regs) * 8.
I think the code is self explained.
>
>> +
>> +/*
>> + * Move the arguments AAPCS64 places somewhere other than the argument slot the
>> + * BPF calling convention gave them. Slot N is X(N) up to the eighth, and the
>> + * outgoing stack argument area from SP beyond it, both for the slot an
>> + * argument comes from and for the one it goes to.
>> + *
>> + * AAPCS64 only ever moves an argument to a higher slot, so no move here ever
>> + * takes BPF_JIT_ARG_TMP: bpf_jit_plan_arg_moves() hands out the scratch only
>> + * for a convention that moves one down, which needs a register to carry the
>> + * value past its own destination.
>> + */
>> +static void emit_kfunc_arg_moves(struct jit_ctx *ctx, const struct btf_func_model *fm)
>> +{
>> + struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
>> + const u8 tmp = bpf2a64[TMP_REG_1];
>> + u32 i, n;
>> +
>> + n = bpf_jit_plan_arg_moves(&arm64_arg_abi, fm, moves);
>> +
>> + for (i = 0; i < n; i++) {
>> + u8 dst = moves[i].dst, src = moves[i].src, reg;
>> +
>> + if (a64_arg_on_stack(src)) {
>> + reg = tmp;
>> + emit(A64_LDR64I(reg, A64_SP, a64_arg_stack_off(src)), ctx);
>> + } else {
>> + reg = src;
>> + }
>> +
>> + if (a64_arg_on_stack(dst))
>> + emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
>> + else if (reg != dst)
>> + emit(A64_MOV(1, dst, reg), ctx);
>> + }
>> +}
> [ ... ]
>
>> @@ -2534,7 +2598,13 @@ struct arg_aux {
>> static int calc_arg_aux(const struct btf_func_model *m,
>> struct arg_aux *a)
>> {
>> - int stack_slots, nregs, slots, i;
>> + int stack_slots, nregs, slots, i, total;
>> +
>> + /* arm64 supports up to MAX_BPF_FUNC_ARGS argument slots */
>> + for (i = 0, total = 0; i < m->nr_args; i++)
>> + total += (m->arg_size[i] + 7) / 8;
>> + if (total > MAX_BPF_FUNC_ARGS)
>> + return -ENOTSUPP;
>>
>> /* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */
>> for (i = 0; i < m->nr_args; i++) {
> This isn't a bug, but could the changelog say explicitly that arm64's
> trampoline path now rejects models whose arguments exceed
> MAX_BPF_FUNC_ARGS eightbytes (previously it accepted up to twice that),
> so the narrowing is visible to anyone bisecting an fentry attach
> failure? The third changelog paragraph describes the calc_arg_aux() hunk
> as adding support ('let us support up to MAX_BPF_FUNC_ARGS argument
> slots'), while on the trampoline path the hunk is a new restriction:
> calc_arg_aux() previously accepted any model btf_distill_func_proto()
> admitted (up to 12 arguments of up to 16 bytes, i.e. up to 24
> eightbytes), and now returns -ENOTSUPP for anything above 12 eightbytes,
> so arch_prepare_bpf_trampoline()/arch_bpf_trampoline_size() fail for
> those targets.
You are right. We shouldn't just reject. I will do proper implementation
to fix the issue.
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH bpf-next v3 12/15] selftests/bpf: Add C tests for by-value arguments up to 16 bytes
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (10 preceding siblings ...)
2026-09-11 15:50 ` [PATCH bpf-next v3 11/15] bpf, arm64: Move kfunc arguments into the arm64 " Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 15:50 ` [PATCH bpf-next v3 13/15] selftests/bpf: Add inline-asm tests for by-value arguments Yonghong Song
` (2 subsequent siblings)
14 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
Extend the by-value argument test with the aggregate cases, written in C
so that they depend on the compiler lowering the argument into a pair of
argument registers rather than on a hand-written register layout.
The programs cover a struct and a union that fill two registers, a
smaller struct that fills one, and two struct arguments in a row,
alongside the __int128 already there. Each has an int argument around it
so that a wrong slot count shows up as a wrong value in the parameters
beside it; two pairs leave room for only one, which follows them. A
global function taking a struct with a pointer member is rejected: the
callee would receive the pointer as an opaque scalar.
A struct the argument registers cannot hold reaches the callee partly on
the stack, which the interpreter does not implement, so that case is
loaded only when the JIT is on.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
.../bpf/progs/verifier_aggregate_arg.c | 165 ++++++++++++++++++
1 file changed, 165 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
index fc7c1b18bc40..b0ecccede47e 100644
--- a/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
+++ b/tools/testing/selftests/bpf/progs/verifier_aggregate_arg.c
@@ -7,6 +7,131 @@
#define MIX_A 0xdeadbeefcafef00dULL
#define MIX_B 0x0123456789abcdefULL
+struct pair {
+ __u64 lo;
+ __u64 hi;
+};
+
+struct small {
+ __u32 a;
+ __u32 b;
+};
+
+union upair {
+ __u64 halves[2];
+ struct {
+ __u64 lo;
+ __u64 hi;
+ } parts;
+};
+
+struct with_ptr {
+ void *p;
+ __u64 x;
+};
+
+static __noinline __u64 take_pair(int a, struct pair p, int c)
+{
+ return (__u64)a + p.lo + p.hi + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_static_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p = { .lo = a, .hi = b };
+
+ if (take_pair(1, p, 2) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+#if defined(__clang__)
+
+__noinline __u64 take_pair_global(int a, struct pair p, int c)
+{
+ return (__u64)a + p.lo + p.hi + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_global_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p = { .lo = a, .hi = b };
+
+ if (take_pair_global(1, p, 2) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+__noinline __u64 take_two_pairs_global(struct pair p, struct pair q, int d)
+{
+ return p.lo + p.hi + q.lo + q.hi + d;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_two_structs_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p = { .lo = a, .hi = b };
+ struct pair q = { .lo = a + 1, .hi = b + 2 };
+
+ if (take_two_pairs_global(p, q, 3) != 2 * a + 2 * b + 6)
+ return 1;
+
+ return 0;
+}
+
+__noinline __u64 take_small_global(int a, struct small s, int c)
+{
+ return (__u64)a + s.a + s.b + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_small_struct_c_test(struct __sk_buff *skb)
+{
+ __u32 a = skb->len ^ (__u32)MIX_A;
+ __u32 b = skb->len ^ (__u32)MIX_B;
+ struct small s = { .a = a, .b = b };
+
+ if (take_small_global(1, s, 2) != (__u64)a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+__noinline __u64 take_upair_global(int a, union upair u, int c)
+{
+ return (__u64)a + u.parts.lo + u.parts.hi + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_union_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ union upair u;
+
+ u.halves[0] = a;
+ u.halves[1] = b;
+ if (take_upair_global(1, u, 2) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+#endif
+
#ifdef __SIZEOF_INT128__
typedef unsigned __int128 u128;
@@ -32,4 +157,44 @@ int aggregate_arg_int128_c_test(struct __sk_buff *skb)
#endif /* __SIZEOF_INT128__ */
+#if defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+static __noinline __u64 take_spilled_pair(int a, int b, int c, int d, struct pair p)
+{
+ return (__u64)a + b + c + d + p.lo + p.hi;
+}
+
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_spilled_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p = { .lo = a, .hi = b };
+ int n = skb->len;
+
+ if (take_spilled_pair(n, n + 1, n + 2, n + 3, p) != a + b + 4 * n + 6)
+ return 1;
+
+ return 0;
+}
+
+#endif
+
+__noinline __u64 take_with_ptr_global(struct with_ptr s)
+{
+ return s.x;
+}
+
+SEC("tc")
+__failure __msg("type STRUCT in take_with_ptr_global() is not composed of scalars")
+int aggregate_arg_ptr_member_fail(struct __sk_buff *skb)
+{
+ struct with_ptr s = { .p = skb, .x = skb->len };
+
+ return take_with_ptr_global(s);
+}
+
char _license[] SEC("license") = "GPL";
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH bpf-next v3 13/15] selftests/bpf: Add inline-asm tests for by-value arguments
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (11 preceding siblings ...)
2026-09-11 15:50 ` [PATCH bpf-next v3 12/15] selftests/bpf: Add C tests for by-value arguments up to 16 bytes Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 16:06 ` sashiko-bot
2026-09-11 15:50 ` [PATCH bpf-next v3 14/15] selftests/bpf: Add tests for by-value kfunc arguments Yonghong Song
2026-09-11 15:50 ` [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64 Yonghong Song
14 siblings, 1 reply; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
The tests cover a struct passed in a register pair, a pointer in one half
of it refused at the call site, a struct too large to pass by value, and
four placements a global function cannot have: six scalars, a struct split
between the last argument register and the stack, one wholly past the
registers, and an __int128 whose two slots push the last parameter out.
The six-scalar case is the one whose slot count is known from the
parameters alone, so it takes the check btf_prepare_func_args() makes
before it walks them, while the other three take the one it makes after.
Both report the same way.
GCC passes an aggregate by invisible reference, so a callee it compiles
expects a pointer where BTF says the halves of the struct are, and those
tests are left to clang.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
.../selftests/bpf/prog_tests/aggregate_arg.c | 9 +
.../selftests/bpf/progs/aggregate_arg_func.c | 187 ++++++++++++++++++
2 files changed, 196 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
create mode 100644 tools/testing/selftests/bpf/progs/aggregate_arg_func.c
diff --git a/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c b/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
new file mode 100644
index 000000000000..b230f3bd3b2a
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "aggregate_arg_func.skel.h"
+
+void test_aggregate_arg(void)
+{
+ RUN_TESTS(aggregate_arg_func);
+}
diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_func.c b/tools/testing/selftests/bpf/progs/aggregate_arg_func.c
new file mode 100644
index 000000000000..ae3faa8a4a94
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_arg_func.c
@@ -0,0 +1,187 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+#ifdef __SIZEOF_INT128__
+typedef unsigned __int128 u128;
+#endif
+
+struct pair {
+ __u64 lo;
+ __u64 hi;
+};
+
+struct too_big {
+ __u64 a;
+ __u64 b;
+ __u64 c;
+};
+
+#if defined(__clang__)
+
+__noinline __u64 global_arg_pair(int a, struct pair p, int c)
+{
+ return (__u64)a + p.lo + p.hi + c;
+}
+
+SEC("tc")
+__success __retval(0x33)
+__naked int aggregate_arg_pair_asm(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 0x10;" /* p.lo */
+ "r3 = 0x20;" /* p.hi */
+ "r4 = 2;"
+ "call %[global_arg_pair];"
+ "exit;"
+ :
+ : __imm(global_arg_pair)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("R2 is not a scalar")
+__naked int aggregate_arg_pair_ptr_fail(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = r10;" /* a stack pointer where p.lo belongs */
+ "r3 = 0x20;"
+ "r4 = 2;"
+ "call %[global_arg_pair];"
+ "exit;"
+ :
+ : __imm(global_arg_pair)
+ : __clobber_all);
+}
+
+#endif
+
+__noinline __u64 global_arg_too_big(struct too_big s)
+{
+ return s.a + s.b + s.c;
+}
+
+SEC("tc")
+__failure __msg("in global_arg_too_big() has size 24, only 1 to 16 bytes can be passed by value")
+__naked int aggregate_arg_too_big_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "r3 = 0;"
+ "call %[global_arg_too_big];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_arg_too_big)
+ : __clobber_all);
+}
+
+#if defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+/* One slot per parameter, so the slot count is settled before the parameters
+ * are walked. The cases below reach the same count only once they have been.
+ */
+__noinline __u64 global_arg_six_scalars(int a, int b, int c, int d, int e, int f)
+{
+ return (__u64)a + b + c + d + e + f;
+}
+
+SEC("tc")
+__failure __msg("global function global_arg_six_scalars() needs 6 > 5 argument slots")
+__naked int aggregate_arg_six_scalars_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "r3 = 0;"
+ "r4 = 0;"
+ "r5 = 0;"
+ "call %[global_arg_six_scalars];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_arg_six_scalars)
+ : __clobber_all);
+}
+
+__noinline __u64 global_arg_split(int a, int b, int c, int d, struct pair p)
+{
+ return (__u64)a + b + c + d + p.lo + p.hi;
+}
+
+SEC("tc")
+__failure __msg("global function global_arg_split() needs 6 > 5 argument slots")
+__naked int aggregate_arg_split_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "r3 = 0;"
+ "r4 = 0;"
+ "r5 = 0;"
+ "call %[global_arg_split];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_arg_split)
+ : __clobber_all);
+}
+
+__noinline __u64 global_arg_past_regs(struct pair p, struct pair q, int a, struct pair r)
+{
+ return p.lo + p.hi + q.lo + q.hi + a + r.lo + r.hi;
+}
+
+SEC("tc")
+__failure __msg("global function global_arg_past_regs() needs 7 > 5 argument slots")
+__naked int aggregate_arg_past_regs_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "r3 = 0;"
+ "r4 = 0;"
+ "r5 = 0;"
+ "call %[global_arg_past_regs];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_arg_past_regs)
+ : __clobber_all);
+}
+
+#ifdef __SIZEOF_INT128__
+
+__noinline __u64 global_arg_i128_slots(u128 v, int a, int b, int c, int d)
+{
+ return (__u64)v + a + b + c + d;
+}
+
+SEC("tc")
+__failure __msg("global function global_arg_i128_slots() needs 6 > 5 argument slots")
+__naked int aggregate_arg_i128_slots_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "r3 = 0;"
+ "r4 = 0;"
+ "r5 = 0;"
+ "call %[global_arg_i128_slots];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_arg_i128_slots)
+ : __clobber_all);
+}
+
+#endif /* __SIZEOF_INT128__ */
+
+#endif
+
+char _license[] SEC("license") = "GPL";
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 13/15] selftests/bpf: Add inline-asm tests for by-value arguments
2026-09-11 15:50 ` [PATCH bpf-next v3 13/15] selftests/bpf: Add inline-asm tests for by-value arguments Yonghong Song
@ 2026-09-11 16:06 ` sashiko-bot
0 siblings, 0 replies; 32+ messages in thread
From: sashiko-bot @ 2026-09-11 16:06 UTC (permalink / raw)
To: Yonghong Song; +Cc: bpf
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Multi-line comment does not follow the required kernel BPF comment style.
--
commit d1826c8bebe9c02679f7bde2ef6a1536675d8bf2
Author: Yonghong Song <yonghong.song@linux.dev>
selftests/bpf: Add inline-asm tests for by-value arguments
This commit adds inline-asm tests to cover struct argument passing,
specifically for structs passed in register pairs, refused pointers, and
structs too large for by-value passing. It also checks various placements
that global functions cannot have.
> diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_func.c b/tools/testing/selftests/bpf/progs/aggregate_arg_func.c
> new file mode 100644
> index 0000000000000..ae3faa8a4a940
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/aggregate_arg_func.c
[ ... ]
> +#if defined(__BPF_FEATURE_STACK_ARGUMENT)
> +
> +/* One slot per parameter, so the slot count is settled before the parameters
> + * are walked. The cases below reach the same count only once they have been.
> + */
> +__noinline __u64 global_arg_six_scalars(int a, int b, int c, int d, int e, int f)
[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF
subsystem comment style? The guidelines require multi-line comments to have
the opening /* on its own line, with the text starting on the next line.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260911154914.2004336-1-yonghong.song@linux.dev?part=13
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH bpf-next v3 14/15] selftests/bpf: Add tests for by-value kfunc arguments
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (12 preceding siblings ...)
2026-09-11 15:50 ` [PATCH bpf-next v3 13/15] selftests/bpf: Add inline-asm tests for by-value arguments Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
2026-09-11 15:50 ` [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64 Yonghong Song
14 siblings, 1 reply; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
Add kfuncs taking a 16-byte struct and an __int128 by value, with
combinations of <= 8 byte arguments and '> 8 && <= 16' byte arguments.
Every test checks the value the kfunc returns, so a misplaced argument
shows up rather than passing quietly.
There are some cases with __int128 as the register argument. For such
arguments, arm64 requires the argument to start at an even slot while
x86-64 has no such requirement.
Two more cover a rejection. An aggregate holding a pointer is refused
everywhere, and in arena_kfunc.c a two-slot struct pushes an arena
pointer past the argument registers, which is refused too. An aggregate
too large to pass by value is already covered in aggregate_arg_func.c.
test_stack_arg_big() in stack_arg_fail.c passed a 16-byte struct as the
sixth argument and asserted the unrecognized stack argument type it used
to be reported as. The JIT places that argument now, so the test is
removed.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
.../selftests/bpf/prog_tests/aggregate_arg.c | 2 +
.../selftests/bpf/progs/aggregate_arg_kfunc.c | 231 ++++++++++++++++++
.../testing/selftests/bpf/progs/arena_kfunc.c | 15 ++
.../selftests/bpf/progs/stack_arg_fail.c | 10 -
.../selftests/bpf/test_kmods/bpf_testmod.c | 77 ++++++
.../bpf/test_kmods/bpf_testmod_kfunc.h | 32 +++
6 files changed, 357 insertions(+), 10 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
diff --git a/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c b/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
index b230f3bd3b2a..aa7562f48737 100644
--- a/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
+++ b/tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
@@ -2,8 +2,10 @@
/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
#include <test_progs.h>
#include "aggregate_arg_func.skel.h"
+#include "aggregate_arg_kfunc.skel.h"
void test_aggregate_arg(void)
{
RUN_TESTS(aggregate_arg_func);
+ RUN_TESTS(aggregate_arg_kfunc);
}
diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
new file mode 100644
index 000000000000..2100f0ab1abe
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+#include "bpf_misc.h"
+
+#ifdef __SIZEOF_INT128__
+typedef unsigned __int128 u128;
+#endif
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+#if defined(__clang__)
+
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_struct(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_pair_arg s = { .lo = a, .hi = b };
+
+ if (bpf_kfunc_call_test_pair_arg(1, s, 2) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+#endif
+
+#ifdef __SIZEOF_INT128__
+
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_int128(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v = ((u128)a << 64) | b;
+
+ if (bpf_kfunc_call_test_i128_arg(1, 2, v) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+/*
+ * arm64 rounds the register number up to an even one for an argument
+ * aligned to 16 bytes, so it wants this __int128 in x2 and x3.
+ * The x86-64 ABI has no such rule.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_int128_odd(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v = ((u128)a << 64) | b;
+
+ if (bpf_kfunc_call_test_i128_arg_odd(1, v, 2) != a + b + 3)
+ return 1;
+
+ return 0;
+}
+
+#endif /* __SIZEOF_INT128__ */
+
+#if defined(__clang__) && defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_last_regs(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_pair_arg s = { .lo = a, .hi = b };
+
+ if (bpf_kfunc_call_test_pair_arg_nofit(1, 2, 3, 4, s) != a + b + 10)
+ return 1;
+
+ return 0;
+}
+
+/*
+ * The x86-64 ABI moves an argument its six remaining registers cannot hold
+ * wholly onto the stack. arm64, with eight argument registers, still has a
+ * pair for it.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_straddle(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_big_arg s = { .a = a, .b = b };
+
+ if (bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s) != a + b + 15)
+ return 1;
+
+ return 0;
+}
+
+/* The same, with an argument after the struct to take the eighth register. */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_tail(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_pair_arg s = { .lo = a, .hi = b };
+
+ if (bpf_kfunc_call_test_pair_arg_tail(1, 2, 3, 4, 5, s, 6) != a + b + 21)
+ return 1;
+
+ return 0;
+}
+
+/*
+ * arm64 gives no register to an argument its eight registers cannot hold,
+ * nor to anything after it. Past its six registers the x86-64 ABI has both
+ * eightbytes on the stack either way.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_split8(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_pair_arg s = { .lo = a, .hi = b };
+
+ if (bpf_kfunc_call_test_pair_arg_split8(1, 2, 3, 4, 5, 6, 7, s) != a + b + 28)
+ return 1;
+
+ return 0;
+}
+
+#ifdef __SIZEOF_INT128__
+
+/*
+ * The same hole, with enough arguments after the __int128 that the shift
+ * reaches the registers the BPF convention counts as stack slots: arm64
+ * wants the last one in x6 where the BPF convention put it in x5.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_int128_shift(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v = ((u128)a << 64) | b;
+
+ if (bpf_kfunc_call_test_i128_arg_shift(1, v, 2, 3, 4) != a + b + 10)
+ return 1;
+
+ return 0;
+}
+
+/*
+ * One argument further and the hole pushes the last one off x7, so the JIT
+ * shifts it into the first arm64 stack slot. The x86-64 ABI packs the
+ * eightbytes, so its last two are on the stack where BPF put them.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_int128_ovf(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v = ((u128)a << 64) | b;
+
+ if (bpf_kfunc_call_test_i128_arg_ovf(1, v, 2, 3, 4, 5, 6) != a + b + 21)
+ return 1;
+
+ return 0;
+}
+
+/*
+ * Both conventions pad the stack to align this __int128, and the BPF
+ * convention pads for neither, so both JITs move it up an eightbyte.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_kfunc_int128_pad(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v = ((u128)a << 64) | b;
+
+ if (bpf_kfunc_call_test_i128_arg_pad(1, 2, 3, 4, 5, 6, 7, v) != a + b + 28)
+ return 1;
+
+ return 0;
+}
+
+#endif /* __SIZEOF_INT128__ */
+
+#endif
+
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__failure __msg("R1 type STRUCT is not composed of scalars")
+int aggregate_arg_kfunc_ptr_member(struct __sk_buff *skb)
+{
+ struct prog_test_ptr_arg s = { .p = skb, .x = 1 };
+
+ return bpf_kfunc_call_test_ptr_arg(s);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/arena_kfunc.c b/tools/testing/selftests/bpf/progs/arena_kfunc.c
index 50609f3b0564..7302d279ead2 100644
--- a/tools/testing/selftests/bpf/progs/arena_kfunc.c
+++ b/tools/testing/selftests/bpf/progs/arena_kfunc.c
@@ -228,6 +228,21 @@ int arena_arg_stack(void *ctx)
bpf_kfunc_arena_stack_arg_test(1, 2, 3, 4, 5, (u64 *)1);
return 0;
}
+
+#if defined(__clang__)
+/* The struct takes two slots, so the arena pointer is the sixth. */
+SEC("syscall")
+__arch_x86_64 __arch_arm64
+__failure __msg("arena pointer cannot be a stack argument")
+int arena_arg_stack_after_pair(void *ctx)
+{
+ struct prog_test_pair_arg s = { .lo = 1, .hi = 2 };
+
+ bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+ bpf_kfunc_call_test_pair_arena_arg(1, 2, 3, s, (u64 *)1);
+ return 0;
+}
+#endif
#else
SEC("syscall")
__arch_x86_64
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_fail.c b/tools/testing/selftests/bpf/progs/stack_arg_fail.c
index eed97d582515..fff2e947ea33 100644
--- a/tools/testing/selftests/bpf/progs/stack_arg_fail.c
+++ b/tools/testing/selftests/bpf/progs/stack_arg_fail.c
@@ -3,20 +3,10 @@
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
-#include "../test_kmods/bpf_testmod_kfunc.h"
#include "bpf_misc.h"
#if defined(__BPF_FEATURE_STACK_ARGUMENT)
-SEC("tc")
-__failure __msg("Unrecognized *(R11-8) type STRUCT")
-int test_stack_arg_big(struct __sk_buff *skb)
-{
- struct prog_test_big_arg s = { .a = 1, .b = 2 };
-
- return bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s);
-}
-
SEC("socket")
__description("r11 in ALU instruction")
__failure __msg("R11 is invalid")
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index f798bbbb4d13..baccee6fdad4 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -1029,6 +1029,72 @@ __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u6
return r;
}
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg(u64 a, struct prog_test_pair_arg s, u64 b)
+{
+ return a + s.lo + s.hi + b;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg(u64 a, u64 b, __int128 v)
+{
+ return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_odd(u64 a, __int128 v, u64 b)
+{
+ return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_shift(u64 a, __int128 v, u64 b, u64 c,
+ u64 d)
+{
+ return a + b + c + d + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_ovf(u64 a, __int128 v, u64 b, u64 c,
+ u64 d, u64 e, u64 f)
+{
+ return a + b + c + d + e + f +
+ (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_pad(u64 a, u64 b, u64 c, u64 d, u64 e,
+ u64 f, u64 g, __int128 v)
+{
+ return a + b + c + d + e + f + g +
+ (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg_nofit(u64 a, u64 b, u64 c, u64 d,
+ struct prog_test_pair_arg s)
+{
+ return a + b + c + d + s.lo + s.hi;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg_tail(u64 a, u64 b, u64 c, u64 d, u64 e,
+ struct prog_test_pair_arg s, u64 f)
+{
+ return a + b + c + d + e + s.lo + s.hi + f;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg_split8(u64 a, u64 b, u64 c, u64 d, u64 e,
+ u64 f, u64 g,
+ struct prog_test_pair_arg s)
+{
+ return a + b + c + d + e + f + g + s.lo + s.hi;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_ptr_arg(struct prog_test_ptr_arg s)
+{
+ return s.x;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arena_arg(u64 a, u64 b, u64 c,
+ struct prog_test_pair_arg s,
+ u64 *f__arena)
+{
+ return a + b + c + s.lo + s.hi + *f__arena;
+}
+
__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
{
struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
@@ -1667,6 +1733,17 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_arr_struct)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_arr2d)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_deep)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_odd)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_shift)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_ovf)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_pad)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_nofit)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_tail)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_split8)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ptr_arg)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arena_arg)
#endif
BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_big)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index b213ef14848b..195ec37d5bbc 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -61,6 +61,16 @@ struct prog_test_big_arg {
__u64 b;
};
+struct prog_test_pair_arg { /* 16 bytes: two argument registers */
+ __u64 lo;
+ __u64 hi;
+};
+
+struct prog_test_ptr_arg { /* 16 bytes, but holds a pointer */
+ void *p;
+ __u64 x;
+};
+
struct prog_test_ret_pair { /* 16 bytes: R0:R2 */
__u64 lo;
__u64 hi;
@@ -221,6 +231,28 @@ __int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;
struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
+__u64 bpf_kfunc_call_test_pair_arg(__u64 a, struct prog_test_pair_arg s, __u64 b) __ksym;
+#ifdef __SIZEOF_INT128__
+__u64 bpf_kfunc_call_test_i128_arg(__u64 a, __u64 b, __int128 v) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
+ __u64 d) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
+ __u64 d, __u64 e, __u64 f) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_pad(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+ __u64 f, __u64 g, __int128 v) __ksym;
+#endif
+__u64 bpf_kfunc_call_test_pair_arg_nofit(__u64 a, __u64 b, __u64 c, __u64 d,
+ struct prog_test_pair_arg s) __ksym;
+__u64 bpf_kfunc_call_test_pair_arg_tail(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+ struct prog_test_pair_arg s, __u64 f) __ksym;
+__u64 bpf_kfunc_call_test_pair_arg_split8(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+ __u64 f, __u64 g,
+ struct prog_test_pair_arg s) __ksym;
+__u64 bpf_kfunc_call_test_ptr_arg(struct prog_test_ptr_arg s) __ksym;
+__u64 bpf_kfunc_call_test_pair_arena_arg(__u64 a, __u64 b, __u64 c,
+ struct prog_test_pair_arg s,
+ __u64 *f__arena) __ksym;
struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;
struct prog_test_ret_nested bpf_kfunc_call_test_ret_nested(__u64 tag) __ksym;
struct prog_test_ret_ptr_arr bpf_kfunc_call_test_ret_ptr_arr(void) __ksym;
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 14/15] selftests/bpf: Add tests for by-value kfunc arguments
2026-09-11 15:50 ` [PATCH bpf-next v3 14/15] selftests/bpf: Add tests for by-value kfunc arguments Yonghong Song
@ 2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 17:24 ` Yonghong Song
0 siblings, 1 reply; 32+ messages in thread
From: bot+bpf-ci @ 2026-09-11 16:47 UTC (permalink / raw)
To: yonghong.song, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2385 bytes --]
> diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
> new file mode 100644
> index 000000000000..8b0e7aed1ff1
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
[ ... ]
> +SEC("tc")
> +__arch_x86_64 __arch_arm64 __load_if_JITed()
> +__description("aggregate kfunc argument: struct at the tail")
> +__success __retval(0)
> +int aggregate_arg_kfunc_tail(struct __sk_buff *skb)
> +{
> + u64 a = 1, b = 2;
> + struct prog_test_pair_arg s = { .lo = a, .hi = b };
> +
> + if (bpf_kfunc_call_test_pair_arg_tail(1, 2, 3, 4, 5, s, 6) != a + b + 21)
> + return 1;
> + return 0;
> +}
Do these tests actually verify that arguments end up in the correct slots?
Looking at the test pattern: bpf_kfunc_call_test_pair_arg_tail(1, 2, 3,
4, 5, s, 6) expects 'a + b + 21', but the kfunc returns the sum of all
its arguments unweighted:
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c:
__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg_tail(
u64 a, u64 b, u64 c, u64 d, u64 e,
struct prog_test_pair_arg s, u64 f)
{
return a + b + c + d + e + s.lo + s.hi + f;
}
The check is invariant under any permutation of values across argument
slots. It will catch a value that is lost, duplicated or garbage, but
not a value that ends up in the wrong slot.
Concrete case in this test: aggregate_arg_kfunc_tail is the one case
where x86-64 moves an argument down into a still-live register (BPF
slots a..e -> 0..4, s -> 6,7, f -> 5, so f travels through
BPF_JIT_ARG_TMP). If the scratch handling swapped f with s.lo, the
callee would read s.lo = 6, f = a, and 1+2+3+4+5 + 6 + b + a still
equals a + b + 21, so the test passes.
The same holds for the two halves of every 16-byte argument:
'(u64)(v >> 64) + (u64)v' and 's.lo + s.hi' are symmetric, so a hi/lo
swap of an __int128 or of prog_test_pair_arg is invisible in all nine
value tests.
The commit message states "Every test checks the value the kfunc
returns, so a misplaced argument shows up rather than passing quietly."
Does this hold if the values are commutative?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 14/15] selftests/bpf: Add tests for by-value kfunc arguments
2026-09-11 16:47 ` bot+bpf-ci
@ 2026-09-12 17:24 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:24 UTC (permalink / raw)
To: bot+bpf-ci, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, martin.lau, mason,
ihor.solodrai
On 9/11/26 9:47 AM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
>> new file mode 100644
>> index 000000000000..8b0e7aed1ff1
>> --- /dev/null
>> +++ b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
> [ ... ]
>
>> +SEC("tc")
>> +__arch_x86_64 __arch_arm64 __load_if_JITed()
>> +__description("aggregate kfunc argument: struct at the tail")
>> +__success __retval(0)
>> +int aggregate_arg_kfunc_tail(struct __sk_buff *skb)
>> +{
>> + u64 a = 1, b = 2;
>> + struct prog_test_pair_arg s = { .lo = a, .hi = b };
>> +
>> + if (bpf_kfunc_call_test_pair_arg_tail(1, 2, 3, 4, 5, s, 6) != a + b + 21)
>> + return 1;
>> + return 0;
>> +}
> Do these tests actually verify that arguments end up in the correct slots?
>
> Looking at the test pattern: bpf_kfunc_call_test_pair_arg_tail(1, 2, 3,
> 4, 5, s, 6) expects 'a + b + 21', but the kfunc returns the sum of all
> its arguments unweighted:
>
> tools/testing/selftests/bpf/test_kmods/bpf_testmod.c:
> __bpf_kfunc u64 bpf_kfunc_call_test_pair_arg_tail(
> u64 a, u64 b, u64 c, u64 d, u64 e,
> struct prog_test_pair_arg s, u64 f)
> {
> return a + b + c + d + e + s.lo + s.hi + f;
> }
>
> The check is invariant under any permutation of values across argument
> slots. It will catch a value that is lost, duplicated or garbage, but
> not a value that ends up in the wrong slot.
>
> Concrete case in this test: aggregate_arg_kfunc_tail is the one case
> where x86-64 moves an argument down into a still-live register (BPF
> slots a..e -> 0..4, s -> 6,7, f -> 5, so f travels through
> BPF_JIT_ARG_TMP). If the scratch handling swapped f with s.lo, the
> callee would read s.lo = 6, f = a, and 1+2+3+4+5 + 6 + b + a still
> equals a + b + 21, so the test passes.
>
> The same holds for the two halves of every 16-byte argument:
> '(u64)(v >> 64) + (u64)v' and 's.lo + s.hi' are symmetric, so a hi/lo
> swap of an __int128 or of prog_test_pair_arg is invisible in all nine
> value tests.
>
> The commit message states "Every test checks the value the kfunc
> returns, so a misplaced argument shows up rather than passing quietly."
> Does this hold if the values are commutative?
I will change kfunc to have different values with different multiplier
for each argument to make the test more robust.
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64
2026-09-11 15:49 [PATCH bpf-next v3 00/15] bpf: Support by-value struct and __int128 arguments Yonghong Song
` (13 preceding siblings ...)
2026-09-11 15:50 ` [PATCH bpf-next v3 14/15] selftests/bpf: Add tests for by-value kfunc arguments Yonghong Song
@ 2026-09-11 15:50 ` Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 3:57 ` Alexei Starovoitov
14 siblings, 2 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-11 15:50 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
There are 3 kfuncs like below:
__u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
__u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
__u64 d) __ksym;
__u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
__u64 d, __u64 e, __u64 f) __ksym;
which requires that '__int128 v' must be 16-byte align on arm64.
Current pahole will reject BTF generation since pahole expects
'__int128 v' has start register 'x1' while arm64 abi requires 'x2'.
Hence, btf generation will fail.
This patch is a hack to disable a few related tests to satisfy CI.
The following is the fix in pahole:
https://lore.kernel.org/bpf/20260911040955.339939-1-yonghong.song@linux.dev/
Once pahole patch is merged, this patch can be discarded.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c | 4 ++++
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c | 4 ++++
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h | 2 ++
3 files changed, 10 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
index 2100f0ab1abe..59f56ab39413 100644
--- a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
+++ b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
@@ -50,6 +50,7 @@ int aggregate_arg_kfunc_int128(struct __sk_buff *skb)
return 0;
}
+#if 0
/*
* arm64 rounds the register number up to an even one for an argument
* aligned to 16 bytes, so it wants this __int128 in x2 and x3.
@@ -70,6 +71,7 @@ int aggregate_arg_kfunc_int128_odd(struct __sk_buff *skb)
return 0;
}
+#endif
#endif /* __SIZEOF_INT128__ */
@@ -152,6 +154,7 @@ int aggregate_arg_kfunc_split8(struct __sk_buff *skb)
#ifdef __SIZEOF_INT128__
+#if 0
/*
* The same hole, with enough arguments after the __int128 that the shift
* reaches the registers the BPF convention counts as stack slots: arm64
@@ -193,6 +196,7 @@ int aggregate_arg_kfunc_int128_ovf(struct __sk_buff *skb)
return 0;
}
+#endif
/*
* Both conventions pad the stack to align this __int128, and the BPF
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index baccee6fdad4..c367442ce086 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -1039,6 +1039,7 @@ __bpf_kfunc u64 bpf_kfunc_call_test_i128_arg(u64 a, u64 b, __int128 v)
return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
}
+#if 0
__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_odd(u64 a, __int128 v, u64 b)
{
return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
@@ -1056,6 +1057,7 @@ __bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_ovf(u64 a, __int128 v, u64 b, u64 c
return a + b + c + d + e + f +
(u64)((unsigned __int128)v >> 64) + (u64)v;
}
+#endif
__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_pad(u64 a, u64 b, u64 c, u64 d, u64 e,
u64 f, u64 g, __int128 v)
@@ -1735,9 +1737,11 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_deep)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg)
+#if 0
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_odd)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_shift)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_ovf)
+#endif
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_pad)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_nofit)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_tail)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index 195ec37d5bbc..0f098045856e 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -234,11 +234,13 @@ struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
__u64 bpf_kfunc_call_test_pair_arg(__u64 a, struct prog_test_pair_arg s, __u64 b) __ksym;
#ifdef __SIZEOF_INT128__
__u64 bpf_kfunc_call_test_i128_arg(__u64 a, __u64 b, __int128 v) __ksym;
+#if 0
__u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
__u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
__u64 d) __ksym;
__u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
__u64 d, __u64 e, __u64 f) __ksym;
+#endif
__u64 bpf_kfunc_call_test_i128_arg_pad(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
__u64 f, __u64 g, __int128 v) __ksym;
#endif
--
2.52.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64
2026-09-11 15:50 ` [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64 Yonghong Song
@ 2026-09-11 16:47 ` bot+bpf-ci
2026-09-12 3:57 ` Alexei Starovoitov
1 sibling, 0 replies; 32+ messages in thread
From: bot+bpf-ci @ 2026-09-11 16:47 UTC (permalink / raw)
To: yonghong.song, bpf
Cc: ast, andrii, daniel, eddyz87, kernel-team, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 7371 bytes --]
> selftests/bpf: Temporary hack to disable register mismatch in arm64
>
> There are 3 kfuncs like below:
> __u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
> __u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
> __u64 d) __ksym;
> __u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
> __u64 d, __u64 e, __u64 f) __ksym;
> which requires that '__int128 v' must be 16-byte align on arm64.
>
> Current pahole will reject BTF generation since pahole expects
> '__int128 v' has start register 'x1' while arm64 abi requires 'x2'.
> Hence, btf generation will fail.
>
> This patch is a hack to disable a few related tests to satisfy CI.
> The following is the fix in pahole:
> https://lore.kernel.org/bpf/20260911040955.339939-1-yonghong.song@linux.dev/
> Once pahole patch is merged, this patch can be discarded.
>
> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
The changelog states the patch is a temporary hack that 'can be
discarded' once the pahole fix is merged, but the patch is submitted as
the last patch of the series with no marker keeping it out of a merge.
Should this have [DO NOT APPLY] or RFC in the subject, or a note under
the --- line where non-applied notes belong?
A few smaller notes: 'must be 16-byte align' should read '16-byte
aligned', and the pahole reference would normally be a Link: tag.
> diff --git a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
> index 2100f0ab1abe..59f56ab39413 100644
> --- a/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
> +++ b/tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
> @@ -50,6 +50,7 @@ int aggregate_arg_kfunc_int128(struct __sk_buff *skb)
> return 0;
> }
>
> +#if 0
> /*
> * arm64 rounds the register number up to an even one for an argument
> * aligned to 16 bytes, so it wants this __int128 in x2 and x3.
Five #if 0 regions are added and none carries an in-tree comment
explaining the disable. The reason (the pahole dependency), the fact
that it's meant to be reverted, and the link to the pahole fix exist
only in the commit message. Once applied, the tree contains unexplained
dead code with nothing that would prompt anyone to re-enable it after
the pahole fix lands. Should each #if 0 have a short comment or TODO
naming the toolchain dependency, or could these use a
version/arch-conditional guard rather than a blanket #if 0?
> @@ -70,6 +71,7 @@ int aggregate_arg_kfunc_int128_odd(struct __sk_buff *skb)
>
> return 0;
> }
> +#endif
>
> #endif /* __SIZEOF_INT128__ */
>
> @@ -152,6 +154,7 @@ int aggregate_arg_kfunc_split8(struct __sk_buff *skb)
>
> #ifdef __SIZEOF_INT128__
>
> +#if 0
> /*
> * The same hole, with enough arguments after the __int128 that the shift
> * reaches the registers the BPF convention counts as stack slots: arm64
The three programs disabled here (aggregate_arg_kfunc_int128_odd,
aggregate_arg_kfunc_int128_shift, and aggregate_arg_kfunc_int128_ovf)
are the only tests that observably exercise the arm64 even-register
round-up added by 7189ec8b9091 in this same series.
Looking at bpf_jit_place_args() in kernel/bpf/core.c, it applies the
round-up only for args flagged BTF_FMODEL_ALIGN16_ARG, i.e. __int128 by
value:
if (align16 && abi->even_reg_align)
nregs_used = round_up(nregs_used, 2);
arm64_arg_abi sets .even_reg_align = true. Of the __int128 kfunc tests
that remain enabled:
- aggregate_arg_kfunc_int128 -> bpf_kfunc_call_test_i128_arg(u64, u64,
__int128): nregs_used is 2 at the __int128, round_up(2, 2) == 2, so
placement is the identity and emit_kfunc_arg_moves() emits nothing.
- aggregate_arg_kfunc_int128_pad ->
bpf_kfunc_call_test_i128_arg_pad(7 x u64, __int128): nregs_used is
7, round_up(7, 2) == 8, but 7 + 2 > 8 already forces the argument
wholly onto the stack, so the round-up changes nothing observable;
only the even_stack_align path is covered.
After this commit, does the arm64 register-shift path (the headline
behaviour of the arm64 patch in this series) and the
alignment-hole-pushes-an-argument-to-the-stack path have any test
coverage on any architecture?
The surrounding code already uses #if defined(__x86_64__) ||
defined(__aarch64__) (bpf_testmod.c and earlier in the same file), and
BPF progs are built with -D__TARGET_ARCH_$(SRCARCH). Would an arm64-only
guard (plus dropping __arch_arm64 from the affected programs) keep the
x86-64 arm of these tests running?
> @@ -193,6 +196,7 @@ int aggregate_arg_kfunc_int128_ovf(struct __sk_buff *skb)
>
> return 0;
> }
> +#endif
>
> /*
> * Both conventions pad the stack to align this __int128, and the BPF
> diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
> index baccee6fdad4..c367442ce086 100644
> --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
> +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
> @@ -1039,6 +1039,7 @@ __bpf_kfunc u64 bpf_kfunc_call_test_i128_arg(u64 a, u64 b, __int128 v)
> return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
> }
>
> +#if 0
> __bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_odd(u64 a, __int128 v, u64 b)
> {
> return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
[ ... ]
> @@ -1735,9 +1737,11 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_deep)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg)
> +#if 0
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_odd)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_shift)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_ovf)
> +#endif
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_pad)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_nofit)
> BTF_ID_FLAGS(func, bpf_kfunc_call_test_pair_arg_tail)
> diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
> index 195ec37d5bbc..0f098045856e 100644
> --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
> +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
> @@ -234,11 +234,13 @@ struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
> __u64 bpf_kfunc_call_test_pair_arg(__u64 a, struct prog_test_pair_arg s, __u64 b) __ksym;
> #ifdef __SIZEOF_INT128__
> __u64 bpf_kfunc_call_test_i128_arg(__u64 a, __u64 b, __int128 v) __ksym;
> +#if 0
> __u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
> __u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
> __u64 d) __ksym;
> __u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
> __u64 d, __u64 e, __u64 f) __ksym;
> +#endif
> __u64 bpf_kfunc_call_test_i128_arg_pad(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
> __u64 f, __u64 g, __int128 v) __ksym;
> #endif
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34620351527
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64
2026-09-11 15:50 ` [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64 Yonghong Song
2026-09-11 16:47 ` bot+bpf-ci
@ 2026-09-12 3:57 ` Alexei Starovoitov
2026-09-12 17:30 ` Yonghong Song
1 sibling, 1 reply; 32+ messages in thread
From: Alexei Starovoitov @ 2026-09-12 3:57 UTC (permalink / raw)
To: Yonghong Song, bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
On Fri Sep 11, 2026 at 8:50 AM PDT, Yonghong Song wrote:
> There are 3 kfuncs like below:
> __u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
> __u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
> __u64 d) __ksym;
> __u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
> __u64 d, __u64 e, __u64 f) __ksym;
> which requires that '__int128 v' must be 16-byte align on arm64.
>
> Current pahole will reject BTF generation since pahole expects
> '__int128 v' has start register 'x1' while arm64 abi requires 'x2'.
> Hence, btf generation will fail.
>
> This patch is a hack to disable a few related tests to satisfy CI.
> The following is the fix in pahole:
> https://lore.kernel.org/bpf/20260911040955.339939-1-yonghong.song@linux.dev/
> Once pahole patch is merged, this patch can be discarded.
You're adding them in patch 14 only to disable them in patch 15?
sure, pahole needs to be fixed, but let's carry such selftests for pahole
out of tree of the time being.
Once it is fixed and we have a mechanism to detect that it is fixed
then we will add these tests.
I'm not sure what would be such pahole detection mechanism.
We don't want to bump the version just for that.
So imo just drop such tests.
Also rebase to bpf-next is needed.
pw-bot: cr
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH bpf-next v3 15/15] selftests/bpf: Temporary hack to disable register mismatch in arm64
2026-09-12 3:57 ` Alexei Starovoitov
@ 2026-09-12 17:30 ` Yonghong Song
0 siblings, 0 replies; 32+ messages in thread
From: Yonghong Song @ 2026-09-12 17:30 UTC (permalink / raw)
To: Alexei Starovoitov, bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, kernel-team
On 9/11/26 8:57 PM, Alexei Starovoitov wrote:
> On Fri Sep 11, 2026 at 8:50 AM PDT, Yonghong Song wrote:
>> There are 3 kfuncs like below:
>> __u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
>> __u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
>> __u64 d) __ksym;
>> __u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
>> __u64 d, __u64 e, __u64 f) __ksym;
>> which requires that '__int128 v' must be 16-byte align on arm64.
>>
>> Current pahole will reject BTF generation since pahole expects
>> '__int128 v' has start register 'x1' while arm64 abi requires 'x2'.
>> Hence, btf generation will fail.
>>
>> This patch is a hack to disable a few related tests to satisfy CI.
>> The following is the fix in pahole:
>> https://lore.kernel.org/bpf/20260911040955.339939-1-yonghong.song@linux.dev/
>> Once pahole patch is merged, this patch can be discarded.
> You're adding them in patch 14 only to disable them in patch 15?
> sure, pahole needs to be fixed, but let's carry such selftests for pahole
> out of tree of the time being.
> Once it is fixed and we have a mechanism to detect that it is fixed
> then we will add these tests.
> I'm not sure what would be such pahole detection mechanism.
> We don't want to bump the version just for that.
> So imo just drop such tests.
>
> Also rebase to bpf-next is needed.
Okay, will remove patch 15 and remove the tests mentioned in patch 15.
Also address with other comments.
And will rebase and re-submit.
>
> pw-bot: cr
^ permalink raw reply [flat|nested] 32+ messages in thread