bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH bpf-next 0/2] bpf, arm64: support for timed may_goto
@ 2025-07-24 12:54 Puranjay Mohan
  2025-07-24 12:54 ` [PATCH bpf-next 1/2] bpf, arm64: Add JIT " Puranjay Mohan
  2025-07-24 12:54 ` [PATCH bpf-next 2/2] selftests/bpf: Enable timed may_goto tests for arm64 Puranjay Mohan
  0 siblings, 2 replies; 5+ messages in thread
From: Puranjay Mohan @ 2025-07-24 12:54 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
	John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
	Puranjay Mohan, Xu Kuohai, Catalin Marinas, Will Deacon,
	Mykola Lysenko, Kumar Kartikeya Dwivedi, bpf

This set adds support for the timed may_goto instruction for the arm64.
The timed may_goto instruction is implemented by the verifier by
reserving 2 8byte slots in the program stack and then calling
arch_bpf_timed_may_goto() in a loop with the stack offset of these two
slots in BPF_REG_AX. It expects the function to put a timestamp in the
first slot and the returned count in BPF_REG_AX is put into the second
slot by a store instruction emitted by the verifier.

arch_bpf_timed_may_goto() is special as it receives the parameter in
BPF_REG_AX and is expected to return the result in BPF_REG_AX as well.
It can't clobber any caller saved registers because verifier doesn't
save anything before emitting the call.

So, arch_bpf_timed_may_goto() is implemented in assembly so the exact
registers that are stored/restored can be controlled (BPF caller saved
registers here) and it also needs to take care of moving arguments and
return values to and from BPF_REG_AX <-> arm64 R0. 

So, arch_bpf_timed_may_goto() acts as a trampoline to call
bpf_check_timed_may_goto() which does the main logic of placing the
timestamp and returning the count.

All tests that use may_goto instruction pass after the changing some of
them in patch 2

 #404     stream_errors:OK
 [...]
 #406/2   stream_success/stream_cond_break:OK
 [...]
 #494/23  verifier_bpf_fastcall/may_goto_interaction_x86_64:SKIP
 #494/24  verifier_bpf_fastcall/may_goto_interaction_arm64:OK
 [...]
 #539/1   verifier_may_goto_1/may_goto 0:OK
 #539/2   verifier_may_goto_1/batch 2 of may_goto 0:OK
 #539/3   verifier_may_goto_1/may_goto batch with offsets 2/1/0:OK
 #539/4   verifier_may_goto_1/may_goto batch with offsets 2/0:OK
 #539     verifier_may_goto_1:OK
 #540/1   verifier_may_goto_2/C code with may_goto 0:OK
 #540     verifier_may_goto_2:OK
 Summary: 7/16 PASSED, 25 SKIPPED, 0 FAILED

Puranjay Mohan (2):
  bpf, arm64: Add JIT support for timed may_goto
  selftests/bpf: Enable timed may_goto tests for arm64

 arch/arm64/net/Makefile                       |  2 +-
 arch/arm64/net/bpf_jit_comp.c                 | 13 ++++++-
 arch/arm64/net/bpf_timed_may_goto.S           | 36 +++++++++++++++++++
 .../testing/selftests/bpf/prog_tests/stream.c |  2 +-
 .../bpf/progs/verifier_bpf_fastcall.c         | 27 ++++++++------
 .../selftests/bpf/progs/verifier_may_goto_1.c | 34 ++++--------------
 6 files changed, 72 insertions(+), 42 deletions(-)
 create mode 100644 arch/arm64/net/bpf_timed_may_goto.S

-- 
2.47.3


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH bpf-next 1/2] bpf, arm64: Add JIT support for timed may_goto
  2025-07-24 12:54 [PATCH bpf-next 0/2] bpf, arm64: support for timed may_goto Puranjay Mohan
@ 2025-07-24 12:54 ` Puranjay Mohan
  2025-08-05 11:28   ` Xu Kuohai
  2025-07-24 12:54 ` [PATCH bpf-next 2/2] selftests/bpf: Enable timed may_goto tests for arm64 Puranjay Mohan
  1 sibling, 1 reply; 5+ messages in thread
From: Puranjay Mohan @ 2025-07-24 12:54 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
	John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
	Puranjay Mohan, Xu Kuohai, Catalin Marinas, Will Deacon,
	Mykola Lysenko, Kumar Kartikeya Dwivedi, bpf

When verifier sees a timed may_goto instruction, it emits a call to
arch_bpf_timed_may_goto() with a stack offset in BPF_REG_AX (arm64 r9)
and expects a count value to be returned in the same register. The
verifier doesn't save or restore any registers before emitting this
call.

arch_bpf_timed_may_goto() should act as a trampoline to call
bpf_check_timed_may_goto() with AAPCS64 calling convention.

To support this custom calling convention, implement
arch_bpf_timed_may_goto() in assembly and make sure BPF caller saved
registers are saved and restored, call bpf_check_timed_may_goto with
arm64 calling convention where first argument and return value both are
in x0, then put the result back into BPF_REG_AX before returning.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
 arch/arm64/net/Makefile             |  2 +-
 arch/arm64/net/bpf_jit_comp.c       | 13 ++++++++++-
 arch/arm64/net/bpf_timed_may_goto.S | 36 +++++++++++++++++++++++++++++
 3 files changed, 49 insertions(+), 2 deletions(-)
 create mode 100644 arch/arm64/net/bpf_timed_may_goto.S

diff --git a/arch/arm64/net/Makefile b/arch/arm64/net/Makefile
index 5c540efb7d9b9..3ae382bfca879 100644
--- a/arch/arm64/net/Makefile
+++ b/arch/arm64/net/Makefile
@@ -2,4 +2,4 @@
 #
 # ARM64 networking code
 #
-obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o
+obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o bpf_timed_may_goto.o
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 89b1b8c248c62..6c954b36f57ea 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1505,7 +1505,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
 		if (ret < 0)
 			return ret;
 		emit_call(func_addr, ctx);
-		emit(A64_MOV(1, r0, A64_R(0)), ctx);
+		/*
+		 * Call to arch_bpf_timed_may_goto() is emitted by the
+		 * verifier and called with custom calling convention with
+		 * first argument and return value in BPF_REG_AX (x9).
+		 */
+		if (func_addr != (u64)arch_bpf_timed_may_goto)
+			emit(A64_MOV(1, r0, A64_R(0)), ctx);
 		break;
 	}
 	/* tail call */
@@ -2914,6 +2920,11 @@ bool bpf_jit_bypass_spec_v4(void)
 	return true;
 }
 
+bool bpf_jit_supports_timed_may_goto(void)
+{
+	return true;
+}
+
 bool bpf_jit_inlines_helper_call(s32 imm)
 {
 	switch (imm) {
diff --git a/arch/arm64/net/bpf_timed_may_goto.S b/arch/arm64/net/bpf_timed_may_goto.S
new file mode 100644
index 0000000000000..45f80e752345c
--- /dev/null
+++ b/arch/arm64/net/bpf_timed_may_goto.S
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2025 Puranjay Mohan <puranjay@kernel.org> */
+
+#include <linux/linkage.h>
+
+SYM_FUNC_START(arch_bpf_timed_may_goto)
+	/* Allocate stack space and emit frame record */
+	stp     x29, x30, [sp, #-64]!
+	mov     x29, sp
+
+	/* Save BPF registers R0 - R5 (x7, x0-x4)*/
+	stp	x7, x0, [sp, #16]
+	stp	x1, x2, [sp, #32]
+	stp	x3, x4, [sp, #48]
+
+	/*
+	 * Stack depth was passed in BPF_REG_AX (x9), add it to the BPF_FP
+	 * (x25) to get the pointer to count and timestamp and pass it as the
+	 * first argument in x0.
+	 */
+	add	x0, x9, x25
+	bl	bpf_check_timed_may_goto
+	/* BPF_REG_AX(x9) will be stored into count, so move return value to it. */
+	mov	x9, x0
+
+
+	/* Restore BPF registers R0 - R5 (x7, x0-x4) */
+	ldp	x7, x0, [sp, #16]
+	ldp	x1, x2, [sp, #32]
+	ldp	x3, x4, [sp, #48]
+
+	/* Restore FP and LR */
+	ldp     x29, x30, [sp], #64
+
+	ret
+SYM_FUNC_END(arch_bpf_timed_may_goto)
-- 
2.47.3


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH bpf-next 2/2] selftests/bpf: Enable timed may_goto tests for arm64
  2025-07-24 12:54 [PATCH bpf-next 0/2] bpf, arm64: support for timed may_goto Puranjay Mohan
  2025-07-24 12:54 ` [PATCH bpf-next 1/2] bpf, arm64: Add JIT " Puranjay Mohan
@ 2025-07-24 12:54 ` Puranjay Mohan
  1 sibling, 0 replies; 5+ messages in thread
From: Puranjay Mohan @ 2025-07-24 12:54 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
	John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
	Puranjay Mohan, Xu Kuohai, Catalin Marinas, Will Deacon,
	Mykola Lysenko, Kumar Kartikeya Dwivedi, bpf

As arm64 JIT now supports timed may_goto instruction, make sure all
relevant tests run on this architecture. Some tests were enabled and
other required modifications to work properly on arm64.

 $ ./test_progs -a "stream*","*may_goto*",verifier_bpf_fastcall

 #404     stream_errors:OK
 [...]
 #406/2   stream_success/stream_cond_break:OK
 [...]
 #494/23  verifier_bpf_fastcall/may_goto_interaction_x86_64:SKIP
 #494/24  verifier_bpf_fastcall/may_goto_interaction_arm64:OK
 [...]
 #539/1   verifier_may_goto_1/may_goto 0:OK
 #539/2   verifier_may_goto_1/batch 2 of may_goto 0:OK
 #539/3   verifier_may_goto_1/may_goto batch with offsets 2/1/0:OK
 #539/4   verifier_may_goto_1/may_goto batch with offsets 2/0:OK
 #539     verifier_may_goto_1:OK
 #540/1   verifier_may_goto_2/C code with may_goto 0:OK
 #540     verifier_may_goto_2:OK
 Summary: 7/16 PASSED, 25 SKIPPED, 0 FAILED

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
 .../testing/selftests/bpf/prog_tests/stream.c |  2 +-
 .../bpf/progs/verifier_bpf_fastcall.c         | 27 +++++++++------
 .../selftests/bpf/progs/verifier_may_goto_1.c | 34 ++++---------------
 3 files changed, 23 insertions(+), 40 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/stream.c b/tools/testing/selftests/bpf/prog_tests/stream.c
index d9f0185dca61b..13f27e3f1f360 100644
--- a/tools/testing/selftests/bpf/prog_tests/stream.c
+++ b/tools/testing/selftests/bpf/prog_tests/stream.c
@@ -77,7 +77,7 @@ void test_stream_errors(void)
 		ASSERT_OK(ret, "ret");
 		ASSERT_OK(opts.retval, "retval");
 
-#if !defined(__x86_64__)
+#if !defined(__x86_64__) && !defined(__aarch64__)
 		ASSERT_TRUE(1, "Timed may_goto unsupported, skip.");
 		if (i == 0) {
 			ret = bpf_prog_stream_read(prog_fd, 2, buf, sizeof(buf), &ropts);
diff --git a/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c b/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c
index c258b0722e045..fb4fa465d67c6 100644
--- a/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c
+++ b/tools/testing/selftests/bpf/progs/verifier_bpf_fastcall.c
@@ -660,19 +660,24 @@ __naked void may_goto_interaction_x86_64(void)
 
 SEC("raw_tp")
 __arch_arm64
-__log_level(4) __msg("stack depth 16")
-/* may_goto counter at -16 */
-__xlated("0: *(u64 *)(r10 -16) =")
-__xlated("1: r1 = 1")
-__xlated("2: call bpf_get_smp_processor_id")
+__log_level(4) __msg("stack depth 24")
+/* may_goto counter at -24 */
+__xlated("0: *(u64 *)(r10 -24) =")
+/* may_goto timestamp at -16 */
+__xlated("1: *(u64 *)(r10 -16) =")
+__xlated("2: r1 = 1")
+__xlated("3: call bpf_get_smp_processor_id")
 /* may_goto expansion starts */
-__xlated("3: r11 = *(u64 *)(r10 -16)")
-__xlated("4: if r11 == 0x0 goto pc+3")
-__xlated("5: r11 -= 1")
-__xlated("6: *(u64 *)(r10 -16) = r11")
+__xlated("4: r11 = *(u64 *)(r10 -24)")
+__xlated("5: if r11 == 0x0 goto pc+6")
+__xlated("6: r11 -= 1")
+__xlated("7: if r11 != 0x0 goto pc+2")
+__xlated("8: r11 = -24")
+__xlated("9: call unknown")
+__xlated("10: *(u64 *)(r10 -24) = r11")
 /* may_goto expansion ends */
-__xlated("7: *(u64 *)(r10 -8) = r1")
-__xlated("8: exit")
+__xlated("11: *(u64 *)(r10 -8) = r1")
+__xlated("12: exit")
 __success
 __naked void may_goto_interaction_arm64(void)
 {
diff --git a/tools/testing/selftests/bpf/progs/verifier_may_goto_1.c b/tools/testing/selftests/bpf/progs/verifier_may_goto_1.c
index 3966d827f2889..08385b6a736de 100644
--- a/tools/testing/selftests/bpf/progs/verifier_may_goto_1.c
+++ b/tools/testing/selftests/bpf/progs/verifier_may_goto_1.c
@@ -9,6 +9,7 @@
 SEC("raw_tp")
 __description("may_goto 0")
 __arch_x86_64
+__arch_arm64
 __xlated("0: r0 = 1")
 __xlated("1: exit")
 __success
@@ -27,6 +28,7 @@ __naked void may_goto_simple(void)
 SEC("raw_tp")
 __description("batch 2 of may_goto 0")
 __arch_x86_64
+__arch_arm64
 __xlated("0: r0 = 1")
 __xlated("1: exit")
 __success
@@ -47,6 +49,7 @@ __naked void may_goto_batch_0(void)
 SEC("raw_tp")
 __description("may_goto batch with offsets 2/1/0")
 __arch_x86_64
+__arch_arm64
 __xlated("0: r0 = 1")
 __xlated("1: exit")
 __success
@@ -69,8 +72,9 @@ __naked void may_goto_batch_1(void)
 }
 
 SEC("raw_tp")
-__description("may_goto batch with offsets 2/0 - x86_64")
+__description("may_goto batch with offsets 2/0")
 __arch_x86_64
+__arch_arm64
 __xlated("0: *(u64 *)(r10 -16) = 65535")
 __xlated("1: *(u64 *)(r10 -8) = 0")
 __xlated("2: r11 = *(u64 *)(r10 -16)")
@@ -84,33 +88,7 @@ __xlated("9: r0 = 1")
 __xlated("10: r0 = 2")
 __xlated("11: exit")
 __success
-__naked void may_goto_batch_2_x86_64(void)
-{
-	asm volatile (
-	".8byte %[may_goto1];"
-	".8byte %[may_goto3];"
-	"r0 = 1;"
-	"r0 = 2;"
-	"exit;"
-	:
-	: __imm_insn(may_goto1, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 2 /* offset */, 0)),
-	  __imm_insn(may_goto3, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 0 /* offset */, 0))
-	: __clobber_all);
-}
-
-SEC("raw_tp")
-__description("may_goto batch with offsets 2/0 - arm64")
-__arch_arm64
-__xlated("0: *(u64 *)(r10 -8) = 8388608")
-__xlated("1: r11 = *(u64 *)(r10 -8)")
-__xlated("2: if r11 == 0x0 goto pc+3")
-__xlated("3: r11 -= 1")
-__xlated("4: *(u64 *)(r10 -8) = r11")
-__xlated("5: r0 = 1")
-__xlated("6: r0 = 2")
-__xlated("7: exit")
-__success
-__naked void may_goto_batch_2_arm64(void)
+__naked void may_goto_batch_2(void)
 {
 	asm volatile (
 	".8byte %[may_goto1];"
-- 
2.47.3


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH bpf-next 1/2] bpf, arm64: Add JIT support for timed may_goto
  2025-07-24 12:54 ` [PATCH bpf-next 1/2] bpf, arm64: Add JIT " Puranjay Mohan
@ 2025-08-05 11:28   ` Xu Kuohai
  2025-08-07 13:27     ` puranjay
  0 siblings, 1 reply; 5+ messages in thread
From: Xu Kuohai @ 2025-08-05 11:28 UTC (permalink / raw)
  To: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu,
	Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
	Hao Luo, Jiri Olsa, Catalin Marinas, Will Deacon, Mykola Lysenko,
	Kumar Kartikeya Dwivedi, bpf

On 7/24/2025 8:54 PM, Puranjay Mohan wrote:
> When verifier sees a timed may_goto instruction, it emits a call to
> arch_bpf_timed_may_goto() with a stack offset in BPF_REG_AX (arm64 r9)
> and expects a count value to be returned in the same register. The
> verifier doesn't save or restore any registers before emitting this
> call.
> 
> arch_bpf_timed_may_goto() should act as a trampoline to call
> bpf_check_timed_may_goto() with AAPCS64 calling convention.
> 
> To support this custom calling convention, implement
> arch_bpf_timed_may_goto() in assembly and make sure BPF caller saved
> registers are saved and restored, call bpf_check_timed_may_goto with
> arm64 calling convention where first argument and return value both are
> in x0, then put the result back into BPF_REG_AX before returning.
> 
> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> ---
>   arch/arm64/net/Makefile             |  2 +-
>   arch/arm64/net/bpf_jit_comp.c       | 13 ++++++++++-
>   arch/arm64/net/bpf_timed_may_goto.S | 36 +++++++++++++++++++++++++++++
>   3 files changed, 49 insertions(+), 2 deletions(-)
>   create mode 100644 arch/arm64/net/bpf_timed_may_goto.S
> 
> diff --git a/arch/arm64/net/Makefile b/arch/arm64/net/Makefile
> index 5c540efb7d9b9..3ae382bfca879 100644
> --- a/arch/arm64/net/Makefile
> +++ b/arch/arm64/net/Makefile
> @@ -2,4 +2,4 @@
>   #
>   # ARM64 networking code
>   #
> -obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o
> +obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o bpf_timed_may_goto.o
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index 89b1b8c248c62..6c954b36f57ea 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -1505,7 +1505,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
>   		if (ret < 0)
>   			return ret;
>   		emit_call(func_addr, ctx);
> -		emit(A64_MOV(1, r0, A64_R(0)), ctx);
> +		/*
> +		 * Call to arch_bpf_timed_may_goto() is emitted by the
> +		 * verifier and called with custom calling convention with
> +		 * first argument and return value in BPF_REG_AX (x9).
> +		 */
> +		if (func_addr != (u64)arch_bpf_timed_may_goto)
> +			emit(A64_MOV(1, r0, A64_R(0)), ctx);
>   		break;
>   	}
>   	/* tail call */
> @@ -2914,6 +2920,11 @@ bool bpf_jit_bypass_spec_v4(void)
>   	return true;
>   }
>   
> +bool bpf_jit_supports_timed_may_goto(void)
> +{
> +	return true;
> +}
> +
>   bool bpf_jit_inlines_helper_call(s32 imm)
>   {
>   	switch (imm) {
> diff --git a/arch/arm64/net/bpf_timed_may_goto.S b/arch/arm64/net/bpf_timed_may_goto.S
> new file mode 100644
> index 0000000000000..45f80e752345c
> --- /dev/null
> +++ b/arch/arm64/net/bpf_timed_may_goto.S
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (c) 2025 Puranjay Mohan <puranjay@kernel.org> */
> +
> +#include <linux/linkage.h>
> +
> +SYM_FUNC_START(arch_bpf_timed_may_goto)
> +	/* Allocate stack space and emit frame record */
> +	stp     x29, x30, [sp, #-64]!
> +	mov     x29, sp
> +
> +	/* Save BPF registers R0 - R5 (x7, x0-x4)*/
> +	stp	x7, x0, [sp, #16]
> +	stp	x1, x2, [sp, #32]
> +	stp	x3, x4, [sp, #48]
> +
> +	/*
> +	 * Stack depth was passed in BPF_REG_AX (x9), add it to the BPF_FP
> +	 * (x25) to get the pointer to count and timestamp and pass it as the
> +	 * first argument in x0.
> +	 */
> +	add	x0, x9, x25

Whether BPF_REG_FP (x25) is set up by the arm64 jit depends on whether
the jit detects any bpf instruction using it. Before generating the
call to arch_bpf_timed_may_goto, the verifier generates a load
instruction using FP, i.e. AX = *(u64 *)(FP - stack_off_cnt),
so FP is always set up in this case.

It seems a bit subtle. Maybe we should add a comment here?

> +	bl	bpf_check_timed_may_goto
> +	/* BPF_REG_AX(x9) will be stored into count, so move return value to it. */
> +	mov	x9, x0
> +
> +

Nit: one extra blank line

> +	/* Restore BPF registers R0 - R5 (x7, x0-x4) */
> +	ldp	x7, x0, [sp, #16]
> +	ldp	x1, x2, [sp, #32]
> +	ldp	x3, x4, [sp, #48]
> +
> +	/* Restore FP and LR */
> +	ldp     x29, x30, [sp], #64
> +
> +	ret
> +SYM_FUNC_END(arch_bpf_timed_may_goto)


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH bpf-next 1/2] bpf, arm64: Add JIT support for timed may_goto
  2025-08-05 11:28   ` Xu Kuohai
@ 2025-08-07 13:27     ` puranjay
  0 siblings, 0 replies; 5+ messages in thread
From: puranjay @ 2025-08-07 13:27 UTC (permalink / raw)
  To: Xu Kuohai, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
	John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
	Catalin Marinas, Will Deacon, Mykola Lysenko,
	Kumar Kartikeya Dwivedi, bpf

Xu Kuohai <xukuohai@huaweicloud.com> writes:

> On 7/24/2025 8:54 PM, Puranjay Mohan wrote:
>> When verifier sees a timed may_goto instruction, it emits a call to
>> arch_bpf_timed_may_goto() with a stack offset in BPF_REG_AX (arm64 r9)
>> and expects a count value to be returned in the same register. The
>> verifier doesn't save or restore any registers before emitting this
>> call.
>> 
>> arch_bpf_timed_may_goto() should act as a trampoline to call
>> bpf_check_timed_may_goto() with AAPCS64 calling convention.
>> 
>> To support this custom calling convention, implement
>> arch_bpf_timed_may_goto() in assembly and make sure BPF caller saved
>> registers are saved and restored, call bpf_check_timed_may_goto with
>> arm64 calling convention where first argument and return value both are
>> in x0, then put the result back into BPF_REG_AX before returning.
>> 
>> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
>> ---
>>   arch/arm64/net/Makefile             |  2 +-
>>   arch/arm64/net/bpf_jit_comp.c       | 13 ++++++++++-
>>   arch/arm64/net/bpf_timed_may_goto.S | 36 +++++++++++++++++++++++++++++
>>   3 files changed, 49 insertions(+), 2 deletions(-)
>>   create mode 100644 arch/arm64/net/bpf_timed_may_goto.S
>> 
>> diff --git a/arch/arm64/net/Makefile b/arch/arm64/net/Makefile
>> index 5c540efb7d9b9..3ae382bfca879 100644
>> --- a/arch/arm64/net/Makefile
>> +++ b/arch/arm64/net/Makefile
>> @@ -2,4 +2,4 @@
>>   #
>>   # ARM64 networking code
>>   #
>> -obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o
>> +obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o bpf_timed_may_goto.o
>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>> index 89b1b8c248c62..6c954b36f57ea 100644
>> --- a/arch/arm64/net/bpf_jit_comp.c
>> +++ b/arch/arm64/net/bpf_jit_comp.c
>> @@ -1505,7 +1505,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
>>   		if (ret < 0)
>>   			return ret;
>>   		emit_call(func_addr, ctx);
>> -		emit(A64_MOV(1, r0, A64_R(0)), ctx);
>> +		/*
>> +		 * Call to arch_bpf_timed_may_goto() is emitted by the
>> +		 * verifier and called with custom calling convention with
>> +		 * first argument and return value in BPF_REG_AX (x9).
>> +		 */
>> +		if (func_addr != (u64)arch_bpf_timed_may_goto)
>> +			emit(A64_MOV(1, r0, A64_R(0)), ctx);
>>   		break;
>>   	}
>>   	/* tail call */
>> @@ -2914,6 +2920,11 @@ bool bpf_jit_bypass_spec_v4(void)
>>   	return true;
>>   }
>>   
>> +bool bpf_jit_supports_timed_may_goto(void)
>> +{
>> +	return true;
>> +}
>> +
>>   bool bpf_jit_inlines_helper_call(s32 imm)
>>   {
>>   	switch (imm) {
>> diff --git a/arch/arm64/net/bpf_timed_may_goto.S b/arch/arm64/net/bpf_timed_may_goto.S
>> new file mode 100644
>> index 0000000000000..45f80e752345c
>> --- /dev/null
>> +++ b/arch/arm64/net/bpf_timed_may_goto.S
>> @@ -0,0 +1,36 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/* Copyright (c) 2025 Puranjay Mohan <puranjay@kernel.org> */
>> +
>> +#include <linux/linkage.h>
>> +
>> +SYM_FUNC_START(arch_bpf_timed_may_goto)
>> +	/* Allocate stack space and emit frame record */
>> +	stp     x29, x30, [sp, #-64]!
>> +	mov     x29, sp
>> +
>> +	/* Save BPF registers R0 - R5 (x7, x0-x4)*/
>> +	stp	x7, x0, [sp, #16]
>> +	stp	x1, x2, [sp, #32]
>> +	stp	x3, x4, [sp, #48]
>> +
>> +	/*
>> +	 * Stack depth was passed in BPF_REG_AX (x9), add it to the BPF_FP
>> +	 * (x25) to get the pointer to count and timestamp and pass it as the
>> +	 * first argument in x0.
>> +	 */
>> +	add	x0, x9, x25
>
> Whether BPF_REG_FP (x25) is set up by the arm64 jit depends on whether
> the jit detects any bpf instruction using it. Before generating the
> call to arch_bpf_timed_may_goto, the verifier generates a load
> instruction using FP, i.e. AX = *(u64 *)(FP - stack_off_cnt),
> so FP is always set up in this case.
>
> It seems a bit subtle. Maybe we should add a comment here?

Yes, a comment would be useful. I will add it in the next version.

>> +	bl	bpf_check_timed_may_goto
>> +	/* BPF_REG_AX(x9) will be stored into count, so move return value to it. */
>> +	mov	x9, x0
>> +
>> +
>
> Nit: one extra blank line
>

Thanks,
Puranjay

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2025-08-07 13:27 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-24 12:54 [PATCH bpf-next 0/2] bpf, arm64: support for timed may_goto Puranjay Mohan
2025-07-24 12:54 ` [PATCH bpf-next 1/2] bpf, arm64: Add JIT " Puranjay Mohan
2025-08-05 11:28   ` Xu Kuohai
2025-08-07 13:27     ` puranjay
2025-07-24 12:54 ` [PATCH bpf-next 2/2] selftests/bpf: Enable timed may_goto tests for arm64 Puranjay Mohan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).