bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter
@ 2025-08-13 12:06 Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames Ilya Leoshkevich
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Ilya Leoshkevich @ 2025-08-13 12:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
	Ilya Leoshkevich

v1: https://lore.kernel.org/bpf/20250812141217.144551-1-iii@linux.ibm.com/
v1 -> v2: Write back tail call counter only for BPF_PSEUDO_CALL

Hi,

This series fixes the tailcall_bpf2bpf_hierarchy test failures on s390.

It takes a simpler approach than x86_64 and aarch64: instead of
introducing a pointer to the tail call counter, it copies the updated
value back from the callee's frame to the caller's frame. This needs to
be done in two locations: after BPF_PSEUDO_CALL and after
BPF_TRAMP_F_CALL_ORIG.

Patch 1 is a cleanup, patches 2 and 3 are the actual fixes, patch 4
improves the hierarchy tests in order to catch issues with accidentally
clobbering the counter.

Best regards,
Ilya

Ilya Leoshkevich (4):
  s390/bpf: Do not write tail call counter into helper and kfunc frames
  s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL
  s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG
  selftests/bpf: Clobber a lot of registers in
    tailcall_bpf2bpf_hierarchy tests

 arch/s390/net/bpf_jit_comp.c                  | 42 ++++++++++++++-----
 .../selftests/bpf/progs/bpf_test_utils.h      | 18 ++++++++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c   |  3 ++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy2.c   |  3 ++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy3.c   |  3 ++
 .../progs/tailcall_bpf2bpf_hierarchy_fentry.c |  3 ++
 6 files changed, 61 insertions(+), 11 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_test_utils.h

-- 
2.50.1


^ permalink raw reply	[flat|nested] 6+ messages in thread
* [PATCH bpf-next v2 1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames
  2025-08-13 12:06 [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter Ilya Leoshkevich
@ 2025-08-13 12:06 ` Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 2/4] s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL Ilya Leoshkevich
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Ilya Leoshkevich @ 2025-08-13 12:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
	Ilya Leoshkevich

Only BPF functions make use of the tail call counter; helpers and
kfuncs ignore and most likely also clobber it. Writing it into these
functions' frames is pointless and misleading, so do not do it.

Fixes: dd691e847d28 ("s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/s390/net/bpf_jit_comp.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index bb17efe29d65..bfac1ddf3447 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -1790,6 +1790,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 
 		REG_SET_SEEN(BPF_REG_5);
 		jit->seen |= SEEN_FUNC;
+
 		/*
 		 * Copy the tail call counter to where the callee expects it.
 		 *
@@ -1800,10 +1801,17 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 		 * Note 2: We assume that the verifier does not let us call the
 		 * main program, which clears the tail call counter on entry.
 		 */
-		/* mvc tail_call_cnt(4,%r15),frame_off+tail_call_cnt(%r15) */
-		_EMIT6(0xd203f000 | offsetof(struct prog_frame, tail_call_cnt),
-		       0xf000 | (jit->frame_off +
-				 offsetof(struct prog_frame, tail_call_cnt)));
+
+		if (insn->src_reg == BPF_PSEUDO_CALL)
+			/*
+			 * mvc tail_call_cnt(4,%r15),
+			 *     frame_off+tail_call_cnt(%r15)
+			 */
+			_EMIT6(0xd203f000 | offsetof(struct prog_frame,
+						     tail_call_cnt),
+			       0xf000 | (jit->frame_off +
+					 offsetof(struct prog_frame,
+						  tail_call_cnt)));
 
 		/* Sign-extend the kfunc arguments. */
 		if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH bpf-next v2 2/4] s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL
  2025-08-13 12:06 [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames Ilya Leoshkevich
@ 2025-08-13 12:06 ` Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 3/4] s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG Ilya Leoshkevich
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Ilya Leoshkevich @ 2025-08-13 12:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
	Ilya Leoshkevich

The tailcall_bpf2bpf_hierarchy_1 test hangs on s390. Its call graph is
as follows:

  entry()
    subprog_tail()
      bpf_tail_call_static(0) -> entry + tail_call_start
    subprog_tail()
      bpf_tail_call_static(0) -> entry + tail_call_start

entry() copies its tail call counter to the subprog_tail()'s frame,
which then increments it. However, the incremented result is discarded,
leading to an astronomically large number of tail calls.

Fix by writing the incremented counter back to the entry()'s frame.

Fixes: dd691e847d28 ("s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/s390/net/bpf_jit_comp.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index bfac1ddf3447..ccb83ac3e6f3 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -1793,13 +1793,6 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 
 		/*
 		 * Copy the tail call counter to where the callee expects it.
-		 *
-		 * Note 1: The callee can increment the tail call counter, but
-		 * we do not load it back, since the x86 JIT does not do this
-		 * either.
-		 *
-		 * Note 2: We assume that the verifier does not let us call the
-		 * main program, which clears the tail call counter on entry.
 		 */
 
 		if (insn->src_reg == BPF_PSEUDO_CALL)
@@ -1833,6 +1826,22 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 		call_r1(jit);
 		/* lgr %b0,%r2: load return value into %b0 */
 		EMIT4(0xb9040000, BPF_REG_0, REG_2);
+
+		/*
+		 * Copy the potentially updated tail call counter back.
+		 */
+
+		if (insn->src_reg == BPF_PSEUDO_CALL)
+			/*
+			 * mvc frame_off+tail_call_cnt(%r15),
+			 *     tail_call_cnt(4,%r15)
+			 */
+			_EMIT6(0xd203f000 | (jit->frame_off +
+					     offsetof(struct prog_frame,
+						      tail_call_cnt)),
+			       0xf000 | offsetof(struct prog_frame,
+						 tail_call_cnt));
+
 		break;
 	}
 	case BPF_JMP | BPF_TAIL_CALL: {
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH bpf-next v2 3/4] s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG
  2025-08-13 12:06 [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 2/4] s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL Ilya Leoshkevich
@ 2025-08-13 12:06 ` Ilya Leoshkevich
  2025-08-13 12:06 ` [PATCH bpf-next v2 4/4] selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests Ilya Leoshkevich
  2025-08-18 13:20 ` [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter patchwork-bot+netdevbpf
  4 siblings, 0 replies; 6+ messages in thread
From: Ilya Leoshkevich @ 2025-08-13 12:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
	Ilya Leoshkevich

The tailcall_bpf2bpf_hierarchy_fentry test hangs on s390. Its call
graph is as follows:

  entry()
    subprog_tail()
      trampoline()
        fentry()
        the rest of subprog_tail()  # via BPF_TRAMP_F_CALL_ORIG
        return to entry()

The problem is that the rest of subprog_tail() increments the tail call
counter, but the trampoline discards the incremented value. This
results in an astronomically large number of tail calls.

Fix by making the trampoline write the incremented tail call counter
back.

Fixes: 528eb2cb87bc ("s390/bpf: Implement arch_prepare_bpf_trampoline()")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/s390/net/bpf_jit_comp.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index ccb83ac3e6f3..b2b8eb62b82e 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2839,6 +2839,9 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 		/* stg %r2,retval_off(%r15) */
 		EMIT6_DISP_LH(0xe3000000, 0x0024, REG_2, REG_0, REG_15,
 			      tjit->retval_off);
+		/* mvc tccnt_off(%r15),tail_call_cnt(4,%r15) */
+		_EMIT6(0xd203f000 | tjit->tccnt_off,
+		       0xf000 | offsetof(struct prog_frame, tail_call_cnt));
 
 		im->ip_after_call = jit->prg_buf + jit->prg;
 
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH bpf-next v2 4/4] selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests
  2025-08-13 12:06 [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter Ilya Leoshkevich
                   ` (2 preceding siblings ...)
  2025-08-13 12:06 ` [PATCH bpf-next v2 3/4] s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG Ilya Leoshkevich
@ 2025-08-13 12:06 ` Ilya Leoshkevich
  2025-08-18 13:20 ` [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter patchwork-bot+netdevbpf
  4 siblings, 0 replies; 6+ messages in thread
From: Ilya Leoshkevich @ 2025-08-13 12:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
	Ilya Leoshkevich

Clobbering a lot of registers and stack slots helps exposing tail call
counter overwrite bugs in JITs.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 .../selftests/bpf/progs/bpf_test_utils.h       | 18 ++++++++++++++++++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c    |  3 +++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy2.c    |  3 +++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy3.c    |  3 +++
 .../progs/tailcall_bpf2bpf_hierarchy_fentry.c  |  3 +++
 5 files changed, 30 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_test_utils.h

diff --git a/tools/testing/selftests/bpf/progs/bpf_test_utils.h b/tools/testing/selftests/bpf/progs/bpf_test_utils.h
new file mode 100644
index 000000000000..f4e67b492dd2
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_test_utils.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __BPF_TEST_UTILS_H__
+#define __BPF_TEST_UTILS_H__
+
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+/* Clobber as many native registers and stack slots as possible. */
+static __always_inline void clobber_regs_stack(void)
+{
+	char tmp_str[] = "123456789";
+	unsigned long tmp;
+
+	bpf_strtoul(tmp_str, sizeof(tmp_str), 0, &tmp);
+	__sink(tmp);
+}
+
+#endif
diff --git a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c
index 327ca395e860..d556b19413d7 100644
--- a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c
+++ b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c
@@ -2,6 +2,7 @@
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_legacy.h"
+#include "bpf_test_utils.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
@@ -24,6 +25,8 @@ int entry(struct __sk_buff *skb)
 {
 	int ret = 1;
 
+	clobber_regs_stack();
+
 	count++;
 	subprog_tail(skb);
 	subprog_tail(skb);
diff --git a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c
index 72fd0d577506..ae94c9c70ab7 100644
--- a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c
+++ b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c
@@ -2,6 +2,7 @@
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_misc.h"
+#include "bpf_test_utils.h"
 
 int classifier_0(struct __sk_buff *skb);
 int classifier_1(struct __sk_buff *skb);
@@ -60,6 +61,8 @@ int tailcall_bpf2bpf_hierarchy_2(struct __sk_buff *skb)
 {
 	int ret = 0;
 
+	clobber_regs_stack();
+
 	subprog_tail0(skb);
 	subprog_tail1(skb);
 
diff --git a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy3.c b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy3.c
index a7fb91cb05b7..56b6b0099840 100644
--- a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy3.c
+++ b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy3.c
@@ -2,6 +2,7 @@
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_misc.h"
+#include "bpf_test_utils.h"
 
 int classifier_0(struct __sk_buff *skb);
 
@@ -53,6 +54,8 @@ int tailcall_bpf2bpf_hierarchy_3(struct __sk_buff *skb)
 {
 	int ret = 0;
 
+	clobber_regs_stack();
+
 	bpf_tail_call_static(skb, &jmp_table0, 0);
 
 	__sink(ret);
diff --git a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c
index c87f9ca982d3..5261395713cd 100644
--- a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c
+++ b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c
@@ -4,6 +4,7 @@
 #include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
+#include "bpf_test_utils.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
@@ -24,6 +25,8 @@ int subprog_tail(void *ctx)
 SEC("fentry/dummy")
 int BPF_PROG(fentry, struct sk_buff *skb)
 {
+	clobber_regs_stack();
+
 	count++;
 	subprog_tail(ctx);
 	subprog_tail(ctx);
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter
  2025-08-13 12:06 [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter Ilya Leoshkevich
                   ` (3 preceding siblings ...)
  2025-08-13 12:06 ` [PATCH bpf-next v2 4/4] selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests Ilya Leoshkevich
@ 2025-08-18 13:20 ` patchwork-bot+netdevbpf
  4 siblings, 0 replies; 6+ messages in thread
From: patchwork-bot+netdevbpf @ 2025-08-18 13:20 UTC (permalink / raw)
  To: Ilya Leoshkevich; +Cc: ast, daniel, andrii, bpf, hca, gor, agordeev

Hello:

This series was applied to bpf/bpf-next.git (master)
by Daniel Borkmann <daniel@iogearbox.net>:

On Wed, 13 Aug 2025 14:06:27 +0200 you wrote:
> v1: https://lore.kernel.org/bpf/20250812141217.144551-1-iii@linux.ibm.com/
> v1 -> v2: Write back tail call counter only for BPF_PSEUDO_CALL
> 
> Hi,
> 
> This series fixes the tailcall_bpf2bpf_hierarchy test failures on s390.
> 
> [...]

Here is the summary with links:
  - [bpf-next,v2,1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames
    https://git.kernel.org/bpf/bpf-next/c/eada40e057fc
  - [bpf-next,v2,2/4] s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL
    https://git.kernel.org/bpf/bpf-next/c/c861a6b14713
  - [bpf-next,v2,3/4] s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG
    https://git.kernel.org/bpf/bpf-next/c/bc3905a71f02
  - [bpf-next,v2,4/4] selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests
    https://git.kernel.org/bpf/bpf-next/c/12741630350c

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-08-18 13:20 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-13 12:06 [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter Ilya Leoshkevich
2025-08-13 12:06 ` [PATCH bpf-next v2 1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames Ilya Leoshkevich
2025-08-13 12:06 ` [PATCH bpf-next v2 2/4] s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL Ilya Leoshkevich
2025-08-13 12:06 ` [PATCH bpf-next v2 3/4] s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG Ilya Leoshkevich
2025-08-13 12:06 ` [PATCH bpf-next v2 4/4] selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests Ilya Leoshkevich
2025-08-18 13:20 ` [PATCH bpf-next v2 0/4] s390/bpf: Write back tail call counter patchwork-bot+netdevbpf

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).