[PATCH bpf-next 0/2] s390/bpf: Write back the tail call counter

[PATCH bpf-next 0/2] s390/bpf: Write back the tail call counter

[Ilya Leoshkevich]

Hi,

This series fixes the tailcall_bpf2bpf_hierarchy test failures on s390.

It takes a simpler approach than x86_64 and aarch64: instead of
introducing a pointer to the tail call counter, it copies the updated
value back from the callee's frame to the caller's frame. This needs to
be done in two locations: after BPF_CALL and after
BPF_TRAMP_F_CALL_ORIG.

Best regards,
Ilya

Ilya Leoshkevich (2):
  s390/bpf: Write back the tail call counter for BPF_CALL
  s390/bpf: Write back the tail call counter for BPF_TRAMP_F_CALL_ORIG

 arch/s390/net/bpf_jit_comp.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

-- 
2.50.1

[PATCH bpf-next 1/2] s390/bpf: Write back the tail call counter for BPF_CALL

[Ilya Leoshkevich]

The tailcall_bpf2bpf_hierarchy_1 test hangs on s390. Its call graph is
as follows:

  entry()
    subprog_tail()
      bpf_tail_call_static(0) -> entry + tail_call_start
    subprog_tail()
      bpf_tail_call_static(0) -> entry + tail_call_start

entry() copies its tail call counter to the subprog_tail()'s frame,
which then increments it. However, the incremented result is discarded,
leading to an astronomically large number of tail calls.

Fix by writing the incremented counter back to the entry()'s frame.

Fixes: dd691e847d28 ("s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/s390/net/bpf_jit_comp.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index bb17efe29d65..85695576df6c 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -1790,16 +1790,11 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 
 		REG_SET_SEEN(BPF_REG_5);
 		jit->seen |= SEEN_FUNC;
+
 		/*
 		 * Copy the tail call counter to where the callee expects it.
-		 *
-		 * Note 1: The callee can increment the tail call counter, but
-		 * we do not load it back, since the x86 JIT does not do this
-		 * either.
-		 *
-		 * Note 2: We assume that the verifier does not let us call the
-		 * main program, which clears the tail call counter on entry.
 		 */
+
 		/* mvc tail_call_cnt(4,%r15),frame_off+tail_call_cnt(%r15) */
 		_EMIT6(0xd203f000 | offsetof(struct prog_frame, tail_call_cnt),
 		       0xf000 | (jit->frame_off +
@@ -1825,6 +1820,17 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 		call_r1(jit);
 		/* lgr %b0,%r2: load return value into %b0 */
 		EMIT4(0xb9040000, BPF_REG_0, REG_2);
+
+		/*
+		 * Copy the potentially updated tail call counter back.
+		 */
+
+		/* mvc frame_off+tail_call_cnt(%r15),tail_call_cnt(4,%r15) */
+		_EMIT6(0xd203f000 | (jit->frame_off +
+				     offsetof(struct prog_frame,
+					      tail_call_cnt)),
+		       0xf000 | offsetof(struct prog_frame, tail_call_cnt));
+
 		break;
 	}
 	case BPF_JMP | BPF_TAIL_CALL: {
-- 
2.50.1

[PATCH bpf-next 2/2] s390/bpf: Write back the tail call counter for BPF_TRAMP_F_CALL_ORIG

[Ilya Leoshkevich]

The tailcall_bpf2bpf_hierarchy_fentry test hangs on s390. Its call
graph is as follows:

  entry()
    subprog_tail()
      trampoline()
        fentry()
        the rest of subprog_tail()  # via BPF_TRAMP_F_CALL_ORIG
        return to entry()

The problem is that the rest of subprog_tail() increments the tail call
counter, but the trampoline discards the incremented value. This
results in an astronomically large number of tail calls.

Fix by making the trampoline write the incremented tail call counter
back.

Fixes: 528eb2cb87bc ("s390/bpf: Implement arch_prepare_bpf_trampoline()")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/s390/net/bpf_jit_comp.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 85695576df6c..b05ddab135e0 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2828,6 +2828,9 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 		/* stg %r2,retval_off(%r15) */
 		EMIT6_DISP_LH(0xe3000000, 0x0024, REG_2, REG_0, REG_15,
 			      tjit->retval_off);
+		/* mvc tccnt_off(%r15),tail_call_cnt(4,%r15) */
+		_EMIT6(0xd203f000 | tjit->tccnt_off,
+		       0xf000 | offsetof(struct prog_frame, tail_call_cnt));
 
 		im->ip_after_call = jit->prg_buf + jit->prg;
 
-- 
2.50.1

Re: [PATCH bpf-next 1/2] s390/bpf: Write back the tail call counter for BPF_CALL

[Ilya Leoshkevich]

On Tue, 2025-08-12 at 16:07 +0200, Ilya Leoshkevich wrote:
> The tailcall_bpf2bpf_hierarchy_1 test hangs on s390. Its call graph
> is
> as follows:
> 
>   entry()
>     subprog_tail()
>       bpf_tail_call_static(0) -> entry + tail_call_start
>     subprog_tail()
>       bpf_tail_call_static(0) -> entry + tail_call_start
> 
> entry() copies its tail call counter to the subprog_tail()'s frame,
> which then increments it. However, the incremented result is
> discarded,
> leading to an astronomically large number of tail calls.
> 
> Fix by writing the incremented counter back to the entry()'s frame.
> 
> Fixes: dd691e847d28 ("s390/bpf: Implement
> bpf_jit_supports_subprog_tailcalls()")
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>  arch/s390/net/bpf_jit_comp.c | 20 +++++++++++++-------
>  1 file changed, 13 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/s390/net/bpf_jit_comp.c
> b/arch/s390/net/bpf_jit_comp.c
> index bb17efe29d65..85695576df6c 100644
> --- a/arch/s390/net/bpf_jit_comp.c
> +++ b/arch/s390/net/bpf_jit_comp.c
> @@ -1790,16 +1790,11 @@ static noinline int bpf_jit_insn(struct
> bpf_jit *jit, struct bpf_prog *fp,
>  
>  		REG_SET_SEEN(BPF_REG_5);
>  		jit->seen |= SEEN_FUNC;
> +
>  		/*
>  		 * Copy the tail call counter to where the callee
> expects it.
> -		 *
> -		 * Note 1: The callee can increment the tail call
> counter, but
> -		 * we do not load it back, since the x86 JIT does
> not do this
> -		 * either.
> -		 *
> -		 * Note 2: We assume that the verifier does not let
> us call the
> -		 * main program, which clears the tail call counter
> on entry.
>  		 */
> +
>  		/* mvc
> tail_call_cnt(4,%r15),frame_off+tail_call_cnt(%r15) */
>  		_EMIT6(0xd203f000 | offsetof(struct prog_frame,
> tail_call_cnt),
>  		       0xf000 | (jit->frame_off +
> @@ -1825,6 +1820,17 @@ static noinline int bpf_jit_insn(struct
> bpf_jit *jit, struct bpf_prog *fp,
>  		call_r1(jit);
>  		/* lgr %b0,%r2: load return value into %b0 */
>  		EMIT4(0xb9040000, BPF_REG_0, REG_2);
> +
> +		/*
> +		 * Copy the potentially updated tail call counter
> back.
> +		 */
> +
> +		/* mvc
> frame_off+tail_call_cnt(%r15),tail_call_cnt(4,%r15) */
> +		_EMIT6(0xd203f000 | (jit->frame_off +
> +				     offsetof(struct prog_frame,
> +					      tail_call_cnt)),
> +		       0xf000 | offsetof(struct prog_frame,
> tail_call_cnt));
> +
>  		break;
>  	}
>  	case BPF_JMP | BPF_TAIL_CALL: {

Hmm, we need to do this only for BPF_PSEUDO_CALLs, otherwise a helper
or a kfunc, which is unaware of the tail call counter convention, will
clobber it with something random, potentially causing a kernel stack
overflow.

I will send a v2 and also provide a test that catches this issue.