Re: [PATCH v2 bpf] bpf: properly verify tail call behavior

[Eduard Zingerman <eddyz87@gmail.com>]

On Tue, 2025-11-04 at 14:30 +0100, Martin Teichmann wrote:
> A successful ebpf tail call does not return to the caller, but to the
> caller-of-the-caller, often just finishing the ebpf program altogether.
> 
> Any restrictions that the verifier needs to take into account - notably
> the fact that the tail call might have modified packet pointers - are to
> be checked on the caller-of-the-caller. Checking it on the caller made
> the verifier refuse perfectly fine programs that would use the packet
> pointers after a tail call, which is no problem as this code is only
> executed if the tail call was unsuccessful, i.e. nothing happened.
> 
> This patch simulates the behavior of a tail call in the verifier. A
> conditional jump to the code after the tail call is added for the case
> of an unsucessful tail call, and a return to the caller is simulated for
> a successful tail call.
> 
> For the successful case we assume that the tail call returns an int,
> as tail calls are currently only allowed in functions that return and
> int. We always assume that the tail call modified the packet pointers,
> as we do not know what the tail call did.
> 
> For the unsuccessful case we know nothing happened, so we do not need to
> add new constraints. Some test are added, notably one corner case found
> by Eduard Zingerman.
> 
> Fixes: 1a4607ffba35 ("bpf: consider that tail calls invalidate packet pointers")
> Link: https://lore.kernel.org/bpf/20251029105828.1488347-1-martin.teichmann@xfel.eu/
> Signed-off-by: Martin Teichmann <martin.teichmann@xfel.eu>
> ---

Hi Martin,

I think this is a viable idea.
Please see a few notes below.

>  kernel/bpf/verifier.c                         | 25 ++++++++++--
>  .../selftests/bpf/progs/verifier_sock.c       | 39 ++++++++++++++++++-
>  2 files changed, 59 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index e4928846e763..9a091e0f2f07 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -11005,6 +11005,10 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
>  	bool in_callback_fn;
>  	int err;
>  
> +	err = bpf_update_live_stack(env);
> +	if (err)
> +		return err;
> +
>  	callee = state->frame[state->curframe];
>  	r0 = &callee->regs[BPF_REG_0];
>  	if (r0->type == PTR_TO_STACK) {
> @@ -11911,6 +11915,24 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
>  		env->prog->call_get_func_ip = true;
>  	}
>  
> +	if (func_id == BPF_FUNC_tail_call) {
> +		if (env->cur_state->curframe) {
> +			struct bpf_verifier_state *branch;
> +			mark_reg_scratched(env, BPF_REG_0);
> +			branch = push_stack(env, env->insn_idx + 1, env->insn_idx, false);
> +			if (IS_ERR(branch))
> +				return PTR_ERR(branch);
> +			clear_all_pkt_pointers(env);
> +			mark_reg_unknown(env, regs, BPF_REG_0);
> +			err = prepare_func_exit(env, &env->insn_idx);
> +			if (err)
> +				return err;
> +			env->insn_idx--;
> +		} else {
> +			changes_data = false;
> +		}
> +	}
> +

Could you please update bpf_insn_successors()?
Technically, it should do something similar to what Anton does in [1].
W/o bpf_insn_successors() reflecting that control flow might jump over
the instructions between tail call and function exit, verifier might
assume that some writes to parent stack always happen, which is not
the case.

I think this is a long standing bug, was here even before my changes
for stack liveness.

[1] https://lore.kernel.org/bpf/20251102205722.3266908-7-a.s.protopopov@gmail.com/

>  	if (changes_data)
>  		clear_all_pkt_pointers(env);
>  	return 0;
> @@ -19876,9 +19898,6 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,
>  		return PROCESS_BPF_EXIT;
>  
>  	if (env->cur_state->curframe) {
> -		err = bpf_update_live_stack(env);
> -		if (err)
> -			return err;
>  		/* exit from nested function */
>  		err = prepare_func_exit(env, &env->insn_idx);
>  		if (err)
> diff --git a/tools/testing/selftests/bpf/progs/verifier_sock.c b/tools/testing/selftests/bpf/progs/verifier_sock.c
> index 2b4610b53382..a2132c72d3b8 100644
> --- a/tools/testing/selftests/bpf/progs/verifier_sock.c
> +++ b/tools/testing/selftests/bpf/progs/verifier_sock.c
> @@ -1117,10 +1117,17 @@ int tail_call(struct __sk_buff *sk)
>  	return 0;
>  }
>  
> -/* Tail calls invalidate packet pointers. */
> +static __noinline
> +int static_tail_call(struct __sk_buff *sk)
> +{
> +	bpf_tail_call_static(sk, &jmp_table, 0);
> +	return 0;
> +}
> +
> +/* Tail calls in sub-programs invalidate packet pointers. */
>  SEC("tc")
>  __failure __msg("invalid mem access")
> -int invalidate_pkt_pointers_by_tail_call(struct __sk_buff *sk)
> +int invalidate_pkt_pointers_by_global_tail_call(struct __sk_buff *sk)
>  {
>  	int *p = (void *)(long)sk->data;
>  
> @@ -1131,4 +1138,32 @@ int invalidate_pkt_pointers_by_tail_call(struct __sk_buff *sk)
>  	return TCX_PASS;
>  }
>

Usually, test cases are added as separate commits.
Changes to tests are bundled with changes to kernel only if that is
necessary to keep bisect happy (tests passing on each commit).

--

Also, could you please add a test case verifying that:
- Precision propagation works correctly when processing program
  fragment containing tail call. (I think that is_jmp_point() logic
  should kick-in automatically here, but would be nice to have a
  test).
- Live stack tracking works correctly under the same circumstances.
  (Test case exercising bpf_update_live_stack() call you inserted).

See verifier_subprog_precision.c and verifier_live_stack.c for
examples.

> +/* Tail calls in static sub-programs invalidate packet pointers. */
> +SEC("tc")
> +__failure __msg("invalid mem access")
> +int invalidate_pkt_pointers_by_static_tail_call(struct __sk_buff *sk)
> +{
> +	int *p = (void *)(long)sk->data;
> +
> +	if ((void *)(p + 1) > (void *)(long)sk->data_end)
> +		return TCX_DROP;
> +	static_tail_call(sk);
> +	*p = 42; /* this is unsafe */
> +	return TCX_PASS;
> +}
> +
> +/* Direct tail calls do not invalidate packet pointers. */
> +SEC("tc")
> +__success
> +int invalidate_pkt_pointers_by_tail_call(struct __sk_buff *sk)
> +{
> +	int *p = (void *)(long)sk->data;
> +
> +	if ((void *)(p + 1) > (void *)(long)sk->data_end)
> +		return TCX_DROP;
> +	bpf_tail_call_static(sk, &jmp_table, 0);
> +	*p = 42; /* this is NOT unsafe: tail calls don't return */
> +	return TCX_PASS;
> +}
> +
>  char _license[] SEC("license") = "GPL";