Re: [PATCH bpf-next v8 1/4] bpf: Relax tracing prog recursive attach rules

[Jiri Olsa <olsajiri@gmail.com>]

On Tue, Dec 12, 2023 at 08:54:06PM +0100, Dmitrii Dolgov wrote:
> Currently, it's not allowed to attach an fentry/fexit prog to another
> one fentry/fexit. At the same time it's not uncommon to see a tracing
> program with lots of logic in use, and the attachment limitation
> prevents usage of fentry/fexit for performance analysis (e.g. with
> "bpftool prog profile" command) in this case. An example could be
> falcosecurity libs project that uses tp_btf tracing programs.
> 
> Following the corresponding discussion [1], the reason for that is to
> avoid tracing progs call cycles without introducing more complex
> solutions. But currently it seems impossible to load and attach tracing
> programs in a way that will form such a cycle. The limitation is coming
> from the fact that attach_prog_fd is specified at the prog load (thus
> making it impossible to attach to a program loaded after it in this
> way), as well as tracing progs not implementing link_detach.
> 
> Replace "no same type" requirement with verification that no more than
> one level of attachment nesting is allowed. In this way only one
> fentry/fexit program could be attached to another fentry/fexit to cover
> profiling use case, and still no cycle could be formed. To implement,
> add a new field into bpf_prog_aux to track nested attachment for tracing
> programs.
> 
> [1]: https://lore.kernel.org/bpf/20191108064039.2041889-16-ast@kernel.org/
> 
> Signed-off-by: Dmitrii Dolgov <9erthalion6@gmail.com>
> ---
> Previous discussion: https://lore.kernel.org/bpf/20231208185557.8477-1-9erthalion6@gmail.com/
> 
> Changes in v8:
>     - Move bookkeping in bpf_tracing_link_release under the tgt_prog
>       condition.
>     - Fix some indentation issues.
> 
> Changes in v7:
>     - Replace attach_depth with a boolean flag to indicate a program is
>       already tracing an fentry/fexit.
> 
> Changes in v6:
>     - Apply nesting level limitation only to tracing programs, otherwise
>       it's possible to apply it in "fentry->extension" case and break it
> 
> Changes in v5:
>     - Remove follower_cnt and drop unreachable cycle prevention condition
>     - Allow only one level of attachment nesting
>     - Do not display attach_depth in bpftool, as it doesn't make sense
>       anymore
> 
> Changes in v3:
>     - Fix incorrect decreasing of attach_depth, setting to 0 instead
>     - Place bookkeeping later, to not miss a cleanup if needed
>     - Display attach_depth in bpftool only if the value is not 0
> 
> Changes in v2:
>     - Verify tgt_prog is not null
>     - Replace boolean followed with number of followers, to handle
>       multiple progs attaching/detaching
> 
>  include/linux/bpf.h   |  1 +
>  kernel/bpf/syscall.c  | 10 +++++++++-
>  kernel/bpf/verifier.c | 39 +++++++++++++++++++++++++--------------
>  3 files changed, 35 insertions(+), 15 deletions(-)
> 
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index eb447b0a9423..e7393674ab94 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -1414,6 +1414,7 @@ struct bpf_prog_aux {
>  	bool dev_bound; /* Program is bound to the netdev. */
>  	bool offload_requested; /* Program is bound and offloaded to the netdev. */
>  	bool attach_btf_trace; /* true if attaching to BTF-enabled raw tp */
> +	bool attach_tracing_prog; /* true if tracing another tracing program */
>  	bool func_proto_unreliable;
>  	bool sleepable;
>  	bool tail_call_reachable;
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 5e43ddd1b83f..af51e97c2c28 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -3040,8 +3040,10 @@ static void bpf_tracing_link_release(struct bpf_link *link)
>  	bpf_trampoline_put(tr_link->trampoline);
>  
>  	/* tgt_prog is NULL if target is a kernel function */
> -	if (tr_link->tgt_prog)
> +	if (tr_link->tgt_prog) {
>  		bpf_prog_put(tr_link->tgt_prog);
> +		link->prog->aux->attach_tracing_prog = false;
> +	}
>  }
>  
>  static void bpf_tracing_link_dealloc(struct bpf_link *link)
> @@ -3243,6 +3245,12 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog,
>  		goto out_unlock;
>  	}
>  
> +	/* Bookkeeping for managing the prog attachment chain */
> +	if (tgt_prog &&
> +		prog->type == BPF_PROG_TYPE_TRACING &&
> +		tgt_prog->type == BPF_PROG_TYPE_TRACING)
> +			prog->aux->attach_tracing_prog = true;

hi,
this still looks bad, I think it should be:

+	if (tgt_prog &&
+	    prog->type == BPF_PROG_TYPE_TRACING &&
+	    tgt_prog->type == BPF_PROG_TYPE_TRACING)
+		prog->aux->attach_tracing_prog = true;

other than that the patchset looks good to me

thanks,
jirka


> +
>  	link->tgt_prog = tgt_prog;
>  	link->trampoline = tr;
>  
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 8e7b6072e3f4..f8c15ce8fd05 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -20077,6 +20077,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
>  			    struct bpf_attach_target_info *tgt_info)
>  {
>  	bool prog_extension = prog->type == BPF_PROG_TYPE_EXT;
> +	bool prog_tracing = prog->type == BPF_PROG_TYPE_TRACING;
>  	const char prefix[] = "btf_trace_";
>  	int ret = 0, subprog = -1, i;
>  	const struct btf_type *t;
> @@ -20147,10 +20148,21 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
>  			bpf_log(log, "Can attach to only JITed progs\n");
>  			return -EINVAL;
>  		}
> -		if (tgt_prog->type == prog->type) {
> -			/* Cannot fentry/fexit another fentry/fexit program.
> -			 * Cannot attach program extension to another extension.
> -			 * It's ok to attach fentry/fexit to extension program.
> +		if (prog_tracing) {
> +			if (aux->attach_tracing_prog) {
> +				/*
> +				 * Target program is an fentry/fexit which is already attached
> +				 * to another tracing program. More levels of nesting
> +				 * attachment are not allowed.
> +				 */
> +				bpf_log(log, "Cannot nest tracing program attach more than once\n");
> +				return -EINVAL;
> +			}
> +		} else if (tgt_prog->type == prog->type) {
> +			/*
> +			 * To avoid potential call chain cycles, prevent attaching of a
> +			 * program extension to another extension. It's ok to attach
> +			 * fentry/fexit to extension program.
>  			 */
>  			bpf_log(log, "Cannot recursively attach\n");
>  			return -EINVAL;
> @@ -20163,16 +20175,15 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
>  			 * except fentry/fexit. The reason is the following.
>  			 * The fentry/fexit programs are used for performance
>  			 * analysis, stats and can be attached to any program
> -			 * type except themselves. When extension program is
> -			 * replacing XDP function it is necessary to allow
> -			 * performance analysis of all functions. Both original
> -			 * XDP program and its program extension. Hence
> -			 * attaching fentry/fexit to BPF_PROG_TYPE_EXT is
> -			 * allowed. If extending of fentry/fexit was allowed it
> -			 * would be possible to create long call chain
> -			 * fentry->extension->fentry->extension beyond
> -			 * reasonable stack size. Hence extending fentry is not
> -			 * allowed.
> +			 * type. When extension program is replacing XDP function
> +			 * it is necessary to allow performance analysis of all
> +			 * functions. Both original XDP program and its program
> +			 * extension. Hence attaching fentry/fexit to
> +			 * BPF_PROG_TYPE_EXT is allowed. If extending of
> +			 * fentry/fexit was allowed it would be possible to create
> +			 * long call chain fentry->extension->fentry->extension
> +			 * beyond reasonable stack size. Hence extending fentry
> +			 * is not allowed.
>  			 */
>  			bpf_log(log, "Cannot extend fentry/fexit\n");
>  			return -EINVAL;
> -- 
> 2.41.0
>