[PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[Jiayuan Chen]

This patch exposes the btf_custom_path feature to bpftool, allowing users
to specify a custom BTF file when loading BPF programs using prog load or
prog loadall commands. This feature is already supported by libbpf, and
this patch makes it accessible through the bpftool command-line interface.

Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
---
 tools/bpf/bpftool/prog.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index f010295350be..63f84e765b34 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -1681,8 +1681,17 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
 		} else if (is_prefix(*argv, "autoattach")) {
 			auto_attach = true;
 			NEXT_ARG();
+		} else if (is_prefix(*argv, "custom_btf")) {
+			NEXT_ARG();
+
+			if (!REQ_ARGS(1))
+				goto err_free_reuse_maps;
+
+			open_opts.btf_custom_path = GET_ARG();
 		} else {
-			p_err("expected no more arguments, 'type', 'map' or 'dev', got: '%s'?",
+			p_err("expected no more arguments, "
+			      "'type', 'map', 'dev', 'offload_dev', 'xdpmeta_dev', 'pinmaps', "
+			      "'autoattach', or 'custom_btf', got: '%s'?",
 			      *argv);
 			goto err_free_reuse_maps;
 		}
-- 
2.47.1

Re: [PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[Alexei Starovoitov]

On Mon, May 12, 2025 at 8:59 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>
> This patch exposes the btf_custom_path feature to bpftool, allowing users
> to specify a custom BTF file when loading BPF programs using prog load or
> prog loadall commands. This feature is already supported by libbpf, and
> this patch makes it accessible through the bpftool command-line interface.
>
> Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
> ---
>  tools/bpf/bpftool/prog.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
> index f010295350be..63f84e765b34 100644
> --- a/tools/bpf/bpftool/prog.c
> +++ b/tools/bpf/bpftool/prog.c
> @@ -1681,8 +1681,17 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
>                 } else if (is_prefix(*argv, "autoattach")) {
>                         auto_attach = true;
>                         NEXT_ARG();
> +               } else if (is_prefix(*argv, "custom_btf")) {
> +                       NEXT_ARG();
> +
> +                       if (!REQ_ARGS(1))
> +                               goto err_free_reuse_maps;
> +
> +                       open_opts.btf_custom_path = GET_ARG();

I don't see a use case yet.
What exactly is the scenario where it's useful ?

Re: [PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[Jiayuan Chen]

2025/5/14 05:19, "Alexei Starovoitov" <alexei.starovoitov@gmail.com> wrote:

> 
> On Mon, May 12, 2025 at 8:59 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
> 
> > 
> > This patch exposes the btf_custom_path feature to bpftool, allowing users
> > 
> >  to specify a custom BTF file when loading BPF programs using prog load or
> > 
> >  prog loadall commands. This feature is already supported by libbpf, and
> > 
> >  this patch makes it accessible through the bpftool command-line interface.
> > 
> >  Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
> > 
> >  ---
> > 
> >  tools/bpf/bpftool/prog.c | 11 ++++++++++-
> > 
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> >  diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
> > 
> >  index f010295350be..63f84e765b34 100644
> > 
> >  --- a/tools/bpf/bpftool/prog.c
> > 
> >  +++ b/tools/bpf/bpftool/prog.c
> > 
> >  @@ -1681,8 +1681,17 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
> > 
> >  } else if (is_prefix(*argv, "autoattach")) {
> > 
> >  auto_attach = true;
> > 
> >  NEXT_ARG();
> > 
> >  + } else if (is_prefix(*argv, "custom_btf")) {
> > 
> >  + NEXT_ARG();
> > 
> >  +
> > 
> >  + if (!REQ_ARGS(1))
> > 
> >  + goto err_free_reuse_maps;
> > 
> >  +
> > 
> >  + open_opts.btf_custom_path = GET_ARG();
> > 
> 
> I don't see a use case yet.
> 
> What exactly is the scenario where it's useful ?
>

This patch just exposes the btf_custom_path feature of libbpf to bpftool.
The argument 'btf_custom_path' in libbpf is used for those kernes that
don't have CONFIG_DEBUG_INFO_BTF enabled but still want to perform CO-RE
relocations. Specifically for older kernels, separate BTF files are already
provided: https://github.com/aquasecurity/btfhub-archive/.
If we want load prog using bpftool on those systems, we have to hack
btf__load_vmlinux_btf() before or write custom loader with libbpf and specify
'btf_custom_path'.

I also found a the similar topic:
https://lore.kernel.org/bpf/20220215225856.671072-1-mauricio@kinvolk.io/

Additionally, pwru supports "--kernel-btf" which serves the same purpose as
this patch.

Therefore, using an external BTF file is a common practice.

Re: [PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[Andrii Nakryiko]

On Tue, May 13, 2025 at 6:51 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>
> 2025/5/14 05:19, "Alexei Starovoitov" <alexei.starovoitov@gmail.com> wrote:
>
> >
> > On Mon, May 12, 2025 at 8:59 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
> >
> > >
> > > This patch exposes the btf_custom_path feature to bpftool, allowing users
> > >
> > >  to specify a custom BTF file when loading BPF programs using prog load or
> > >
> > >  prog loadall commands. This feature is already supported by libbpf, and
> > >
> > >  this patch makes it accessible through the bpftool command-line interface.
> > >
> > >  Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
> > >
> > >  ---
> > >
> > >  tools/bpf/bpftool/prog.c | 11 ++++++++++-
> > >
> > >  1 file changed, 10 insertions(+), 1 deletion(-)
> > >
> > >  diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
> > >
> > >  index f010295350be..63f84e765b34 100644
> > >
> > >  --- a/tools/bpf/bpftool/prog.c
> > >
> > >  +++ b/tools/bpf/bpftool/prog.c
> > >
> > >  @@ -1681,8 +1681,17 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
> > >
> > >  } else if (is_prefix(*argv, "autoattach")) {
> > >
> > >  auto_attach = true;
> > >
> > >  NEXT_ARG();
> > >
> > >  + } else if (is_prefix(*argv, "custom_btf")) {
> > >
> > >  + NEXT_ARG();
> > >
> > >  +
> > >
> > >  + if (!REQ_ARGS(1))
> > >
> > >  + goto err_free_reuse_maps;
> > >
> > >  +
> > >
> > >  + open_opts.btf_custom_path = GET_ARG();
> > >
> >
> > I don't see a use case yet.
> >
> > What exactly is the scenario where it's useful ?
> >
>
> This patch just exposes the btf_custom_path feature of libbpf to bpftool.
> The argument 'btf_custom_path' in libbpf is used for those kernes that
> don't have CONFIG_DEBUG_INFO_BTF enabled but still want to perform CO-RE
> relocations. Specifically for older kernels, separate BTF files are already
> provided: https://github.com/aquasecurity/btfhub-archive/.
> If we want load prog using bpftool on those systems, we have to hack
> btf__load_vmlinux_btf() before or write custom loader with libbpf and specify
> 'btf_custom_path'.
>
> I also found a the similar topic:
> https://lore.kernel.org/bpf/20220215225856.671072-1-mauricio@kinvolk.io/
>
> Additionally, pwru supports "--kernel-btf" which serves the same purpose as
> this patch.
>
> Therefore, using an external BTF file is a common practice.

I think it's fine to expose this to bpftool. But maybe call the option
"kernel_btf" to make it more obvious that this is BTF representing
kernel types, as opposed to program BTF itself.

Re: [PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[Quentin Monnet]

2025-05-14 09:39 UTC-0700 ~ Andrii Nakryiko <andrii.nakryiko@gmail.com>
> On Tue, May 13, 2025 at 6:51 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>>
>> 2025/5/14 05:19, "Alexei Starovoitov" <alexei.starovoitov@gmail.com> wrote:
>>
>>>
>>> On Mon, May 12, 2025 at 8:59 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>>>
>>>>
>>>> This patch exposes the btf_custom_path feature to bpftool, allowing users
>>>>
>>>>  to specify a custom BTF file when loading BPF programs using prog load or
>>>>
>>>>  prog loadall commands. This feature is already supported by libbpf, and
>>>>
>>>>  this patch makes it accessible through the bpftool command-line interface.
>>>>
>>>>  Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
>>>>
>>>>  ---
>>>>
>>>>  tools/bpf/bpftool/prog.c | 11 ++++++++++-
>>>>
>>>>  1 file changed, 10 insertions(+), 1 deletion(-)
>>>>
>>>>  diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
>>>>
>>>>  index f010295350be..63f84e765b34 100644
>>>>
>>>>  --- a/tools/bpf/bpftool/prog.c
>>>>
>>>>  +++ b/tools/bpf/bpftool/prog.c
>>>>
>>>>  @@ -1681,8 +1681,17 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
>>>>
>>>>  } else if (is_prefix(*argv, "autoattach")) {
>>>>
>>>>  auto_attach = true;
>>>>
>>>>  NEXT_ARG();
>>>>
>>>>  + } else if (is_prefix(*argv, "custom_btf")) {
>>>>
>>>>  + NEXT_ARG();
>>>>
>>>>  +
>>>>
>>>>  + if (!REQ_ARGS(1))
>>>>
>>>>  + goto err_free_reuse_maps;
>>>>
>>>>  +
>>>>
>>>>  + open_opts.btf_custom_path = GET_ARG();
>>>>
>>>
>>> I don't see a use case yet.
>>>
>>> What exactly is the scenario where it's useful ?
>>>
>>
>> This patch just exposes the btf_custom_path feature of libbpf to bpftool.
>> The argument 'btf_custom_path' in libbpf is used for those kernes that
>> don't have CONFIG_DEBUG_INFO_BTF enabled but still want to perform CO-RE
>> relocations. Specifically for older kernels, separate BTF files are already
>> provided: https://github.com/aquasecurity/btfhub-archive/.
>> If we want load prog using bpftool on those systems, we have to hack
>> btf__load_vmlinux_btf() before or write custom loader with libbpf and specify
>> 'btf_custom_path'.
>>
>> I also found a the similar topic:
>> https://lore.kernel.org/bpf/20220215225856.671072-1-mauricio@kinvolk.io/
>>
>> Additionally, pwru supports "--kernel-btf" which serves the same purpose as
>> this patch.
>>
>> Therefore, using an external BTF file is a common practice.
> 
> I think it's fine to expose this to bpftool. But maybe call the option
> "kernel_btf" to make it more obvious that this is BTF representing
> kernel types, as opposed to program BTF itself.


Hi Jiayuan, we'll also need to update the documentation (the man page,
in the summary at the top and in the subcommand description), the
interactive help message at the bottom of bpftool's prog.c, and the bash
completion (I can help with it if necessary), please.

Thanks,
Quentin

Re: [PATCH bpf-next v1] bpftool: Add support for custom BTF path in prog load/loadall

[Jiayuan Chen]

2025/5/15 24:52, "Quentin Monnet" <qmo@kernel.org> wrote:

> 
> 2025-05-14 09:39 UTC-0700 ~ Andrii Nakryiko <andrii.nakryiko@gmail.com>
> 
> > 
> > On Tue, May 13, 2025 at 6:51 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
> > 
> > > 
> > > 2025/5/14 05:19, "Alexei Starovoitov" <alexei.starovoitov@gmail.com> wrote:
> > > 
> > 
> >  On Mon, May 12, 2025 at 8:59 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
> > 
> >  This patch exposes the btf_custom_path feature to bpftool, allowing users
> > 
> >  to specify a custom BTF file when loading BPF programs using prog load or
> > 
> >  prog loadall commands. This feature is already supported by libbpf, and
> > 
> >  this patch makes it accessible through the bpftool command-line interface.
> > 
> >  Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
> > 
> >  ---
> > 
> >  tools/bpf/bpftool/prog.c | 11 ++++++++++-
> > 
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> >  diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
> > 
> >  index f010295350be..63f84e765b34 100644
> > 
> >  --- a/tools/bpf/bpftool/prog.c
> > 
> >  +++ b/tools/bpf/bpftool/prog.c
> > 
> >  @@ -1681,8 +1681,17 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
> > 
> >  } else if (is_prefix(*argv, "autoattach")) {
> > 
> >  auto_attach = true;
> > 
> >  NEXT_ARG();
> > 
> >  + } else if (is_prefix(*argv, "custom_btf")) {
> > 
> >  + NEXT_ARG();
> > 
> >  +
> > 
> >  + if (!REQ_ARGS(1))
> > 
> >  + goto err_free_reuse_maps;
> > 
> >  +
> > 
> >  + open_opts.btf_custom_path = GET_ARG();
> > 
> >  I don't see a use case yet.
> > 
> >  What exactly is the scenario where it's useful ?
> > 
> > > 
> > > This patch just exposes the btf_custom_path feature of libbpf to bpftool.
> > > 
> > >  The argument 'btf_custom_path' in libbpf is used for those kernes that
> > > 
> > >  don't have CONFIG_DEBUG_INFO_BTF enabled but still want to perform CO-RE
> > > 
> > >  relocations. Specifically for older kernels, separate BTF files are already
> > > 
> > >  provided: https://github.com/aquasecurity/btfhub-archive/.
> > > 
> > >  If we want load prog using bpftool on those systems, we have to hack
> > > 
> > >  btf__load_vmlinux_btf() before or write custom loader with libbpf and specify
> > > 
> > >  'btf_custom_path'.
> > > 
> > >  I also found a the similar topic:
> > > 
> > >  https://lore.kernel.org/bpf/20220215225856.671072-1-mauricio@kinvolk.io/
> > > 
> > >  Additionally, pwru supports "--kernel-btf" which serves the same purpose as
> > > 
> > >  this patch.
> > > 
> > >  Therefore, using an external BTF file is a common practice.
> > > 
> > 
> >  
> > 
> >  I think it's fine to expose this to bpftool. But maybe call the option
> > 
> >  "kernel_btf" to make it more obvious that this is BTF representing
> > 
> >  kernel types, as opposed to program BTF itself.
> > 
> 

"kernel_btf" is better, thanks.

> Hi Jiayuan, we'll also need to update the documentation (the man page,
> 
> in the summary at the top and in the subcommand description), the
> 
> interactive help message at the bottom of bpftool's prog.c, and the bash
> 
> completion (I can help with it if necessary), please.
> 
> Thanks,
> 
> Quentin
>

Thank you for pointing that out. I did miss those parts, and I also
noticed that others had overlooked similar issues. I will address all
the subcommands of "load prog" in this update.