* Pahole/BTF issue with __int128
@ 2025-05-07 20:02 Alexis Lothoré
2025-05-07 20:39 ` Tony Ambardar
0 siblings, 1 reply; 3+ messages in thread
From: Alexis Lothoré @ 2025-05-07 20:02 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Alan Maguire, bpf, dwarves
Hello,
I am working on some ebpf feature for ARM64 (improving trampolines to
attach tracing programs to functions with more arguments than the current
limit), and I am facing an issue with the generated BTF information when
playing with large int types like __int128 (I need to use those large types
to properly test some architecture-specific alignment expectations). I
suspect the issue to be in pahole, but I would like to get some opinions on
my observations, and maybe some guidance on where to look at to go further.
I would like to attach some fentry/fexit programs to the following kind of
function, which is currently defined in a kernel module (bpf_testmod.ko in
bpf selftests):
struct bpf_testmod_struct_arg_7 {
_int128 a;
};
noinline int bpf_testmod_test_struct_arg_11(
struct bpf_testmod_struct_arg_7 a,
struct bpf_testmod_struct_arg_7 b,
struct bpf_testmod_struct_arg_7 c,
struct bpf_testmod_struct_arg_7 d,
short e,
struct bpf_testmod_struct_arg_7 f)
{
[...]
}
This one works well (let's call it case 1), I am able to attach
fentry/fexit programs to such function through libbpf.
However, if, in a case 2, I change the bpf_testmod_test_struct_arg_11
prototype to use __in128 arguments instead of struct arguments, like the
following one:
noinline int bpf_testmod_test_struct_arg_11(
__int128 a,
__int128 b,
__int128 c,
__int128 d,
short e,
__int128 f)
{
[...]
}
and rebuild the module/run my test, this does not work anymore, and libbpf
complains with the following error:
libbpf: prog 'test_struct_many_args_9': failed to find kernel BTF type ID
of 'bpf_testmod_test_struct_arg_11': -ESRCH
Inspecting the generated BTF information in bpf_testmod.ko file with bpftool, I
indeed find some BTF info related to my target func in case 1 but not in
case 2:
[...]
[118] STRUCT 'bpf_testmod_struct_arg_7' size=16 vlen=1
'a' type_id=10 bits_offset=0
[...]
[371] FUNC_PROTO '(anon)' ret_type_id=6 vlen=6
'a' type_id=118
'b' type_id=118
'c' type_id=118
'd' type_id=118
'e' type_id=5
'f' type_id=118
[372] FUNC 'bpf_testmod_test_struct_arg_11' type_id=371 linkage=static
[...]
I checked the command executed by the kernel build system to generate BTF
info for the module, and got the following one:
pahole -J -j\
--btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs\
--btf_features=attributes --lang_exclude=rust\
--btf_features=distilled_base --btf_base vmlinux\
tools/testing/selftests/bpf/bpf_testmod.ko
I ran the same command before/after switching the struct arguments to
__int128, and made the same observation (I am running pahole 1.30). I then
took a look at available DWARF info available in bpf_testmod.ko for pahole
to generate BTF info, and AFAICT, it looks ok (to be confirmed ?) in both
cases (I am using an aarch64-linux-gcc toolchain, v13.2.0 from
https://toolchains.bootlin.com/)
Case 1:
[...]
<1><262>: Abbrev Number: 106 (DW_TAG_base_type)
<263> DW_AT_byte_size : 16
<264> DW_AT_encoding : 5 (signed)
<265> DW_AT_name : (indirect string, offset: 0x193bc): __int128
[...]
<1><23429>: Abbrev Number: 11 (DW_TAG_structure_type)
<2342a> DW_AT_name : (indirect string, offset: 0xe98d): bpf_testmod_struct_arg_7
<2342e> DW_AT_byte_size : 16
<2342f> DW_AT_decl_file : 1
<23430> DW_AT_decl_line : 70
<23431> DW_AT_decl_column : 8
<23432> DW_AT_sibling : <0x23442>
<2><23436>: Abbrev Number: 12 (DW_TAG_member)
<23437> DW_AT_name : a
<23439> DW_AT_decl_file : 1
<2343a> DW_AT_decl_line : 71
<2343b> DW_AT_decl_column : 11
<2343c> DW_AT_type : <0x262>
<23440> DW_AT_data_member_location: 0
[...]
<1><295c1>: Abbrev Number: 99 (DW_TAG_subprogram)
<295c2> DW_AT_external : 1
<295c2> DW_AT_name : (indirect string, offset: 0x5e20): bpf_testmod_test_struct_arg_11
<295c6> DW_AT_decl_file : 1
<295c7> DW_AT_decl_line : 152
<295c8> DW_AT_decl_column : 14
<295c9> DW_AT_prototyped : 1
<295c9> DW_AT_type : <0xdd>
<295cd> DW_AT_low_pc : 0x1380
<295d5> DW_AT_high_pc : 0x34
<295dd> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
<295df> DW_AT_GNU_all_call_sites: 1
<295df> DW_AT_sibling : <0x2964a>
<2><295e3>: Abbrev Number: 45 (DW_TAG_formal_parameter)
<295e4> DW_AT_name : a
<295e6> DW_AT_decl_file : 1
<295e7> DW_AT_decl_line : 152
<295e8> DW_AT_decl_column : 77
<295e9> DW_AT_type : <0x23429>
<295ed> DW_AT_location : 0x6196 (location list)
<295f1> DW_AT_GNU_locviews: 0x6194
[...]
Case 2:
[...]
<1><262>: Abbrev Number: 106 (DW_TAG_base_type)
<263> DW_AT_byte_size : 16
<264> DW_AT_encoding : 5 (signed)
<265> DW_AT_name : (indirect string, offset: 0x1935d): __int128
[...]
<1><29552>: Abbrev Number: 98 (DW_TAG_subprogram)
<29553> DW_AT_external : 1
<29553> DW_AT_name : (indirect string, offset: 0x5e20): bpf_testmod_test_struct_arg_11
<29557> DW_AT_decl_file : 1
<29558> DW_AT_decl_line : 148
<29559> DW_AT_decl_column : 14
<2955a> DW_AT_prototyped : 1
<2955a> DW_AT_type : <0xdd>
<2955e> DW_AT_low_pc : 0x1380
<29566> DW_AT_high_pc : 0x34
<2956e> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
<29570> DW_AT_GNU_all_call_sites: 1
<29570> DW_AT_sibling : <0x295d6>
<2><29574>: Abbrev Number: 46 (DW_TAG_formal_parameter)
<29575> DW_AT_name : a
<29577> DW_AT_decl_file : 1
<29578> DW_AT_decl_line : 148
<29579> DW_AT_decl_column : 54
<2957a> DW_AT_type : <0x262>
<2957e> DW_AT_location : 0x6158 (location list)
<29582> DW_AT_GNU_locviews: 0x6154
[...]
Am I missing some constraint or limitation that would prevent the case 2
function from being described with BTF info ? If not, any advice about how
to debug this further ?
Thanks,
Alexis
--
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: Pahole/BTF issue with __int128
2025-05-07 20:02 Pahole/BTF issue with __int128 Alexis Lothoré
@ 2025-05-07 20:39 ` Tony Ambardar
2025-05-07 22:20 ` Alexis Lothoré
0 siblings, 1 reply; 3+ messages in thread
From: Tony Ambardar @ 2025-05-07 20:39 UTC (permalink / raw)
To: Alexis Lothoré; +Cc: Arnaldo Carvalho de Melo, Alan Maguire, bpf, dwarves
On Wed, May 07, 2025 at 10:02:51PM +0200, Alexis Lothoré wrote:
> Hello,
>
> I am working on some ebpf feature for ARM64 (improving trampolines to
> attach tracing programs to functions with more arguments than the current
> limit), and I am facing an issue with the generated BTF information when
> playing with large int types like __int128 (I need to use those large types
> to properly test some architecture-specific alignment expectations). I
> suspect the issue to be in pahole, but I would like to get some opinions on
> my observations, and maybe some guidance on where to look at to go further.
>
> I would like to attach some fentry/fexit programs to the following kind of
> function, which is currently defined in a kernel module (bpf_testmod.ko in
> bpf selftests):
>
> struct bpf_testmod_struct_arg_7 {
> _int128 a;
> };
>
> noinline int bpf_testmod_test_struct_arg_11(
> struct bpf_testmod_struct_arg_7 a,
> struct bpf_testmod_struct_arg_7 b,
> struct bpf_testmod_struct_arg_7 c,
> struct bpf_testmod_struct_arg_7 d,
> short e,
> struct bpf_testmod_struct_arg_7 f)
> {
> [...]
> }
>
> This one works well (let's call it case 1), I am able to attach
> fentry/fexit programs to such function through libbpf.
>
> However, if, in a case 2, I change the bpf_testmod_test_struct_arg_11
> prototype to use __in128 arguments instead of struct arguments, like the
> following one:
>
> noinline int bpf_testmod_test_struct_arg_11(
> __int128 a,
> __int128 b,
> __int128 c,
> __int128 d,
> short e,
> __int128 f)
> {
> [...]
> }
>
> and rebuild the module/run my test, this does not work anymore, and libbpf
> complains with the following error:
> libbpf: prog 'test_struct_many_args_9': failed to find kernel BTF type ID
> of 'bpf_testmod_test_struct_arg_11': -ESRCH
>
> Inspecting the generated BTF information in bpf_testmod.ko file with bpftool, I
> indeed find some BTF info related to my target func in case 1 but not in
> case 2:
>
> [...]
> [118] STRUCT 'bpf_testmod_struct_arg_7' size=16 vlen=1
> 'a' type_id=10 bits_offset=0
> [...]
> [371] FUNC_PROTO '(anon)' ret_type_id=6 vlen=6
> 'a' type_id=118
> 'b' type_id=118
> 'c' type_id=118
> 'd' type_id=118
> 'e' type_id=5
> 'f' type_id=118
> [372] FUNC 'bpf_testmod_test_struct_arg_11' type_id=371 linkage=static
> [...]
>
> I checked the command executed by the kernel build system to generate BTF
> info for the module, and got the following one:
> pahole -J -j\
> --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs\
> --btf_features=attributes --lang_exclude=rust\
> --btf_features=distilled_base --btf_base vmlinux\
> tools/testing/selftests/bpf/bpf_testmod.ko
>
> I ran the same command before/after switching the struct arguments to
> __int128, and made the same observation (I am running pahole 1.30). I then
> took a look at available DWARF info available in bpf_testmod.ko for pahole
> to generate BTF info, and AFAICT, it looks ok (to be confirmed ?) in both
> cases (I am using an aarch64-linux-gcc toolchain, v13.2.0 from
> https://toolchains.bootlin.com/)
>
> Case 1:
>
> [...]
> <1><262>: Abbrev Number: 106 (DW_TAG_base_type)
> <263> DW_AT_byte_size : 16
> <264> DW_AT_encoding : 5 (signed)
> <265> DW_AT_name : (indirect string, offset: 0x193bc): __int128
> [...]
> <1><23429>: Abbrev Number: 11 (DW_TAG_structure_type)
> <2342a> DW_AT_name : (indirect string, offset: 0xe98d): bpf_testmod_struct_arg_7
> <2342e> DW_AT_byte_size : 16
> <2342f> DW_AT_decl_file : 1
> <23430> DW_AT_decl_line : 70
> <23431> DW_AT_decl_column : 8
> <23432> DW_AT_sibling : <0x23442>
> <2><23436>: Abbrev Number: 12 (DW_TAG_member)
> <23437> DW_AT_name : a
> <23439> DW_AT_decl_file : 1
> <2343a> DW_AT_decl_line : 71
> <2343b> DW_AT_decl_column : 11
> <2343c> DW_AT_type : <0x262>
> <23440> DW_AT_data_member_location: 0
> [...]
> <1><295c1>: Abbrev Number: 99 (DW_TAG_subprogram)
> <295c2> DW_AT_external : 1
> <295c2> DW_AT_name : (indirect string, offset: 0x5e20): bpf_testmod_test_struct_arg_11
> <295c6> DW_AT_decl_file : 1
> <295c7> DW_AT_decl_line : 152
> <295c8> DW_AT_decl_column : 14
> <295c9> DW_AT_prototyped : 1
> <295c9> DW_AT_type : <0xdd>
> <295cd> DW_AT_low_pc : 0x1380
> <295d5> DW_AT_high_pc : 0x34
> <295dd> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
> <295df> DW_AT_GNU_all_call_sites: 1
> <295df> DW_AT_sibling : <0x2964a>
> <2><295e3>: Abbrev Number: 45 (DW_TAG_formal_parameter)
> <295e4> DW_AT_name : a
> <295e6> DW_AT_decl_file : 1
> <295e7> DW_AT_decl_line : 152
> <295e8> DW_AT_decl_column : 77
> <295e9> DW_AT_type : <0x23429>
> <295ed> DW_AT_location : 0x6196 (location list)
> <295f1> DW_AT_GNU_locviews: 0x6194
> [...]
>
> Case 2:
>
> [...]
> <1><262>: Abbrev Number: 106 (DW_TAG_base_type)
> <263> DW_AT_byte_size : 16
> <264> DW_AT_encoding : 5 (signed)
> <265> DW_AT_name : (indirect string, offset: 0x1935d): __int128
> [...]
> <1><29552>: Abbrev Number: 98 (DW_TAG_subprogram)
> <29553> DW_AT_external : 1
> <29553> DW_AT_name : (indirect string, offset: 0x5e20): bpf_testmod_test_struct_arg_11
> <29557> DW_AT_decl_file : 1
> <29558> DW_AT_decl_line : 148
> <29559> DW_AT_decl_column : 14
> <2955a> DW_AT_prototyped : 1
> <2955a> DW_AT_type : <0xdd>
> <2955e> DW_AT_low_pc : 0x1380
> <29566> DW_AT_high_pc : 0x34
> <2956e> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
> <29570> DW_AT_GNU_all_call_sites: 1
> <29570> DW_AT_sibling : <0x295d6>
> <2><29574>: Abbrev Number: 46 (DW_TAG_formal_parameter)
> <29575> DW_AT_name : a
> <29577> DW_AT_decl_file : 1
> <29578> DW_AT_decl_line : 148
> <29579> DW_AT_decl_column : 54
> <2957a> DW_AT_type : <0x262>
> <2957e> DW_AT_location : 0x6158 (location list)
> <29582> DW_AT_GNU_locviews: 0x6154
> [...]
>
Hi Alexis,
> Am I missing some constraint or limitation that would prevent the case 2
> function from being described with BTF info ? If not, any advice about how
> to debug this further ?
>
I suspect this might be related to an issue I ran into where pahole may
mis-encode types larger than register-size [1]. Out of curiosity, could
you try rebuilding and using a pahole with my latest patch [2]?
1: https://lore.kernel.org/dwarves/20250410083359.198724-1-tony.ambardar@gmail.com/
2: https://lore.kernel.org/dwarves/20250502070318.1561924-1-tony.ambardar@gmail.com/
Cheers,
Tony
> Thanks,
>
> Alexis
>
> --
> Alexis Lothoré, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: Pahole/BTF issue with __int128
2025-05-07 20:39 ` Tony Ambardar
@ 2025-05-07 22:20 ` Alexis Lothoré
0 siblings, 0 replies; 3+ messages in thread
From: Alexis Lothoré @ 2025-05-07 22:20 UTC (permalink / raw)
To: Tony Ambardar; +Cc: Arnaldo Carvalho de Melo, Alan Maguire, bpf, dwarves
Hi Tony,
thanks a lot for the prompt answer !
On Wed May 7, 2025 at 10:39 PM CEST, Tony Ambardar wrote:
> On Wed, May 07, 2025 at 10:02:51PM +0200, Alexis Lothoré wrote:
>> Hello,
[...]
> Hi Alexis,
>
>> Am I missing some constraint or limitation that would prevent the case 2
>> function from being described with BTF info ? If not, any advice about how
>> to debug this further ?
>>
>
> I suspect this might be related to an issue I ran into where pahole may
> mis-encode types larger than register-size [1]. Out of curiosity, could
> you try rebuilding and using a pahole with my latest patch [2]?
>
> 1: https://lore.kernel.org/dwarves/20250410083359.198724-1-tony.ambardar@gmail.com/
> 2: https://lore.kernel.org/dwarves/20250502070318.1561924-1-tony.ambardar@gmail.com/
I gave a try to your patch on top of pahole 1.30, and indeed it seems that
my issue is fixed with your solution. I now have some BTF info for my
bpf_testmod_test_struct_arg_11 func:
[...]
[370] FUNC_PROTO '(anon)' ret_type_id=6 vlen=6
'a' type_id=10
'b' type_id=10
'c' type_id=10
'd' type_id=10
'e' type_id=5
'f' type_id=10
[371] FUNC 'bpf_testmod_test_struct_arg_11' type_id=370 linkage=static
[...]
I also did some quick tests around Alan's request in your series, I'll
report to your series' thread.
Thanks for the help !
Alexis
>
> Cheers,
> Tony
>
>> Thanks,
>>
>> Alexis
>>
>> --
>> Alexis Lothoré, Bootlin
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
--
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2025-05-07 22:20 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-05-07 20:02 Pahole/BTF issue with __int128 Alexis Lothoré
2025-05-07 20:39 ` Tony Ambardar
2025-05-07 22:20 ` Alexis Lothoré
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox