Using gcc-bpf for bpftool: problems with CO-RE feature detection

Using gcc-bpf for bpftool: problems with CO-RE feature detection

[Holger Hoffstätte]

Hi,

I'm trying to use Gentoo's bpf-toolchain - basically just gcc built for
the BPF target - to build the CO-RE support in bpftool, in order to
provide an alternative to clang.

This currently fails because the feature detection relies on a comment
in the generated BPF assembly, which gcc does not seem to generate.

While I'm using the Github mirror for bpftool, the same check is
being done in the kernel build, so it affects both.

Our tracker bug with full output etc. is: https://bugs.gentoo.org/943113

Basically the problem boils down to:

	.long	16777248                        # 0x1000020
	.long	9                               # BTF_KIND_VAR(id = 3)
	.long	234881024                       # 0xe000000

generated by clang (19.1.3)

vs.

	.4byte	0x1000020
	.4byte	0x9
	.4byte	0xe000000

generated by gcc (14.2.0).

As the values themselves are correct, the problem is really just
the missing debug information in gcc's output. So far I've tried
every option I could find, but to no avail. I have no idea whether
this is because I'm holding it wrong, gcc cannot do it for the bpf
target (yet?) or anything else.

Does anybody know how I can convince gcc to generate symbol comments?
Alternatively can we find a better way to verify the generated output
instead of grepping for a comment?

This is not really a bug, but IMHO having an alternative toolchain to
build BPF seems like a good idea in general. Gentoo's bpf-toolchain
package was initially made to build dtrace, and seems to be working
fine so far.

Thanks for any suggestions!

cheers
Holger

Re: Using gcc-bpf for bpftool: problems with CO-RE feature detection

[Jose E. Marchesi]

Hello.

> I'm trying to use Gentoo's bpf-toolchain - basically just gcc built for
> the BPF target - to build the CO-RE support in bpftool, in order to
> provide an alternative to clang.
>
> This currently fails because the feature detection relies on a comment
> in the generated BPF assembly, which gcc does not seem to generate.
>
> While I'm using the Github mirror for bpftool, the same check is
> being done in the kernel build, so it affects both.
>
> Our tracker bug with full output etc. is: https://bugs.gentoo.org/943113
>
> Basically the problem boils down to:
>
> 	.long	16777248                        # 0x1000020
> 	.long	9                               # BTF_KIND_VAR(id = 3)
> 	.long	234881024                       # 0xe000000
>
> generated by clang (19.1.3)
>
> vs.
>
> 	.4byte	0x1000020
> 	.4byte	0x9
> 	.4byte	0xe000000
>
> generated by gcc (14.2.0).
>
> As the values themselves are correct, the problem is really just
> the missing debug information in gcc's output. So far I've tried
> every option I could find, but to no avail. I have no idea whether
> this is because I'm holding it wrong, gcc cannot do it for the bpf
> target (yet?) or anything else.
>
> Does anybody know how I can convince gcc to generate symbol comments?
> Alternatively can we find a better way to verify the generated output
> instead of grepping for a comment?

GCC can generate similar comments if you pass the -dA option.  These are
intended for testing the generated BTF in the GCC testsuite, however,
and right now I don't remember whether the comments mimic what clang
generates.  You can give it a try...

>
> This is not really a bug, but IMHO having an alternative toolchain to
> build BPF seems like a good idea in general. Gentoo's bpf-toolchain
> package was initially made to build dtrace, and seems to be working
> fine so far.
>
> Thanks for any suggestions!
>
> cheers
> Holger

Re: Using gcc-bpf for bpftool: problems with CO-RE feature detection

[Holger Hoffstätte]

On 2024-11-09 17:02, Jose E. Marchesi wrote:
> 
> Hello.
> 
>> I'm trying to use Gentoo's bpf-toolchain - basically just gcc built for
>> the BPF target - to build the CO-RE support in bpftool, in order to
>> provide an alternative to clang.
>>
>> This currently fails because the feature detection relies on a comment
>> in the generated BPF assembly, which gcc does not seem to generate.
>>
>> While I'm using the Github mirror for bpftool, the same check is
>> being done in the kernel build, so it affects both.
>>
>> Our tracker bug with full output etc. is: https://bugs.gentoo.org/943113
>>
>> Basically the problem boils down to:
>>
>> 	.long	16777248                        # 0x1000020
>> 	.long	9                               # BTF_KIND_VAR(id = 3)
>> 	.long	234881024                       # 0xe000000
>>
>> generated by clang (19.1.3)
>>
>> vs.
>>
>> 	.4byte	0x1000020
>> 	.4byte	0x9
>> 	.4byte	0xe000000
>>
>> generated by gcc (14.2.0).
>>
>> As the values themselves are correct, the problem is really just
>> the missing debug information in gcc's output. So far I've tried
>> every option I could find, but to no avail. I have no idea whether
>> this is because I'm holding it wrong, gcc cannot do it for the bpf
>> target (yet?) or anything else.
>>
>> Does anybody know how I can convince gcc to generate symbol comments?
>> Alternatively can we find a better way to verify the generated output
>> instead of grepping for a comment?
> 
> GCC can generate similar comments if you pass the -dA option.  These are
> intended for testing the generated BTF in the GCC testsuite, however,
> and right now I don't remember whether the comments mimic what clang
> generates.  You can give it a try...

Thank you so much for your reply, that's the one flag I hadn't tried yet,
and it does indeed output exactly the needed value.

After a bit of creative Makefile surgery I now get:

..
Makefile.feature:32: Probing: feature-clang-bpf-co-re
Makefile.feature:33: printf '%s\n' 'struct s { int i; } __attribute__((preserve_access_index)); struct s foo = {};' | bpf-unknown-none-gcc -g -dA -S -o - -x c -  | grep -q BTF_KIND_VAR && (echo 1 && >&2 echo result: 1) || (echo 0 && >&2 echo result: 0)
result: 1
..

There's a few things to clean up like clang-specific annotations and LLVM_STRIP,
but so far the resulting binary says:

   bpftool v7.5.0
   using libbpf v1.5
   features: libbfd, skeletons

so it seems to have worked. \o/

cheers
Holger