Support for gcc

Support for gcc

[SuHsueyu]

Hello, I use gcc 12.1.0 to compile a source file:
t.c
struct t {
  int a:2;
  int b:3;
  int c:2;
} g;
with gcc -c -gbtf t.c
and try to use libbpf API btf__parse_split, bpf_object__open, and
bpf_object__open to parse and load into the kernel, but it failed with
"libbpf: elf: /path/to/t.o is not a valid eBPF object file".

Is it wrong for me to do so? Due to some constraint, I cannot use
clang but gcc. How to parse and load gcc compiled object file with
libbpf?

Re: Support for gcc

[sdf]

On 12/20, SuHsueyu wrote:
> Hello, I use gcc 12.1.0 to compile a source file:
> t.c
> struct t {
>    int a:2;
>    int b:3;
>    int c:2;
> } g;
> with gcc -c -gbtf t.c
> and try to use libbpf API btf__parse_split, bpf_object__open, and
> bpf_object__open to parse and load into the kernel, but it failed with
> "libbpf: elf: /path/to/t.o is not a valid eBPF object file".

> Is it wrong for me to do so? Due to some constraint, I cannot use
> clang but gcc. How to parse and load gcc compiled object file with
> libbpf?

I haven't used gcc myself, so can't really help. All I can do is to
point you to these two places:

* https://gcc.gnu.org/wiki/BPFBackEnd
* tools/testing/selftests/bpf/Makefile

Maybe try to follow the instructions on that BPFBackEnd page and try
to compile kernel's selftests to see what's missing?

Re: Support for gcc

[Andrii Nakryiko]

On Tue, Dec 20, 2022 at 3:56 AM SuHsueyu <anolasc13@gmail.com> wrote:
>
> Hello, I use gcc 12.1.0 to compile a source file:
> t.c
> struct t {
>   int a:2;
>   int b:3;
>   int c:2;
> } g;
> with gcc -c -gbtf t.c
> and try to use libbpf API btf__parse_split, bpf_object__open, and
> bpf_object__open to parse and load into the kernel, but it failed with
> "libbpf: elf: /path/to/t.o is not a valid eBPF object file".

if (ehdr->e_type != ET_REL || (ehdr->e_machine && ehdr->e_machine != EM_BPF))

This check is failing in libbpf. So check which of those two are not
set appropriately. cc Jose to point where to report GCC-BPF specific
issues.

>
> Is it wrong for me to do so? Due to some constraint, I cannot use
> clang but gcc. How to parse and load gcc compiled object file with
> libbpf?

Re: Support for gcc

[SuHsueyu]

I used clang and gcc to generate the obj files, and then readelf these
two files, I found that there is a problem with the machine item, the
machine item generated by clang is "Linux BPF", and the one generated
by gcc is my physical machine architecture "x86-64", which makes the
check of "ehdr->e_machine != EM_BPF" fail.
This seems to be a problem with my not specifying a compilation option
similar to clang target  when compiling with gcc.

On Wed, Dec 21, 2022 at 8:34 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Dec 20, 2022 at 3:56 AM SuHsueyu <anolasc13@gmail.com> wrote:
> >
> > Hello, I use gcc 12.1.0 to compile a source file:
> > t.c
> > struct t {
> >   int a:2;
> >   int b:3;
> >   int c:2;
> > } g;
> > with gcc -c -gbtf t.c
> > and try to use libbpf API btf__parse_split, bpf_object__open, and
> > bpf_object__open to parse and load into the kernel, but it failed with
> > "libbpf: elf: /path/to/t.o is not a valid eBPF object file".
>
> if (ehdr->e_type != ET_REL || (ehdr->e_machine && ehdr->e_machine != EM_BPF))
>
> This check is failing in libbpf. So check which of those two are not
> set appropriately. cc Jose to point where to report GCC-BPF specific
> issues.
>
> >
> > Is it wrong for me to do so? Due to some constraint, I cannot use
> > clang but gcc. How to parse and load gcc compiled object file with
> > libbpf?

Re: Support for gcc

[Alan Maguire]

On 21/12/2022 11:03, SuHsueyu wrote:
> I used clang and gcc to generate the obj files, and then readelf these
> two files, I found that there is a problem with the machine item, the
> machine item generated by clang is "Linux BPF", and the one generated
> by gcc is my physical machine architecture "x86-64", which makes the
> check of "ehdr->e_machine != EM_BPF" fail.
> This seems to be a problem with my not specifying a compilation option
> similar to clang target  when compiling with gcc.
>
> On Wed, Dec 21, 2022 at 8:34 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Tue, Dec 20, 2022 at 3:56 AM SuHsueyu <anolasc13@gmail.com> wrote:
>>>
>>> Hello, I use gcc 12.1.0 to compile a source file:
>>> t.c
>>> struct t {
>>>   int a:2;
>>>   int b:3;
>>>   int c:2;
>>> } g;
>>> with gcc -c -gbtf t.c
>>> and try to use libbpf API btf__parse_split, bpf_object__open, and
>>> bpf_object__open to parse and load into the kernel, but it failed with
>>> "libbpf: elf: /path/to/t.o is not a valid eBPF object file".
>>
>> if (ehdr->e_type != ET_REL || (ehdr->e_machine && ehdr->e_machine != EM_BPF))
>>
>> This check is failing in libbpf. So check which of those two are not
>> set appropriately. cc Jose to point where to report GCC-BPF specific
>> issues.
>>
>>>
>>> Is it wrong for me to do so? Due to some constraint, I cannot use
>>> clang but gcc. How to parse and load gcc compiled object file with
>>> libbpf?

As mentioned https://gcc.gnu.org/wiki/BPFBackEnd has some details;
I used those to install gcc-bpf-unknown-none/binutils-bpf-unknown-none
(package names differ for different distros so use the wiki page to
find the ones you need) and ran

$ bpf-unknown-none-gcc t.c -c -o t.o -gbtf
$ bpftool btf dump file t.o
[1] STRUCT 't' size=4 vlen=3
	'a' type_id=2 bits_offset=0 bitfield_size=2
	'b' type_id=2 bits_offset=2 bitfield_size=3
	'c' type_id=2 bits_offset=5 bitfield_size=2
[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] VAR 'g' type_id=1, linkage=global
[4] DATASEC '.bss' size=0 vlen=1
	type_id=3 offset=0 size=4 (VAR 'g')

$ file t.o
t.o: ELF 64-bit LSB relocatable, eBPF, version 1 (SYSV), not stripped

Hope this helps,

Alan

Re: Support for gcc

[Jose E. Marchesi]

> On Tue, Dec 20, 2022 at 3:56 AM SuHsueyu <anolasc13@gmail.com> wrote:
>>
>> Hello, I use gcc 12.1.0 to compile a source file:
>> t.c
>> struct t {
>>   int a:2;
>>   int b:3;
>>   int c:2;
>> } g;
>> with gcc -c -gbtf t.c
>> and try to use libbpf API btf__parse_split, bpf_object__open, and
>> bpf_object__open to parse and load into the kernel, but it failed with
>> "libbpf: elf: /path/to/t.o is not a valid eBPF object file".
>
> if (ehdr->e_type != ET_REL || (ehdr->e_machine && ehdr->e_machine != EM_BPF))
>
> This check is failing in libbpf. So check which of those two are not
> set appropriately. cc Jose to point where to report GCC-BPF specific
> issues.

Thanks for the CC.
That would be https://gcc.gnu.org/bugzilla.

(I don't think this is a GCC issue and certainly not a libbpf one.
 Seems like the op is building a x86_64 object and not a BPF object.)

>
>>
>> Is it wrong for me to do so? Due to some constraint, I cannot use
>> clang but gcc. How to parse and load gcc compiled object file with
>> libbpf?

Re: Support for gcc

[Jose E. Marchesi]

> Hello, I use gcc 12.1.0 to compile a source file:
> t.c
> struct t {
>   int a:2;
>   int b:3;
>   int c:2;
> } g;
> with gcc -c -gbtf t.c
> and try to use libbpf API btf__parse_split, bpf_object__open, and
> bpf_object__open to parse and load into the kernel, but it failed with
> "libbpf: elf: /path/to/t.o is not a valid eBPF object file".
>
> Is it wrong for me to do so? Due to some constraint, I cannot use
> clang but gcc. How to parse and load gcc compiled object file with
> libbpf?

It seems to me that you are most likely using a GCC targetted at your
local architecture (x86_64-linux-gnu perhaps?) instead of
bpf-unknown-none.

(Note that GCC can generate BTF for any target, not just BPF.)

So you need a bpf-unknown-none-gcc toolchain.
You can either:

a) Install a pre-compiled cross available in your distro.
   Debian ships gcc-bpf, for example.  See
   https://gcc.gnu.org/wiki/BPFBackEnd for a list.

or,

b) Build crossed versions of binutils and gcc, configuring with
   --target=bpf-unknown-none.

or,

c) Use crosstool-ng to build a GCC BPF cross.  We recently added support
   for bpf-unknown-none there.

Hope this helps.

Re: Support for gcc

[Jose E. Marchesi]

> So you need a bpf-unknown-none-gcc toolchain.
> You can either:
>
> a) Install a pre-compiled cross available in your distro.
>    Debian ships gcc-bpf, for example.  See
>    https://gcc.gnu.org/wiki/BPFBackEnd for a list.
>
> or,
>
> b) Build crossed versions of binutils and gcc, configuring with
>    --target=bpf-unknown-none.
>
> or,
>
> c) Use crosstool-ng to build a GCC BPF cross.  We recently added support
>    for bpf-unknown-none there.

Incidentally, thanks to Marc Poulhiès godbolt.org has now support for
nightly builds of GCC BPF.

Re: Support for gcc

[SuHsueyu]

Thank you very much. It solved my problem; what bpf-gcc build can now
be loaded by libbpf. BTW, when I tried to use bpf-gcc in godbolt.org,
I did not add any additional compile options, and it reported an
error:

/opt/compiler-explorer/bpf/gcc-trunk-20221225/bpf-unknown-none/lib/gcc/bpf-unknown-none/13.0.0/../../../../bpf-unknown-none/bin/ld:
-pie not supported
collect2: error: ld returned 1 exit status
Compiler returned: 1

On Thu, Dec 22, 2022 at 2:26 AM Jose E. Marchesi
<jose.marchesi@oracle.com> wrote:
>
>
> > So you need a bpf-unknown-none-gcc toolchain.
> > You can either:
> >
> > a) Install a pre-compiled cross available in your distro.
> >    Debian ships gcc-bpf, for example.  See
> >    https://gcc.gnu.org/wiki/BPFBackEnd for a list.
> >
> > or,
> >
> > b) Build crossed versions of binutils and gcc, configuring with
> >    --target=bpf-unknown-none.
> >
> > or,
> >
> > c) Use crosstool-ng to build a GCC BPF cross.  We recently added support
> >    for bpf-unknown-none there.
>
> Incidentally, thanks to Marc Poulhiès godbolt.org has now support for
> nightly builds of GCC BPF.

Re: Support for gcc

[Jose E. Marchesi]

> Thank you very much. It solved my problem; what bpf-gcc build can now
> be loaded by libbpf.

Glad it helped :)

> BTW, when I tried to use bpf-gcc in godbolt.org, I did not add any
> additional compile options, and it reported an error:
>
> /opt/compiler-explorer/bpf/gcc-trunk-20221225/bpf-unknown-none/lib/gcc/bpf-unknown-none/13.0.0/../../../../bpf-unknown-none/bin/ld:
> -pie not supported
> collect2: error: ld returned 1 exit status
> Compiler returned: 1

Hmm, I just tried and it didn't add -pie to the command line options.  I
think that somehow godbolt.org remembers and re-uses the settings used
by the last user.  At least when it comes to select the cross compiler.
Maybe it is the same with the compilation options...

>
> On Thu, Dec 22, 2022 at 2:26 AM Jose E. Marchesi
> <jose.marchesi@oracle.com> wrote:
>>
>>
>> > So you need a bpf-unknown-none-gcc toolchain.
>> > You can either:
>> >
>> > a) Install a pre-compiled cross available in your distro.
>> >    Debian ships gcc-bpf, for example.  See
>> >    https://gcc.gnu.org/wiki/BPFBackEnd for a list.
>> >
>> > or,
>> >
>> > b) Build crossed versions of binutils and gcc, configuring with
>> >    --target=bpf-unknown-none.
>> >
>> > or,
>> >
>> > c) Use crosstool-ng to build a GCC BPF cross.  We recently added support
>> >    for bpf-unknown-none there.
>>
>> Incidentally, thanks to Marc Poulhiès godbolt.org has now support for
>> nightly builds of GCC BPF.

Re: Support for gcc

[Marc Poulhiès]

>> BTW, when I tried to use bpf-gcc in godbolt.org, I did not add any
>> additional compile options, and it reported an error:
>>
>> /opt/compiler-explorer/bpf/gcc-trunk-20221225/bpf-unknown-none/lib/gcc/bpf-unknown-none/13.0.0/../../../../bpf-unknown-none/bin/ld:
>> -pie not supported
>> collect2: error: ld returned 1 exit status
>> Compiler returned: 1
>
> Hmm, I just tried and it didn't add -pie to the command line options.  I
> think that somehow godbolt.org remembers and re-uses the settings used
> by the last user.  At least when it comes to select the cross compiler.
> Maybe it is the same with the compilation options...

Hi,

Happened to find this discussion and am taking the liberty to reply,
hope that's ok :)

Compiler-Explorer does not reuse anything from another user. It may
reuse something you've used in a previous session stored in your local
browser, but then this will show in the "Compiler Options...." text
field.

Here's my findings about this pie. By default, we only emit
assembly (so you won't see any ld error). But if you ask for linking to
a binary (Output >> Compile to binary), you will get this error:

https://godbolt.org/z/za3sbr5qn

Looks like our toolchain defaults to PIE. I'll look into that
(https://github.com/compiler-explorer/compiler-explorer/issues/4517).

You can use -fno-pie or uncheck "compile to binary" in the meantime.

Thanks,
Marc

PS: If you think you have a bug in compiler-explorer, click the "share"
menu (top right) and provide us with a link (short or full, both work,
but short will be easier to share in a mail).