Re: Errors compiling BPF programs from Linux selftests/bpf with GCC

["Jose E. Marchesi" <jose.marchesi@oracle.com>]

> On Thursday, January 2nd, 2025 at 1:47 AM, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:
>
>> Hi Ihor.
>> Thanks for working on this! :)
>> 
>> > [...]
>> > Older versions compile the dummy program without errors, however on
>> > attempt to build the selftests there is a different issue: conflicting
>> > int64 definitions (full log at [6]).
>> > 
>> > In file included from /usr/include/x86_64-linux-gnu/sys/types.h:155,
>> > from /usr/include/x86_64-linux-gnu/bits/socket.h:29,
>> > from /usr/include/x86_64-linux-gnu/sys/socket.h:33,
>> > from /usr/include/linux/if.h:28,
>> > from /usr/include/linux/icmp.h:23,
>> > from progs/test_cls_redirect_dynptr.c:10:
>> > /usr/include/x86_64-linux-gnu/bits/stdint-intn.h:27:19: error:
>> > conflicting types for ‘int64_t’; have ‘__int64_t’ {aka ‘long long
>> > int’}
>> > 27 | typedef __int64_t int64_t;
>> > | ^~~~~~~
>> > In file included from progs/test_cls_redirect_dynptr.c:6:
>> > /ci/workspace/bpfgcc.20240922/lib/gcc/bpf-unknown-none/15.0.0/include/stdint.h:43:24:
>> > note: previous declaration of ‘int64_t’ with type ‘int64_t’ {aka ‘long
>> > int’}
>> > 43 | typedef INT64_TYPE int64_t;
>> > | ^~~~~~~
>> 
>> 
>> I think this is what is going on:
>> 
>> The BPF selftest is indirectly including glibc headers from the host
>> where it is being compiled. In this case your x86_64 ubuntu system.
>> 
>> Many glibc headers include bits/wordsize.h, which in the case of x86_64
>> is:
>> 
>> #if defined x86_64 && !defined ILP32
>> # define __WORDSIZE 64
>> #else
>> # define __WORDSIZE 32
>> #define __WORDSIZE32_SIZE_ULONG 0
>> #define __WORDSIZE32_PTRDIFF_LONG 0
>> #endif
>> 
>> and then in bits/types.h:
>> 
>> #if __WORDSIZE == 64
>> typedef signed long int __int64_t;
>> typedef unsigned long int __uint64_t;
>> #else
>> extension typedef signed long long int __int64_t;
>> extension typedef unsigned long long int __uint64_t;
>> #endif
>> 
>> i.e. your BPF program ends using __WORDSIZE 32. This eventually leads
>> to int64_t being defined as `signed long long int' in stdint-intn.h, as
>> it would correspond to a x86_64 program running in 32-bit mode.
>> 
>> GCC BPF, on the other hand, is a "baremetal" compiler and it provides a
>> small set of headers (including stdint.h) that implement standard C99
>> types like int64_t, adjusted to the BPF architecture.
>> 
>> In this case there is a conflict between the 32-bit x86_64 definition of
>> int64_t and the one of BPF.
>
> Hi Jose, thanks for breaking this down.
>
> I was able to mitigate int64_t declaration conflict by passing
> -nostdinc to gcc.
>
> Currently system-installed headers are being passed via -idirafter in
> a compilation command:
>
>     /ci/workspace/bpfgcc.20241229/bin/bpf-unknown-none-gcc \
>         -g -Wall -Werror -D__TARGET_ARCH_x86 -mlittle-endian \
>         -I/ci/workspace/tools/testing/selftests/bpf/tools/include
>         -I/ci/workspace/tools/testing/selftests/bpf \
>         -I/ci/workspace/tools/include/uapi \
>         -I/ci/workspace/tools/testing/selftests/usr/include \
>         -Wno-compare-distinct-pointer-types \
>         -idirafter /usr/lib/gcc/x86_64-linux-gnu/13/include \
>         -idirafter /usr/local/include \
>         -idirafter /usr/include/x86_64-linux-gnu \
>         -idirafter /usr/include \
>         -DBPF_NO_PRESERVE_ACCESS_INDEX \
>         -Wno-attributes \
>         -O2 -std=gnu17 \                  # -nostdinc here helps
>         -c progs/test_cls_redirect.c \
>         -o /ci/workspace/tools/testing/selftests/bpf/bpf_gcc/test_cls_redirect.bpf.o
>
> Passing -nostdinc makes gcc to pick compiler-installed header, if I
> understand correctly.

I think that passing -nostdinc is having the opposite effect: it makes
GCC to not use its own distributed stdint.h, so it finds the host's
stdint.h via idirafter.

With that option GCC BPF behaves like clang BPF, include-wise.

>> 
>> PS: the other headers installed by GCC BPF are:
>> float.h iso646.h limits.h stdalign.h stdarg.h stdatomic.h stdbool.h
>> stdckdint.h stddef.h stdfix.h stdint.h stdnoreturn.h syslimits.h
>> tgmath.h unwind.h varargs.h
>
> From your comments, it seems that BPF programs *must not* include
> system glibc headers when compiled with GCC. Or is this only true for
> the headers you listed above?
>
> I wonder what is the proper way to build BPF programs with gcc then.
> In the source code the includes are what you'd expect:
>
>     #include <stdbool.h>
>     #include <stddef.h>
>     #include <stdint.h>       // conflict is between this
>     #include <string.h>
>
>     #include <linux/bpf.h>
>     #include <linux/icmp.h>   // and this
>     #include <linux/icmpv6.h>
>     ...
>
> Any suggestions?

IMO the BPP selftest (and BPF programs in general) must not include host
glibc headers at all, regardless of what BPF compiler is used.  The
glibc headers installed in the host are tailored to some particular
architecture, be it x86_64 or whatever, not necessarily compatible with
what the compilers assume for the BPF target.

This particular case shows the problem well: all the glibc headers
included by that BPF selftest assume that `long' is 32 bits, not 64
bits, because x86_64 is not defined.  This conflicts with both clang's
and GCC's assumption that in BPF a `long' is 64 bits.  This may or may
not be a problem, depending on whether the BPF program uses the stuff
defined in the headers and how it uses it.  Had you be using an arm or
sparc host instead of x86_64, you may be including macros and stuff that
assume chars are unsigned.  But chars are signed in bpf.

>
> Thanks.