Re: EF_BPF_GNU_XBPF

["Jose E. Marchesi" <jose.marchesi@oracle.com>]

Hi Daniel.

>>>> As such, the property of being verifiable is irrelevant.
>>>
>>> No. It's a fundamental property of BPF.
>>> If it's not verifiable it's not BPF.
>> Sure.
>> 
>>> It's not xBPF either.
>> Heh, beg to differ :)
>> 
>>> Please call it something else and don't confuse people that your ISA
>>> has any overlap with BPF. It doesn't. It's not verifiable.
>> Nonsense.  xBPF has as much overlap with BPF as it can have: around
>> 99%.
>> The purpose of having the e_flag is to avoid confusion, not to
>> increase
>> it.  xBPF objects are mainly used to test the GCC BPF backend (and other
>> purposes we have in mind, like ease the debugging of BPF programs) but
>> we want to eliminate the chance of these objects to be confused with
>> legit BPF files, and used as such.
>
> I fully agree with Alexei. Looking at [0], if some of these extensions are
> useful and help/optimize code generation, why not add them to the BPF runtime
> in the kernel so they can be properly used in general for code generation
> from gcc/llvm in BPF backend?

The reasons why xBPF came to existence are:

1) Due to BPF being so restrictive, many hundreds of GCC tests won't
   even build, because they use functions having more than 5 arguments,
   or functions with too big stack frames, or indirect calls, etc.  We
   want to be able to test our backend properly, so we added the -mxbpf
   option in order to relax some of these restrictions.

2) We are working on a BPF simulator that works with GDB.  For that to
   work, we needed to add a "breakpoint" instruction that GDB can patch
   in the program.  Having a simulator also allows us to run more GCC
   tests.

3) With some extensions, it becomes possible to support DWARF call frame
   information, and therefore to debug BPF programs in GDB with
   unwinding support.  You can build with -mxbpf, debug, then build
   again without -mxbpf.

[We have received messages from people saying that a more relaxed
 variant of BPF would be useful in some userland contexts, and xbpf
 could certainly be used for that, but that's secondary.]

So, xBPF is mainly about compiler validation and debugging of BPF
programs.  It is not about helping with code optimization, and the
extensions it implements are clearly unsuitable for the kernel.

That's why it is important to clearly flag the ELF files that make use
of these extensions, and therefore our intention of using a bit in
e_flags for that purpose, and this thread.

The LLVM backend could also benefit from xBPF, for exactly the same
reasons than GCC.  In fact, right now the LLVM backend generates
non-conforming BPF instructions under certain circumstances, and the
resulting (invalid) objects are not annotated at all.

> xBPF would indeed be highly confusing if it cannot be used from the
> runtime (unless these are properly integrated into the kernel,
> verified and thus become a fixed part of eBPF ISA).
>
>   [0] https://linuxplumbersconf.org/event/7/contributions/724/attachments/636/1166/bpf.pdf

My intention was to discuss about these matters during LPC in the
Toolchain MC:

https://linuxplumbersconf.org/event/7/contributions/752/attachments/689/1288/toolchain-MC-bpf-discussion.pdf

Unfortunately no BPF people were present during the session, probably
because of scheduling, but we can do so via email.

We just want to provide good tools to you people :)