EF_BPF_GNU_XBPF

EF_BPF_GNU_XBPF

[Jose E. Marchesi]

Hello BPF people!

In order to ease the testing of the GCC bpf port we are adding a number
of extensions to the BPF ISA.

We would like to use one bit in the e_flags field of the ELF header in
order to flag that the code in the ELF file is not plain eBPF:

For EM_BPF:

#define EF_BPF_GNU_XBPF 0x00000001

Any objection?
Salud!

Re: EF_BPF_GNU_XBPF

[Alexei Starovoitov]

On Wed, Sep 2, 2020 at 11:22 AM Jose E. Marchesi
<jose.marchesi@oracle.com> wrote:
>
>
> Hello BPF people!
>
> In order to ease the testing of the GCC bpf port we are adding a number
> of extensions to the BPF ISA.
>
> We would like to use one bit in the e_flags field of the ELF header in
> order to flag that the code in the ELF file is not plain eBPF:
>
> For EM_BPF:
>
> #define EF_BPF_GNU_XBPF 0x00000001
>
> Any objection?

I've looked at your lpc slides and the extensions don't look like BPF
extensions.
At least I didn't see any attempt to make them verifiable.
In that sense it's not BPF and it's not correct to use EM_BPF for it.
I suggest to define your own EM for your ISA.

Re: EF_BPF_GNU_XBPF

[Jose E. Marchesi]

>> Hello BPF people!
>>
>> In order to ease the testing of the GCC bpf port we are adding a number
>> of extensions to the BPF ISA.
>>
>> We would like to use one bit in the e_flags field of the ELF header in
>> order to flag that the code in the ELF file is not plain eBPF:
>>
>> For EM_BPF:
>>
>> #define EF_BPF_GNU_XBPF 0x00000001
>>
>> Any objection?
>
> I've looked at your lpc slides and the extensions don't look like BPF
> extensions.
> At least I didn't see any attempt to make them verifiable.

It is an extension in the sense it is a superset of BPF.  Call it a
variant if you want.

As such, the property of being verifiable is irrelevant.  If we wanted
the extensions to be verifiable we would probably ask for them to be
added to BPF proper.  That's not the case, and that's precisely the
reason why we need to use that bit.

> In that sense it's not BPF and it's not correct to use EM_BPF for it.
> I suggest to define your own EM for your ISA.

Sorry, but that's not how ELF machine numbers work :)

It is perfectly normal, and also common practice, to use the same
machine code for several variants of the same base architecture, and I
see no reason for EM_BPF to be handled differently.  It would be silly,
and very inconvenient, to allocate a new machine number.

I don't think you people are using that bit in e_flags (or any bit, for
that matter).  So we just need to agree in reserving that bit for
EF_BPF_GNU_XBPF.

Shouldn't be a big matter surely?  There are 31 bits left ;)

Re: EF_BPF_GNU_XBPF

[Alexei Starovoitov]

On Wed, Sep 02, 2020 at 10:19:58PM +0200, Jose E. Marchesi wrote:
> 
> As such, the property of being verifiable is irrelevant.

No. It's a fundamental property of BPF.
If it's not verifiable it's not BPF. It's not xBPF either.
Please call it something else and don't confuse people that your ISA
has any overlap with BPF. It doesn't. It's not verifiable.

Re: EF_BPF_GNU_XBPF

[Jose E. Marchesi]

>> As such, the property of being verifiable is irrelevant.
>
> No. It's a fundamental property of BPF.
> If it's not verifiable it's not BPF.

Sure.

> It's not xBPF either.

Heh, beg to differ :)

> Please call it something else and don't confuse people that your ISA
> has any overlap with BPF. It doesn't. It's not verifiable.

Nonsense.  xBPF has as much overlap with BPF as it can have: around 99%.

The purpose of having the e_flag is to avoid confusion, not to increase
it.  xBPF objects are mainly used to test the GCC BPF backend (and other
purposes we have in mind, like ease the debugging of BPF programs) but
we want to eliminate the chance of these objects to be confused with
legit BPF files, and used as such.

Re: EF_BPF_GNU_XBPF

[Daniel Borkmann]

On 9/2/20 11:17 PM, Jose E. Marchesi wrote:
> 
>>> As such, the property of being verifiable is irrelevant.
>>
>> No. It's a fundamental property of BPF.
>> If it's not verifiable it's not BPF.
> 
> Sure.
> 
>> It's not xBPF either.
> 
> Heh, beg to differ :)
> 
>> Please call it something else and don't confuse people that your ISA
>> has any overlap with BPF. It doesn't. It's not verifiable.
> 
> Nonsense.  xBPF has as much overlap with BPF as it can have: around 99%.
> 
> The purpose of having the e_flag is to avoid confusion, not to increase
> it.  xBPF objects are mainly used to test the GCC BPF backend (and other
> purposes we have in mind, like ease the debugging of BPF programs) but
> we want to eliminate the chance of these objects to be confused with
> legit BPF files, and used as such.

I fully agree with Alexei. Looking at [0], if some of these extensions are
useful and help/optimize code generation, why not add them to the BPF runtime
in the kernel so they can be properly used in general for code generation
from gcc/llvm in BPF backend? xBPF would indeed be highly confusing if it cannot
be used from the runtime (unless these are properly integrated into the kernel,
verified and thus become a fixed part of eBPF ISA).

   [0] https://linuxplumbersconf.org/event/7/contributions/724/attachments/636/1166/bpf.pdf

Re: EF_BPF_GNU_XBPF

[Jose E. Marchesi]

Hi Daniel.

>>>> As such, the property of being verifiable is irrelevant.
>>>
>>> No. It's a fundamental property of BPF.
>>> If it's not verifiable it's not BPF.
>> Sure.
>> 
>>> It's not xBPF either.
>> Heh, beg to differ :)
>> 
>>> Please call it something else and don't confuse people that your ISA
>>> has any overlap with BPF. It doesn't. It's not verifiable.
>> Nonsense.  xBPF has as much overlap with BPF as it can have: around
>> 99%.
>> The purpose of having the e_flag is to avoid confusion, not to
>> increase
>> it.  xBPF objects are mainly used to test the GCC BPF backend (and other
>> purposes we have in mind, like ease the debugging of BPF programs) but
>> we want to eliminate the chance of these objects to be confused with
>> legit BPF files, and used as such.
>
> I fully agree with Alexei. Looking at [0], if some of these extensions are
> useful and help/optimize code generation, why not add them to the BPF runtime
> in the kernel so they can be properly used in general for code generation
> from gcc/llvm in BPF backend?

The reasons why xBPF came to existence are:

1) Due to BPF being so restrictive, many hundreds of GCC tests won't
   even build, because they use functions having more than 5 arguments,
   or functions with too big stack frames, or indirect calls, etc.  We
   want to be able to test our backend properly, so we added the -mxbpf
   option in order to relax some of these restrictions.

2) We are working on a BPF simulator that works with GDB.  For that to
   work, we needed to add a "breakpoint" instruction that GDB can patch
   in the program.  Having a simulator also allows us to run more GCC
   tests.

3) With some extensions, it becomes possible to support DWARF call frame
   information, and therefore to debug BPF programs in GDB with
   unwinding support.  You can build with -mxbpf, debug, then build
   again without -mxbpf.

[We have received messages from people saying that a more relaxed
 variant of BPF would be useful in some userland contexts, and xbpf
 could certainly be used for that, but that's secondary.]

So, xBPF is mainly about compiler validation and debugging of BPF
programs.  It is not about helping with code optimization, and the
extensions it implements are clearly unsuitable for the kernel.

That's why it is important to clearly flag the ELF files that make use
of these extensions, and therefore our intention of using a bit in
e_flags for that purpose, and this thread.

The LLVM backend could also benefit from xBPF, for exactly the same
reasons than GCC.  In fact, right now the LLVM backend generates
non-conforming BPF instructions under certain circumstances, and the
resulting (invalid) objects are not annotated at all.

> xBPF would indeed be highly confusing if it cannot be used from the
> runtime (unless these are properly integrated into the kernel,
> verified and thus become a fixed part of eBPF ISA).
>
>   [0] https://linuxplumbersconf.org/event/7/contributions/724/attachments/636/1166/bpf.pdf

My intention was to discuss about these matters during LPC in the
Toolchain MC:

https://linuxplumbersconf.org/event/7/contributions/752/attachments/689/1288/toolchain-MC-bpf-discussion.pdf

Unfortunately no BPF people were present during the session, probably
because of scheduling, but we can do so via email.

We just want to provide good tools to you people :)

Re: EF_BPF_GNU_XBPF

[David Miller]

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date: Wed, 2 Sep 2020 13:32:06 -0700

> On Wed, Sep 02, 2020 at 10:19:58PM +0200, Jose E. Marchesi wrote:
>> 
>> As such, the property of being verifiable is irrelevant.
> 
> No. It's a fundamental property of BPF.
> If it's not verifiable it's not BPF. It's not xBPF either.
> Please call it something else and don't confuse people that your ISA
> has any overlap with BPF. It doesn't. It's not verifiable.

I have to agree with Alexei here.  You are trying to create something
which is not fundamentally BPF and it will create a lot of confusion
and hardship on people who are working on BPF when you publish
binaries with this machine type.

Please don't do stuff like this, thank you.

Re: EF_BPF_GNU_XBPF

[David Miller]

From: Daniel Borkmann <daniel@iogearbox.net>
Date: Wed, 2 Sep 2020 23:33:17 +0200

> if some of these extensions are useful and help/optimize code
> generation, why not add them to the BPF runtime in the kernel so
> they can be properly used in general for code generation from
> gcc/llvm in BPF backend?

+1

Re: EF_BPF_GNU_XBPF

[David Miller]

From: "Jose E. Marchesi" <jose.marchesi@oracle.com>
Date: Thu, 03 Sep 2020 00:10:07 +0200

> 1) Due to BPF being so restrictive, many hundreds of GCC tests won't
>    even build, because they use functions having more than 5 arguments,
>    or functions with too big stack frames, or indirect calls, etc.  We
>    want to be able to test our backend properly, so we added the -mxbpf
>    option in order to relax some of these restrictions.
> 
> 2) We are working on a BPF simulator that works with GDB.  For that to
>    work, we needed to add a "breakpoint" instruction that GDB can patch
>    in the program.  Having a simulator also allows us to run more GCC
>    tests.
> 
> 3) With some extensions, it becomes possible to support DWARF call frame
>    information, and therefore to debug BPF programs in GDB with
>    unwinding support.  You can build with -mxbpf, debug, then build
>    again without -mxbpf.

All sounds like features to propose for BPF itself, rather than throw
into some weird extension.

Why not come to the bpf community and discuss the need for these
features?

Re: EF_BPF_GNU_XBPF

[Jose E. Marchesi]

Hi David.

>> On Wed, Sep 02, 2020 at 10:19:58PM +0200, Jose E. Marchesi wrote:
>>> 
>>> As such, the property of being verifiable is irrelevant.
>> 
>> No. It's a fundamental property of BPF.
>> If it's not verifiable it's not BPF. It's not xBPF either.
>> Please call it something else and don't confuse people that your ISA
>> has any overlap with BPF. It doesn't. It's not verifiable.
>
> I have to agree with Alexei here.  You are trying to create something
> which is not fundamentally BPF and it will create a lot of confusion
> and hardship on people who are working on BPF when you publish
> binaries with this machine type.

How would that create confusion and harship if the binaries are clearly
marked as containing extensions?

It is the lack of flagging that would create such confusion, and that
situation is precisely what I'm trying to avoid with this proposal of
using a bit in e_flags.

Re: EF_BPF_GNU_XBPF

[Jose E. Marchesi]

>> 1) Due to BPF being so restrictive, many hundreds of GCC tests won't
>>    even build, because they use functions having more than 5 arguments,
>>    or functions with too big stack frames, or indirect calls, etc.  We
>>    want to be able to test our backend properly, so we added the -mxbpf
>>    option in order to relax some of these restrictions.
>> 
>> 2) We are working on a BPF simulator that works with GDB.  For that to
>>    work, we needed to add a "breakpoint" instruction that GDB can patch
>>    in the program.  Having a simulator also allows us to run more GCC
>>    tests.
>> 
>> 3) With some extensions, it becomes possible to support DWARF call frame
>>    information, and therefore to debug BPF programs in GDB with
>>    unwinding support.  You can build with -mxbpf, debug, then build
>>    again without -mxbpf.
>
> All sounds like features to propose for BPF itself, rather than throw
> into some weird extension.
>
> Why not come to the bpf community and discuss the need for these
> features?

Well, as I said these features are mainly intended for the benefit of
the tooling side.  But of course if some of them are deemed useful
enough (and feasible) to be added to BPF proper, then great!

As for discussion, here I am, as I was in LPC :)