Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Jean-Philippe Brucker]

Hi,

[Adding the linux-arm-kernel list on Cc]

On Fri, Aug 07, 2020 at 04:20:58PM +0200, Jakov Petrina wrote:
> Hi everyone,
> 
> recently we have begun extensive research into eBPF and related
> technologies. Seeking an easier development process, we have switched over
> to using the eBPF CO-RE [0] approach internally which has enabled us to
> simplify most aspects of eBPF development, especially those related to
> cross-compilation.
> 
> However, as part of these efforts we have stumbled upon several problems
> that we feel would benefit from a community discussion where we may share
> our solutions and discuss alternatives moving forward.
> 
> As a reference point, we have started researching and modifying several eBPF
> CO-RE samples that have been developed or migrated from existing `bcc`
> tooling. Most notable examples are those present in `bcc`'s `libbpf-tools`
> directory [1]. Some of these samples have just recently been converted to
> respective eBPF CO-RE variants, of which the `tcpconnect` tracing sample has
> proven to be very interesting.
> 
> First showstopper for cross-compiling aforementioned example on the ARM
> 32-bit platform has been with regards to generation of the required
> `vmlinux.h` kernel header from the BTF information. More specifically, our
> initial approach to have e.g. a compilation target dependency which would
> invoke `bpftool` at configure time was not appropriate due to several
> issues: a) CO-RE requires host kernel to have been compiled in such a way to
> expose BTF information which may not available, and b) the generated
> `vmlinux.h` was actually architecture-specific.
> 
> The second point proved interesting because `tcpconnect` makes use of the
> `BPF_KPROBE` and `BPF_KRETPROBE` macros, which pass `struct pt_regs *ctx` as
> the first function parameter. The `pt_regs` structure is defined by the
> kernel and is architecture-specific. Since `libbpf` does have
> architecture-specific conditionals, pairing it with an "invalid" `vmlinux.h`
> resulted in cross-compilation failure as `libbpf` provided macros that work
> with ARM `pt_regs`, and `vmlinux.h` had an x86 `pt_regs` definition. To
> resolve this issue, we have resorted to including pre-generated
> `<arch>_vmlinux.h` files in our CO-RE build system.
> 
> However, there are certainly drawbacks to this approach: a) (relatively)
> large file size of the generated headers, b) regular maintenance to
> re-generate the header files for various architectures and kernel versions,
> and c) incompatible definitions being generated, to name a few. This last
> point relates to the the fact that our `aarch64`/`arm64` kernel generates
> the following definition using `bpftool`, which has resulted in compilation
> failure:
> 
> ```
> typedef __Poly8_t poly8x16_t[16];
> ```
> 
> AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> have opted to comment out this line as there was no additional `poly8x16_t`
> usage in the header file.

It looks like this "__Poly8_t" type is internal to GCC (provided in
arm_neon.h) and clang has its own internals. I managed to reproduce this
with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
the moment. Maybe libbpf should generate defines to translate these
intrinsics between clang and gcc? Not very elegant. I'll take another
look next week.

> Given various issues we have encountered so far (among which is a kernel
> panic/crash on a specific device), additional input and feedback regarding
> cross-compilation of the eBPF utilities would be greatly appreciated.

I don't know if there is a room for improvement regarding your a) and b)
points, as I think the added complexity is inherent to cross-building. But
kernel crashes definitely need to be fixed, as well as the above problem.

Thanks,
Jean

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Andrii Nakryiko]

On Fri, Aug 7, 2020 at 10:24 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Hi,
>
> [Adding the linux-arm-kernel list on Cc]
>
> On Fri, Aug 07, 2020 at 04:20:58PM +0200, Jakov Petrina wrote:
> > Hi everyone,
> >
> > recently we have begun extensive research into eBPF and related
> > technologies. Seeking an easier development process, we have switched over
> > to using the eBPF CO-RE [0] approach internally which has enabled us to
> > simplify most aspects of eBPF development, especially those related to
> > cross-compilation.
> >
> > However, as part of these efforts we have stumbled upon several problems
> > that we feel would benefit from a community discussion where we may share
> > our solutions and discuss alternatives moving forward.
> >
> > As a reference point, we have started researching and modifying several eBPF
> > CO-RE samples that have been developed or migrated from existing `bcc`
> > tooling. Most notable examples are those present in `bcc`'s `libbpf-tools`
> > directory [1]. Some of these samples have just recently been converted to
> > respective eBPF CO-RE variants, of which the `tcpconnect` tracing sample has
> > proven to be very interesting.
> >
> > First showstopper for cross-compiling aforementioned example on the ARM
> > 32-bit platform has been with regards to generation of the required
> > `vmlinux.h` kernel header from the BTF information. More specifically, our
> > initial approach to have e.g. a compilation target dependency which would
> > invoke `bpftool` at configure time was not appropriate due to several
> > issues: a) CO-RE requires host kernel to have been compiled in such a way to
> > expose BTF information which may not available, and b) the generated
> > `vmlinux.h` was actually architecture-specific.
> >
> > The second point proved interesting because `tcpconnect` makes use of the
> > `BPF_KPROBE` and `BPF_KRETPROBE` macros, which pass `struct pt_regs *ctx` as
> > the first function parameter. The `pt_regs` structure is defined by the
> > kernel and is architecture-specific. Since `libbpf` does have
> > architecture-specific conditionals, pairing it with an "invalid" `vmlinux.h`
> > resulted in cross-compilation failure as `libbpf` provided macros that work
> > with ARM `pt_regs`, and `vmlinux.h` had an x86 `pt_regs` definition. To
> > resolve this issue, we have resorted to including pre-generated
> > `<arch>_vmlinux.h` files in our CO-RE build system.
> >
> > However, there are certainly drawbacks to this approach: a) (relatively)
> > large file size of the generated headers, b) regular maintenance to
> > re-generate the header files for various architectures and kernel versions,
> > and c) incompatible definitions being generated, to name a few. This last
> > point relates to the the fact that our `aarch64`/`arm64` kernel generates
> > the following definition using `bpftool`, which has resulted in compilation
> > failure:
> >
> > ```
> > typedef __Poly8_t poly8x16_t[16];
> > ```
> >
> > AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> > have opted to comment out this line as there was no additional `poly8x16_t`
> > usage in the header file.
>
> It looks like this "__Poly8_t" type is internal to GCC (provided in
> arm_neon.h) and clang has its own internals. I managed to reproduce this
> with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> the moment. Maybe libbpf should generate defines to translate these
> intrinsics between clang and gcc? Not very elegant. I'll take another
> look next week.

libbpf is already blacklisting __builtin_va_list for GCC, so we can
just add __Poly8_t to the list. See [0].
Are there any other types like that? If you guys can provide me this,
I'll gladly update libbpf to take those compiler-provided
types/built-ins into account.

  [0] https://github.com/torvalds/linux/blob/master/tools/lib/bpf/btf_dump.c#L585-L598

>
> > Given various issues we have encountered so far (among which is a kernel
> > panic/crash on a specific device), additional input and feedback regarding
> > cross-compilation of the eBPF utilities would be greatly appreciated.
>
> I don't know if there is a room for improvement regarding your a) and b)
> points, as I think the added complexity is inherent to cross-building. But
> kernel crashes definitely need to be fixed, as well as the above problem.
>
> Thanks,
> Jean

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Alexei Starovoitov]

On Fri, Aug 7, 2020 at 11:41 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Fri, Aug 7, 2020 at 10:24 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > Hi,
> >
> > [Adding the linux-arm-kernel list on Cc]
> >
> > On Fri, Aug 07, 2020 at 04:20:58PM +0200, Jakov Petrina wrote:
> > > Hi everyone,
> > >
> > > recently we have begun extensive research into eBPF and related
> > > technologies. Seeking an easier development process, we have switched over
> > > to using the eBPF CO-RE [0] approach internally which has enabled us to
> > > simplify most aspects of eBPF development, especially those related to
> > > cross-compilation.
> > >
> > > However, as part of these efforts we have stumbled upon several problems
> > > that we feel would benefit from a community discussion where we may share
> > > our solutions and discuss alternatives moving forward.
> > >
> > > As a reference point, we have started researching and modifying several eBPF
> > > CO-RE samples that have been developed or migrated from existing `bcc`
> > > tooling. Most notable examples are those present in `bcc`'s `libbpf-tools`
> > > directory [1]. Some of these samples have just recently been converted to
> > > respective eBPF CO-RE variants, of which the `tcpconnect` tracing sample has
> > > proven to be very interesting.
> > >
> > > First showstopper for cross-compiling aforementioned example on the ARM
> > > 32-bit platform has been with regards to generation of the required
> > > `vmlinux.h` kernel header from the BTF information. More specifically, our
> > > initial approach to have e.g. a compilation target dependency which would
> > > invoke `bpftool` at configure time was not appropriate due to several
> > > issues: a) CO-RE requires host kernel to have been compiled in such a way to
> > > expose BTF information which may not available, and b) the generated
> > > `vmlinux.h` was actually architecture-specific.
> > >
> > > The second point proved interesting because `tcpconnect` makes use of the
> > > `BPF_KPROBE` and `BPF_KRETPROBE` macros, which pass `struct pt_regs *ctx` as
> > > the first function parameter. The `pt_regs` structure is defined by the
> > > kernel and is architecture-specific. Since `libbpf` does have
> > > architecture-specific conditionals, pairing it with an "invalid" `vmlinux.h`
> > > resulted in cross-compilation failure as `libbpf` provided macros that work
> > > with ARM `pt_regs`, and `vmlinux.h` had an x86 `pt_regs` definition. To
> > > resolve this issue, we have resorted to including pre-generated
> > > `<arch>_vmlinux.h` files in our CO-RE build system.
> > >
> > > However, there are certainly drawbacks to this approach: a) (relatively)
> > > large file size of the generated headers, b) regular maintenance to
> > > re-generate the header files for various architectures and kernel versions,
> > > and c) incompatible definitions being generated, to name a few. This last
> > > point relates to the the fact that our `aarch64`/`arm64` kernel generates
> > > the following definition using `bpftool`, which has resulted in compilation
> > > failure:
> > >
> > > ```
> > > typedef __Poly8_t poly8x16_t[16];
> > > ```
> > >
> > > AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> > > have opted to comment out this line as there was no additional `poly8x16_t`
> > > usage in the header file.
> >
> > It looks like this "__Poly8_t" type is internal to GCC (provided in
> > arm_neon.h) and clang has its own internals. I managed to reproduce this
> > with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> > the moment. Maybe libbpf should generate defines to translate these
> > intrinsics between clang and gcc? Not very elegant. I'll take another
> > look next week.
>
> libbpf is already blacklisting __builtin_va_list for GCC, so we can
> just add __Poly8_t to the list. See [0].
> Are there any other types like that? If you guys can provide me this,
> I'll gladly update libbpf to take those compiler-provided
> types/built-ins into account.

Shouldn't __Int8x16_t and friends cause the same trouble?
There is a bunch more in gcc/config/arm/arm-simd-builtin-types.def.
May be there is a way to detect compiler builtin types by pattern matching
their dwarf/btf shape and skip them automatically?
The simplest, of course, is to only add a few that caused this known
trouble to blocklist.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Jean-Philippe Brucker]

On Fri, Aug 07, 2020 at 01:54:02PM -0700, Alexei Starovoitov wrote:
[...]
> > > > ```
> > > > typedef __Poly8_t poly8x16_t[16];
> > > > ```
> > > >
> > > > AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> > > > have opted to comment out this line as there was no additional `poly8x16_t`
> > > > usage in the header file.
> > >
> > > It looks like this "__Poly8_t" type is internal to GCC (provided in
> > > arm_neon.h) and clang has its own internals. I managed to reproduce this
> > > with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> > > the moment. Maybe libbpf should generate defines to translate these
> > > intrinsics between clang and gcc? Not very elegant. I'll take another
> > > look next week.
> >
> > libbpf is already blacklisting __builtin_va_list for GCC, so we can
> > just add __Poly8_t to the list. See [0].
> > Are there any other types like that? If you guys can provide me this,
> > I'll gladly update libbpf to take those compiler-provided
> > types/built-ins into account.
> 
> Shouldn't __Int8x16_t and friends cause the same trouble?

I think these do get properly defined, for example in my vmlinux.h:

	typedef signed char int8x16_t[16];

From a cursory reading of the "ARM C Language Extension" doc (IHI0053D) it
looks like only the poly8/16/64/128_t types are unspecified. It's safe to
drop them as long as they're not used in structs or function parameters,
but I sent a more generic fix [1] that copies the clang defintions. When
building the kernel with clang, the polyX_t types do get typedefs.

Thanks,
Jean

[1] https://lore.kernel.org/bpf/20200810122835.2309026-1-jean-philippe@linaro.org/

> There is a bunch more in gcc/config/arm/arm-simd-builtin-types.def.
> May be there is a way to detect compiler builtin types by pattern matching
> their dwarf/btf shape and skip them automatically?
> The simplest, of course, is to only add a few that caused this known
> trouble to blocklist.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Andrii Nakryiko]

On Mon, Aug 10, 2020 at 5:58 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Fri, Aug 07, 2020 at 01:54:02PM -0700, Alexei Starovoitov wrote:
> [...]
> > > > > ```
> > > > > typedef __Poly8_t poly8x16_t[16];
> > > > > ```
> > > > >
> > > > > AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> > > > > have opted to comment out this line as there was no additional `poly8x16_t`
> > > > > usage in the header file.
> > > >
> > > > It looks like this "__Poly8_t" type is internal to GCC (provided in
> > > > arm_neon.h) and clang has its own internals. I managed to reproduce this
> > > > with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> > > > the moment. Maybe libbpf should generate defines to translate these
> > > > intrinsics between clang and gcc? Not very elegant. I'll take another
> > > > look next week.
> > >
> > > libbpf is already blacklisting __builtin_va_list for GCC, so we can
> > > just add __Poly8_t to the list. See [0].
> > > Are there any other types like that? If you guys can provide me this,
> > > I'll gladly update libbpf to take those compiler-provided
> > > types/built-ins into account.
> >
> > Shouldn't __Int8x16_t and friends cause the same trouble?
>
> I think these do get properly defined, for example in my vmlinux.h:
>
>         typedef signed char int8x16_t[16];
>
> From a cursory reading of the "ARM C Language Extension" doc (IHI0053D) it
> looks like only the poly8/16/64/128_t types are unspecified. It's safe to
> drop them as long as they're not used in structs or function parameters,
> but I sent a more generic fix [1] that copies the clang defintions. When
> building the kernel with clang, the polyX_t types do get typedefs.
>
> Thanks,
> Jean
>

Hi Jean,

Would you be so kind to build some simple C repro code that uses those
polyX_t types? Ideally built by both GCC and Clang. And then run
`pahole -J` on them to get .BTF into them as well. If you can share
those two with me, I'd love to look at how DWARF and BTF look like.

I'm, unfortunately, having trouble making something like that to
cross-compile on my x86-64 machine, I've spent a bunch of time already
on this unsuccessfully and it's really frustrating at this point. If
you have an ARM system (or cross-compilation set up properly), it
shouldn't take much time for you, hopefully. Just make sure that those
polyX_t types do make it into DWARF, so, e.g., use them with static
variable or something, e.g.,:

int main() {
    static poly8_t a = 12;
    return a + 10;
}

Or something along those lines. Thanks!

> [1] https://lore.kernel.org/bpf/20200810122835.2309026-1-jean-philippe@linaro.org/
>
> > There is a bunch more in gcc/config/arm/arm-simd-builtin-types.def.
> > May be there is a way to detect compiler builtin types by pattern matching
> > their dwarf/btf shape and skip them automatically?
> > The simplest, of course, is to only add a few that caused this known
> > trouble to blocklist.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Jean-Philippe Brucker]

On Mon, Aug 10, 2020 at 11:54:54PM -0700, Andrii Nakryiko wrote:
> On Mon, Aug 10, 2020 at 5:58 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > On Fri, Aug 07, 2020 at 01:54:02PM -0700, Alexei Starovoitov wrote:
> > [...]
> > > > > > ```
> > > > > > typedef __Poly8_t poly8x16_t[16];
> > > > > > ```
> > > > > >
> > > > > > AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> > > > > > have opted to comment out this line as there was no additional `poly8x16_t`
> > > > > > usage in the header file.
> > > > >
> > > > > It looks like this "__Poly8_t" type is internal to GCC (provided in
> > > > > arm_neon.h) and clang has its own internals. I managed to reproduce this
> > > > > with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> > > > > the moment. Maybe libbpf should generate defines to translate these
> > > > > intrinsics between clang and gcc? Not very elegant. I'll take another
> > > > > look next week.
> > > >
> > > > libbpf is already blacklisting __builtin_va_list for GCC, so we can
> > > > just add __Poly8_t to the list. See [0].
> > > > Are there any other types like that? If you guys can provide me this,
> > > > I'll gladly update libbpf to take those compiler-provided
> > > > types/built-ins into account.
> > >
> > > Shouldn't __Int8x16_t and friends cause the same trouble?
> >
> > I think these do get properly defined, for example in my vmlinux.h:
> >
> >         typedef signed char int8x16_t[16];
> >
> > From a cursory reading of the "ARM C Language Extension" doc (IHI0053D) it
> > looks like only the poly8/16/64/128_t types are unspecified. It's safe to
> > drop them as long as they're not used in structs or function parameters,
> > but I sent a more generic fix [1] that copies the clang defintions. When
> > building the kernel with clang, the polyX_t types do get typedefs.
> >
> > Thanks,
> > Jean
> >
> 
> Hi Jean,
> 
> Would you be so kind to build some simple C repro code that uses those
> polyX_t types? Ideally built by both GCC and Clang. And then run
> `pahole -J` on them to get .BTF into them as well. If you can share
> those two with me, I'd love to look at how DWARF and BTF look like.
> 
> I'm, unfortunately, having trouble making something like that to
> cross-compile on my x86-64 machine, I've spent a bunch of time already
> on this unsuccessfully and it's really frustrating at this point. If
> you have an ARM system (or cross-compilation set up properly), it
> shouldn't take much time for you, hopefully. Just make sure that those
> polyX_t types do make it into DWARF, so, e.g., use them with static
> variable or something, e.g.,:
> 
> int main() {
>     static poly8_t a = 12;
>     return a + 10;
> }
> 
> Or something along those lines. Thanks!

No problem, I put the source and clang+gcc binaries in a tarball here:
https://jpbrucker.net/tmp/test-poly-neon.tar.bz2

These contain all the base types defined by arm_neon.h (minus the new
bfloat16, which I don't think matters at the moment)

Thanks,
Jean

> 
> > [1] https://lore.kernel.org/bpf/20200810122835.2309026-1-jean-philippe@linaro.org/
> >
> > > There is a bunch more in gcc/config/arm/arm-simd-builtin-types.def.
> > > May be there is a way to detect compiler builtin types by pattern matching
> > > their dwarf/btf shape and skip them automatically?
> > > The simplest, of course, is to only add a few that caused this known
> > > trouble to blocklist.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Andrii Nakryiko]

On Tue, Aug 11, 2020 at 2:54 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Mon, Aug 10, 2020 at 11:54:54PM -0700, Andrii Nakryiko wrote:
> > On Mon, Aug 10, 2020 at 5:58 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > >
> > > On Fri, Aug 07, 2020 at 01:54:02PM -0700, Alexei Starovoitov wrote:
> > > [...]
> > > > > > > ```
> > > > > > > typedef __Poly8_t poly8x16_t[16];
> > > > > > > ```
> > > > > > >
> > > > > > > AFAICT these are ARM NEON intrinsic definitions which are GCC-specific. We
> > > > > > > have opted to comment out this line as there was no additional `poly8x16_t`
> > > > > > > usage in the header file.
> > > > > >
> > > > > > It looks like this "__Poly8_t" type is internal to GCC (provided in
> > > > > > arm_neon.h) and clang has its own internals. I managed to reproduce this
> > > > > > with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> > > > > > the moment. Maybe libbpf should generate defines to translate these
> > > > > > intrinsics between clang and gcc? Not very elegant. I'll take another
> > > > > > look next week.
> > > > >
> > > > > libbpf is already blacklisting __builtin_va_list for GCC, so we can
> > > > > just add __Poly8_t to the list. See [0].
> > > > > Are there any other types like that? If you guys can provide me this,
> > > > > I'll gladly update libbpf to take those compiler-provided
> > > > > types/built-ins into account.
> > > >
> > > > Shouldn't __Int8x16_t and friends cause the same trouble?
> > >
> > > I think these do get properly defined, for example in my vmlinux.h:
> > >
> > >         typedef signed char int8x16_t[16];
> > >
> > > From a cursory reading of the "ARM C Language Extension" doc (IHI0053D) it
> > > looks like only the poly8/16/64/128_t types are unspecified. It's safe to
> > > drop them as long as they're not used in structs or function parameters,
> > > but I sent a more generic fix [1] that copies the clang defintions. When
> > > building the kernel with clang, the polyX_t types do get typedefs.
> > >
> > > Thanks,
> > > Jean
> > >
> >
> > Hi Jean,
> >
> > Would you be so kind to build some simple C repro code that uses those
> > polyX_t types? Ideally built by both GCC and Clang. And then run
> > `pahole -J` on them to get .BTF into them as well. If you can share
> > those two with me, I'd love to look at how DWARF and BTF look like.
> >
> > I'm, unfortunately, having trouble making something like that to
> > cross-compile on my x86-64 machine, I've spent a bunch of time already
> > on this unsuccessfully and it's really frustrating at this point. If
> > you have an ARM system (or cross-compilation set up properly), it
> > shouldn't take much time for you, hopefully. Just make sure that those
> > polyX_t types do make it into DWARF, so, e.g., use them with static
> > variable or something, e.g.,:
> >
> > int main() {
> >     static poly8_t a = 12;
> >     return a + 10;
> > }
> >
> > Or something along those lines. Thanks!
>
> No problem, I put the source and clang+gcc binaries in a tarball here:
> https://jpbrucker.net/tmp/test-poly-neon.tar.bz2
>
> These contain all the base types defined by arm_neon.h (minus the new
> bfloat16, which I don't think matters at the moment)
>

Thanks a lot! It was very helpful. I wonder why there was never
poly32_t defined?

> Thanks,
> Jean
>
> >
> > > [1] https://lore.kernel.org/bpf/20200810122835.2309026-1-jean-philippe@linaro.org/
> > >
> > > > There is a bunch more in gcc/config/arm/arm-simd-builtin-types.def.
> > > > May be there is a way to detect compiler builtin types by pattern matching
> > > > their dwarf/btf shape and skip them automatically?
> > > > The simplest, of course, is to only add a few that caused this known
> > > > trouble to blocklist.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Russell King - ARM Linux admin]

On Fri, Aug 07, 2020 at 07:23:53PM +0200, Jean-Philippe Brucker wrote:
> On Fri, Aug 07, 2020 at 04:20:58PM +0200, Jakov Petrina wrote:
> > Hi everyone,
> > 
> > recently we have begun extensive research into eBPF and related
> > technologies. Seeking an easier development process, we have switched over
> > to using the eBPF CO-RE [0] approach internally which has enabled us to
> > simplify most aspects of eBPF development, especially those related to
> > cross-compilation.

For those of us not familiar with what CO-RE is, this doesn't help.
I assume the [0] was a reference to something that explained it,
but that isn't included.

> > However, as part of these efforts we have stumbled upon several problems
> > that we feel would benefit from a community discussion where we may share
> > our solutions and discuss alternatives moving forward.
> > 
> > As a reference point, we have started researching and modifying several eBPF
> > CO-RE samples that have been developed or migrated from existing `bcc`
> > tooling. Most notable examples are those present in `bcc`'s `libbpf-tools`
> > directory [1]. Some of these samples have just recently been converted to
> > respective eBPF CO-RE variants, of which the `tcpconnect` tracing sample has
> > proven to be very interesting.
> > 
> > First showstopper for cross-compiling aforementioned example on the ARM
> > 32-bit platform has been with regards to generation of the required
> > `vmlinux.h` kernel header from the BTF information. More specifically, our
> > initial approach to have e.g. a compilation target dependency which would
> > invoke `bpftool` at configure time was not appropriate due to several
> > issues: a) CO-RE requires host kernel to have been compiled in such a way to
> > expose BTF information which may not available, and b) the generated

What is "BTF information"?  Google suggests it's something to do with
the British Thyroid Foundation.

Please don't use three letter abbreviations unless they are widely
understood, or if you wish to, please ensure that you explain them.
TLAs otherwise are an exclusion mechanism.

> > `vmlinux.h` was actually architecture-specific.

What is this "vmlinux.h" ?  It isn't something that the kernel provides
afaics.  It doesn't seem to be present on my existing x86 Debian system.
I've seen it on Fedora systems in the dim and distant past.

Where do you think it comes from?  Where are you finding it?
-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Jakov Petrina]

Hi,

On 07/08/2020 21:00, Russell King - ARM Linux admin wrote:
> 
> For those of us not familiar with what CO-RE is, this doesn't help.
> I assume the [0] was a reference to something that explained it,
> but that isn't included.
> 

the reference [0] is link to a blog post which explains the eBPF CO-RE 
concept; I have added this link as a reference below.

> 
> What is "BTF information"?  Google suggests it's something to do with
> the British Thyroid Foundation.
> 
> Please don't use three letter abbreviations unless they are widely
> understood, or if you wish to, please ensure that you explain them.
> TLAs otherwise are an exclusion mechanism.
> >
> What is this "vmlinux.h" ?  It isn't something that the kernel provides
> afaics.  It doesn't seem to be present on my existing x86 Debian system.
> I've seen it on Fedora systems in the dim and distant past.
> 
> Where do you think it comes from?  Where are you finding it?
> 

The blog post [0] provides description and context for the references 
and abbreviations used, but in the future I will be sure to avoid using 
abbreviations unless they are commonly understood.

[0] 
https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html

Regards,
-- 
Jakov Petrina

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Russell King - ARM Linux admin]

On Mon, Aug 10, 2020 at 09:52:17AM +0200, Jakov Petrina wrote:
> Hi,
> 
> On 07/08/2020 21:00, Russell King - ARM Linux admin wrote:
> > 
> > For those of us not familiar with what CO-RE is, this doesn't help.
> > I assume the [0] was a reference to something that explained it,
> > but that isn't included.
> > 
> 
> the reference [0] is link to a blog post which explains the eBPF CO-RE
> concept; I have added this link as a reference below.
> 
> > 
> > What is "BTF information"?  Google suggests it's something to do with
> > the British Thyroid Foundation.
> > 
> > Please don't use three letter abbreviations unless they are widely
> > understood, or if you wish to, please ensure that you explain them.
> > TLAs otherwise are an exclusion mechanism.
> > >
> > What is this "vmlinux.h" ?  It isn't something that the kernel provides
> > afaics.  It doesn't seem to be present on my existing x86 Debian system.
> > I've seen it on Fedora systems in the dim and distant past.
> > 
> > Where do you think it comes from?  Where are you finding it?
> > 
> 
> The blog post [0] provides description and context for the references and
> abbreviations used, but in the future I will be sure to avoid using
> abbreviations unless they are commonly understood.
> 
> [0] https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html

Okay, you've addressed one point I raised, but you have not addressed
any of the questions I raised.  I'll take this thread as just noise on
the mailing list since it seems to contain nothing of any relevance to
the Linux kernel, and no one seems willing to explain why they think it
is relevant.

Thanks.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: eBPF CO-RE cross-compilation for 32-bit ARM platforms

[Jakov Petrina]

Hi,

On 07/08/2020 19:23, Jean-Philippe Brucker wrote:
> Hi,
> 
> [Adding the linux-arm-kernel list on Cc]
> 
> 
> It looks like this "__Poly8_t" type is internal to GCC (provided in
> arm_neon.h) and clang has its own internals. I managed to reproduce this
> with an arm64 allyesconfig kernel (+BTF), but don't know how to fix it at
> the moment. Maybe libbpf should generate defines to translate these
> intrinsics between clang and gcc? Not very elegant. I'll take another
> look next week.
> 

indeed, this has only been present in our arm64 kernel builds but I 
suppose it may surface for different configurations as well. Per 
Andrii's suggestion, I think blacklisting such types during the dump 
would be a safe bet for now.

> 
> I don't know if there is a room for improvement regarding your a) and b)
> points, as I think the added complexity is inherent to cross-building. But
> kernel crashes definitely need to be fixed, as well as the above problem.
> 

If that is the case, I suppose an additional step would be to configure 
the kernel in such a way so that the generated header files result in 
the smallest possible file size, while retaining all structures that we 
require in our eBPF programs.

The kernel crash seems to elude our attempts to reproduce it; it occurs 
only on a target 32-bit platform device, but a reproducible program does 
not crash the kernel in a QEMU ARM environment. When we investigate this 
further we will definitely share our results.

> Thanks,
> Jean
> 

Thanks,
-- 
Jakov Petrina

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel