Re: ebpf: issue with clang

[Eric Leblond <eric@regit.org>]

Hello,

On Thu, 2016-06-09 at 17:34 -0700, Alexei Starovoitov wrote:
> On Thu, Jun 09, 2016 at 11:10:05PM +0200, Eric Leblond wrote:
> > Hello,
> > 
> > I'm working on integrating ebpf cluster load balancing for
> > AF_PACKET
> > and I've got some problem to get real code inside the EBPF filter.
> > 
> > I've tried different command lines in the build process. One of
> > them
> > is:
> > clang-3.9 -Wall -O2 -emit-llvm -c hash_ports.c -o - | llc-3.9
> > -march=bpf -filetype=obj -o hash_ports.bpf
> > 
> > If I use that one, then the generated code is almost void. If I
> > remove
> > the -O2 then I've got a generated code that fails during load. When
> > not
> > using -O2, I manage to load a trivial filter (return of static
> > value).
> >  
> > The C code is the following (a derivative of http-simple-filter.c
> > used
> > for testing):
> > 
> > int filter(struct __sk_buff *skb) {
> > 	uint8_t *cursor = 0;
> > 	struct ethernet_t *ethernet = cursor_advance(cursor,
> > sizeof(*ethernet));
> 
> this is bcc C syntax that is hiding the explicit load_byte/half/word
> operations
> we have to do when using plain C.
> If you want to compile C code with clang -O2 -target bpf file.c -c -o
> file.o
> and copy .o around to be used in tc like:
> tc filter add dev eth0 ingress bpf da obj file.o
> then plain C should be used like in all samples/bpf/*_kern.c
> examples.
> Other folks like the convenience of bcc that hides clang/llvm
> invocation.
> It mostly applicable to tracing tools where both bcc-C and
> corresponding python or lua bits are in the same file
> like in iovisor/bcc/tools/* scripts.
> The iovisor/bcc/examples/networking/* (where this http-simple-
> filter.c came from)
> are also suitable for networking and relying on pyroute2 to talk to
> kernel to create netns, veth and to attach bpf to qdisc.
> 
> In summary there are several ways to write bpf C code:
> 1. plain C syntax as in samples/bpf/*_kern.c
> Pro: compiles with clang into .o
> Con: .o requires elf loader (integrated into tc already for
> networking),

Yes, that's not an easy part. I've devel one loader for suricata but I
will check the one in tc to see if I can take advantage of it.

> but not friendly for tracing that needs recompile for every kernel
> due to unstable kprobes
> 2. bcc C syntax that compiles C on the fly in memory and loads
> directly
> Pro: there is no .o, no extra files, no need to install clang/llvm
> Con: bcc is not widely available yet. ubuntu and others already have
> it in apt.
> python and lua may not be for everyone. c++ api is not stable yet.

I need to include it into suricata which is C code. I've played with
bcc and it is a great tool but installation on the different platform
may be complicated for our users.

> 3. perf+bpf, it is similar to samples/pbf/ C style with few
> extensions.
> If .c file is passed, the perf calls external clang and loads .o
> eventually
> Pro: out-of-the-box perf and clang work well
> Con: not available for networking

Out of scope for me then.

> Sounds like you want to use it with af_packet then
> tools/testing/selftests/net/reuseport_bpf.c
> could be a good start too, but there bpf is written in asm.

Yes, bad point, asm is not really what I want. I want "normal advanced"
users to be able to edit the load balancing function.

> If you pick bcc style then iovisor-dev@lists.iovisor.org mailing list
> is a good place to ask questions. Be sure to subscribe first, since
> it rejects non-subscriber emails to reduce spam.

Thanks a lot for all these explanations. You saved me days!

BR,
-- 

Eric Leblond <eric@regit.org>