Re: Build error of samples/bpf

[sdf@google.com]

On 07/13, Zeng, Oak wrote:
> Thank you sdf for the reply.

> It is news to me that samples/bpf tend to go stale. When I looked the  
> samples/bpf folder git history, the last update is from only 2 months  
> ago. And yes I can see samples/bfp is not actively updated recently. We  
> are from Intel's GPU group and we are working on some bpf tools for GPU  
> profiling purpose. We made our work based on the structure of samples/bpf  
> because we can conveniently use libbpf. We chose bpf c frontend (vs  
> python frontend) because python bpf program seems can't execute under  
> non-root leading to some security concerns. This work is not yet upstream  
> but we planned to upstream it.

I'm mostly talking from the following perspective:
http://vger.kernel.org/bpfconf2022_material/lsfmmbpf2022-bpf-ci.pdf

BPF maintainers have a CI that continuously builds and runs
tools/testing/selftests/bpf. I don't think it includes samples/bpf;
that's what I mean by "go stale". Eventually, people fix samples, but
there is no continuous system to verify they are healthy.

> Now if samples/bpf is going stale, we need to change our plan.  
> Specifically I have a few questions for our future planning:

> 1. does bpf community accept more bpf samples such as using bpf to  
> kprobe/tracepoint GPU activities? Is such work helpful/welcome to/by the  
> community? I can see the existing samples and selftest are mainly for  
> general linux profiling such as fs, network etc. Do you accept more  
> samples for specific driver profiling - in our case it is profiling of  
> Intel's i915 GPU driver.

> 2. Should we port our samples/bpf to tools/testing/selftests/bpf for  
> upstream purpose? I am not sure whether tools/testing/selftests/bpf is a  
> good place for our gpu profiling samples. If tools/testing/selftests/bpf  
> is not a good place for gpu profilers, any suggestion where we can  
> upstream our gpu profilers?

> Cc other bpf reviewers. Thank you for your considerations!

I'll let others chime in, but IMO it's a gray area. This can probably
go into samples, but if you want to have a profiler that you want others
to use, why not distribute it as part of real profiling tools like perf or
bpftrace?

> Thanks,
> Oak

> > -----Original Message-----
> > From: sdf@google.com <sdf@google.com>
> > Sent: July 12, 2022 1:23 PM
> > To: Zeng, Oak <oak.zeng@intel.com>
> > Cc: bpf@vger.kernel.org
> > Subject: Re: Build error of samples/bpf
> >
> > On 07/12, Zeng, Oak wrote:
> > > Hello all,
> >
> > > I tried to build the latest samples/bpf following instructions in the
> > > README.rst in samples/bpf folder. I ran into various issue such as:
> >
> > > samples/bpf/Makefile:375: *** Cannot find a vmlinux for VMLINUX_BTF at
> > > any of "  /home/szeng/dii-tools/linux/vmlinux", build the kernel or
> > > set VMLINUX_BTF or VMLINUX_H variable
> >
> > > I was able to fix above issue by enable CONFIG_DEBUG_INFO_BTF in
> > > kernel .config file.
> >
> > > But I eventually ran into other errors.  I had to fix those errors by
> > > install dwarves, updating my clang/llvm to version 10.
> >
> > > I was able to build it if I comment out all the xdp programs from
> > > Makefile. It seems those xdp programs require advanced features such
> > > as data structure layout in vmlinux.h (dumped from vmlinux using
> > > bpftool) and this require special kernel config support.
> >
> > > So I thought instead of fixing those errors one by one, I should ask
> > > those who are working in this area, is there any instructions on how
> > > to build samples/bpf? The README.rst seems out-of-date, for example,
> > > it doesn't mention CONFIG_DEBUG_INFO_BTF. The required llvm/clang
> > > version in README.rst is also out-of-date.
> >
> > > More specifically, to build samples/bpf, is there an example kernel
> > > .config to use? I tried those config here
> > > https://github.com/torvalds/linux/blob/master/tools/testing/selftests/
> > > bpf/config
> > > but build errors persist.
> >
> > > Or is there any other tools I need to install/update on my system?
> >
> > In general, I'd say, samples/bpf tend to go stale. We mostly work on the
> > selftests and don't pay too much attention to the samples :-( Ideally,  
> the
> > samples should be part of selftests so they get exercised by the CI.
> >
> > One thing I can suggest is to look at  
> tools/testing/selftests/bpf/vmtest.sh
> > script. It builds the kernels with vetted configs capable of running  
> selftests.
> > That should be, in theory, be enough to compile the samples.
> >
> > > My whole build log is as below:
> >
> > > szeng@linux:~/dii-tools/linux$ make M=samples/bpf
> > > readelf: Error: Missing knowledge of 32-bit reloc types used in DWARF
> > > sections of machine number 247
> > > readelf: Warning: unable to apply unsupported reloc type 10 to section
> > > .debug_info
> > > readelf: Warning: unable to apply unsupported reloc type 1 to section
> > > .debug_info
> > > readelf: Warning: unable to apply unsupported reloc type 10 to section
> > > .debug_info make -C
> > > /home/szeng/dii-tools/linux/samples/bpf/../../tools/lib/bpf RM='rm  
> -rf'
> > > EXTRA_CFLAGS="-Wall -O2 -Wmissing-prototypes -Wstrict-prototypes
> > > -I./usr/include -I./tools/testing/selftests/bpf/
> > > -I/home/szeng/dii-tools/linux/samples/bpf/libbpf/include
> > > -I./tools/include -I./tools/perf -DHAVE_ATTR_TEST=0" \
> > >          LDFLAGS=  
> srctree=/home/szeng/dii-tools/linux/samples/bpf/../../ \
> > >          O= OUTPUT=/home/szeng/dii-tools/linux/samples/bpf/libbpf/
> > > DESTDIR=/home/szeng/dii-tools/linux/samples/bpf/libbpf prefix= \
> > >          /home/szeng/dii-tools/linux/samples/bpf/libbpf/libbpf.a
> > > install_headers
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/libbpf.o
> > >    CC       
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/bpf.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/nlattr.o
> > >    CC       
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/btf.o
> > >    CC
> > >  
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/libbpf_errno.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/str_error.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/netlink.o
> > >    CC
> > > /home/szeng/dii-
> > tools/linux/samples/bpf/libbpf/staticobjs/bpf_prog_linfo.o
> > >    CC
> > > /home/szeng/dii-
> > tools/linux/samples/bpf/libbpf/staticobjs/libbpf_probes.o
> > >    CC       
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/xsk.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/hashmap.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/btf_dump.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/ringbuf.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/strset.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/linker.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/gen_loader.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/relo_core.o
> > >    LD
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/libbpf-in.o
> > >    LINK    /home/szeng/dii-tools/linux/samples/bpf/libbpf/libbpf.a
> > >    INSTALL headers
> > >    CC  samples/bpf/test_lru_dist
> > >    CC  samples/bpf/sock_example
> > >    CC  samples/bpf/../../tools/testing/selftests/bpf/cgroup_helpers.o
> > >    CC  samples/bpf/../../tools/testing/selftests/bpf/trace_helpers.o
> > >    CC  samples/bpf/cookie_uid_helper_example.o
> > >    CC  samples/bpf/cpustat_user.o
> > >    CC  samples/bpf/fds_example.o
> > >    CC  samples/bpf/hbm.o
> > >    CC  samples/bpf/i915_latency_hist_user.o
> > >    CC  samples/bpf/i915_stat_user.o
> > >    CC  samples/bpf/ibumad_user.o
> > >    CC  samples/bpf/lathist_user.o
> > >    CC  samples/bpf/lwt_len_hist_user.o
> > >    CC  samples/bpf/map_perf_test_user.o
> > >    CC  samples/bpf/offwaketime_user.o
> > >    CC  samples/bpf/sampleip_user.o
> > >    CC  samples/bpf/sockex1_user.o
> > >    CC  samples/bpf/sockex2_user.o
> > >    CC  samples/bpf/sockex3_user.o
> > >    CC  samples/bpf/spintest_user.o
> > >    CC  samples/bpf/syscall_tp_user.o
> > >    CC  samples/bpf/task_fd_query_user.o
> > >    CC  samples/bpf/tc_l2_redirect_user.o
> > >    CC  samples/bpf/test_cgrp2_array_pin.o
> > >    CC  samples/bpf/test_cgrp2_attach.o
> > >    CC  samples/bpf/test_cgrp2_sock.o
> > >    CC  samples/bpf/test_cgrp2_sock2.o
> > >    CC  samples/bpf/test_current_task_under_cgroup_user.o
> > >    CC  samples/bpf/test_map_in_map_user.o
> > >    CC  samples/bpf/test_overhead_user.o
> > >    CC  samples/bpf/test_probe_write_user_user.o
> > >    CC  samples/bpf/trace_event_user.o
> > >    CC  samples/bpf/trace_output_user.o
> > >    CC  samples/bpf/tracex1_user.o
> > >    CC  samples/bpf/tracex2_user.o
> > >    CC  samples/bpf/tracex3_user.o
> > >    CC  samples/bpf/tracex4_user.o
> > >    CC  samples/bpf/tracex5_user.o
> > >    CC  samples/bpf/tracex6_user.o
> > >    CC  samples/bpf/tracex7_user.o
> > >    CC  samples/bpf/xdp1_user.o
> > >    CC  samples/bpf/xdp_adjust_tail_user.o
> > >    CC  samples/bpf/xdp_fwd_user.o
> > > make -C
> > > /home/szeng/dii-tools/linux/samples/bpf/../../tools/bpf/bpftool
> > > srctree=/home/szeng/dii-tools/linux/samples/bpf/../../ \
> > >          OUTPUT=/home/szeng/dii-tools/linux/samples/bpf/bpftool/ \
> > >           
> LIBBPF_OUTPUT=/home/szeng/dii-tools/linux/samples/bpf/libbpf/ \
> > >
> > > LIBBPF_DESTDIR=/home/szeng/dii-tools/linux/samples/bpf/libbpf/
> >
> > > Auto-detecting system features:
> > > ...                        libbfd: [ OFF ]
> > > ...        disassembler-four-args: [ OFF ]
> > > ...                          zlib: [ on  ]
> > > ...                        libcap: [ OFF ]
> > > ...               clang-bpf-co-re: [ on  ]
> >
> >
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/libbpf.o
> > >    CC       
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/bpf.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/nlattr.o
> > >    CC       
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/btf.o
> > >    CC
> > >  
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/libbpf_errno.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/str_error.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/netlink.o
> > >    CC
> > > /home/szeng/dii-
> > tools/linux/samples/bpf/libbpf/staticobjs/bpf_prog_linfo.o
> > >    CC
> > > /home/szeng/dii-
> > tools/linux/samples/bpf/libbpf/staticobjs/libbpf_probes.o
> > >    CC       
> /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/xsk.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/hashmap.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/btf_dump.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/ringbuf.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/strset.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/linker.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/gen_loader.o
> > >    CC
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/relo_core.o
> > >    LD
> > > /home/szeng/dii-tools/linux/samples/bpf/libbpf/staticobjs/libbpf-in.o
> > >    LINK    /home/szeng/dii-tools/linux/samples/bpf/libbpf/libbpf.a
> > >    CLANG    
> /home/szeng/dii-tools/linux/samples/bpf/bpftool/profiler.bpf.o
> > >    GEN      
> /home/szeng/dii-tools/linux/samples/bpf/bpftool/profiler.skel.h
> > >    CC      /home/szeng/dii-tools/linux/samples/bpf/bpftool/prog.o
> > >    CLANG    
> /home/szeng/dii-tools/linux/samples/bpf/bpftool/pid_iter.bpf.o
> > >    GEN      
> /home/szeng/dii-tools/linux/samples/bpf/bpftool/pid_iter.skel.h
> > >    CC      /home/szeng/dii-tools/linux/samples/bpf/bpftool/pids.o
> > >    LINK    /home/szeng/dii-tools/linux/samples/bpf/bpftool/bpftool
> > >    CC  samples/bpf/xdp_router_ipv4_user.o
> > >    CC  samples/bpf/xdp_rxq_info_user.o
> > >    CC  samples/bpf/xdp_sample_pkts_user.o
> > >    CC  samples/bpf/xdp_tx_iptunnel_user.o
> > >    CC  samples/bpf/xdpsock_ctrl_proc.o
> > >    CC  samples/bpf/xsk_fwd.o
> > >    CLANG-BPF  samples/bpf/xdp_sample.bpf.o
> > >    CLANG-BPF  samples/bpf/xdp_redirect_map_multi.bpf.o
> > >    CLANG-BPF  samples/bpf/xdp_redirect_cpu.bpf.o
> > >    CLANG-BPF  samples/bpf/xdp_redirect_map.bpf.o
> > >    CLANG-BPF  samples/bpf/xdp_monitor.bpf.o
> > >    CLANG-BPF  samples/bpf/xdp_redirect.bpf.o
> > >    BPF GEN-OBJ  samples/bpf/xdp_monitor
> > >    BPF GEN-SKEL samples/bpf/xdp_monitor
> > > libbpf: map 'rx_cnt': unexpected def kind var.
> > > Error: failed to open BPF object file: Invalid argument
> > > samples/bpf/Makefile:430: recipe for target
> > > 'samples/bpf/xdp_monitor.skel.h' failed
> > > make[1]: *** [samples/bpf/xdp_monitor.skel.h] Error 255
> > > make[1]: *** Deleting file 'samples/bpf/xdp_monitor.skel.h'
> > > Makefile:1868: recipe for target 'samples/bpf' failed
> > > make: *** [samples/bpf] Error 2
> >
> >
> > > Thanks,
> > > Oak