Re: [PATCH v2 bpf-next 1/2] bpftool: introduce "prog profile" command

[Yonghong Song <yhs@fb.com>]

On 3/1/20 2:37 PM, Song Liu wrote:
> 
> 
>> On Feb 29, 2020, at 7:52 PM, Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 2/28/20 3:40 PM, Song Liu wrote:
>>> With fentry/fexit programs, it is possible to profile BPF program with
>>> hardware counters. Introduce bpftool "prog profile", which measures key
>>> metrics of a BPF program.
>>> bpftool prog profile command creates per-cpu perf events. Then it attaches
>>> fentry/fexit programs to the target BPF program. The fentry program saves
>>> perf event value to a map. The fexit program reads the perf event again,
>>> and calculates the difference, which is the instructions/cycles used by
>>> the target program.
>>> Example input and output:
>>>    ./bpftool prog profile 3 id 337 cycles instructions llc_misses
>>>          4228 run_cnt
>>>       3403698 cycles                                              (84.08%)
>>>       3525294 instructions   #  1.04 insn per cycle               (84.05%)
>>>            13 llc_misses     #  3.69 LLC misses per million isns  (83.50%)
>>> This command measures cycles and instructions for BPF program with id
>>> 337 for 3 seconds. The program has triggered 4228 times. The rest of the
>>> output is similar to perf-stat. In this example, the counters were only
>>> counting ~84% of the time because of time multiplexing of perf counters.
>>> Note that, this approach measures cycles and instructions in very small
>>> increments. So the fentry/fexit programs introduce noticeable errors to
>>> the measurement results.
>>> The fentry/fexit programs are generated with BPF skeletons. Therefore, we
>>> build bpftool twice. The first time _bpftool is built without skeletons.
>>> Then, _bpftool is used to generate the skeletons. The second time, bpftool
>>> is built with skeletons.
>>> Signed-off-by: Song Liu <songliubraving@fb.com>
>>> ---
>>>   tools/bpf/bpftool/Makefile                |  18 +
>>>   tools/bpf/bpftool/prog.c                  | 428 +++++++++++++++++++++-
>>>   tools/bpf/bpftool/skeleton/profiler.bpf.c | 171 +++++++++
>>>   tools/bpf/bpftool/skeleton/profiler.h     |  47 +++
>>>   tools/scripts/Makefile.include            |   1 +
>>>   5 files changed, 664 insertions(+), 1 deletion(-)
>>>   create mode 100644 tools/bpf/bpftool/skeleton/profiler.bpf.c
>>>   create mode 100644 tools/bpf/bpftool/skeleton/profiler.h
>>> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
>>> index c4e810335810..c035fc107027 100644
>>> --- a/tools/bpf/bpftool/Makefile
>>> +++ b/tools/bpf/bpftool/Makefile
>>> @@ -59,6 +59,7 @@ LIBS = $(LIBBPF) -lelf -lz
>>>     INSTALL ?= install
>>>   RM ?= rm -f
>>> +CLANG ?= clang
>>>     FEATURE_USER = .bpftool
>>>   FEATURE_TESTS = libbfd disassembler-four-args reallocarray zlib
>>> @@ -110,6 +111,22 @@ SRCS += $(BFD_SRCS)
>>>   endif
>>>     OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
>>> +_OBJS = $(filter-out $(OUTPUT)prog.o,$(OBJS)) $(OUTPUT)_prog.o
>>> +
>>> +$(OUTPUT)_prog.o: prog.c
>>> +	$(QUIET_CC)$(COMPILE.c) -MMD -DBPFTOOL_WITHOUT_SKELETONS -o $@ $<
>>> +
>>> +$(OUTPUT)_bpftool: $(_OBJS) $(LIBBPF)
>>> +	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(_OBJS) $(LIBS)
>>> +
>>> +skeleton/profiler.bpf.o: skeleton/profiler.bpf.c
>>> +	$(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -c $< -o $@
>>
>> With a fresh checkout, applying this patch and just selftests like
>>   make -C tools/testing/selftests/bpf
>>
>> I got the following build error:
>>
>> make[2]: Leaving directory `/data/users/yhs/work/net-next/tools/lib/bpf'
>> clang -g -O2 -target bpf -c skeleton/profiler.bpf.c -o skeleton/profiler.bpf.o
>> skeleton/profiler.bpf.c:5:10: fatal error: 'bpf/bpf_helpers.h' file not found
>> #include <bpf/bpf_helpers.h>
>>          ^~~~~~~~~~~~~~~~~~~
>> 1 error generated.
>> make[1]: *** [skeleton/profiler.bpf.o] Error 1
>>
>> I think Makefile should be tweaked to avoid selftest failure.
> 
> Hmm... I am not seeing this error. The build succeeded in the test.

Just tried again with a *fresh* checkout of tools/ directory with the patch.
   -bash-4.4$ make -C tools/testing/selftests/bpf
   ...
   LINK 
/data/users/yhs/work/net-next/tools/testing/selftests/bpf/tools/build/bpftool//libbpf/libbpf.a
   LINK 
/data/users/yhs/work/net-next/tools/testing/selftests/bpf/tools/build/bpftool/_bpftool
make[1]: *** No rule to make target `skeleton/profiler.bpf.c', needed by 
`skeleton/profiler.bpf.o'.  Stop.
make: *** 
[/data/users/yhs/work/net-next/tools/testing/selftests/bpf/tools/sbin/bpftool] 
Error 2
make: Leaving directory 
`/data/users/yhs/work/net-next/tools/testing/selftests/bpf

The error is different from my previous try, but the build still fails.

> 
> Thanks,
> Song
>