Re: [GIT PULL 00/16] perf/ebpf improvements and fixes

[Ingo Molnar <mingo@kernel.org>]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Hi Ingo,
> 
> 	This is on top of my last perf-urgent-for-mingo pull req, that is
> outstanding, please check if this can get into this merge window.
> 
> - Arnaldo
> 
> The following changes since commit 092b1f0b5f9f797812da0de927c3aa26acbe8762:
> 
>   perf probe: Clear probe_trace_event when add_probe_trace_event() fails (2015-11-13 12:28:09 -0300)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-ebpf-for-mingo
> 
> for you to fetch changes up to 0fbcd50e1385d3a78ddb3cf52dcdff802bfb835e:
> 
>   perf test: Mute test cases error messages if verbose == 0 (2015-11-17 11:45:06 -0300)
> 
> ----------------------------------------------------------------
> perf/ebpf improvements and fixes:
> 
> User visible:
> 
> - Allows BPF scriptlets specify arguments to be fetched using
>   DWARF info, using a prologue generated at compile/build time (He Kuang, Wang Nan)
> 
> - Allow attaching BPF scriptlets to module symbols (Wang Nan)
> 
> - Allow attaching BPF scriptlets to userspace code using uprobe (Wang Nan)
> 
> - BPF programs now can specify 'perf probe' tunables via its section name,
>   separating key=val values using semicolons (Wang Nan)
> 
> Testing some of these new BPF features:
> 
> Use case: get callchains when receiving SSL packets, filter then in the
>           kernel, at arbitrary place.
> 
>   # cat ssl.bpf.c
>   #define SEC(NAME) __attribute__((section(NAME), used))
> 
>   struct pt_regs;
> 
>   SEC("func=__inet_lookup_established hnum")
>   int func(struct pt_regs *ctx, int err, unsigned short port)
>   {
>           return err == 0 && port == 443;
>   }
> 
>   char _license[] SEC("license") = "GPL";
>   int  _version   SEC("version") = LINUX_VERSION_CODE;
>   #
>   # perf record -a -g -e ssl.bpf.c
>   ^C[ perf record: Woken up 1 times to write data ]
>   [ perf record: Captured and wrote 0.787 MB perf.data (3 samples) ]
>   # perf script | head -30
>   swapper     0 [000] 58783.268118: perf_bpf_probe:func: (ffffffff816a0f60) hnum=0x1bb
> 	 8a0f61 __inet_lookup_established (/lib/modules/4.3.0+/build/vmlinux)
> 	 896def ip_rcv_finish (/lib/modules/4.3.0+/build/vmlinux)
> 	 8976c2 ip_rcv (/lib/modules/4.3.0+/build/vmlinux)
> 	 855eba __netif_receive_skb_core (/lib/modules/4.3.0+/build/vmlinux)
> 	 8565d8 __netif_receive_skb (/lib/modules/4.3.0+/build/vmlinux)
> 	 8572a8 process_backlog (/lib/modules/4.3.0+/build/vmlinux)
> 	 856b11 net_rx_action (/lib/modules/4.3.0+/build/vmlinux)
> 	 2a284b __do_softirq (/lib/modules/4.3.0+/build/vmlinux)
> 	 2a2ba3 irq_exit (/lib/modules/4.3.0+/build/vmlinux)
> 	 96b7a4 do_IRQ (/lib/modules/4.3.0+/build/vmlinux)
> 	 969807 ret_from_intr (/lib/modules/4.3.0+/build/vmlinux)
> 	 2dede5 cpu_startup_entry (/lib/modules/4.3.0+/build/vmlinux)
> 	 95d5bc rest_init (/lib/modules/4.3.0+/build/vmlinux)
> 	1163ffa start_kernel ([kernel.vmlinux].init.text)
> 	11634d7 x86_64_start_reservations ([kernel.vmlinux].init.text)
> 	1163623 x86_64_start_kernel ([kernel.vmlinux].init.text)
> 
>   qemu-system-x86  9178 [003] 58785.792417: perf_bpf_probe:func: (ffffffff816a0f60) hnum=0x1bb
> 	 8a0f61 __inet_lookup_established (/lib/modules/4.3.0+/build/vmlinux)
> 	 896def ip_rcv_finish (/lib/modules/4.3.0+/build/vmlinux)
> 	 8976c2 ip_rcv (/lib/modules/4.3.0+/build/vmlinux)
> 	 855eba __netif_receive_skb_core (/lib/modules/4.3.0+/build/vmlinux)
> 	 8565d8 __netif_receive_skb (/lib/modules/4.3.0+/build/vmlinux)
> 	 856660 netif_receive_skb_internal (/lib/modules/4.3.0+/build/vmlinux)
> 	 8566ec netif_receive_skb_sk (/lib/modules/4.3.0+/build/vmlinux)
> 	   430a br_handle_frame_finish ([bridge])
> 	   48bc br_handle_frame ([bridge])
> 	 855f44 __netif_receive_skb_core (/lib/modules/4.3.0+/build/vmlinux)
> 	 8565d8 __netif_receive_skb (/lib/modules/4.3.0+/build/vmlinux)
>   #
> 
>   Use 'perf probe' various options to list functions, see what variables can
>   be collected at any given point, experiment first collecting without a filter,
>   then filter, use it together with 'perf trace', 'perf top', with or without
>   callchains, if it explodes, please tell us!
> 
> Infrastructure:
> 
> - Add further 'perf test' entries for BPF and LLVM (Wang Nan)
> 
> - Improve 'perf test' to suport subtests, so that the series of tests
>   performed in the LLVM and BPF main tests appear in the default 'perf test'
>   output (Wang Nan)
> 
> - Move memdup() from tools/perf to tools/lib/string.c (Arnaldo Carvalho de Melo)
> 
> - Adopt strtobool() from the kernel into tools/lib/ (Wang Nan)
> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> ----------------------------------------------------------------
> Arnaldo Carvalho de Melo (1):
>       tools: Adopt memdup() from tools/perf, moving it to tools/lib/string.c
> 
> He Kuang (1):
>       perf bpf: Add prologue for BPF programs for fetching arguments
> 
> Wang Nan (14):
>       tools: Clone the kernel's strtobool function
>       bpf tools: Load a program with different instances using preprocessor
>       perf bpf: Add BPF_PROLOGUE config options for further patches
>       perf bpf: Compile dwarf-regs.c if CONFIG_BPF_PROLOGUE is on
>       perf bpf: Allow BPF program attach to uprobe events
>       perf bpf: Allow attaching BPF programs to modules symbols
>       perf bpf: Allow BPF program config probing options
>       perf bpf: Generate prologue for BPF programs
>       perf test: Test the BPF prologue adding infrastructure
>       perf test: Fix 'perf test BPF' when it fails to find a suitable vmlinux
>       perf bpf: Use same BPF program if arguments are identical
>       perf test: Print result for each LLVM subtest
>       perf test: Print result for each BPF subtest
>       perf test: Mute test cases error messages if verbose == 0
> 
>  tools/include/linux/string.h                |  11 +
>  tools/lib/bpf/libbpf.c                      | 146 ++++++++-
>  tools/lib/bpf/libbpf.h                      |  64 ++++
>  tools/lib/string.c                          |  62 ++++
>  tools/perf/MANIFEST                         |   2 +
>  tools/perf/arch/x86/util/Build              |   1 +
>  tools/perf/config/Makefile                  |  12 +
>  tools/perf/tests/.gitignore                 |   1 +
>  tools/perf/tests/Build                      |   9 +-
>  tools/perf/tests/bpf-script-test-prologue.c |  35 +++
>  tools/perf/tests/bpf.c                      |  79 ++++-
>  tools/perf/tests/builtin-test.c             | 134 ++++++--
>  tools/perf/tests/llvm.c                     |  67 ++--
>  tools/perf/tests/llvm.h                     |   2 +
>  tools/perf/tests/tests.h                    |  27 +-
>  tools/perf/util/Build                       |   7 +
>  tools/perf/util/bpf-loader.c                | 434 +++++++++++++++++++++++++-
>  tools/perf/util/bpf-loader.h                |   4 +
>  tools/perf/util/bpf-prologue.c              | 455 ++++++++++++++++++++++++++++
>  tools/perf/util/bpf-prologue.h              |  34 +++
>  tools/perf/util/include/linux/string.h      |   3 -
>  tools/perf/util/probe-event.c               |   7 +-
>  tools/perf/util/string.c                    |  16 -
>  23 files changed, 1502 insertions(+), 110 deletions(-)
>  create mode 100644 tools/include/linux/string.h
>  create mode 100644 tools/lib/string.c
>  create mode 100644 tools/perf/tests/bpf-script-test-prologue.c
>  create mode 100644 tools/perf/util/bpf-prologue.c
>  create mode 100644 tools/perf/util/bpf-prologue.h
>  delete mode 100644 tools/perf/util/include/linux/string.h

So considering the relatively large diffstat and the fact that we are 3 days after 
the closing of the merge window, I think we should separate this into minimal 
fixes for perf/urgent, and more involved changes for v4.5.

Nice progress with eBPF tooling!

Thanks,

	Ingo