[PATCH v5 bpf-next 00/13] bpf: Add btf func info support

[Martin KaFai Lau <kafai@fb.com>]

The BTF support was added to kernel by Commit 69b693f0aefa
("bpf: btf: Introduce BPF Type Format (BTF)"), which introduced
.BTF section into ELF file and is primarily
used for map pretty print.
pahole is used to convert dwarf to BTF for ELF files.

This patch added func info support to the kernel so we can
get better ksym's for bpf function calls. Basically,
function call types are passed to kernel and the kernel
extract function names from these types in order to contruct ksym
for these functions.

The llvm patch at https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D53736&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=VQnoQ7LvghIj0gVEaiQSUw&m=CGaBxxwoBOirzU2H_1txiXzzOWCa2Z8ihJYI3JUJhOI&s=kwigBzxoDmvGui52AFHvn7a3oTNYBUEWKdxyTaHKjpA&e=
will generate .BTF section and one more section .BTF.ext.
The .BTF.ext section encodes function type
information. The following is a sample output for selftests
test_btf with file test_btf_haskv.o for translated insns
and jited insns respectively.

  $ bpftool prog dump xlated id 1
  int _dummy_tracepoint(struct dummy_tracepoint_args * arg):
     0: (85) call pc+2#bpf_prog_2dcecc18072623fc_test_long_fname_1
     1: (b7) r0 = 0
     2: (95) exit
  int test_long_fname_1(struct dummy_tracepoint_args * arg):
     3: (85) call pc+1#bpf_prog_89d64e4abf0f0126_test_long_fname_2
     4: (95) exit
  int test_long_fname_2(struct dummy_tracepoint_args * arg):
     5: (b7) r2 = 0
     6: (63) *(u32 *)(r10 -4) = r2
     7: (79) r1 = *(u64 *)(r1 +8)
     ...
     22: (07) r1 += 1
     23: (63) *(u32 *)(r0 +4) = r1
     24: (95) exit

  $ bpftool prog dump jited id 1
  int _dummy_tracepoint(struct dummy_tracepoint_args * arg):
  bpf_prog_b07ccb89267cf242__dummy_tracepoint:
     0:   push   %rbp
     1:   mov    %rsp,%rbp
    ......
    3c:   add    $0x28,%rbp
    40:   leaveq
    41:   retq

  int test_long_fname_1(struct dummy_tracepoint_args * arg):
  bpf_prog_2dcecc18072623fc_test_long_fname_1:
     0:   push   %rbp
     1:   mov    %rsp,%rbp
    ......
    3a:   add    $0x28,%rbp
    3e:   leaveq
    3f:   retq

  int test_long_fname_2(struct dummy_tracepoint_args * arg):
  bpf_prog_89d64e4abf0f0126_test_long_fname_2:
     0:   push   %rbp
     1:   mov    %rsp,%rbp
    ......
    80:   add    $0x28,%rbp
    84:   leaveq
    85:   retq

Changelogs:
  v4 -> v5:
    . Add back BTF_KIND_FUNC_PROTO as v1 did.  The difference
      is BTF_KIND_FUNC_PROTO cannot have t->name_off now.
      All param metadata is defined in BTF_KIND_FUNC_PROTO.
      BTF_KIND_FUNC must have t->name_off != 0 and t->type
      refers to a BTF_KIND_FUNC_PROTO.

      The above is the conclusion after the discussion between
      Edward Cree, Alexei, Daniel, Yonghong and Martin.
  v3 -> v4:
    . Remove BTF_KIND_FUNC_PROTO. BTF_KIND_FUNC is used for
      both function pointer and subprogram. The name_off field
      is used to distinguish both.
    . The record size is added to the func_info subsection
      in .BTF.ext to enable future extension.
    . The bpf_prog_info interface change to make it similar
      bpf_prog_load.
    . Related kernel and libbpf changes to accommodate the
      new .BTF.ext and kernel interface changes.
  v2 -> v3:
    . Removed kernel btf extern functions btf_type_id_func()
      and btf_get_name_by_id(). Instead, exposing existing
      functions btf_type_by_id() and btf_name_by_offset().
    . Added comments about ELF section .BTF.ext layout.
    . Better codes in btftool as suggested by Edward Cree.
  v1 -> v2:
    . Added missing sign-off.
    . Limited the func_name/struct_member_name length for validity test.
    . Removed/changed several verifier messages.
    . Modified several commit messages to remove line_off reference.

Martin KaFai Lau (4):
  bpf: btf: Break up btf_type_is_void()
  bpf: btf: Add BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO
  tools/bpf: Sync kernel btf.h header
  tools/bpf: Add tests for BTF_KIND_FUNC_PROTO and BTF_KIND_FUNC

Yonghong Song (9):
  bpf: Introduce bpf_func_info
  tools/bpf: sync kernel uapi bpf.h header to tools directory
  tools/bpf: add new fields for program load in lib/bpf
  tools/bpf: extends test_btf to test load/retrieve func_type info
  tools/bpf: add support to read .BTF.ext sections
  tools/bpf: do not use pahole if clang/llvm can generate BTF sections
  tools/bpf: refactor to implement btf_get_from_id() in lib/bpf
  tools/bpf: enhance test_btf file testing to test func info
  tools/bpf: bpftool: add support for func types

 include/linux/bpf.h                          |   5 +-
 include/linux/bpf_verifier.h                 |   1 +
 include/linux/btf.h                          |   2 +
 include/uapi/linux/bpf.h                     |  13 +
 include/uapi/linux/btf.h                     |  18 +-
 kernel/bpf/btf.c                             | 420 +++++++--
 kernel/bpf/core.c                            |  13 +
 kernel/bpf/syscall.c                         |  59 +-
 kernel/bpf/verifier.c                        | 120 ++-
 samples/bpf/Makefile                         |   8 +
 tools/bpf/bpftool/btf_dumper.c               | 136 +++
 tools/bpf/bpftool/main.h                     |   2 +
 tools/bpf/bpftool/map.c                      |  68 +-
 tools/bpf/bpftool/prog.c                     |  56 ++
 tools/bpf/bpftool/xlated_dumper.c            |  33 +
 tools/bpf/bpftool/xlated_dumper.h            |   3 +
 tools/include/uapi/linux/bpf.h               |  13 +
 tools/include/uapi/linux/btf.h               |  18 +-
 tools/lib/bpf/bpf.c                          |  50 +-
 tools/lib/bpf/bpf.h                          |   4 +
 tools/lib/bpf/btf.c                          | 347 +++++++
 tools/lib/bpf/btf.h                          |  51 +
 tools/lib/bpf/libbpf.c                       |  87 +-
 tools/testing/selftests/bpf/Makefile         |   8 +
 tools/testing/selftests/bpf/test_btf.c       | 923 ++++++++++++++++++-
 tools/testing/selftests/bpf/test_btf_haskv.c |  16 +-
 tools/testing/selftests/bpf/test_btf_nokv.c  |  16 +-
 27 files changed, 2317 insertions(+), 173 deletions(-)

-- 
2.17.1