Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu:
> > > This patch introduces BPF Type Format (BTF).
> > > 
> > > BTF (BPF Type Format) is the meta data format which describes
> > > the data types of BPF program/map.  Hence, it basically focus
> > > on the C programming language which the modern BPF is primary
> > > using.  The first use case is to provide a generic pretty print
> > > capability for a BPF map.
> > > 
> > > A modified pahole that can convert dwarf to BTF is here:
> > > https://github.com/iamkafai/pahole/tree/btf
> > > (Arnaldo, there is some BTF_KIND numbering changes on
> > >  Apr 18th, d61426c1571)
> > 
> > Thanks for letting me know, I'm starting to look at this,
> Hi Arnaldo,
> 
> Do you have a chance to take a look and pull it?  The kernel
> changes will be in 4.18, so it will be handy if it is available in
> the pahole repository.
> 
> [ btw, the latest commit (1 commit) should be 94a11b59e592 ].

Yeah, the one I had before had:

    It also raises the number of types (and functions) limit from 0x7fff to
    0x7fffffff.

----

And on this last one I see that:

 /* Max # of type identifier */
-#define BTF_MAX_TYPE   0x7fffffff
+#define BTF_MAX_TYPE   0x0000ffff
 /* Max offset into the string section */
-#define BTF_MAX_NAME_OFFSET    0x7fffffff
+#define BTF_MAX_NAME_OFFSET    0x0000ffff

So somehow (still reading) you'll be able to get more space, if we find
necessary, to have more types and names, ok.

Continuing...

- Arnaldo
 
> > 
> > - Arnaldo
> >  
> > > Please see individual patch for details.
> > > 
> > > v5:
> > > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not
> > >   currently used.  They can be added in the future.
> > >   Some bpf_df_xxx() are removed together.
> > > - Add comment in patch 7 to clarify that the new bpffs_map_fops
> > >   should not be extended further.
> > > 
> > > v4:
> > > - Fix warning (remove unneeded semicolon)
> > > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in
> > >   patch 1.  Caught by W=1.
> > > 
> > > v3:
> > > - Rebase to bpf-next
> > > - Fix sparse warning (by adding static)
> > > - Add BTF header logging: btf_verifier_log_hdr()
> > > - Fix the alignment test on btf->type_off
> > > - Add tests for the BTF header
> > > - Lower the max BTF size to 16MB.  It should be enough
> > >   for some time.  We could raise it later if it would
> > >   be needed.
> > > 
> > > v2:
> > > - Use kvfree where needed in patch 1 and 2
> > > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> > >   in patch 1
> > > - Fix an incorrect goto target in map_create() during
> > >   the btf-error-path in patch 7
> > > - re-org some local vars to keep the rev xmas tree in btf.c
> > > 
> > > Martin KaFai Lau (10):
> > >   bpf: btf: Introduce BPF Type Format (BTF)
> > >   bpf: btf: Validate type reference
> > >   bpf: btf: Check members of struct/union
> > >   bpf: btf: Add pretty print capability for data with BTF type info
> > >   bpf: btf: Add BPF_BTF_LOAD command
> > >   bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
> > >   bpf: btf: Add pretty print support to the basic arraymap
> > >   bpf: btf: Sync bpf.h and btf.h to tools/
> > >   bpf: btf: Add BTF support to libbpf
> > >   bpf: btf: Add BTF tests
> > > 
> > >  include/linux/bpf.h                          |   20 +-
> > >  include/linux/btf.h                          |   48 +
> > >  include/uapi/linux/bpf.h                     |   12 +
> > >  include/uapi/linux/btf.h                     |  130 ++
> > >  kernel/bpf/Makefile                          |    1 +
> > >  kernel/bpf/arraymap.c                        |   50 +
> > >  kernel/bpf/btf.c                             | 2064 ++++++++++++++++++++++++++
> > >  kernel/bpf/inode.c                           |  156 +-
> > >  kernel/bpf/syscall.c                         |   51 +-
> > >  tools/include/uapi/linux/bpf.h               |   12 +
> > >  tools/include/uapi/linux/btf.h               |  130 ++
> > >  tools/lib/bpf/Build                          |    2 +-
> > >  tools/lib/bpf/bpf.c                          |   92 +-
> > >  tools/lib/bpf/bpf.h                          |   16 +
> > >  tools/lib/bpf/btf.c                          |  374 +++++
> > >  tools/lib/bpf/btf.h                          |   22 +
> > >  tools/lib/bpf/libbpf.c                       |  148 +-
> > >  tools/lib/bpf/libbpf.h                       |    3 +
> > >  tools/testing/selftests/bpf/Makefile         |   26 +-
> > >  tools/testing/selftests/bpf/test_btf.c       | 1669 +++++++++++++++++++++
> > >  tools/testing/selftests/bpf/test_btf_haskv.c |   48 +
> > >  tools/testing/selftests/bpf/test_btf_nokv.c  |   43 +
> > >  22 files changed, 5076 insertions(+), 41 deletions(-)
> > >  create mode 100644 include/linux/btf.h
> > >  create mode 100644 include/uapi/linux/btf.h
> > >  create mode 100644 kernel/bpf/btf.c
> > >  create mode 100644 tools/include/uapi/linux/btf.h
> > >  create mode 100644 tools/lib/bpf/btf.c
> > >  create mode 100644 tools/lib/bpf/btf.h
> > >  create mode 100644 tools/testing/selftests/bpf/test_btf.c
> > >  create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
> > >  create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
> > > 
> > > -- 
> > > 2.9.5