Re: [RFC PATCH dwarves] btf: add support for split BTF loading and encoding

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Thu, Nov 05, 2020 at 11:10:14AM -0800, Andrii Nakryiko escreveu:
> On Thu, Nov 5, 2020 at 3:42 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > Em Wed, Nov 04, 2020 at 08:39:36PM -0800, Andrii Nakryiko escreveu:
> > > @@ -679,11 +681,11 @@ static int btf_elf__write(const char *filename, struct btf *btf)
> > >  {
> > >       GElf_Shdr shdr_mem, *shdr;
> > >       GElf_Ehdr ehdr_mem, *ehdr;
> > > -     Elf_Data *btf_elf = NULL;
> > > +     Elf_Data *btf_data = NULL;

> > Can you please split this into two patches, one doing just the rename
> > of btf_elf to btf_data and then moving to btf__new_empty_split()? Eases
> > reviewing.
 
> sure, will do in the next version

Thanks!
 
> > With this split btf code would it be possible to paralelize the encoding
> > of the modules BTF? I have to check the other patches and how this gets
> > used in the kernel build process... :-)
 
> Yes, each module's BTF is generated completely independently. See some
> numbers in [0].
> 
>   [0] https://patchwork.kernel.org/project/netdevbpf/patch/20201105045140.2589346-4-andrii@kernel.org/

I saw it, very good. I wonder if we could manage to also paralelize the
processing of DWARF compile units in the BTF encoder, like start
processing and at the end just figure out how many types were in a CU,
get the highest type id and bump it to + the number of types in the
current CU, adjust the types, continue, something like that.

- Arnaldo