Re: [PATCH bpf-next v2 00/11] bpf: add support for new btf kind BTF_KIND_TAG

["Jose E. Marchesi" <jose.marchesi@oracle.com>]

> On 12/20/21 1:49 AM, Jose E. Marchesi wrote:
>> 
>>> On 12/17/21 5:44 PM, Alexei Starovoitov wrote:
>>>> On Fri, Dec 17, 2021 at 11:40:10AM +0100, Jose E. Marchesi wrote:
>>>>>
>>>>> 2) The need for DWARF to convey free-text tags on certain elements, such
>>>>>      as members of struct types.
>>>>>
>>>>>      The motivation for this was originally the way the Linux kernel
>>>>>      generates its BTF information, using pahole, using DWARF as a source.
>>>>>      As we discussed in our last exchange on this topic, this is
>>>>>      accidental, i.e. if the kernel switched to generate BTF directly from
>>>>>      the compiler and the linker could merge/deduplicate BTF, there would
>>>>>      be no need for using DWARF to act as the "unwilling conveyer" of this
>>>>>      information.  There are additional benefits of this second approach.
>>>>>      Thats why we didn't plan to add these extended DWARF DIEs to GCC.
>>>>>
>>>>>      However, it now seems that a DWARF consumer, the drgn project, would
>>>>>      also benefit from having such a support in DWARF to distinguish
>>>>>      between different kind of pointers.
>>>> drgn can use .percpu section in vmlinux for global percpu vars.
>>>> For pointers the annotation is indeed necessary.
>>>>
>>>>>      So it seems to me that now we have two use-cases for adding support
>>>>>      for these free-text tags to DWARF, as a proper extension to the
>>>>>      format, strictly unrelated to BTF, BPF or even the kernel, since:
>>>>>      - This is not kernel specific.
>>>>>      - This is not directly related to BTF.
>>>>>      - This is not directly related to BPF.
>>>> __percpu annotation is kernel specific.
>>>> __user and __rcu are kernel specific too.
>>>> Only BPF and BTF can meaningfully consume all three.
>>>> drgn can consume __percpu.
>>>> In that sense if GCC follows LLVM and emits compiler specific DWARF
>>>> tag
>>>> pahole can convert it to the same BTF regardless whether kernel
>>>> was compiled with clang or gcc.
>>>> drgn can consume dwarf generated by clang or gcc as well even when BTF
>>>> is not there. That is the fastest way forward.
>>>> In that sense it would be nice to have common DWARF tag for pointer
>>>> annotations, but it's not mandatory. The time is the most valuable asset.
>>>> Implementing GCC specific DWARF tag doesn't require committee voting
>>>> and the mailing list bikeshedding.
>>>>
>>>>> 3) Addition of C-family language-level constructions to specify
>>>>>      free-text tags on certain language elements, such as struct fields.
>>>>>
>>>>>      These are the attributes, or built-ins or whatever syntax.
>>>>>
>>>>>      Note that, strictly speaking:
>>>>>      - This is orthogonal to both DWARF and BTF, and any other supported
>>>>>        debugging format, which may or may not be expressive enough to
>>>>>        convey the free-form text tag.
>>>>>      - This is not specific to BPF.
>>>>>
>>>>>      Therefore I would avoid any reference to BTF or BPF in the attribute
>>>>>      names.  Something like `__attribute__((btf_tag("arbitrary_str")))'
>>>>>      makes very little sense to me; the attribute name ought to be more
>>>>>      generic.
>>>> Let's agree to disagree.
>>>> When BPF ISA was designed we didn't go to Intel, Arm, Mips, etc in order to
>>>> come up with the best ISA that would JIT to those architectures the best
>>>> possible way. Same thing with btf_tag. Today it is specific to BTF and BPF
>>>> only. Hence it's called this way. Whenever actual users will appear that need
>>>> free-text tags on a struct field then and only then will be the time to discuss
>>>> generic tag name. Just because "free-text tag on a struct field" sounds generic
>>>> it doesn't mean that it has any use case beyond what we're using it for in BPF
>>>> land. It goes back to the point of coding now instead of talking about coding.
>>>> If gcc wants to call it __attribute__((my_precious_gcc_tag("arbitrary_str")))
>>>> go ahead and code it this way. The include/linux/compiler.h can accommodate it.
>>>
>>> Just want to add a little bit context for this. In the beginning when
>>> we proposed to add the attribute, we named as a generic name like
>>> 'tag' (or something like that). But eventually upstream suggested
>>> 'btf_tag' since the use case we proposed is for bpf. At that point, we
>>> don't know drgn use cases yet. Even with that, the use cases are still
>>> just for linux kernel.
>>>
>>> At that time, some *similar* use cases did came up, e.g., for
>>> swift<->C++ conversion encoding ("tag name", "attribute info") for
>>> attributes in the source code, will help a lot. But they will use a
>>> different "tag name" than btf_tag to differentiate.
>> Thanks for the info.
>> I find it very interesting that the LLVM people prefers to have
>> several
>> "use case specific" tag names instead of something more generic, which
>> is the exact opposite of what I would have done :) They may have
>> appealing reasons for doing so.  Do you have a pointer to the dicussion
>> you had upstream at hand?
>> Anyway, I will taste the waters with the other GCC hackers about
>> both
>> DIEs and attribute and see what we can come out with.  Thanks again for
>> reaching out Yonghong.
>
> Hi, Jose,
>
> Any progress on gcc btf_tag support discussion? If possible, could
> you add me to the discussion mailing list so I may help to move
> the project forward? Thanks a lot!

We are in the process of implementing the support of the BTF extensions
(which is done) and the C language attributes (which is WIP.)

I haven't started the discussion about DWARF yet.  Will do shortly.  You
will be in CC :)