Re: [PATCH bpf-next 0/9] bpf: add support for new btf kind BTF_KIND_TAG

["Jose E. Marchesi" <jose.marchesi@oracle.com>]

> On 9/9/21 7:19 PM, Jose E. Marchesi wrote:
>> 
>>> On 9/9/21 3:45 PM, Jose E. Marchesi wrote:
>>>> Hi Yonghong.
>>>>
>>>>> LLVM14 added support for a new C attribute ([1])
>>>>>     __attribute__((btf_tag("arbitrary_str")))
>>>>> This attribute will be emitted to dwarf ([2]) and pahole
>>>>> will convert it to BTF. Or for bpf target, this
>>>>> attribute will be emitted to BTF directly ([3]).
>>>>> The attribute is intended to provide additional
>>>>> information for
>>>>>     - struct/union type or struct/union member
>>>>>     - static/global variables
>>>>>     - static/global function or function parameter.
>>>>>
>>>>> This new attribute can be used to add attributes
>>>>> to kernel codes, e.g., pre- or post- conditions,
>>>>> allow/deny info, or any other info in which only
>>>>> the kernel is interested. Such attributes will
>>>>> be processed by clang frontend and emitted to
>>>>> dwarf, converting to BTF by pahole. Ultimiately
>>>>> the verifier can use these information for
>>>>> verification purpose.
>>>>>
>>>>> The new attribute can also be used for bpf
>>>>> programs, e.g., tagging with __user attributes
>>>>> for function parameters, specifying global
>>>>> function preconditions, etc. Such information
>>>>> may help verifier to detect user program
>>>>> bugs.
>>>>>
>>>>> After this series, pahole dwarf->btf converter
>>>>> will be enhanced to support new llvm tag
>>>>> for btf_tag attribute. With pahole support,
>>>>> we will then try to add a few real use case,
>>>>> e.g., __user/__rcu tagging, allow/deny list,
>>>>> some kernel function precondition, etc,
>>>>> in the kernel.
>>>> We are looking into implementing this in the GCC BPF port.
>>>
>>> Hi, Jose, thanks for your reply. It would be great if the
>>> btf_tag can be implemented in gcc.
>>>
>>>> Supporting the new C attribute in BPF programs as a target-specific
>>>> attribute, and the new BTF kind, is straightforward enough.
>>>> However, I am afraid it will be difficult to upstream to GCC support
>>>> for
>>>> a target-independent C attribute called `btf_tag' that emits a
>>>> LLVM-specific DWARF tag.  Even if we proposed to use a GCC-specific
>>>
>>> Are you concerned with the name? The btf_tag name cames from the
>>> discussion in
>>> https://lore.kernel.org/bpf/CAADnVQJa=b=hoMGU213wMxyZzycPEKjAPFArKNatbVe4FvzVUA@mail.gmail.com/
>>> as llvm guys want this attribute to be explicitly referring to bpf echo
>>> system because we didn't implement for C++, and we didn't try to
>>> annotate everywhere. Since its main purpose is to eventually encode in
>>> btf (for different architectures), so we settled with btf_tag instead of
>>> bpf_tag.
>>>
>>> But if you have suggestion to change to a different name which can
>>> be acceptable by both gcc and llvm community, I am okay with that.
>> I think the name of the attribute is very fine when BTF is generated
>> directly, like when compiling BPF programs.  My concern is that the
>> connection `btf_tag () -> DWARF -> kernel/pahole -> BTF' may be seen as
>> too indirect and application-specific (the kernel) for a general-purpose
>> compiler attribute.
>
> For llvm, btf_tag implies implementation scope as it *only covers* btf
> use cases. There are some other use cases which may utilize the same
> IR/dwarf implementation, but they may use a flag to control or different
> attribute. And this has been agreed upon with llvm community, so we
> should be okay here.
>
>>>> DWARF tag like DW_TAG_GNU_annotation using the same number, or better a
>>>> compiler neutral tag like DW_TAG_annotation or DW_TAG_BPF_annotation,
>>>> adding such an attribute for all targets would still likely to be much
>>>> controversial...
>>>
>>> This is okay too. If gcc settles with DW_TAG_GNU_annotation with a
>>> different number (not conflict with existing other llvm tag numbers),
>>> I think llvm can change to have the same name and number since we are
>>> still in the release.
>> Thanks, that is very nice and appreciated :) I don't think the
>> particular number used to encode the tag matters much, provided it
>> doesn't collide with any existing one of course...
>> However, there may be a way to entirely avoid creating a new DWARF
>> tag... see below.
>> 
>>>> Would you be open to explore other, more generic, ways to convey
>>>> these
>>>> annotations to pahole, something that could be easily supported by GCC,
>>>> and potentially other C compilers?
>>>
>>> Could you share your proposal in detail? I think some kind of difference
>>> might be okay if it is handled by pahole and invisible to users,
>>> although it would be good if pahole only needs to handle single
>>> interface w.r.t. btf_tag support.
>> GCC can currently generate BTF for any target, not just BPF.  For
>> example, you can compile an object foo.o with both DWARF and BTF with:
>> $ gcc -c -gdwarf -gbtf foo.c
>> or with just BTF with:
>> $ gcc -c -gbtf foo.c
>> Binutils (ld) also supports full type deduplication for CTF, which
>> is
>> very similar to BTF.  We use it to build kernels in-house with CTF
>> enabled for dtrace.  It is certainly possible to add support to ld to
>> also merge and deduplicate BTF sections... it is similar enough to CTF
>> to (hopefully) not require much work, and we were already considering
>> doing it anyway for other purposes.
>> So the proposal would be:
>> For GCC, we can implement the btf_tag for any target, but impacting
>> only
>> the BTF output as the name implies.  No effect in DWARF.  Once ld is
>> able to merge and deduplicate BTF, it shall then be possible to build
>> the kernel and obtain the BTF for it without the aid of pahole, just
>> building it with -gdwarf -gbtf and linking normally. (We know this works
>> with CTF.)
>
> This should be okay.

Super.  We will work in this direction.

>> For LLVM, nothing would have to be done in the short term: just use
>> the
>> DWARF DIE + pahole approach already implemented.  But in the medium term
>> LLVM could be made to 1) support emitting BTF for any target (not sure
>> how difficult would that be, maybe it already can do that?) and 2) to
>> support the -gbtf command-line option.
>> Then the generation of BTF for the kernel could be done in the same
>> way
>> (same build rules) with both compilers, and there would be no need for
>> conveying the extra tags (nor any future extensions to aid the verifier
>> on the kernel side) to pahole via DWARF.  Pure BTF all the way up (or
>> down) without diversion to DWARF :)
>> Does this make sense? WDYT?
>
> During discussion to implement btf_tag attribute, I actually have a
> prototype to emit BTF with non-BPF targets in llvm.
> see https://reviews.llvm.org/D103549
>
> But since we get a simpler solution to emit the info to llvm, so we went
> there. We will keep this in mind, it is totally possible in the future
> we may start to directly generate BTF from llvm for all architectures.

Nice :)
Thanks.