Re: [PATCH bpf-next 0/5] BTF: arbitrary __attribute__ encoding

["Jose E. Marchesi" <jose.marchesi@oracle.com>]

> On Wednesday, January 22nd, 2025 at 3:44 AM, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:
>
>> 
>> 
>> > This patch series extends BPF Type Format (BTF) to support arbitrary
>> > attribute encoding.
>> > 
>> > Setting the kind_flag to 1 in BTF type tags and decl tags now changes
>> > the meaning for the encoded tag, in particular with respect to
>> > btf_dump in libbpf.
>> > 
>> > If the kflag is set, then the string encoded by the tag represents the
>> > full attribute-list of an attribute specifier [1].
>> 
>> 
>> Why is extending BTF necessary for this? Type and declaration tags
>> contain arbitrary strings, and AFAIK you can have more than one type tag
>> associated with a single type or declaration. Why coupling the
>> interpretation of the contents of the string with the transport format?
>> 
>> Something like "cattribute:always_inline".
>
> Hi Jose. Good questions.
>
> You are correct that the tags can contain arbitrary strings already,
> and that multiple tags can be associated with the same type or
> declaration.
>
> A specific problem I'm trying to solve is how to direct btf_dump in
> interpreting tags as attributes, and do it in a generic way, as it's a
> part of libbpf.
>
> I discussed with Andrii, Eduard and Alexei a couple of approaches, and
> tried some of them.
>
> For example, a set of dump options could be introduced to handle
> specific use-cases, similar to what you suggested in a
> ATTR_PRESERVE_ACCESS_INDEX patch [1]. This is a valid approach,
> however it is not very generic. An option will have to be introduced
> and implemented for every new use-case.
>
> A more generic approach is adding a set of callbacks to btf_dump. This
> is a big design task, which I think should be avoided unless
> absolutely necessary.
>
> The benefit of this change - defining flagged tags as attributes - is
> that it enables BTF to natively encode attributes as part of a type,
> which is not possible currently. And it's a simple change.
>
> Using the contents of the tag to indicate it's meaning (such as
> "cattrubite:always_inline") will work too. However I don't think it's
> desirable to have to parse the tag strings within libbpf, even more so
> in BPF verifier.

I expect the verifier will in any case have to distinguish the different
strings it gets in the tags, for other purposes, right, so this wouldn't
be introducing anything different?

Also FWIW DWARF doesn't have a kind_flag.

>
> In a discussion with Andrii we briefly entertained an idea of allowing
> btf_dump to print the tag string directly (without requiring it to be
> a tag or attribute), which would allow all kinds of hacks. Tempting,
> but probably very bug-prone.
>
> [1] https://lore.kernel.org/bpf/20240503111836.25275-1-jose.marchesi@oracle.com/
>
>> 
>> > This feature will allow extending tools such as pahole and bpftool to
>> > capture and use more granular type information, and make it easier to
>> > manage compatibility between clang and gcc BPF compilers.
>> > 
>> > [1] https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gcc/Attribute-Syntax.html
>> > 
>> > Ihor Solodrai (5):
>> > libbpf: introduce kflag for type_tags and decl_tags in BTF
>> > libbpf: check the kflag of type tags in btf_dump
>> > selftests/bpf: add a btf_dump test for type_tags
>> > bpf: allow kind_flag for BTF type and decl tags
>> > selftests/bpf: add a BTF verification test for kflagged type_tag
>> > 
>> > Documentation/bpf/btf.rst | 27 +++-
>> > kernel/bpf/btf.c | 7 +-
>> > tools/include/uapi/linux/btf.h | 3 +-
>> > tools/lib/bpf/btf.c | 87 +++++++---
>> > tools/lib/bpf/btf.h | 3 +
>> > tools/lib/bpf/btf_dump.c | 5 +-
>> > tools/lib/bpf/libbpf.map | 2 +
>> > tools/testing/selftests/bpf/prog_tests/btf.c | 23 ++-
>> > .../selftests/bpf/prog_tests/btf_dump.c | 148 +++++++++++++-----
>> > tools/testing/selftests/bpf/test_btf.h | 6 +
>> > 10 files changed, 234 insertions(+), 77 deletions(-)