Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire <alan.maguire@oracle.com>]

On 19/12/2025 18:19, Andrii Nakryiko wrote:
> On Fri, Dec 19, 2025 at 10:14 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 19/12/2025 17:53, Andrii Nakryiko wrote:
>>> On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> On 16/12/2025 19:23, Andrii Nakryiko wrote:
>>>>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>
>>>>>> BTF kind layouts provide information to parse BTF kinds. By separating
>>>>>> parsing BTF from using all the information it provides, we allow BTF
>>>>>> to encode new features even if they cannot be used by readers. This
>>>>>> will be helpful in particular for cases where older tools are used
>>>>>> to parse newer BTF with kinds the older tools do not recognize;
>>>>>> the BTF can still be parsed in such cases using kind layout.
>>>>>>
>>>>>> The intent is to support encoding of kind layouts optionally so that
>>>>>> tools like pahole can add this information. For each kind, we record
>>>>>>
>>>>>> - length of singular element following struct btf_type
>>>>>> - length of each of the btf_vlen() elements following
>>>>>>
>>>>>> The ideas here were discussed at [1], [2]; hence
>>>>>>
>>>>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>>>>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>>>>>
>>>>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
>>>>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
>>>>>> ---
>>>>>>  include/uapi/linux/btf.h       | 11 +++++++++++
>>>>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>>>>>>  2 files changed, 22 insertions(+)
>>>>>>
>>>>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
>>>>>> index 266d4ffa6c07..c1854a1c7b38 100644
>>>>>> --- a/include/uapi/linux/btf.h
>>>>>> +++ b/include/uapi/linux/btf.h
>>>>>> @@ -8,6 +8,15 @@
>>>>>>  #define BTF_MAGIC      0xeB9F
>>>>>>  #define BTF_VERSION    1
>>>>>>
>>>>>> +/*
>>>>>> + * kind layout section consists of a struct btf_kind_layout for each known
>>>>>> + * kind at BTF encoding time.
>>>>>> + */
>>>>>> +struct btf_kind_layout {
>>>>>> +       __u8 info_sz;           /* size of singular element after btf_type */
>>>>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
>>>>>
>>>>> So Eduard pointed out that at some point we discussed having a name of
>>>>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
>>>>> what were the arguments, do you remember? I'm not sure how I feel now
>>>>> about having extra 4 bytes per kind, but that's not really a lot of
>>>>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
>>>>>
>>>>
>>>> Yeah we went back and forth on that; I think it's on balance worthwhile
>>>> to be honest; tools can be a bit more expressive about what's missing.
>>>>
>>>>> I think we were also discussing having flags per kind to designate
>>>>> some extra semantics, where applicable. Again, don't remember
>>>>> arguments for or against, but one case where I think this would be
>>>>> very beneficial is when we add something like type_tag, which is
>>>>> inevitably used from "normal" struct and will be almost inevitable in
>>>>> normal vmlinux BTF. Think about it, we have some field which will be
>>>>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
>>>>> totally break (old) bpftool's dump, as it really can be easily ignored
>>>>> **if we know TYPE_TAG can be ignored and it is just a reference
>>>>> type**. That reference type means that there is another type pointed
>>>>> to using struct btf_type::type field (instead of that field being a
>>>>> size).
>>>>>
>>>>> So I think it would be nice to encode this as a flag that says a) kind
>>>>> can be ignored without compromising type integrity (i.e., memory
>>>>> layout is preserved) which will be true for all kinds of modifier
>>>>> kinds (const/volatile/restrict/type_tag, even for typedef that should
>>>>> be true) and b) kind is reference type, so struct btf_type::type is a
>>>>> "pointer" to a valid other underlying type.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>
>>>> Again we did go back and forth here but to me there's much more value in
>>>> being both able to parse _and_ sanitize BTF, at least for the simple cases.
>>>> What we can include are as you say types in the type graph that are optional
>>>> reference kinds (like type tag), and kinds that are not implicated in the
>>>> known type graph like the location stuff (it only points _to_ known kinds,
>>>> no known kinds will point to location data). So any case where known
>>>> types + optional ref types constitute the type graph we are good.
>>>> Anything more complex than these would involve having to represent the
>>>> layout of type references within unknown kinds (kind of like what we do for
>>>> field iteration) which seems a bit much.
>>>>
>>>> Now one thing that we might want to introduce here is a sanitization-friendly
>>>> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
>>>> which can be used to overwrite kinds we don't want in the sanitized output.
>>>> We need this to preserve the type ids for the kernel BTF we sanitize.
>>>> I get that it seems weird to add a new incompatibility to handle incompatibility,
>>>> but the sooner we do it the better I guess. The reason I suggest it now is we'd
>>>> potentially need some more complex sanitization for the location stuff for
>>>> cases like large location sections, and it might be cleaner to have a special
>>>> "ignore this it's just sanitization info" kind, especially for cases like
>>>> BTF C dump.
>>>
>>> So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
>>> we can "overwrite" any possible unknown BTF kind?.. As you said,
>>> though, this would only work for new kernels, so that's sad... I don't
>>> know, I don't hate the idea, but curious what others think.
>>>
>>> Alternatively, we can just try to never add kinds where the vlen
>>> element is not a multiple of 8 or 12. We can then use ENUM
>>> (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
>>> types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
>>> are other options. We just don't have 4-bytes-per-vlen for the most
>>> universal "filler", unfortunately.
>>>
>>> The advantage of the latter is full backwards compatibility with old kernels.
>>>
>>
>> True. And I guess during sanitization we can just handle intermediate
>> types in a type graph by adjusting type ids to skip over them, so we
>> likely have everything we need already. Funnily enough the BTF location
>> stuff will give us a vlen-specified 4 byte object (specifying the
>> location parameters associated with an inline), so that will help in
>> the future for cases where it is recognized but other kinds are not.
> 
> So coming back to flags? Let's do two flags: "safe modifier-like
> reference kind" (for type_tag-like things where they can be dropped
> from the chain of types) and "safe to ignore non-structural type" that
> can't be part of any struct/union and are more like decl_tag where
> they only reference other types, but can be dropped/replaced with
> something? And if kind doesn't have either of those, we won't attempt
> to sanitize (and hopefully we won't even have kinds like that, but if
> necessary we can add more flags with some other "safe" semantics, if
> necessary?)
>

Sounds perfect, thanks!