Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire <alan.maguire@oracle.com>]

On 19/12/2025 17:53, Andrii Nakryiko wrote:
> On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 16/12/2025 19:23, Andrii Nakryiko wrote:
>>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> BTF kind layouts provide information to parse BTF kinds. By separating
>>>> parsing BTF from using all the information it provides, we allow BTF
>>>> to encode new features even if they cannot be used by readers. This
>>>> will be helpful in particular for cases where older tools are used
>>>> to parse newer BTF with kinds the older tools do not recognize;
>>>> the BTF can still be parsed in such cases using kind layout.
>>>>
>>>> The intent is to support encoding of kind layouts optionally so that
>>>> tools like pahole can add this information. For each kind, we record
>>>>
>>>> - length of singular element following struct btf_type
>>>> - length of each of the btf_vlen() elements following
>>>>
>>>> The ideas here were discussed at [1], [2]; hence
>>>>
>>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>>>
>>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
>>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
>>>> ---
>>>>  include/uapi/linux/btf.h       | 11 +++++++++++
>>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>>>>  2 files changed, 22 insertions(+)
>>>>
>>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
>>>> index 266d4ffa6c07..c1854a1c7b38 100644
>>>> --- a/include/uapi/linux/btf.h
>>>> +++ b/include/uapi/linux/btf.h
>>>> @@ -8,6 +8,15 @@
>>>>  #define BTF_MAGIC      0xeB9F
>>>>  #define BTF_VERSION    1
>>>>
>>>> +/*
>>>> + * kind layout section consists of a struct btf_kind_layout for each known
>>>> + * kind at BTF encoding time.
>>>> + */
>>>> +struct btf_kind_layout {
>>>> +       __u8 info_sz;           /* size of singular element after btf_type */
>>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
>>>
>>> So Eduard pointed out that at some point we discussed having a name of
>>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
>>> what were the arguments, do you remember? I'm not sure how I feel now
>>> about having extra 4 bytes per kind, but that's not really a lot of
>>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
>>>
>>
>> Yeah we went back and forth on that; I think it's on balance worthwhile
>> to be honest; tools can be a bit more expressive about what's missing.
>>
>>> I think we were also discussing having flags per kind to designate
>>> some extra semantics, where applicable. Again, don't remember
>>> arguments for or against, but one case where I think this would be
>>> very beneficial is when we add something like type_tag, which is
>>> inevitably used from "normal" struct and will be almost inevitable in
>>> normal vmlinux BTF. Think about it, we have some field which will be
>>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
>>> totally break (old) bpftool's dump, as it really can be easily ignored
>>> **if we know TYPE_TAG can be ignored and it is just a reference
>>> type**. That reference type means that there is another type pointed
>>> to using struct btf_type::type field (instead of that field being a
>>> size).
>>>
>>> So I think it would be nice to encode this as a flag that says a) kind
>>> can be ignored without compromising type integrity (i.e., memory
>>> layout is preserved) which will be true for all kinds of modifier
>>> kinds (const/volatile/restrict/type_tag, even for typedef that should
>>> be true) and b) kind is reference type, so struct btf_type::type is a
>>> "pointer" to a valid other underlying type.
>>>
>>> Thoughts?
>>>
>>
>> Again we did go back and forth here but to me there's much more value in
>> being both able to parse _and_ sanitize BTF, at least for the simple cases.
>> What we can include are as you say types in the type graph that are optional
>> reference kinds (like type tag), and kinds that are not implicated in the
>> known type graph like the location stuff (it only points _to_ known kinds,
>> no known kinds will point to location data). So any case where known
>> types + optional ref types constitute the type graph we are good.
>> Anything more complex than these would involve having to represent the
>> layout of type references within unknown kinds (kind of like what we do for
>> field iteration) which seems a bit much.
>>
>> Now one thing that we might want to introduce here is a sanitization-friendly
>> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
>> which can be used to overwrite kinds we don't want in the sanitized output.
>> We need this to preserve the type ids for the kernel BTF we sanitize.
>> I get that it seems weird to add a new incompatibility to handle incompatibility,
>> but the sooner we do it the better I guess. The reason I suggest it now is we'd
>> potentially need some more complex sanitization for the location stuff for
>> cases like large location sections, and it might be cleaner to have a special
>> "ignore this it's just sanitization info" kind, especially for cases like
>> BTF C dump.
> 
> So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
> we can "overwrite" any possible unknown BTF kind?.. As you said,
> though, this would only work for new kernels, so that's sad... I don't
> know, I don't hate the idea, but curious what others think.
> 
> Alternatively, we can just try to never add kinds where the vlen
> element is not a multiple of 8 or 12. We can then use ENUM
> (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
> types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
> are other options. We just don't have 4-bytes-per-vlen for the most
> universal "filler", unfortunately.
> 
> The advantage of the latter is full backwards compatibility with old kernels.
>

True. And I guess during sanitization we can just handle intermediate
types in a type graph by adjusting type ids to skip over them, so we
likely have everything we need already. Funnily enough the BTF location
stuff will give us a vlen-specified 4 byte object (specifying the
location parameters associated with an inline), so that will help in
the future for cases where it is recognized but other kinds are not.

Alan