Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire <alan.maguire@oracle.com>]

On 09/01/2026 01:40, Alexei Starovoitov wrote:
> On Thu, Jan 8, 2026 at 5:24 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>
>>> On 06/01/2026 01:19, Andrii Nakryiko wrote:
>>>> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
>>>> <alexei.starovoitov@gmail.com> wrote:
>>>>>
>>>>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
>>>>> <andrii.nakryiko@gmail.com> wrote:
>>>>>>
>>>>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>
>>>>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
>>>>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>>>>>>>>
>>>>>>>>> The sanitization we can get from layout info is for handling a kernel built with
>>>>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>>>>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
>>>>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>>>>>>>>> to be usable for some cases (where the type graph is not fatally compromised
>>>>>>>>> by the lack of a kind). This will always be somewhat limited, but it
>>>>>>>>> does provide more usability than we have today.
>>>>>>>>
>>>>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
>>>>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
>>>>>>>> only to
>>>>>>>
>>>>>>> Right; it's an extension of the sanitization concept from what it does today.
>>>>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
>>>>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
>>>>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
>>>>>>> latest kind info kernel has) to make that kernel BTF usable.
>>>>>>
>>>>>> Wait, is that really a goal? I get why Alexei is confused now :)
>>>>>>
>>>>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
>>>>>> for the sake of loading it into the kernel. If some user space tool is
>>>>>> trying to work with kernel BTF that has BTF features that tool doesn't
>>>>>> support, then we only have two reasonable options: a) tool just fails
>>>>>> to process that BTF altogether or b) the tool is smart enough to
>>>>>> utilize BTF layout information to know which BTF types it can safely
>>>>>> skip (that's where those flags I argue for would be useful). In both
>>>>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
>>>>>> layout info to construct a lookup table for btf__type_by_id(). And
>>>>>> libbpf doesn't need to do anything beyond that, IMO.
>>>>>>
>>>>>> We'll teach bpftool to dump as much of BTF as possible (I mean
>>>>>> `bpftool btf dump file`), so it's possible to get an idea of what part
>>>>>> of BTF is not supported and show those that we know about. We could
>>>>>> teach btf_dump to ignore those types that are "safe modifier-like
>>>>>> reference kind" (as marked with that flag I proposed earlier), so that
>>>>>> `format c` works as well (though I wouldn't recommend using such
>>>>>> output as a proper vmlinux.h, users should update bpftool ASAP for
>>>>>> such use cases).
>>>>>>
>>>>>> As far as the kernel is concerned, BTF layout is not used and should
>>>>>> not be used or trusted (it can be "spoofed" by the user). It can
>>>>>> validate it for sanity, but that's pretty much it. Other than that, if
>>>>>> the kernel doesn't *completely* understand every single piece of BTF,
>>>>>> it should reject it (and that's also why libbpf should sanitize BPF
>>>>>> object's BTF, of course).
>>>>>
>>>>> +1 to all of the above, except ok-to-skip flag, since I feel
>>>>> it will cause more bike sheding and arguing whether a particular
>>>>> new addition to BTF is skippable or not. Like upcoming location info.
>>>>
>>>> I was thinking about something like TYPE_TAG, where it's in the chain
>>>> of types and is unavoidable when processing STRUCT and its field.
>>>> Having a flag specifying that it's ref-like (so btf_type::type field
>>>> points to a valid type ID) would allow it to still make sense of the
>>>> entire struct and its fields, though you might be missing some
>>>> (presumably) optional and highly-specialized extra annotation.
>>>>
>>>> But it's fine not to add it, just some type graphs will be completely
>>>> unprocessable using old tools. Perhaps not such a big deal.
>>>>
>>>> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
>>>> whether they are more like DECL_TAG (roots pointing to other types) or
>>>> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
>>>> fields, PROTO args, etc).
>>>>
>>>>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
>>>>> Skippable? One might argue that they are while they are mandatory
>>>>> for some other use case.
>>>>> Looking at it differently, if the kernel is old and cannot understand that
>>>>> BTF feature the libbpf has to sanitize it no matter skippable or not.
>>>>> While from btf__parse() pov it also doesn't matter.
>>>>> btf_new()->btf_parse_hdr() will remember kind layout,
>>>>> and btf_parse_type_sec() can construct the index for the whole thing
>>>>> with layout info,
>>>>> while btf_validate_type() has to warn about unknown kind regardless
>>>>> of skippable flag. The tool (bpftool or else) needs to yell when
>>>>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
>>>>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
>>>>> though one might argue that they are.
>>>>
>>>> Yeah, I agree about vmlinux.h. One way to enforce this would be to
>>>> have btf_dump emit something uncompilable as
>>>> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
>>>> modified.
>>>>
>>>> But yeah, we don't want bikeshedding. It's fine.
>>>>
>>>
>>> Ok so is it best to leave out flags entirely then? If so where we
>>> are now is to have each kind layout entry have a string name offset,
>>> a singular element size and a vlen-specified object size. To be
>>> conservative it might make sense to allow 16 bits for each size field,
>>> leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
>>> We could cut down further by leaving out kind name strings if needed.
>>
>> Are we sure we will *never* need flags? I'd probably stick to
>> single-byte sizes and have 2 bytes reserved for flags or whatever we
>> might need in the future?
> 
> Just to clarify what I was saying.
> I think it's a good thing to have flags space and reserve it.
> I just struggle to see the value of 'ok-to-skip' flag.
> 
> So 2 bytes of reserved space for flags makes sense to me.

Ok sounds good; I think there is still value in having the single flag
that tells us that the type/size field in struct btf_type refers
to a type though, right?