Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire <alan.maguire@oracle.com>]

On 06/01/2026 01:19, Andrii Nakryiko wrote:
> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
>>
>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
>> <andrii.nakryiko@gmail.com> wrote:
>>>
>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>>>>>
>>>>>> The sanitization we can get from layout info is for handling a kernel built with
>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>>>>>> to be usable for some cases (where the type graph is not fatally compromised
>>>>>> by the lack of a kind). This will always be somewhat limited, but it
>>>>>> does provide more usability than we have today.
>>>>>
>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
>>>>> only to
>>>>
>>>> Right; it's an extension of the sanitization concept from what it does today.
>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
>>>> latest kind info kernel has) to make that kernel BTF usable.
>>>
>>> Wait, is that really a goal? I get why Alexei is confused now :)
>>>
>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
>>> for the sake of loading it into the kernel. If some user space tool is
>>> trying to work with kernel BTF that has BTF features that tool doesn't
>>> support, then we only have two reasonable options: a) tool just fails
>>> to process that BTF altogether or b) the tool is smart enough to
>>> utilize BTF layout information to know which BTF types it can safely
>>> skip (that's where those flags I argue for would be useful). In both
>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
>>> layout info to construct a lookup table for btf__type_by_id(). And
>>> libbpf doesn't need to do anything beyond that, IMO.
>>>
>>> We'll teach bpftool to dump as much of BTF as possible (I mean
>>> `bpftool btf dump file`), so it's possible to get an idea of what part
>>> of BTF is not supported and show those that we know about. We could
>>> teach btf_dump to ignore those types that are "safe modifier-like
>>> reference kind" (as marked with that flag I proposed earlier), so that
>>> `format c` works as well (though I wouldn't recommend using such
>>> output as a proper vmlinux.h, users should update bpftool ASAP for
>>> such use cases).
>>>
>>> As far as the kernel is concerned, BTF layout is not used and should
>>> not be used or trusted (it can be "spoofed" by the user). It can
>>> validate it for sanity, but that's pretty much it. Other than that, if
>>> the kernel doesn't *completely* understand every single piece of BTF,
>>> it should reject it (and that's also why libbpf should sanitize BPF
>>> object's BTF, of course).
>>
>> +1 to all of the above, except ok-to-skip flag, since I feel
>> it will cause more bike sheding and arguing whether a particular
>> new addition to BTF is skippable or not. Like upcoming location info.
> 
> I was thinking about something like TYPE_TAG, where it's in the chain
> of types and is unavoidable when processing STRUCT and its field.
> Having a flag specifying that it's ref-like (so btf_type::type field
> points to a valid type ID) would allow it to still make sense of the
> entire struct and its fields, though you might be missing some
> (presumably) optional and highly-specialized extra annotation.
> 
> But it's fine not to add it, just some type graphs will be completely
> unprocessable using old tools. Perhaps not such a big deal.
> 
> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
> whether they are more like DECL_TAG (roots pointing to other types) or
> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
> fields, PROTO args, etc).
> 
>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
>> Skippable? One might argue that they are while they are mandatory
>> for some other use case.
>> Looking at it differently, if the kernel is old and cannot understand that
>> BTF feature the libbpf has to sanitize it no matter skippable or not.
>> While from btf__parse() pov it also doesn't matter.
>> btf_new()->btf_parse_hdr() will remember kind layout,
>> and btf_parse_type_sec() can construct the index for the whole thing
>> with layout info,
>> while btf_validate_type() has to warn about unknown kind regardless
>> of skippable flag. The tool (bpftool or else) needs to yell when
>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
>> though one might argue that they are.
> 
> Yeah, I agree about vmlinux.h. One way to enforce this would be to
> have btf_dump emit something uncompilable as
> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
> modified.
> 
> But yeah, we don't want bikeshedding. It's fine.
> 

Ok so is it best to leave out flags entirely then? If so where we
are now is to have each kind layout entry have a string name offset,
a singular element size and a vlen-specified object size. To be
conservative it might make sense to allow 16 bits for each size field,
leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
We could cut down further by leaving out kind name strings if needed.