Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire <alan.maguire@oracle.com>]

On 09/01/2026 18:34, Alexei Starovoitov wrote:
> On Fri, Jan 9, 2026 at 5:21 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 09/01/2026 01:40, Alexei Starovoitov wrote:
>>> On Thu, Jan 8, 2026 at 5:24 PM Andrii Nakryiko
>>> <andrii.nakryiko@gmail.com> wrote:
>>>>
>>>> On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>
>>>>> On 06/01/2026 01:19, Andrii Nakryiko wrote:
>>>>>> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
>>>>>> <alexei.starovoitov@gmail.com> wrote:
>>>>>>>
>>>>>>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
>>>>>>> <andrii.nakryiko@gmail.com> wrote:
>>>>>>>>
>>>>>>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>>>
>>>>>>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
>>>>>>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>>>>>>>>>>
>>>>>>>>>>> The sanitization we can get from layout info is for handling a kernel built with
>>>>>>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>>>>>>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
>>>>>>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>>>>>>>>>>> to be usable for some cases (where the type graph is not fatally compromised
>>>>>>>>>>> by the lack of a kind). This will always be somewhat limited, but it
>>>>>>>>>>> does provide more usability than we have today.
>>>>>>>>>>
>>>>>>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
>>>>>>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
>>>>>>>>>> only to
>>>>>>>>>
>>>>>>>>> Right; it's an extension of the sanitization concept from what it does today.
>>>>>>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
>>>>>>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
>>>>>>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
>>>>>>>>> latest kind info kernel has) to make that kernel BTF usable.
>>>>>>>>
>>>>>>>> Wait, is that really a goal? I get why Alexei is confused now :)
>>>>>>>>
>>>>>>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
>>>>>>>> for the sake of loading it into the kernel. If some user space tool is
>>>>>>>> trying to work with kernel BTF that has BTF features that tool doesn't
>>>>>>>> support, then we only have two reasonable options: a) tool just fails
>>>>>>>> to process that BTF altogether or b) the tool is smart enough to
>>>>>>>> utilize BTF layout information to know which BTF types it can safely
>>>>>>>> skip (that's where those flags I argue for would be useful). In both
>>>>>>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
>>>>>>>> layout info to construct a lookup table for btf__type_by_id(). And
>>>>>>>> libbpf doesn't need to do anything beyond that, IMO.
>>>>>>>>
>>>>>>>> We'll teach bpftool to dump as much of BTF as possible (I mean
>>>>>>>> `bpftool btf dump file`), so it's possible to get an idea of what part
>>>>>>>> of BTF is not supported and show those that we know about. We could
>>>>>>>> teach btf_dump to ignore those types that are "safe modifier-like
>>>>>>>> reference kind" (as marked with that flag I proposed earlier), so that
>>>>>>>> `format c` works as well (though I wouldn't recommend using such
>>>>>>>> output as a proper vmlinux.h, users should update bpftool ASAP for
>>>>>>>> such use cases).
>>>>>>>>
>>>>>>>> As far as the kernel is concerned, BTF layout is not used and should
>>>>>>>> not be used or trusted (it can be "spoofed" by the user). It can
>>>>>>>> validate it for sanity, but that's pretty much it. Other than that, if
>>>>>>>> the kernel doesn't *completely* understand every single piece of BTF,
>>>>>>>> it should reject it (and that's also why libbpf should sanitize BPF
>>>>>>>> object's BTF, of course).
>>>>>>>
>>>>>>> +1 to all of the above, except ok-to-skip flag, since I feel
>>>>>>> it will cause more bike sheding and arguing whether a particular
>>>>>>> new addition to BTF is skippable or not. Like upcoming location info.
>>>>>>
>>>>>> I was thinking about something like TYPE_TAG, where it's in the chain
>>>>>> of types and is unavoidable when processing STRUCT and its field.
>>>>>> Having a flag specifying that it's ref-like (so btf_type::type field
>>>>>> points to a valid type ID) would allow it to still make sense of the
>>>>>> entire struct and its fields, though you might be missing some
>>>>>> (presumably) optional and highly-specialized extra annotation.
>>>>>>
>>>>>> But it's fine not to add it, just some type graphs will be completely
>>>>>> unprocessable using old tools. Perhaps not such a big deal.
>>>>>>
>>>>>> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
>>>>>> whether they are more like DECL_TAG (roots pointing to other types) or
>>>>>> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
>>>>>> fields, PROTO args, etc).
>>>>>>
>>>>>>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
>>>>>>> Skippable? One might argue that they are while they are mandatory
>>>>>>> for some other use case.
>>>>>>> Looking at it differently, if the kernel is old and cannot understand that
>>>>>>> BTF feature the libbpf has to sanitize it no matter skippable or not.
>>>>>>> While from btf__parse() pov it also doesn't matter.
>>>>>>> btf_new()->btf_parse_hdr() will remember kind layout,
>>>>>>> and btf_parse_type_sec() can construct the index for the whole thing
>>>>>>> with layout info,
>>>>>>> while btf_validate_type() has to warn about unknown kind regardless
>>>>>>> of skippable flag. The tool (bpftool or else) needs to yell when
>>>>>>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
>>>>>>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
>>>>>>> though one might argue that they are.
>>>>>>
>>>>>> Yeah, I agree about vmlinux.h. One way to enforce this would be to
>>>>>> have btf_dump emit something uncompilable as
>>>>>> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
>>>>>> modified.
>>>>>>
>>>>>> But yeah, we don't want bikeshedding. It's fine.
>>>>>>
>>>>>
>>>>> Ok so is it best to leave out flags entirely then? If so where we
>>>>> are now is to have each kind layout entry have a string name offset,
>>>>> a singular element size and a vlen-specified object size. To be
>>>>> conservative it might make sense to allow 16 bits for each size field,
>>>>> leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
>>>>> We could cut down further by leaving out kind name strings if needed.
>>>>
>>>> Are we sure we will *never* need flags? I'd probably stick to
>>>> single-byte sizes and have 2 bytes reserved for flags or whatever we
>>>> might need in the future?
>>>
>>> Just to clarify what I was saying.
>>> I think it's a good thing to have flags space and reserve it.
>>> I just struggle to see the value of 'ok-to-skip' flag.
>>>
>>> So 2 bytes of reserved space for flags makes sense to me.
>>
>> Ok sounds good; I think there is still value in having the single flag
>> that tells us that the type/size field in struct btf_type refers
>> to a type though, right?
> 
> What's the value?
> 
> I sort-of kinda see small value of a set of flags like:
> - this kind is a type (int, ptr, array, struct, func_proto)
> - this kind is a modifier of a type (volatile, const, restrict, type_tag)
> 
> but then we cannot quite classify var, datasec, decl_tag, func..
> 
> So it feels like it's getting into the bikeshed category again.

I suppose one value would be it would be easier to recognize pure reference kinds
since they'd have such a flag set and have no extra info. But I think at this point
we should just focus on support parsing of unknown BTF kinds via kind layout. I'll add
an empty flags field for now.

Alan