Re: Question: missing vmlinux BTF variable declarations

[Alan Maguire <alan.maguire@oracle.com>]

On Tue, 3 May 2022, Stephen Brennan wrote:

> >> Ideally we structure BTFs as a multi level tree.  Where BTF with
> >> global vars and other non essential BTF info can be added to vmlinux
> >> BTF at run-time. BTF of kernel mods can add on top and mods can have
> >> split BTF too.
> 
> I see what you mean. It does sound a bit frustrating to have an
> additional BTF module to augment every external module, which would be
> the third level of that tree.
> 
> We'll need to allocate more module structs and pages within the kernel
> for that data, I wonder whether it would be cheaper for the
> "non-essential" module BTF to just reside in the same BTF section of
> that module.
> 
> I suppose I can run my modified pahole on some sample modules and see
> the BTF size difference, rather than just speculating, I'll do that in a
> follow-up here.
> 
> > Yeah, reuses existing mechanizm, doesn't increase the kernel BTF
> > footprint by default, allows for debuggers, profilers, tracers, etc to
> > ask for extra info in the form of just loading btf_global_variables.ko.
> 
> I agree, this is a quite elegant solution. Though it'll require a fair
> bit of work to achieve, I do think it's important to keep the footprint
> down. One thing I'd like to see in this world is a way to instruct the
> kernel that "I always want the non-essential BTF loaded", maybe via
> cmdline or sysctl. This way, the module loader would know to search for
> "$MODNAME-btf" for each module which doesn't end with "-btf".
>

Hmm, could we just have a tristate CONFIG_DEBUG_INFO_BTF_EXTRA?
If set to y, the extra vars are builtin to vmlinux BTF and
modules, and if set to m, they reside in /sys/kernel/btf/vmlinux-btf-extra
courtesy of the vmlinux-btf-extra.ko module (or whatever naming
scheme makes sense). Looks like pahole already has an option to
store encoded BTF elsewhere:

--btf_encode_detached=FILENAME

...so maybe all we need is something like --btf_gen_var_only for
the case where we build the btf-extra module

pahole -J --btf_base vmlinux  --btf_gen_var_only 
--btf_encode_detached=vmlinux_btf_extra

?

That's still only 2-way split BTF (base vmlinux BTF
plus vmlinux variables); we'd only need the
three-way split for the case where modules use the
-extra approach too, and I'd wonder about the viability
of having an -extra BTF module for each module, especially
if space-saving is the goal.

Alan