Is fentry/fexit support possible with an external BTF?

Is fentry/fexit support possible with an external BTF?

[Jason Ling]

Some context:

The devices I am interested in have a small kernel partition (~16MB).
Building the kernel with CONFIG_DEBUG_INFO_BTF=y increases the kernel
size by about ~1.7M.
So I've tried to use an external BTF (generating my own vmlinux BTF
and placing it on a more spacious partition) but it seems like my
eBPFs that attach to fexit hooks now fail loading. According to some
comments in libbpf this seems to be expected.

e.g an eBPF program that looks like this

SEC("fexit/ksys_unshare")
int BPF_PROG(handle_exit, unsigned long unshare_flags, int rv) {
}

fails the loading process.


My guess is that there is additional debug/BTF information beyond what
is available in vmlinux BTF that gets linked into vmlinuz and without
this information the attaching to certain hooks fail.

So my question is:

is there a way to achieve my goal of using a kernel that has been
built with CONFIG_DEBUG_INFO_BTF=n and still be able to use
fentry/fexit type hooks?

Re: Is fentry/fexit support possible with an external BTF?

[Kui-Feng Lee]

On 1/23/23 16:21, Jason Ling wrote:
> Some context:
>
> The devices I am interested in have a small kernel partition (~16MB).
> Building the kernel with CONFIG_DEBUG_INFO_BTF=y increases the kernel
> size by about ~1.7M.
> So I've tried to use an external BTF (generating my own vmlinux BTF
> and placing it on a more spacious partition) but it seems like my
> eBPFs that attach to fexit hooks now fail loading. According to some
> comments in libbpf this seems to be expected.
>
> e.g an eBPF program that looks like this
>
> SEC("fexit/ksys_unshare")
> int BPF_PROG(handle_exit, unsigned long unshare_flags, int rv) {
> }
>
> fails the loading process.
>
>
> My guess is that there is additional debug/BTF information beyond what
> is available in vmlinux BTF that gets linked into vmlinuz and without
> this information the attaching to certain hooks fail.

IIRC, the verifier running in the kernel also needs BTF to verify the code.

That means you need a way to load BTF to kernel manually as well.


>
> So my question is:
>
> is there a way to achieve my goal of using a kernel that has been
> built with CONFIG_DEBUG_INFO_BTF=n and still be able to use
> fentry/fexit type hooks?

Re: Is fentry/fexit support possible with an external BTF?

[Alan Maguire]

On 24/01/2023 18:26, Kui-Feng Lee wrote:
> On 1/23/23 16:21, Jason Ling wrote:
>> Some context:
>>
>> The devices I am interested in have a small kernel partition (~16MB).
>> Building the kernel with CONFIG_DEBUG_INFO_BTF=y increases the kernel
>> size by about ~1.7M.

Would it help in this environment if vmlinux BTF was in a module?
I've been exploring supporting CONFIG_DEBUG_INFO_BTF=m which
would involve having a vmlinux_btf module that could be loaded
on demand.

Alan  

>> So I've tried to use an external BTF (generating my own vmlinux BTF
>> and placing it on a more spacious partition) but it seems like my
>> eBPFs that attach to fexit hooks now fail loading. According to some
>> comments in libbpf this seems to be expected.
>>
>> e.g an eBPF program that looks like this
>>
>> SEC("fexit/ksys_unshare")
>> int BPF_PROG(handle_exit, unsigned long unshare_flags, int rv) {
>> }
>>
>> fails the loading process.
>>
>>
>> My guess is that there is additional debug/BTF information beyond what
>> is available in vmlinux BTF that gets linked into vmlinuz and without
>> this information the attaching to certain hooks fail.
> 
> IIRC, the verifier running in the kernel also needs BTF to verify the code.
> 
> That means you need a way to load BTF to kernel manually as well.
> 
> 
>>
>> So my question is:
>>
>> is there a way to achieve my goal of using a kernel that has been
>> built with CONFIG_DEBUG_INFO_BTF=n and still be able to use
>> fentry/fexit type hooks?

Re: Is fentry/fexit support possible with an external BTF?

[Jason Ling]

>Would it help in this environment if vmlinux BTF was in a module?

Yes, that would fix our issues as we place our modules on a separate partition.