Re: Dynamic kfunc discovery

[Jiri Olsa <olsajiri@gmail.com>]

On Wed, Dec 20, 2023 at 09:44:10AM -0700, Daniel Xu wrote:
> Hi Jiri,
> 
> On Wed, Dec 20, 2023 at 10:07:34AM +0100, Jiri Olsa wrote:
> > On Tue, Dec 19, 2023 at 07:15:42PM -0800, Alexei Starovoitov wrote:
> > > On Tue, Dec 19, 2023 at 9:29 AM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I was chatting w/ Quentin [0] about how bpftool could:
> > > >
> > > > 1. Support a "feature dump" of all supported kfuncs on running kernel
> > > > 2. Generate vmlinux.h with kfunc prototypes
> > > >
> > > > I had another idea this morning so I thought I'd bounce it around
> > > > on the list in case others had better ones. 3 vague ideas:
> > > >
> > > > 1. Add a BTF type tag annotation in __bpf_kfunc macro. This would
> > > >    let bpftool parse BTF to do discovery. It would be fairly clean and
> > > >    straightforward, except that I don't think GCC supports these type
> > > >    tags. So only clang-built-linux would work.
> > > >
> > > > 2. Do the same thing as above, except rather than tagging src code,
> > > >    teach pahole about the .BTF_ids section in vmlinux. pahole could then
> > > >    construct BTF with the appropriate type tags.
> > 
> > I thought it'd be nice to have this in BTF, but to generate the .BTF_ids
> > section we need the BTF data (for BTF IDs), so that might be tricky
> 
> Isn't .BTF_ids already present in vmlinux before getting to
> resolve_btfids? It looks to me like all resolve_btfids does is patch
> symbols to the read BTF ID values.

yes, it's there but it's empty.. IDs are zero

> To inject BTF type tags from pahole, I don't think it needs a patched
> .BTF_ids section, right? After pahole has generated all the regular
> entries, it could walk .BTF_ids and try to match up symbol names with
> BTF function entries. And then inject the BTF type tag.

so what resolve_btfids does is to lookup all __BTF_ID__set8__* symbols,
finds their BTF IDs and stores them where the symbol points

there's explanation on the symbol name in tools/bpf/resolve_btfids/main.c
header

pahole could do the same and once it has the IDs it could add the type
tag to them, initially I thought having extra BTF section with kfuncs
BTF ids, but type tag seems like better way to do that

jirka

> 
> > 
> > > 
> > > resolve_btfids knows about all of them already.
> > > The best is to teach bpftool about them as well.
> > > It can look for BTF_SET8_START and there it can find btf_ids
> > 
> > with the access to vmlinux, bpftool could get the addresses of all
> > set8s, read all btf ids and generate the header
> > 
> > $ nm vmlinux | grep BTF_ID__set8 
> > ffffffff843bf044 r __BTF_ID__set8__bpf_kfunc_check_set_skb
> > ffffffff843bf064 r __BTF_ID__set8__bpf_kfunc_check_set_sock_addr
> > ffffffff843bf054 r __BTF_ID__set8__bpf_kfunc_check_set_xdp
> > ffffffff843be940 r __BTF_ID__set8__bpf_map_iter_kfunc_ids
> > ffffffff843bf22c r __BTF_ID__set8__bpf_mptcp_fmodret_ids
> > ffffffff843be604 r __BTF_ID__set8__bpf_rstat_kfunc_ids
> > ffffffff843bf074 r __BTF_ID__set8__bpf_sk_iter_kfunc_ids
> > ffffffff843bf1c4 r __BTF_ID__set8__bpf_tcp_ca_check_kfunc_ids
> > ffffffff843bf0bc r __BTF_ID__set8__bpf_test_modify_return_ids
> > ffffffff843be864 r __BTF_ID__set8__common_btf_ids
> > ffffffff843be9a8 r __BTF_ID__set8__cpumask_kfunc_btf_ids
> > ffffffff843bf174 r __BTF_ID__set8__fou_kfunc_set
> > ffffffff843be678 r __BTF_ID__set8__fs_kfunc_set_ids
> > ffffffff843be794 r __BTF_ID__set8__generic_btf_ids
> > ffffffff843be650 r __BTF_ID__set8__key_sig_kfunc_set
> > ffffffff843bf10c r __BTF_ID__set8__nf_ct_kfunc_set
> > ffffffff843bf164 r __BTF_ID__set8__nf_nat_kfunc_set
> > ffffffff843bf18c r __BTF_ID__set8__tcp_cubic_check_kfunc_ids
> > ffffffff843bf0dc r __BTF_ID__set8__test_sk_check_kfunc_ids
> > ffffffff843bf084 r __BTF_ID__set8__xdp_metadata_kfunc_ids
> > ffffffff843bf1f4 r __BTF_ID__set8__xfrm_ifc_kfunc_set
> > ffffffff843bf20c r __BTF_ID__set8__xfrm_state_kfunc_set
> > 
> > jirka
> > 
> > > of all kfuncs.
> > > From there it can generate them into vmlinux.h
> > > 
> > > We wanted kfuncs to appear in vmlinux.h for quite some time,
> > > but no one had cycles to do it.
> > > Still an awesome feature to have.
> > > 
> 
> Thanks,
> Daniel