Re: Replicating kfunc_call_test kernel test on standalone bpf program (calling kernel function is not allowed)

[Jiri Olsa <olsajiri@gmail.com>]

On Fri, Sep 30, 2022 at 05:03:30PM -0500, Henrique Fingler wrote:
> > > > Hi all,
> > > >
> > > > I'm trying to replicate a bpf test in the kernel that calls a function
> > > > defined in the kernel itself.
> > > > Source code is here:
> > > > https://github.com/torvalds/linux/blob/v5.15/tools/testing/selftests/bpf/progs/kfunc_call_test.c
> > > >
> > > > I think I have all dependencies:
> > > >  Running within a qemu VM (Ubuntu 18.04)
> > > >  Kernel v 5.15 compiled from scratch with configs from
> > > > tools/bpf/bpftool/feature.c
> > > >  pahole v1.22 (1.24 has a reported bug that doesn't allow me to use it)
> > > >  libbpf v1.0
> > > >  Installed bpf tool from 5.15 kernel directory at `tools/bpf`
> > > >  clang and llvm 15
> > > >
> > > > The goal is to call `bpf_kfunc_call_test1`, which is defined in
> > > > net/bpf/test_run.c.
> > > > I have two BPF programs and neither works. The first one is as is from
> > > > the kernel:
> > > >
> > > > #include "vmlinux.h"
> > > > #include <bpf/bpf_helpers.h>
> > > >
> > > > extern __u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b,
> > > >                   __u32 c, __u64 d) __ksym;
> > > >
> > > > SEC("classifier")
> > > > int kfunc_call_test1(struct __sk_buff *skb)
> > > > {
> > > >     struct sock *sk = 0;
> > > >     __u64 a;
> > > >     a = bpf_kfunc_call_test1(sk, 1, 2, 3, 4);
> > >
> > > hi,
> > > IIUC you are passing 'sk' pointer defined on the stack, while
> > > bpf_kfunc_call_test1 expects kernel pointer
> > >
> > > the kernel selftest test takes it from the skb with:
> > >
> > >         struct bpf_sock *sk = skb->sk;
> >
> > I see. So even if the kernel function (bpf_kfunc_call_test1) does not
> > use the argument, bpf is checking if it's a kernel pointer? Is the bpf
> > compiler doing this check?

it's the verifier check.. that the function is called with proper
argument types/pointers

> > I assumed that passing a constant 0 pointer would work since the other
> > parameters are just constants, even in the kernel test.
> > After changing the first program to the original code, the error
> > changed, so that's progress. Now it says, even when running with root
> > permissions:
> > "libbpf: prog 'kfunc_call_test1': BPF program load failed: Permission denied"
> > Would be interesting to know why, but not necessary since I won't use
> > it this way.

some of the verifier's failure can return -EACCES and has nothing
to do with root permissions

> 
> Following up on this, I have moved to kernel 5.19.12 and I'm trying to
> make any kfunc work.
> I changed net/bpf/test_run.c to allow for more prog types, like master
> branch does, since originally it only had the first line for
> BPF_PROG_TYPE_SCHED_CLS.
> 
> ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS,
> &bpf_prog_test_kfunc_set);
> ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING,
> &bpf_prog_test_kfunc_set);
> ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_KPROBE,
> &bpf_prog_test_kfunc_set);
> ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACEPOINT,
> &bpf_prog_test_kfunc_set);
> return ret ?: register_btf_id_dtor_kfuncs(bpf_prog_test_dtor_kfunc,
>                           ARRAY_SIZE(bpf_prog_test_dtor_kfunc),
>                           THIS_MODULE);
> 
> I also added my own function to test it out and added it to the SET
> 
> u64 noinline bpf_kfunc_call_test4(u32 a, u64 b, u32 c, u64 d)
> {
>     return a + b + c + d;
> }
> //this is inside  BTF_SET_START(test_sk_check_kfunc_ids)
> BTF_ID(func, bpf_kfunc_call_test4)
> 
> Now I'm spraying every BPF program I can find to call either function:
> 
> extern __u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b,
> __u32 c, __u64 d) __ksym;
> extern __u64 bpf_kfunc_call_test4(__u32 a, __u64 b, __u32 c, __u64 d) __ksym;
> 
> Compiling works, and when I run it I get no errors, *except* Permission denied.
> gist here: https://gist.github.com/hfingler/5c2c0b713299daa6b0ba07fa92ff29de
> 
> libbpf: prog 'kfunc_call_test1': BPF program load failed: Permission denied

seems like the kfunc is not found on the set for the program type, I guess
your change above did not go as expected

not sure what is your goal exactly, but perhaps better than starting from
scratch would be to take prog_tests/kfunc_call.c and progs/kfunc_call_test.c
and change them accordingly?

jirka

> ...
> calling kernel function bpf_kfunc_call_test1 is not allowed
> -- END PROG LOAD LOG --
> libbpf: prog 'kfunc_call_test1': failed to load: -13
> libbpf: failed to load object 'hello_bpf'
> libbpf: failed to load BPF skeleton 'hello_bpf': -13
> 
> I've tried a few programs and it would be too much to paste here, so
> here's a gist with all of them, which I tried each separately, not all
> at once:
> https://gist.github.com/hfingler/eb544b23cc36d57b8e9723cd36fbf243
> Basically, I've tried SEC("tc"),
> SEC("tracepoint/syscalls/sys_enter_open"),
> SEC("kprobe/__x64_sys_write")
> 
> What permission is being denied? I've tried running as root and I get
> the same thing, is there bpf permission checking somewhere else in the
> kernel? Do I have to have some sort of capability? Am I missing some
> kernel config?
> These are the configs I'm enabling in the kernel:
> https://gist.github.com/hfingler/ed780bd52b751625f52bbb08eb853641