Re: one prog multi fentry. Was: [PATCH bpf-next] libbpf: support module BTF for BPF_TYPE_ID_TARGET CO-RE relocation

[Alexei Starovoitov <alexei.starovoitov@gmail.com>]

On Wed, Dec 16, 2020 at 02:27:23PM -0800, Andrii Nakryiko wrote:
> 
> But this seems more "verifiable" and nicer to use, even though it
> won't substituting an arbitrary btf_id and btf_obj (but that's sort of
> a goal, I think):
> 
> skb = bpf_get_btf_arg(ctx, 1, bpf_core_type_id_kernel(skb));

yep. makes sense to me.
Assuming that ctx has both:
- BTF of the func and the helper will follow to arg's BTF at run-time
  to check that it matches 3rd arg btf_id.
- and the actual arg values as well. So that helper will return them.

> > - default mode where we trace function arguments for kprobe and return value
> >   for kretprobe; that's covered by the above; and
> > - a mode where the user specifies what they want. For example running
> >
> > $ ksnoop "ip_send_skb"
> >
> > ...is an example of default mode, this will trace entry/return and print
> > arguments and return values, while
> >
> > $ ksnoop "ip_send_skb(skb)"
> >
> > ...will trace the skb argument only, and
> >
> > $ ksnoop "ip_send_skb(skb->sk)"
> >
> > ...will trace the skb->sk value.  The user-space side of the program
> > matches the function/arg name and looks up the referenced type, setting it
> > in the function's map.  For field references such as skb->sk, it also
> > records offset and whether that offset is a pointer (as is the case for
> > skb->sk) - in such cases we need to read the offset value via bpf_probe_read()
> > and use it in bpf_snprintf_btf() along with the referenced type.  Only a
> > single simple reference like the above is supported currently, but
> > multiple levels of reference could be made to work too.

Alan,

I'm not sure why the last example is so different form the first two.
I think ksnoop tool will generate the program on the fly, right?
So it can generate normal LDX insn with CO-RE relocation (instead of bpf_probe_read)
to access skb->sk. It can also add relo for that LDX to point to
struct sk_buff's btf_id defined inside prog's BTF.
The 'sk' offset inside bpf program and inside BTF can be anything: 0, 4, ...
libbpf relocation logic will find the right offset in kernel's sk_buff.
If ksnoop doesn't have an ability to parse vmlinux.h file or kernel's BTF
it can 'cheat'.
If the cmdline looks like:
$ ksnoop "ip_send_skb(skb->sk)"
It can generate BTF:
struct sk_buff {
   struct sock *sk;
};

If cmdline looks like:
$ ksnoop "ip_send_skb(skb->sock)"
It can generate BTF:
struct sk_buff {
   struct sock *sock;
};
Obviously there is no 'sock' field inside kernel's struct sk_buff, but tool
doesn't need to care. It can let libbpf do the checking and match
fields properly.

> > into that a bit more if you don't mind because I think some form of
> > user-space-specified BTF ids may be the easiest approach for more flexible
> > generic tracing that covers more than function arguments.

I think you're trying to figure out kernel's btf_ids in ksnoop tool.
I suggest to leave that job to libbpf. Generate local BTFs in ksnoop
with CO-RE relocs and let libbpf handle insn patching.
No FDs to worry about from ksnoop side either.