Re: [PATCH bpf-next] libbpf: support module BTF for BPF_TYPE_ID_TARGET CO-RE relocation

[Alan Maguire <alan.maguire@oracle.com>]

On Tue, 8 Dec 2020, Alexei Starovoitov wrote:

> On Tue, Dec 08, 2020 at 10:13:35PM +0000, Alan Maguire wrote:
> > 
> > Does this approach prevent more complex run-time specification of BTF 
> > object fd though?  For example, I've been working on a simple tracer 
> > focused on kernel debugging; it uses a BPF map entry for each kernel 
> > function that is traced. User-space populates the map entry with BTF type 
> > ids for the function arguments/return value, and when the BPF program 
> > runs it uses the instruction pointer to look up the map entry for that
> > function, and uses bpf_snprintf_btf() to write the string representations 
> > of the function arguments/return values.  I'll send out an RFC soon, 
> > but longer-term I was hoping to extend it to support module-specific 
> > types.  Would a dynamic case like that - where the BTF module fd is looked 
> > up in a map entry during program execution (rather than derived via 
> > __btf_builtin_type_id()) work too? Thanks!
> 
> fd has to be resolved in the process context. bpf prog can read fd
> number from the map, but that number is meaningless.
> Say we allow using btf_obj_id+btf_id, how user space will know these
> two numbers? Some new libbpf api that searches for it?
> An extension to libbpf_find_vmlinux_btf_id() ? I was hoping that this api
> will stay semi-internal. But say it's extended.
> The user space will store a pair of numbers into a map and
> what program are going to do with it?
> If it's printing struct veth_stats contents it should have attached to
> a corresponding function in the veth module via fentry or something.
> The prog has hard coded logic in C with specific pointer to print.
> The prog has its type right there. Why would the prog take a pointer
> from one place, but it's type_id from the map? That's not realistic.
> Where it would potentially make sense is what I think you're descring
> where single kprobe style prog attached to many places and args of
> those places are stored in a map and the prog selects them with
> map_lookup with key=PT_REGS_IP ?

Right, that's exactly it.  A pair of generic tracing BPF programs are
used, and they attach to kprobe/kretprobes, and when they run they use 
the arguments plus the map details about BTF ids of those arguments to 
run bpf_snprintf_btf(), and send perf events to userspace containing
the results.

> And passes pointers into bpf_snprintf_btf() from PT_REGS_PARM1() ?

Exactly.

> I see why that is useful, but it's so racy. By the time the map
> is populated those btf_obj_id+btf_id could be invalid.
> I think instead of doing this in user space the program needs an access
> to vmlinux+mods BTFs. Sort-of like proposed bpf helper to return ksym
> based on IP there could be a helper to figure out btf_id+btf_obj_POINTER
> based on IP. Then there will no need for external map to populate.
> Would that solve your use case?

That would be fantastic! We could do that from the context passed into a
kprobe program as the IP in struct pt_regs points at the function.  
kretprobes seems a bit trickier as in that case the IP in struct pt_regs 
is actually set to kretprobe_trampoline rather than the function we're
returning from due to how kretprobes work; maybe there's another way to 
get it in that case though..

Alan