Re: [PATCH bpf-next 1/2] bpf, x86: patch tail-call fentry slot on non-IBT JITs

[Leon Hwang <leon.hwang@linux.dev>]

On 31/3/26 00:46, Takeru Hayasaka wrote:
>> Sounds like you are developing/maintaining an XDP project.
>>
>> If so, and the kernel carries the patches in
>> https://lore.kernel.org/all/20230912150442.2009-1-hffilwlqm@gmail.com/,
>> recommend modifying the XDP project using dispatcher like libxdp [1].
>> Then, you are able to trace the subprogs which aim to run tail calls;
>> meanwhile, you are able to filter packets using pcap-filter, and to
>> output packets using bpf_xdp_output() helper.
>>
>> [1]
>> https://github.com/xdp-project/xdp-tools/blob/main/lib/libxdp/xdp-dispatcher.c.in
> 
> Thank you very much for your wonderful comment, Leon.
> This was the first time I learned that such a mechanism exists.
> 
> It is a very interesting ecosystem.
> If I understand correctly, the idea is to invoke a component that
> dumps pcap data as one of the tail-called components, right?

It is similar to xdp-ninja/xdp-dump.

However, this idea has one more step forward: it is to trace the
subprogs instead of only the main prog.

For example,

__noinline int subprog0(struct xdp_md *xdp) { bpf_tail_call_static(xdp,
&m, 0); }
__noinline int subprog1(struct xdp_md *xdp) { bpf_tail_call_static(xdp,
&m, 1); }
__noinline int subprog2(struct xdp_md *xdp) { bpf_tail_call_static(xdp,
&m, 2); }
SEC("xdp") int main(struct xdp_md *xdp)
{
	subprog0(xdp);
	subprog1(xdp);
	subprog2(xdp);
	return XDP_PASS;
}

All of them, subprog{0,1,2} and main, will be traced.

In this idea, it is to inject pcap-filter expression, the cbpf, using
elibpcap [1], and to output packets like your xdp-ninja.

It works well during the time I maintained an XDP project.

[1] https://github.com/jschwinger233/elibpcap

> Thank you very much for sharing this idea with me.
> If I have a chance to write a new XDP program in the future, I would
> definitely like to try it.
> 
> On the other hand, I feel that it is somewhat difficult to apply this
> idea directly to existing codebases, or to cases where the code is> written in Go using something like cilium/ebpf.
> Also, when it comes to code running in production environments, making
> changes itself can be difficult.

Correct. If cannot modify the code, and the tail calls are not called
inner subprogs, the aforementioned idea is helpless to trace the tail
callees.

> 
> For that reason, I prototyped a tool like this.
> It is something like a middle ground between xdpdump and xdpcap.
> I built it so that only packets matched by cbpf are sent up through
> perf, and while testing it, I noticed that it does not work well for
> targets invoked via tail call.
> This is what motivated me to send the patch.
> 

I have similar idea years ago, a more generic tracer for tail calls.
However, as Alexei's concern, I won't post it.

> https://github.com/takehaya/xdp-ninja
> 

It looks wonderful.

I developed a similar tool, bpfsnoop [1], to trace BPF progs/subprogs
and kernel functions with filtering packets/arguments and outputting
packets/arguments info. However, it lacks the ability of outputting
packets to pcap file.

[1] https://github.com/bpfsnoop/bpfsnoop

Thanks,
Leon

> Once again, thank you for sharing the idea.
> Takeru