Re: Assistance needed with TCP BBR to BPF Conversion Issue

[Martin KaFai Lau <martin.lau@linux.dev>]

On 7/26/24 2:15 PM, Mingrui Zhang wrote:
> Dear BPF community,
> 
> I am a student currently trying to use the BPF interface in the TCP congestion control study for faster Linux system integration without compiling the entire kernel.
> 
> I've encountered a challenge while attempting to convert TCP BBR to BPF format and would greatly appreciate your guidance.
> 
> My modifications to the original tcp_bbr is as follow:
> * change u8,u32,u64,etc to __u8, __u32, __u64, etc.
> * Defined external kernel functions
> * Removed compiler flags using macro (e.g., "unlikely", "READ_ONCE")
> * Borrowed some time definitions from bpf_cubic (e.g., HZ and JIFFY)
> * Defined constant values not included in vmlinux.h (e.g., "TCP_INFINITE_SSTHRESH")
> * Implemented do_div() and cmpxchg() from assembly to C
> * Changed min_t() macro to min()
> This is the link to my modified tcp_bbr file https://github.com/zmrui/bbr-bpf/blob/main/tcp_bbr.c
> 
> I use "clang -O2 -target bpf -c -g bpf_cubic.c" command to compile and it doesn't output any warning or error,
> and the "sudo bpftool struct_ops register tcp_bbr.o" command does not have any output
> 
> Then the "bpftool -debug" option displays the following debug message at the last line:
> "libbpf: sec '.rodata': failed to determine size from ELF: size 0, err -2"

Good to see works in trying tcp_bbr.c with struct_ops.

It is likely the .o is invalid. Are you sure the program was compiled successfully?

 From looking at the following lines, the kernel you are using is not the 
upstream kernel.

extern unsigned int tcp_left_out(const struct tcp_sock *tp) __ksym;
extern unsigned int tcp_packets_in_flight(const struct tcp_sock *tp) __ksym;
extern __u32 tcp_stamp_us_delta(__u64 t1, __u64 t0) __ksym;
extern __u32 get_random_u32_below(__u32 ceil) __ksym;
extern __u32 tcp_min_rtt(const struct tcp_sock *tp) __ksym;
extern unsigned long msecs_to_jiffies(const unsigned int m)  __ksym;
extern __u32 tcp_snd_cwnd(const struct tcp_sock *tp) __ksym;
extern void tcp_snd_cwnd_set(struct tcp_sock *tp, __u32 val) __ksym;
extern __u32 minmax_running_max(struct minmax *m, __u32 win, __u32 t, __u32 
meas) __ksym;
extern __u32 minmax_reset(struct minmax *m, u32 t, u32 meas) __ksym;
extern  __u32 minmax_get(const struct minmax *m) __ksym;

They are not kfunc in the upstream kernel. Most of them don't have to be kfunc. 
Try to implement them in the bpf program itself (i.e. the tcp_bbr.c in your 
github link).

It is hard for the community to help without something reproducible in the 
upstream kernel. Lets target for getting tcp_bbr.c compiled in the selftests 
first (under tools/testing/selftests/bpf/progs like the bpf_cubic.c) and post 
the patch to the mailing list. bpf_devel_QA.rst has some guides.

> Additionally, the new algorithm doesn't appear in "net.ipv4.tcp_available_congestion_control" or in "bpftool struct_ops list".
> 
> I did not find much related content for this debug error message on the Internet.
> I would be very grateful for any suggestions or insights you might have regarding this issue.
> Thank you in advance for your time and expertise.
> 
> For context, here's my system information:
> Ubuntu 22.04
> 6.5.0-41-generic
> $ bpftool -V
> 	bpftool v7.3.0
> 	using libbpf v1.3
> 	features: llvm, skeletons
> -$ clang -v
> 	-Ubuntu clang version 14.0.0-1ubuntu1.1
> 
> Best,
> Mingrui
>