Re: [PATCH bpf-next v3 0/8] Support bpf trampoline for s390x - CI issue

Ilya Leoshkevich
2023-01-31T13:37:03Z

On Mon, 2023-01-30 at 19:13 -0800, Alexei Starovoitov wrote:
> On Mon, Jan 30, 2023 at 10:56 AM Ilya Leoshkevich <iii@linux.ibm.com>
> wrote:
> >
> > On Sun, 2023-01-29 at 19:28 -0800, Alexei Starovoitov wrote:
> > > On Sun, Jan 29, 2023 at 11:05 AM Ilya Leoshkevich
> > > <iii@linux.ibm.com>
> > > wrote:
> > > >
> > > > v2:
> > > > https://lore.kernel.org/bpf/20230128000650.1516334-1-iii@linux.ibm.com/#t
> > > > v2 -> v3:
> > > > - Make __arch_prepare_bpf_trampoline static.
> > > >   (Reported-by: kernel test robot <lkp@intel.com>)
> > > > - Support both old- and new- style map definitions in
> > > > sk_assign.
> > > > (Alexei)
> > > > - Trim DENYLIST.s390x. (Alexei)
> > > > - Adjust s390x vmlinux path in vmtest.sh.
> > > > - Drop merged fixes.
> > >
> > > It looks great. Applied.
> > >
> > > Sadly clang repo is unreliable today. I've kicked BPF CI multiple
> > > times,
> > > but it didn't manage to fetch the clang. Pushed anyway.
> > > Pls watch for BPF CI failures in future runs.
> >
> > I think this is because llvm-toolchain-focal contains llvm 17 now.
> > So we need to either use llvm-toolchain-focal-16, or set
> > llvm_default_version=16 in libbpf/ci.
>
> Yep. That was fixed.
> Looks like only one test is failing on s390:
> test_synproxy:PASS:./xdp_synproxy --iface tmp1 --single 0 nsec
> expect_str:FAIL:SYNACKs after connection unexpected SYNACKs after
> connection: actual '' != expected 'Total SYNACKs generated: 1\x0A'
>
> #284/1 xdp_synproxy/xdp:FAIL
> #284 xdp_synproxy:FAIL
> Summary: 260/1530 PASSED, 31 SKIPPED, 1 FAILED

Thanks! Where do you see the xdp_synproxy failure? I checked the jobs
at [1] and rather see two migrate_reuseport failures ([2], [3]):

count_requests:FAIL:count in BPF prog unexpected count in BPF prog:
actual 10 != expected 25
#127/7 migrate_reuseport/IPv6 TCP_NEW_SYN_RECV
reqsk_timer_handler:FAIL

count_requests:FAIL:count in BPF prog unexpected count in BPF prog:
actual 14 != expected 25
#127/4 migrate_reuseport/IPv4 TCP_NEW_SYN_RECV
inet_csk_complete_hashdance:FAIL

I tried running vmtest.sh in a loop, and could not reproduce neither
the xdp_synproxy nor the migrate_reuseport failure.

In migrate_reuseport, from the userspace perspective everything works,
(count_requests:PASS:count in userspace 0 nsec). This means that we
always get to the bpf_sk_select_reuseport() call and it succeeds.
The eBPF program still records at least some migrations while the
connection is in the TCP_NEW_SYN_RECV state, so I wonder if other
migrations, for whatever reason, happen in a different state?

[1] https://github.com/libbpf/libbpf/actions/workflows/test.yml
[2]
https://github.com/libbpf/libbpf/actions/runs/4049227053/jobs/6965361085#step:30:8908
[3]