From: David Vernet <void@manifault.com>
To: Stanislav Fomichev <sdf@google.com>
Cc: Magnus Karlsson <magnus.karlsson@gmail.com>,
magnus.karlsson@intel.com, bjorn@kernel.org, ast@kernel.org,
daniel@iogearbox.net, netdev@vger.kernel.org,
maciej.fijalkowski@intel.com, bpf@vger.kernel.org, yhs@fb.com,
andrii@kernel.org, martin.lau@linux.dev, song@kernel.org,
john.fastabend@gmail.com, kpsingh@kernel.org, haoluo@google.com,
jolsa@kernel.org, tirthendu.sarkar@intel.com,
jonathan.lemon@gmail.com
Subject: Re: [PATCH bpf-next v2 11/15] selftests/xsk: get rid of built-in XDP program
Date: Wed, 4 Jan 2023 13:13:59 -0600 [thread overview]
Message-ID: <Y7XP9924wTevHcBT@maniforge.lan> (raw)
In-Reply-To: <CAKH8qBt1HVcpxUMV0+gWN7eptr2+V899TRk39yYZwcoYMgkYCg@mail.gmail.com>
On Wed, Jan 04, 2023 at 10:19:37AM -0800, Stanislav Fomichev wrote:
> On Wed, Jan 4, 2023 at 10:14 AM David Vernet <void@manifault.com> wrote:
> >
> > On Wed, Jan 04, 2023 at 01:17:40PM +0100, Magnus Karlsson wrote:
> > > From: Magnus Karlsson <magnus.karlsson@intel.com>
> > >
> > > Get rid of the built-in XDP program that was part of the old libbpf
> > > code in xsk.c and replace it with an eBPF program build using the
> > > framework by all the other bpf selftests. This will form the base for
> > > adding more programs in later commits.
> > >
> > > Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
> > > ---
> > > tools/testing/selftests/bpf/Makefile | 2 +-
> > > .../selftests/bpf/progs/xsk_xdp_progs.c | 19 ++++
> > > tools/testing/selftests/bpf/xsk.c | 88 ++++---------------
> > > tools/testing/selftests/bpf/xsk.h | 6 +-
> > > tools/testing/selftests/bpf/xskxceiver.c | 72 ++++++++-------
> > > tools/testing/selftests/bpf/xskxceiver.h | 7 +-
> > > 6 files changed, 88 insertions(+), 106 deletions(-)
> > > create mode 100644 tools/testing/selftests/bpf/progs/xsk_xdp_progs.c
> > >
> > > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> > > index 205e8c3c346a..a0193a8f9da6 100644
> > > --- a/tools/testing/selftests/bpf/Makefile
> > > +++ b/tools/testing/selftests/bpf/Makefile
> > > @@ -240,7 +240,7 @@ $(OUTPUT)/flow_dissector_load: $(TESTING_HELPERS)
> > > $(OUTPUT)/test_maps: $(TESTING_HELPERS)
> > > $(OUTPUT)/test_verifier: $(TESTING_HELPERS) $(CAP_HELPERS)
> > > $(OUTPUT)/xsk.o: $(BPFOBJ)
> > > -$(OUTPUT)/xskxceiver: $(OUTPUT)/xsk.o
> > > +$(OUTPUT)/xskxceiver: $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_progs.skel.h
> >
> > Hi Magnus,
> >
> > This seems to break the selftests build for clang:
> >
> > $ pwd
> > <redacted>/bpf-next/tools/testing/selftests/bpf
> >
> > $ make LLVM=1 CC=clang
> > MKDIR libbpf
> > HOSTCC /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep.o
> > HOSTLD /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep-in.o
> > LINK /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep
> >
> > ...
> >
> > GEN-SKEL [test_progs-no_alu32] test_static_linked.skel.h
> > LINK-BPF [test_progs-no_alu32] test_usdt.bpf.o
> > GEN-SKEL [test_progs-no_alu32] linked_vars.skel.h
> > GEN-SKEL [test_progs-no_alu32] linked_funcs.skel.h
> > EXT-COPY [test_progs-no_alu32] urandom_read bpf_testmod.ko liburandom_read.so xdp_synproxy sign-file ima_setup.sh verify_sig_setup.sh btf_dump_test_case_bitfields.c btf_dump_test_case_multidim.c btf_dump_test_case_namespacing.c btf_dump_test_case_ordering.c btf_dump_test_case_packing.c btf_dump_test_case_padding.c btf_dump_test_case_syntax.c
> > GEN-SKEL [test_progs-no_alu32] linked_maps.skel.h
> > GEN-SKEL [test_progs-no_alu32] test_subskeleton.skel.h
> > BINARY xskxceiver
> > BINARY bench
> > GEN-SKEL [test_progs-no_alu32] test_subskeleton_lib.skel.h
> > GEN-SKEL [test_progs-no_alu32] test_usdt.skel.h
> > clang-15: error: cannot specify -o when generating multiple output files
> > make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xskxceiver] Error 1
> > make: *** Waiting for unfinished jobs....
> > make[1]: Nothing to be done for 'docs'.
> > $
> >
> > It's also broken on CI: https://github.com/kernel-patches/bpf/actions/runs/3837984934/jobs/6533917001
> >
> > Could you please look into this?
>
> Ugh, that's the same issue I'm getting for my xdp_hw_metadata binary.
> And I'm still at loss on how to reproduce it locally. (I've tried 'apt
> install clang-16 in ubuntu latest docker container' and it's still
> fine).
I was able to reproduce this issue locally:
[void@maniforge bpf]$ make -j LLVM=1 CC=clang
GEN-SKEL [test_progs] pyperf600.skel.h
GEN-SKEL [test_progs] test_verif_scale2.skel.h
LINK-BPF [test_progs] test_static_linked.bpf.o
LINK-BPF [test_progs] linked_funcs.bpf.o
...
LINK-BPF [test_progs-no_alu32] test_usdt.bpf.o
EXT-COPY [test_progs-no_alu32] urandom_read bpf_testmod.ko liburandom_read.so xdp_synproxy sign-file ima_setup.sh verify_sig_setup.sh btf_dump_test_case_bitfields.c btf_dump_test_case_multidim.c btf_dump_test_case_namespacing.c btf_dump_test_case_ordering.c btf_dump_test_case_packing.c btf_dump_test_case_padding.c btf_dump_test_case_syntax.c
GEN-SKEL [test_progs-no_alu32] linked_funcs.skel.h
BINARY bench
GEN-SKEL [test_progs-no_alu32] test_subskeleton.skel.h
BINARY xdp_hw_metadata
GEN-SKEL [test_progs-no_alu32] test_subskeleton_lib.skel.h
GEN-SKEL [test_progs-no_alu32] test_usdt.skel.h
clang-15: error: cannot specify -o when generating multiple output files
make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata] Error 1
make: *** Waiting for unfinished jobs....
make[1]: Nothing to be done for 'docs'.
Here's the actual clang command being executed:
[void@maniforge bpf]$ make LLVM=1 --dry-run xdp_hw_metadata
printf ' %-8s%s %s%s\n' "BINARY" "" "xdp_hw_metadata" "";
clang --target=x86_64-linux-gnu -fintegrated-as -g -O0 -rdynamic -Wall -Werror -DHAVE_GENHDR -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/include -I/home/void/upstream/bpf-next/include/generated -I/home/void/upstream/bpf-next/tools/lib -I/home/void/upstream/bpf-next/tools/include -I/home/void/upstream/bpf-next/tools/include/uapi -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf -Wno-unused-command-line-argument -static xdp_hw_metadata.c /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xsk.o /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata.skel.h /home/void/upstream/bpf-next/tools/testing/selftests/bpf/network_helpers.o -lelf -lz -lrt -lpthread -o /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata
and the output using --debug=j
[void@maniforge bpf]$ make LLVM=1 --debug=j xdp_hw_metadata
Putting child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693804 on the chain.
Live child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693804
BINARY xdp_hw_metadata
Reaping winning child 0x55cc78cd6670 PID 693804
Live child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693805
clang-15: error: cannot specify -o when generating multiple output files
Reaping losing child 0x55cc78cd6670 PID 693805
make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata] Error 1
Removing child 0x55cc78cd6670 PID 693805 from chain.
make is taking the xdp_hw_metadata.skel.h file and providing it as an
input to clang. So I believe what's going on here is that the clang
command above is actually creating two output files:
1. xdp_hw_metadata
2. The precompiled header generated from xdp_hw_metadata.skel.h
and the error is clang reasonably saying: "I don't know which output
file you're referring to with -o". I'm surprised that gcc doesn't
complain about this, but I assume that it's doing the far more
non-intuitive thing of first outputting the precompiled header as
xdp_hw_metadata, and then immediately overwriting it with the actual
xdp_hw_metadata binary.
> Any pointers on how to debug those github actions locally?
>
> > Thanks,
> > David
next prev parent reply other threads:[~2023-01-04 19:14 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-04 12:17 [PATCH bpf-next v2 00/15] selftests/xsk: speed-ups, fixes, and new XDP programs Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 01/15] selftests/xsk: print correct payload for packet dump Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 02/15] selftests/xsk: do not close unused file descriptors Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 03/15] selftests/xsk: submit correct number of frames in populate_fill_ring Magnus Karlsson
2023-01-09 23:01 ` Maciej Fijalkowski
2023-01-04 12:17 ` [PATCH bpf-next v2 04/15] selftests/xsk: print correct error codes when exiting Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 05/15] selftests/xsk: remove unused variable outstanding_tx Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 06/15] selftests/xsk: add debug option for creating netdevs Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 07/15] selftests/xsk: replace asm acquire/release implementations Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 08/15] selftests/xsk: remove namespaces Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 09/15] selftests/xsk: load and attach XDP program only once per mode Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 10/15] selftests/xsk: remove unnecessary code in control path Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 11/15] selftests/xsk: get rid of built-in XDP program Magnus Karlsson
2023-01-04 18:14 ` David Vernet
2023-01-04 18:19 ` Stanislav Fomichev
2023-01-04 19:13 ` David Vernet [this message]
2023-01-04 19:46 ` Stanislav Fomichev
2023-01-05 8:38 ` Magnus Karlsson
2023-01-04 23:15 ` Andrii Nakryiko
2023-01-04 23:44 ` Stanislav Fomichev
2023-01-11 19:20 ` Andrii Nakryiko
2023-01-04 12:17 ` [PATCH bpf-next v2 12/15] selftests/xsk: add test when some packets are XDP_DROPed Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 13/15] selftests/xsk: merge dual and single thread dispatchers Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 14/15] selftests/xsk: automatically restore packet stream Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 15/15] selftests/xsk: automatically switch XDP programs Magnus Karlsson
2023-01-10 11:56 ` Maciej Fijalkowski
2023-01-10 12:08 ` Magnus Karlsson
2023-01-10 12:00 ` [PATCH bpf-next v2 00/15] selftests/xsk: speed-ups, fixes, and new " Maciej Fijalkowski
2023-01-10 12:10 ` Magnus Karlsson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y7XP9924wTevHcBT@maniforge.lan \
--to=void@manifault.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bjorn@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=haoluo@google.com \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=jonathan.lemon@gmail.com \
--cc=kpsingh@kernel.org \
--cc=maciej.fijalkowski@intel.com \
--cc=magnus.karlsson@gmail.com \
--cc=magnus.karlsson@intel.com \
--cc=martin.lau@linux.dev \
--cc=netdev@vger.kernel.org \
--cc=sdf@google.com \
--cc=song@kernel.org \
--cc=tirthendu.sarkar@intel.com \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.