[PATCH bpf] tools/runqslower: Fix cross-build

[PATCH bpf] tools/runqslower: Fix cross-build

[Jean-Philippe Brucker]

Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
building") uses the target libbpf to build the host bpftool, which
doesn't work when cross-building:

  make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
  ...
    LINK    /tmp/runqslower/bpftool/bpftool
  /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
  /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
  collect2: error: ld returned 1 exit status

When cross-building, the target architecture differs from the host. The
bpftool used for building runqslower is executed on the host, and thus
must use a different libbpf than that used for runqslower itself.
Remove the LIBBPF_OUTPUT and LIBBPF_DESTDIR parameters, so the bpftool
build makes its own library if necessary.

In the selftests, pass the host bpftool, already a prerequisite for the
runqslower recipe, as BPFTOOL_OUTPUT. The runqslower Makefile will use
the bpftool that's already built for selftests instead of making a new
one.

Fixes: be79505caf3f ("tools/runqslower: Install libbpf headers when building")
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 tools/bpf/runqslower/Makefile        | 3 +--
 tools/testing/selftests/bpf/Makefile | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index bbd1150578f7..8791d0e2762b 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -88,5 +88,4 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
 
 $(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
 	$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)   \
-		    LIBBPF_OUTPUT=$(BPFOBJ_OUTPUT)			       \
-		    LIBBPF_DESTDIR=$(BPF_DESTDIR) CC=$(HOSTCC) LD=$(HOSTLD)
+		    CC=$(HOSTCC) LD=$(HOSTLD)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 54b0a41a3775..62fafbeb4672 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -187,7 +187,7 @@ DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
 $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
 	$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower	       \
 		    OUTPUT=$(RUNQSLOWER_OUTPUT) VMLINUX_BTF=$(VMLINUX_BTF)     \
-		    BPFTOOL_OUTPUT=$(BUILD_DIR)/bpftool/		       \
+		    BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/		       \
 		    BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf			       \
 		    BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) &&	       \
 		    cp $(RUNQSLOWER_OUTPUT)runqslower $@
-- 
2.33.1

Re: [PATCH bpf] tools/runqslower: Fix cross-build

[Quentin Monnet]

2021-11-12 15:51 UTC+0000 ~ Jean-Philippe Brucker <jean-philippe@linaro.org>
> Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
> building") uses the target libbpf to build the host bpftool, which
> doesn't work when cross-building:
> 
>   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
>   ...
>     LINK    /tmp/runqslower/bpftool/bpftool
>   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
>   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
>   collect2: error: ld returned 1 exit status
> 
> When cross-building, the target architecture differs from the host. The
> bpftool used for building runqslower is executed on the host, and thus
> must use a different libbpf than that used for runqslower itself.
> Remove the LIBBPF_OUTPUT and LIBBPF_DESTDIR parameters, so the bpftool
> build makes its own library if necessary.
> 
> In the selftests, pass the host bpftool, already a prerequisite for the
> runqslower recipe, as BPFTOOL_OUTPUT. The runqslower Makefile will use
> the bpftool that's already built for selftests instead of making a new
> one.
> 
> Fixes: be79505caf3f ("tools/runqslower: Install libbpf headers when building")
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Reviewed-by: Quentin Monnet <quentin@isovalent.com>

I realised too late I should have cc-ed you on those patches, apologies
for not doing so. Thank you for the fix!

Quentin

Re: [PATCH bpf] tools/runqslower: Fix cross-build

[Jean-Philippe Brucker]

On Fri, Nov 12, 2021 at 04:17:21PM +0000, Quentin Monnet wrote:
> 2021-11-12 15:51 UTC+0000 ~ Jean-Philippe Brucker <jean-philippe@linaro.org>
> > Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
> > building") uses the target libbpf to build the host bpftool, which
> > doesn't work when cross-building:
> > 
> >   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
> >   ...
> >     LINK    /tmp/runqslower/bpftool/bpftool
> >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
> >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
> >   collect2: error: ld returned 1 exit status
> > 
> > When cross-building, the target architecture differs from the host. The
> > bpftool used for building runqslower is executed on the host, and thus
> > must use a different libbpf than that used for runqslower itself.
> > Remove the LIBBPF_OUTPUT and LIBBPF_DESTDIR parameters, so the bpftool
> > build makes its own library if necessary.
> > 
> > In the selftests, pass the host bpftool, already a prerequisite for the
> > runqslower recipe, as BPFTOOL_OUTPUT. The runqslower Makefile will use
> > the bpftool that's already built for selftests instead of making a new
> > one.
> > 
> > Fixes: be79505caf3f ("tools/runqslower: Install libbpf headers when building")
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Reviewed-by: Quentin Monnet <quentin@isovalent.com>
> 
> I realised too late I should have cc-ed you on those patches, apologies
> for not doing so. Thank you for the fix!

No worries, I usually try to catch build issues in bpf-next but missed it
this time. I'm still slowly working towards getting automated testing on
Arm targets, which will catch regressions quicker.

Thanks,
Jean

Re: [PATCH bpf] tools/runqslower: Fix cross-build

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to bpf/bpf.git (master)
by Daniel Borkmann <daniel@iogearbox.net>:

On Fri, 12 Nov 2021 15:51:30 +0000 you wrote:
> Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
> building") uses the target libbpf to build the host bpftool, which
> doesn't work when cross-building:
> 
>   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
>   ...
>     LINK    /tmp/runqslower/bpftool/bpftool
>   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
>   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
>   collect2: error: ld returned 1 exit status
> 
> [...]

Here is the summary with links:
  - [bpf] tools/runqslower: Fix cross-build
    https://git.kernel.org/bpf/bpf/c/c9213bcf2fe1

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH bpf] tools/runqslower: Fix cross-build

[Andrii Nakryiko]

On Fri, Nov 12, 2021 at 8:28 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Fri, Nov 12, 2021 at 04:17:21PM +0000, Quentin Monnet wrote:
> > 2021-11-12 15:51 UTC+0000 ~ Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
> > > building") uses the target libbpf to build the host bpftool, which
> > > doesn't work when cross-building:
> > >
> > >   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
> > >   ...
> > >     LINK    /tmp/runqslower/bpftool/bpftool
> > >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
> > >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
> > >   collect2: error: ld returned 1 exit status
> > >
> > > When cross-building, the target architecture differs from the host. The
> > > bpftool used for building runqslower is executed on the host, and thus
> > > must use a different libbpf than that used for runqslower itself.
> > > Remove the LIBBPF_OUTPUT and LIBBPF_DESTDIR parameters, so the bpftool
> > > build makes its own library if necessary.
> > >
> > > In the selftests, pass the host bpftool, already a prerequisite for the
> > > runqslower recipe, as BPFTOOL_OUTPUT. The runqslower Makefile will use
> > > the bpftool that's already built for selftests instead of making a new
> > > one.
> > >
> > > Fixes: be79505caf3f ("tools/runqslower: Install libbpf headers when building")
> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > Reviewed-by: Quentin Monnet <quentin@isovalent.com>
> >
> > I realised too late I should have cc-ed you on those patches, apologies
> > for not doing so. Thank you for the fix!
>
> No worries, I usually try to catch build issues in bpf-next but missed it
> this time. I'm still slowly working towards getting automated testing on
> Arm targets, which will catch regressions quicker.

Are you planning to contribute it to BPF CI? Or you meant you have
your own separate system?

>
> Thanks,
> Jean

Re: [PATCH bpf] tools/runqslower: Fix cross-build

[Jean-Philippe Brucker]

On Mon, Nov 15, 2021 at 10:10:03PM -0800, Andrii Nakryiko wrote:
> On Fri, Nov 12, 2021 at 8:28 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > On Fri, Nov 12, 2021 at 04:17:21PM +0000, Quentin Monnet wrote:
> > > 2021-11-12 15:51 UTC+0000 ~ Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > > Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
> > > > building") uses the target libbpf to build the host bpftool, which
> > > > doesn't work when cross-building:
> > > >
> > > >   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
> > > >   ...
> > > >     LINK    /tmp/runqslower/bpftool/bpftool
> > > >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
> > > >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
> > > >   collect2: error: ld returned 1 exit status
> > > >
> > > > When cross-building, the target architecture differs from the host. The
> > > > bpftool used for building runqslower is executed on the host, and thus
> > > > must use a different libbpf than that used for runqslower itself.
> > > > Remove the LIBBPF_OUTPUT and LIBBPF_DESTDIR parameters, so the bpftool
> > > > build makes its own library if necessary.
> > > >
> > > > In the selftests, pass the host bpftool, already a prerequisite for the
> > > > runqslower recipe, as BPFTOOL_OUTPUT. The runqslower Makefile will use
> > > > the bpftool that's already built for selftests instead of making a new
> > > > one.
> > > >
> > > > Fixes: be79505caf3f ("tools/runqslower: Install libbpf headers when building")
> > > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > Reviewed-by: Quentin Monnet <quentin@isovalent.com>
> > >
> > > I realised too late I should have cc-ed you on those patches, apologies
> > > for not doing so. Thank you for the fix!
> >
> > No worries, I usually try to catch build issues in bpf-next but missed it
> > this time. I'm still slowly working towards getting automated testing on
> > Arm targets, which will catch regressions quicker.
> 
> Are you planning to contribute it to BPF CI? Or you meant you have
> your own separate system?

At the moment I'm looking at adding it to LKFT, which does regression
testing on various Arm boards (https://lkft.linaro.org/about/). The only
bit missing for that is support for cross-building tools with clang, which
I'll send shortly.

Thanks,
Jean

Re: [PATCH bpf] tools/runqslower: Fix cross-build

[Andrii Nakryiko]

On Tue, Nov 16, 2021 at 12:40 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Mon, Nov 15, 2021 at 10:10:03PM -0800, Andrii Nakryiko wrote:
> > On Fri, Nov 12, 2021 at 8:28 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > >
> > > On Fri, Nov 12, 2021 at 04:17:21PM +0000, Quentin Monnet wrote:
> > > > 2021-11-12 15:51 UTC+0000 ~ Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > > > Commit be79505caf3f ("tools/runqslower: Install libbpf headers when
> > > > > building") uses the target libbpf to build the host bpftool, which
> > > > > doesn't work when cross-building:
> > > > >
> > > > >   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/runqslower O=/tmp/runqslower
> > > > >   ...
> > > > >     LINK    /tmp/runqslower/bpftool/bpftool
> > > > >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a(libbpf-in.o): Relocations in generic ELF (EM: 183)
> > > > >   /usr/bin/ld: /tmp/runqslower/libbpf/libbpf.a: error adding symbols: file in wrong format
> > > > >   collect2: error: ld returned 1 exit status
> > > > >
> > > > > When cross-building, the target architecture differs from the host. The
> > > > > bpftool used for building runqslower is executed on the host, and thus
> > > > > must use a different libbpf than that used for runqslower itself.
> > > > > Remove the LIBBPF_OUTPUT and LIBBPF_DESTDIR parameters, so the bpftool
> > > > > build makes its own library if necessary.
> > > > >
> > > > > In the selftests, pass the host bpftool, already a prerequisite for the
> > > > > runqslower recipe, as BPFTOOL_OUTPUT. The runqslower Makefile will use
> > > > > the bpftool that's already built for selftests instead of making a new
> > > > > one.
> > > > >
> > > > > Fixes: be79505caf3f ("tools/runqslower: Install libbpf headers when building")
> > > > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > > Reviewed-by: Quentin Monnet <quentin@isovalent.com>
> > > >
> > > > I realised too late I should have cc-ed you on those patches, apologies
> > > > for not doing so. Thank you for the fix!
> > >
> > > No worries, I usually try to catch build issues in bpf-next but missed it
> > > this time. I'm still slowly working towards getting automated testing on
> > > Arm targets, which will catch regressions quicker.
> >
> > Are you planning to contribute it to BPF CI? Or you meant you have
> > your own separate system?
>
> At the moment I'm looking at adding it to LKFT, which does regression
> testing on various Arm boards (https://lkft.linaro.org/about/). The only
> bit missing for that is support for cross-building tools with clang, which
> I'll send shortly.

Detecting regressions is good, but preventing regressions is even
better. That's what we are doing with BPF CI. We create a Github PR
and run BPF selftests before patches are applied (see [0] for the
current list of patch sets being reviewed and thus tested). It would
be good to have more support for various architectures there. We
currently have x86-64 and s390x was added just recently. Regressions
still happen, obviously (we don't also control all the other parts of
the kernel so after tree merges things quite often are a bit unstable
and broken, but the system has caught many issues already), but it's
been a tremendous help and time saver. So consider contributing, if
you care about those architectures.

  [0] https://github.com/kernel-patches/bpf/pulls

>
> Thanks,
> Jean