[PATCH bpf-next v2 0/3] Use lightweigt version of bpftool

[PATCH bpf-next v2 0/3] Use lightweigt version of bpftool

[Pu Lehui]

Currently, samples/bpf, tools/runqslower and bpf/iterators use bpftool
for vmlinux.h, skeleton, and static linking only. We can uselightweight
bootstrap version of bpftool to handle these, and it will be faster.

v2:
- make libbpf and bootstrap bpftool independent. and make it simple.

v1: https://lore.kernel.org/bpf/20220712030813.865410-1-pulehui@huawei.com

Pu Lehui (3):
  samples: bpf: Fix cross-compiling error by using bootstrap bpftool
  tools: runqslower: build and use lightweight bootstrap version of
    bpftool
  bpf: iterators: build and use lightweight bootstrap version of bpftool

 kernel/bpf/preload/iterators/Makefile | 10 +++-------
 samples/bpf/Makefile                  | 10 ++++------
 tools/bpf/runqslower/Makefile         |  7 +++----
 3 files changed, 10 insertions(+), 17 deletions(-)

-- 
2.25.1

[PATCH bpf-next v2 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool

[Pu Lehui]

Currently, when cross compiling bpf samples, the host side cannot
use arch-specific bpftool to generate vmlinux.h or skeleton. Since
samples/bpf use bpftool for vmlinux.h, skeleton, and static linking
only, we can use lightweight bootstrap version of bpftool to handle
these, and it's always host-native.

Signed-off-by: Pu Lehui <pulehui@huawei.com>
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
---
 samples/bpf/Makefile | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 5002a5b9a7da..727da3c5879b 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -282,12 +282,10 @@ $(LIBBPF): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
 
 BPFTOOLDIR := $(TOOLS_PATH)/bpf/bpftool
 BPFTOOL_OUTPUT := $(abspath $(BPF_SAMPLES_PATH))/bpftool
-BPFTOOL := $(BPFTOOL_OUTPUT)/bpftool
-$(BPFTOOL): $(LIBBPF) $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
-	    $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
-		OUTPUT=$(BPFTOOL_OUTPUT)/ \
-		LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
-		LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/
+BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
+$(BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
+	$(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ 		\
+		OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
 
 $(LIBBPF_OUTPUT) $(BPFTOOL_OUTPUT):
 	$(call msg,MKDIR,$@)
-- 
2.25.1

[PATCH bpf-next v2 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool

[Pu Lehui]

tools/runqslower use bpftool for vmlinux.h, skeleton, and static linking
only. So we can use lightweight bootstrap version of bpftool to handle
these, and it will be faster.

Signed-off-by: Pu Lehui <pulehui@huawei.com>
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
---
 tools/bpf/runqslower/Makefile | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index da6de16a3dfb..8b3d87b82b7a 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -4,7 +4,7 @@ include ../../scripts/Makefile.include
 OUTPUT ?= $(abspath .output)/
 
 BPFTOOL_OUTPUT := $(OUTPUT)bpftool/
-DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bpftool
+DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bootstrap/bpftool
 BPFTOOL ?= $(DEFAULT_BPFTOOL)
 LIBBPF_SRC := $(abspath ../../lib/bpf)
 BPFOBJ_OUTPUT := $(OUTPUT)libbpf/
@@ -86,6 +86,5 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
 	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) \
 		    DESTDIR=$(BPFOBJ_OUTPUT) prefix= $(abspath $@) install_headers
 
-$(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
-	$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)   \
-		    ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)
+$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
+	$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) bootstrap
-- 
2.25.1

[PATCH bpf-next v2 3/3] bpf: iterators: build and use lightweight bootstrap version of bpftool

[Pu Lehui]

kernel/bpf/preload/iterators use bpftool for vmlinux.h, skeleton, and
static linking only. So we can use lightweight bootstrap version of
bpftool to handle these, and it will be faster.

Signed-off-by: Pu Lehui <pulehui@huawei.com>
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
---
 kernel/bpf/preload/iterators/Makefile | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/kernel/bpf/preload/iterators/Makefile b/kernel/bpf/preload/iterators/Makefile
index bfe24f8c5a20..6762b1260f2f 100644
--- a/kernel/bpf/preload/iterators/Makefile
+++ b/kernel/bpf/preload/iterators/Makefile
@@ -9,7 +9,7 @@ LLVM_STRIP ?= llvm-strip
 TOOLS_PATH := $(abspath ../../../../tools)
 BPFTOOL_SRC := $(TOOLS_PATH)/bpf/bpftool
 BPFTOOL_OUTPUT := $(abs_out)/bpftool
-DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
+DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
 BPFTOOL ?= $(DEFAULT_BPFTOOL)
 
 LIBBPF_SRC := $(TOOLS_PATH)/lib/bpf
@@ -61,9 +61,5 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
 		    OUTPUT=$(abspath $(dir $@))/ prefix=		       \
 		    DESTDIR=$(LIBBPF_DESTDIR) $(abspath $@) install_headers
 
-$(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
-	$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC)			       \
-		    OUTPUT=$(BPFTOOL_OUTPUT)/				       \
-		    LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/			       \
-		    LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/			       \
-		    prefix= DESTDIR=$(abs_out)/ install-bin
+$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
+	$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC)	OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
-- 
2.25.1

Re: [PATCH bpf-next v2 0/3] Use lightweigt version of bpftool

[Andrii Nakryiko]

On Wed, Jul 13, 2022 at 7:16 PM Pu Lehui <pulehui@huawei.com> wrote:
>
> Currently, samples/bpf, tools/runqslower and bpf/iterators use bpftool
> for vmlinux.h, skeleton, and static linking only. We can uselightweight
> bootstrap version of bpftool to handle these, and it will be faster.
>
> v2:
> - make libbpf and bootstrap bpftool independent. and make it simple.
>

Quentin, does this patch set look good to you?

> v1: https://lore.kernel.org/bpf/20220712030813.865410-1-pulehui@huawei.com
>
> Pu Lehui (3):
>   samples: bpf: Fix cross-compiling error by using bootstrap bpftool
>   tools: runqslower: build and use lightweight bootstrap version of
>     bpftool
>   bpf: iterators: build and use lightweight bootstrap version of bpftool
>
>  kernel/bpf/preload/iterators/Makefile | 10 +++-------
>  samples/bpf/Makefile                  | 10 ++++------
>  tools/bpf/runqslower/Makefile         |  7 +++----
>  3 files changed, 10 insertions(+), 17 deletions(-)
>
> --
> 2.25.1
>

Re: [PATCH bpf-next v2 0/3] Use lightweigt version of bpftool

[Quentin Monnet]

On 15/07/2022 17:56, Andrii Nakryiko wrote:
> On Wed, Jul 13, 2022 at 7:16 PM Pu Lehui <pulehui@huawei.com> wrote:
>>
>> Currently, samples/bpf, tools/runqslower and bpf/iterators use bpftool
>> for vmlinux.h, skeleton, and static linking only. We can uselightweight
>> bootstrap version of bpftool to handle these, and it will be faster.
>>
>> v2:
>> - make libbpf and bootstrap bpftool independent. and make it simple.
>>
> 
> Quentin, does this patch set look good to you?

[Apologies, the mail server has been filtering Pu's emails as spam for
some reason and I had missed the discussion :s]

Looks OK to me:
Acked-by: Quentin Monnet <quentin@isovalent.com>

Although I'm a bit sorry to see the sharing of libbpf between bpftool
and libbpf go away. But OK. We can maybe reintroduce it through
bpftool's Makefile or a separate include Makefile in the future.

Quentin

Re: [PATCH bpf-next v2 0/3] Use lightweigt version of bpftool

[Pu Lehui]

On 2022/7/16 1:15, Quentin Monnet wrote:
> On 15/07/2022 17:56, Andrii Nakryiko wrote:
>> On Wed, Jul 13, 2022 at 7:16 PM Pu Lehui <pulehui@huawei.com> wrote:
>>>
>>> Currently, samples/bpf, tools/runqslower and bpf/iterators use bpftool
>>> for vmlinux.h, skeleton, and static linking only. We can uselightweight
>>> bootstrap version of bpftool to handle these, and it will be faster.
>>>
>>> v2:
>>> - make libbpf and bootstrap bpftool independent. and make it simple.
>>>
>>
>> Quentin, does this patch set look good to you?
> 
> [Apologies, the mail server has been filtering Pu's emails as spam for
> some reason and I had missed the discussion :s]
> 
> Looks OK to me:
> Acked-by: Quentin Monnet <quentin@isovalent.com>
> 
> Although I'm a bit sorry to see the sharing of libbpf between bpftool
> and libbpf go away. But OK. We can maybe reintroduce it through
> bpftool's Makefile or a separate include Makefile in the future.
> 
> Quentin
> .

Hi, Quentin,

so much thanks for your review and ack. and apologies for not receiving 
your advice in time due to the misconfiguration of mail server[0]. and 
looking forward to your improvement for bpftool.

[0]https://lore.kernel.org/all/20220523152516.7sr247i3bzwhr44w@quack3.lan/

> 

Re: [PATCH bpf-next v2 0/3] Use lightweigt version of bpftool

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Thu, 14 Jul 2022 10:46:09 +0800 you wrote:
> Currently, samples/bpf, tools/runqslower and bpf/iterators use bpftool
> for vmlinux.h, skeleton, and static linking only. We can uselightweight
> bootstrap version of bpftool to handle these, and it will be faster.
> 
> v2:
> - make libbpf and bootstrap bpftool independent. and make it simple.
> 
> [...]

Here is the summary with links:
  - [bpf-next,v2,1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool
    https://git.kernel.org/bpf/bpf-next/c/2e4966288c16
  - [bpf-next,v2,2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool
    https://git.kernel.org/bpf/bpf-next/c/3a2a58c4479a
  - [bpf-next,v2,3/3] bpf: iterators: build and use lightweight bootstrap version of bpftool
    https://git.kernel.org/bpf/bpf-next/c/3848636b4a88

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html