[PATCH v1] sched_ext: Improve cross-compilation support in Makefile

[PATCH v1] sched_ext: Improve cross-compilation support in Makefile

[yangsonghua]

Modify the tools/sched_ext/Makefile to better handle cross-compilation
environments by:

1. Fix host tools build directory structure by separating obj/ from output
   (HOST_BUILD_DIR now points to $(OBJ_DIR)/host/obj)
2. Properly propagate CROSS_COMPILE to libbpf sub-make invocation
3. Add missing $(HOST_BPFOBJ) build rule with proper host toolchain flags
   (ARCH=, CROSS_COMPILE=, explicit HOSTCC/HOSTLD)
4. Consistently quote $(HOSTCC) in bpftool build rule
5. Change LDFLAGS assignment to += to allow external extensions

The changes ensure proper cross-compilation behavior while maintaining
backward compatibility with native builds. Host tools are now correctly
built with the host toolchain while target binaries use the cross-toolchain.

Signed-off-by: yangsonghua <yangsonghua@lixiang.com>
---
 tools/sched_ext/Makefile | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/tools/sched_ext/Makefile b/tools/sched_ext/Makefile
index ca3815e572d8..d68780e2e03d 100644
--- a/tools/sched_ext/Makefile
+++ b/tools/sched_ext/Makefile
@@ -61,8 +61,8 @@ SCXOBJ_DIR := $(OBJ_DIR)/sched_ext
 BINDIR := $(OUTPUT_DIR)/bin
 BPFOBJ := $(BPFOBJ_DIR)/libbpf.a
 ifneq ($(CROSS_COMPILE),)
-HOST_BUILD_DIR		:= $(OBJ_DIR)/host
-HOST_OUTPUT_DIR	:= host-tools
+HOST_BUILD_DIR		:= $(OBJ_DIR)/host/obj
+HOST_OUTPUT_DIR	:= $(OBJ_DIR)/host
 HOST_INCLUDE_DIR	:= $(HOST_OUTPUT_DIR)/include
 else
 HOST_BUILD_DIR		:= $(OBJ_DIR)
@@ -98,7 +98,7 @@ ifneq ($(LLVM),)
 CFLAGS += -Wno-unused-command-line-argument
 endif
 
-LDFLAGS = -lelf -lz -lpthread
+LDFLAGS += -lelf -lz -lpthread
 
 IS_LITTLE_ENDIAN = $(shell $(CC) -dM -E - </dev/null |				\
 			grep 'define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__')
@@ -136,14 +136,25 @@ $(MAKE_DIRS):
 $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)			\
 	   $(APIDIR)/linux/bpf.h						\
 	   | $(OBJ_DIR)/libbpf
-	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(OBJ_DIR)/libbpf/	\
+	$(Q)$(MAKE) $(submake_extras) CROSS_COMPILE=$(CROSS_COMPILE) 		\
+		    -C $(BPFDIR) OUTPUT=$(OBJ_DIR)/libbpf/			\
 		    EXTRA_CFLAGS='-g -O0 -fPIC'					\
+		    LDFLAGS="$(LDFLAGS)"					\
 		    DESTDIR=$(OUTPUT_DIR) prefix= all install_headers
 
+$(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		\
+	   $(APIDIR)/linux/bpf.h						\
+	   | $(HOST_BUILD_DIR)/libbpf
+	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) 				\
+		    OUTPUT=$(HOST_BUILD_DIR)/libbpf/				\
+		    ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD=$(HOSTLD)		\
+		    EXTRA_CFLAGS='-g -O0 -fPIC'					\
+		    DESTDIR=$(HOST_OUTPUT_DIR) prefix= all install_headers
+
 $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
 		    $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool
 	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
-		    ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)		\
+		    ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD=$(HOSTLD)		\
 		    EXTRA_CFLAGS='-g -O0'					\
 		    OUTPUT=$(HOST_BUILD_DIR)/bpftool/				\
 		    LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/			\
@@ -185,7 +196,7 @@ $(addprefix $(BINDIR)/,$(c-sched-targets)): \
 		$(SCX_COMMON_DEPS)
 	$(eval sched=$(notdir $@))
 	$(CC) $(CFLAGS) -c $(sched).c -o $(SCXOBJ_DIR)/$(sched).o
-	$(CC) -o $@ $(SCXOBJ_DIR)/$(sched).o $(HOST_BPFOBJ) $(LDFLAGS)
+	$(CC) -o $@ $(SCXOBJ_DIR)/$(sched).o $(BPFOBJ) $(LDFLAGS)
 
 $(c-sched-targets): %: $(BINDIR)/%
 
-- 
2.25.1

Re: [PATCH v1] sched_ext: Improve cross-compilation support in Makefile

[Tejun Heo]

On Mon, Apr 14, 2025 at 04:14:36PM +0800, yangsonghua wrote:
> Modify the tools/sched_ext/Makefile to better handle cross-compilation
> environments by:
> 
> 1. Fix host tools build directory structure by separating obj/ from output
>    (HOST_BUILD_DIR now points to $(OBJ_DIR)/host/obj)
> 2. Properly propagate CROSS_COMPILE to libbpf sub-make invocation
> 3. Add missing $(HOST_BPFOBJ) build rule with proper host toolchain flags
>    (ARCH=, CROSS_COMPILE=, explicit HOSTCC/HOSTLD)
> 4. Consistently quote $(HOSTCC) in bpftool build rule
> 5. Change LDFLAGS assignment to += to allow external extensions
> 
> The changes ensure proper cross-compilation behavior while maintaining
> backward compatibility with native builds. Host tools are now correctly
> built with the host toolchain while target binaries use the cross-toolchain.
> 
> Signed-off-by: yangsonghua <yangsonghua@lixiang.com>

Doesn't break native builds at least. I take your words that this improves
cross-builds. Applied to sched_ext/for-6.16.

Thanks.

-- 
tejun

Re: [PATCH v1] sched_ext: Improve cross-compilation support in Makefile

[Eric yang]

Hi Tejun,

I am pleased to receive your feedback. Based on your suggestions, I
have submitted a v2 patch.
Link：https://lore.kernel.org/lkml/20250415054642.3878839-1-yangsonghua@lixiang.com/

Additionally, I have tested both v1 patch and v2 patch, they work
correctly in native builds or
cross-compilation scenarios.

Thank you for your guidance.
- Eric


Tejun Heo <tj@kernel.org> 于2025年4月15日周二 01:01写道：
>
> On Mon, Apr 14, 2025 at 04:14:36PM +0800, yangsonghua wrote:
> > Modify the tools/sched_ext/Makefile to better handle cross-compilation
> > environments by:
> >
> > 1. Fix host tools build directory structure by separating obj/ from output
> >    (HOST_BUILD_DIR now points to $(OBJ_DIR)/host/obj)
> > 2. Properly propagate CROSS_COMPILE to libbpf sub-make invocation
> > 3. Add missing $(HOST_BPFOBJ) build rule with proper host toolchain flags
> >    (ARCH=, CROSS_COMPILE=, explicit HOSTCC/HOSTLD)
> > 4. Consistently quote $(HOSTCC) in bpftool build rule
> > 5. Change LDFLAGS assignment to += to allow external extensions
> >
> > The changes ensure proper cross-compilation behavior while maintaining
> > backward compatibility with native builds. Host tools are now correctly
> > built with the host toolchain while target binaries use the cross-toolchain.
> >
> > Signed-off-by: yangsonghua <yangsonghua@lixiang.com>
>
> Doesn't break native builds at least. I take your words that this improves
> cross-builds. Applied to sched_ext/for-6.16.
>
> Thanks.
>
> --
> tejun