* [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT)
@ 2019-03-06 19:59 Stanislav Fomichev
2019-03-06 19:59 ` [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build Stanislav Fomichev
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Stanislav Fomichev @ 2019-03-06 19:59 UTC (permalink / raw)
To: netdev, bpf; +Cc: davem, ast, daniel, Stanislav Fomichev
A bunch of related changes lumped together:
* Create prog_tests and verifier output directories; these don't exist with
out-of-tree $(OUTPUT)
* Add missing -I (via separate TEST_{PROGS,VERIFIER}_CFLAGS) for the main tree
($(PWD) != $(OUTPUT) for out-of-tree)
* Add libbpf.a dependency for test_progs_32 (parallel make fails otherwise)
* Add missing "; \" after "cd" when generating test.h headers
Tested by:
$ alias m="make -s -j$(nproc)"
$ m -C tools/testing/selftests/bpf/ clean
$ m -C tools/lib/bpf/ clean
$ rm -rf xxx; mkdir xxx; m -C tools/testing/selftests/bpf/ OUTPUT=$PWD/xxx
$ m -C tools/testing/selftests/bpf/
Fixes: 3f30658830f3 ("selftests: bpf: break up test_progs - preparations")
Fixes: 2dfb40121ee8 ("selftests: bpf: prepare for break up of verifier tests")
Fixes: 3ef84346c561 ("selftests: bpf: makefile support sub-register code-gen test mode")
Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
tools/testing/selftests/bpf/Makefile | 33 +++++++++++++++++++---------
1 file changed, 23 insertions(+), 10 deletions(-)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 518cd587cd63..2aed37ea61a4 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -153,6 +153,9 @@ endif
endif
endif
+TEST_PROGS_CFLAGS := -I. -I$(OUTPUT)
+TEST_VERIFIER_CFLAGS := -I. -I$(OUTPUT) -Iverifier
+
ifneq ($(SUBREG_CODEGEN),)
ALU32_BUILD_DIR = $(OUTPUT)/alu32
TEST_CUSTOM_PROGS += $(ALU32_BUILD_DIR)/test_progs_32
@@ -162,13 +165,15 @@ TEST_CUSTOM_PROGS += $(ALU32_BUILD_DIR)/test_progs_32
$(ALU32_BUILD_DIR)/urandom_read: $(OUTPUT)/urandom_read
cp $< $@
-$(ALU32_BUILD_DIR)/test_progs_32: test_progs.c $(ALU32_BUILD_DIR) \
+$(ALU32_BUILD_DIR)/test_progs_32: test_progs.c $(OUTPUT)/libbpf.a\
+ $(ALU32_BUILD_DIR) \
$(ALU32_BUILD_DIR)/urandom_read
- $(CC) $(CFLAGS) -o $(ALU32_BUILD_DIR)/test_progs_32 $< \
- trace_helpers.c prog_tests/*.c $(OUTPUT)/libbpf.a $(LDLIBS)
+ $(CC) $(TEST_PROGS_CFLAGS) $(CFLAGS) \
+ -o $(ALU32_BUILD_DIR)/test_progs_32 \
+ test_progs.c trace_helpers.c prog_tests/*.c \
+ $(OUTPUT)/libbpf.a $(LDLIBS)
$(ALU32_BUILD_DIR)/test_progs_32: $(PROG_TESTS_H)
-$(ALU32_BUILD_DIR)/test_progs_32: CFLAGS += -I$(OUTPUT)
$(ALU32_BUILD_DIR)/test_progs_32: prog_tests/*.c
$(ALU32_BUILD_DIR)/%.o: progs/%.c $(ALU32_BUILD_DIR) \
@@ -202,12 +207,16 @@ endif
PROG_TESTS_H := $(OUTPUT)/prog_tests/tests.h
$(OUTPUT)/test_progs: $(PROG_TESTS_H)
-$(OUTPUT)/test_progs: CFLAGS += -I$(OUTPUT)
+$(OUTPUT)/test_progs: CFLAGS += $(TEST_PROGS_CFLAGS)
$(OUTPUT)/test_progs: prog_tests/*.c
+PROG_TESTS_DIR = $(OUTPUT)/prog_tests
+$(PROG_TESTS_DIR):
+ mkdir -p $@
+
PROG_TESTS_FILES := $(wildcard prog_tests/*.c)
-$(PROG_TESTS_H): $(PROG_TESTS_FILES)
- $(shell ( cd prog_tests/
+$(PROG_TESTS_H): $(PROG_TESTS_DIR) $(PROG_TESTS_FILES)
+ $(shell ( cd prog_tests/; \
echo '/* Generated header, do not edit */'; \
echo '#ifdef DECLARE'; \
ls *.c 2> /dev/null | \
@@ -221,11 +230,15 @@ $(PROG_TESTS_H): $(PROG_TESTS_FILES)
VERIFIER_TESTS_H := $(OUTPUT)/verifier/tests.h
$(OUTPUT)/test_verifier: $(VERIFIER_TESTS_H)
-$(OUTPUT)/test_verifier: CFLAGS += -I$(OUTPUT)
+$(OUTPUT)/test_verifier: CFLAGS += $(TEST_VERIFIER_CFLAGS)
+
+VERIFIER_TESTS_DIR = $(OUTPUT)/verifier
+$(VERIFIER_TESTS_DIR):
+ mkdir -p $@
VERIFIER_TEST_FILES := $(wildcard verifier/*.c)
-$(OUTPUT)/verifier/tests.h: $(VERIFIER_TEST_FILES)
- $(shell ( cd verifier/
+$(OUTPUT)/verifier/tests.h: $(VERIFIER_TESTS_DIR) $(VERIFIER_TEST_FILES)
+ $(shell ( cd verifier/; \
echo '/* Generated header, do not edit */'; \
echo '#ifdef FILL_ARRAY'; \
ls *.c 2> /dev/null | \
--
2.21.0.352.gf09ad66450-goog
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build
2019-03-06 19:59 [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Stanislav Fomichev
@ 2019-03-06 19:59 ` Stanislav Fomichev
2019-03-07 1:13 ` Y Song
2019-03-07 1:02 ` [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Y Song
2019-03-07 10:29 ` Daniel Borkmann
2 siblings, 1 reply; 5+ messages in thread
From: Stanislav Fomichev @ 2019-03-06 19:59 UTC (permalink / raw)
To: netdev, bpf; +Cc: davem, ast, daniel, Stanislav Fomichev, Eric Dumazet
libbpf targets don't explicitly depend on fixdep target, so when
we do 'make -j$(nproc)', there is a high probability, that some
objects will be built before fixdep binary is available.
Fix this by running sub-make; this makes sure that fixdep dependency
is properly accounted for.
For the same issue in perf, see commit abb26210a395 ("perf tools: Force
fixdep compilation at the start of the build").
Before:
$ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/
Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]
HOSTCC /tmp/bld/fixdep.o
CC /tmp/bld/libbpf.o
CC /tmp/bld/bpf.o
CC /tmp/bld/btf.o
CC /tmp/bld/nlattr.o
CC /tmp/bld/libbpf_errno.o
CC /tmp/bld/str_error.o
CC /tmp/bld/netlink.o
CC /tmp/bld/bpf_prog_linfo.o
CC /tmp/bld/libbpf_probes.o
CC /tmp/bld/xsk.o
HOSTLD /tmp/bld/fixdep-in.o
LINK /tmp/bld/fixdep
LD /tmp/bld/libbpf-in.o
LINK /tmp/bld/libbpf.a
LINK /tmp/bld/libbpf.so
LINK /tmp/bld/test_libbpf
$ head /tmp/bld/.libbpf.o.cmd
# cannot find fixdep (/usr/local/google/home/sdf/src/linux/xxx//fixdep)
# using basic dep data
/tmp/bld/libbpf.o: libbpf.c /usr/include/stdc-predef.h \
/usr/include/stdlib.h /usr/include/features.h \
/usr/include/x86_64-linux-gnu/sys/cdefs.h \
/usr/include/x86_64-linux-gnu/bits/wordsize.h \
/usr/include/x86_64-linux-gnu/gnu/stubs.h \
/usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
/usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h \
After:
$ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/
Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]
HOSTCC /tmp/bld/fixdep.o
HOSTLD /tmp/bld/fixdep-in.o
LINK /tmp/bld/fixdep
CC /tmp/bld/libbpf.o
CC /tmp/bld/bpf.o
CC /tmp/bld/nlattr.o
CC /tmp/bld/btf.o
CC /tmp/bld/libbpf_errno.o
CC /tmp/bld/str_error.o
CC /tmp/bld/netlink.o
CC /tmp/bld/bpf_prog_linfo.o
CC /tmp/bld/libbpf_probes.o
CC /tmp/bld/xsk.o
LD /tmp/bld/libbpf-in.o
LINK /tmp/bld/libbpf.a
LINK /tmp/bld/libbpf.so
LINK /tmp/bld/test_libbpf
$ head /tmp/bld/.libbpf.o.cmd
cmd_/tmp/bld/libbpf.o := gcc -Wp,-MD,/tmp/bld/.libbpf.o.d -Wp,-MT,/tmp/bld/libbpf.o -g -Wall -DHAVE_LIBELF_MMAP_SUPPORT -DCOMPAT_NEED_REALLOCARRAY -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -Wstrict-aliasing=3 -Werror -Wall -fPIC -I. -I/usr/local/google/home/sdf/src/linux/tools/include -I/usr/local/google/home/sdf/src/linux/tools/arch/x86/include/uapi -I/usr/local/google/home/sdf/src/linux/tools/include/uapi -fvisibility=hidden -D"BUILD_STR(s)=$(pound)s" -c -o /tmp/bld/libbpf.o libbpf.c
source_/tmp/bld/libbpf.o := libbpf.c
deps_/tmp/bld/libbpf.o := \
/usr/include/stdc-predef.h \
/usr/include/stdlib.h \
/usr/include/features.h \
/usr/include/x86_64-linux-gnu/sys/cdefs.h \
/usr/include/x86_64-linux-gnu/bits/wordsize.h \
Reported-by: Eric Dumazet <edumazet@google.com>
Fixes: 7c422f557266 ("tools build: Build fixdep helper from perf and basic libs")
Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
tools/lib/bpf/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index a05c43468bd0..61aaacf0cfa1 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -147,7 +147,8 @@ endif
TARGETS = $(CMD_TARGETS)
-all: fixdep all_cmd
+all: fixdep
+ $(Q)$(MAKE) all_cmd
all_cmd: $(CMD_TARGETS) check
--
2.21.0.352.gf09ad66450-goog
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT)
2019-03-06 19:59 [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Stanislav Fomichev
2019-03-06 19:59 ` [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build Stanislav Fomichev
@ 2019-03-07 1:02 ` Y Song
2019-03-07 10:29 ` Daniel Borkmann
2 siblings, 0 replies; 5+ messages in thread
From: Y Song @ 2019-03-07 1:02 UTC (permalink / raw)
To: Stanislav Fomichev
Cc: netdev, bpf, David Miller, Alexei Starovoitov, Daniel Borkmann
On Wed, Mar 6, 2019 at 1:14 PM Stanislav Fomichev <sdf@google.com> wrote:
>
> A bunch of related changes lumped together:
> * Create prog_tests and verifier output directories; these don't exist with
> out-of-tree $(OUTPUT)
> * Add missing -I (via separate TEST_{PROGS,VERIFIER}_CFLAGS) for the main tree
> ($(PWD) != $(OUTPUT) for out-of-tree)
> * Add libbpf.a dependency for test_progs_32 (parallel make fails otherwise)
> * Add missing "; \" after "cd" when generating test.h headers
>
> Tested by:
> $ alias m="make -s -j$(nproc)"
> $ m -C tools/testing/selftests/bpf/ clean
> $ m -C tools/lib/bpf/ clean
> $ rm -rf xxx; mkdir xxx; m -C tools/testing/selftests/bpf/ OUTPUT=$PWD/xxx
> $ m -C tools/testing/selftests/bpf/
>
> Fixes: 3f30658830f3 ("selftests: bpf: break up test_progs - preparations")
> Fixes: 2dfb40121ee8 ("selftests: bpf: prepare for break up of verifier tests")
> Fixes: 3ef84346c561 ("selftests: bpf: makefile support sub-register code-gen test mode")
>
> Signed-off-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Yonghong Song <yhs@fb.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build
2019-03-06 19:59 ` [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build Stanislav Fomichev
@ 2019-03-07 1:13 ` Y Song
0 siblings, 0 replies; 5+ messages in thread
From: Y Song @ 2019-03-07 1:13 UTC (permalink / raw)
To: Stanislav Fomichev
Cc: netdev, bpf, David Miller, Alexei Starovoitov, Daniel Borkmann,
Eric Dumazet
On Wed, Mar 6, 2019 at 1:14 PM Stanislav Fomichev <sdf@google.com> wrote:
>
> libbpf targets don't explicitly depend on fixdep target, so when
> we do 'make -j$(nproc)', there is a high probability, that some
> objects will be built before fixdep binary is available.
>
> Fix this by running sub-make; this makes sure that fixdep dependency
> is properly accounted for.
>
> For the same issue in perf, see commit abb26210a395 ("perf tools: Force
> fixdep compilation at the start of the build").
>
> Before:
>
> $ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/
>
> Auto-detecting system features:
> ... libelf: [ on ]
> ... bpf: [ on ]
>
> HOSTCC /tmp/bld/fixdep.o
> CC /tmp/bld/libbpf.o
> CC /tmp/bld/bpf.o
> CC /tmp/bld/btf.o
> CC /tmp/bld/nlattr.o
> CC /tmp/bld/libbpf_errno.o
> CC /tmp/bld/str_error.o
> CC /tmp/bld/netlink.o
> CC /tmp/bld/bpf_prog_linfo.o
> CC /tmp/bld/libbpf_probes.o
> CC /tmp/bld/xsk.o
> HOSTLD /tmp/bld/fixdep-in.o
> LINK /tmp/bld/fixdep
> LD /tmp/bld/libbpf-in.o
> LINK /tmp/bld/libbpf.a
> LINK /tmp/bld/libbpf.so
> LINK /tmp/bld/test_libbpf
>
> $ head /tmp/bld/.libbpf.o.cmd
> # cannot find fixdep (/usr/local/google/home/sdf/src/linux/xxx//fixdep)
> # using basic dep data
>
> /tmp/bld/libbpf.o: libbpf.c /usr/include/stdc-predef.h \
> /usr/include/stdlib.h /usr/include/features.h \
> /usr/include/x86_64-linux-gnu/sys/cdefs.h \
> /usr/include/x86_64-linux-gnu/bits/wordsize.h \
> /usr/include/x86_64-linux-gnu/gnu/stubs.h \
> /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
> /usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h \
>
> After:
>
> $ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/
>
> Auto-detecting system features:
> ... libelf: [ on ]
> ... bpf: [ on ]
>
> HOSTCC /tmp/bld/fixdep.o
> HOSTLD /tmp/bld/fixdep-in.o
> LINK /tmp/bld/fixdep
> CC /tmp/bld/libbpf.o
> CC /tmp/bld/bpf.o
> CC /tmp/bld/nlattr.o
> CC /tmp/bld/btf.o
> CC /tmp/bld/libbpf_errno.o
> CC /tmp/bld/str_error.o
> CC /tmp/bld/netlink.o
> CC /tmp/bld/bpf_prog_linfo.o
> CC /tmp/bld/libbpf_probes.o
> CC /tmp/bld/xsk.o
> LD /tmp/bld/libbpf-in.o
> LINK /tmp/bld/libbpf.a
> LINK /tmp/bld/libbpf.so
> LINK /tmp/bld/test_libbpf
>
> $ head /tmp/bld/.libbpf.o.cmd
> cmd_/tmp/bld/libbpf.o := gcc -Wp,-MD,/tmp/bld/.libbpf.o.d -Wp,-MT,/tmp/bld/libbpf.o -g -Wall -DHAVE_LIBELF_MMAP_SUPPORT -DCOMPAT_NEED_REALLOCARRAY -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -Wstrict-aliasing=3 -Werror -Wall -fPIC -I. -I/usr/local/google/home/sdf/src/linux/tools/include -I/usr/local/google/home/sdf/src/linux/tools/arch/x86/include/uapi -I/usr/local/google/home/sdf/src/linux/tools/include/uapi -fvisibility=hidden -D"BUILD_STR(s)=$(pound)s" -c -o /tmp/bld/libbpf.o libbpf.c
>
> source_/tmp/bld/libbpf.o := libbpf.c
>
> deps_/tmp/bld/libbpf.o := \
> /usr/include/stdc-predef.h \
> /usr/include/stdlib.h \
> /usr/include/features.h \
> /usr/include/x86_64-linux-gnu/sys/cdefs.h \
> /usr/include/x86_64-linux-gnu/bits/wordsize.h \
>
> Reported-by: Eric Dumazet <edumazet@google.com>
> Fixes: 7c422f557266 ("tools build: Build fixdep helper from perf and basic libs")
>
> Signed-off-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Yonghong Song <yhs@fb.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT)
2019-03-06 19:59 [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Stanislav Fomichev
2019-03-06 19:59 ` [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build Stanislav Fomichev
2019-03-07 1:02 ` [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Y Song
@ 2019-03-07 10:29 ` Daniel Borkmann
2 siblings, 0 replies; 5+ messages in thread
From: Daniel Borkmann @ 2019-03-07 10:29 UTC (permalink / raw)
To: Stanislav Fomichev, netdev, bpf; +Cc: davem, ast
On 03/06/2019 08:59 PM, Stanislav Fomichev wrote:
> A bunch of related changes lumped together:
> * Create prog_tests and verifier output directories; these don't exist with
> out-of-tree $(OUTPUT)
> * Add missing -I (via separate TEST_{PROGS,VERIFIER}_CFLAGS) for the main tree
> ($(PWD) != $(OUTPUT) for out-of-tree)
> * Add libbpf.a dependency for test_progs_32 (parallel make fails otherwise)
> * Add missing "; \" after "cd" when generating test.h headers
>
> Tested by:
> $ alias m="make -s -j$(nproc)"
> $ m -C tools/testing/selftests/bpf/ clean
> $ m -C tools/lib/bpf/ clean
> $ rm -rf xxx; mkdir xxx; m -C tools/testing/selftests/bpf/ OUTPUT=$PWD/xxx
> $ m -C tools/testing/selftests/bpf/
>
> Fixes: 3f30658830f3 ("selftests: bpf: break up test_progs - preparations")
> Fixes: 2dfb40121ee8 ("selftests: bpf: prepare for break up of verifier tests")
> Fixes: 3ef84346c561 ("selftests: bpf: makefile support sub-register code-gen test mode")
>
Btw, please no newline between these tags, I've fixed it up.
> Signed-off-by: Stanislav Fomichev <sdf@google.com>
Applied, thanks!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-03-07 10:29 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-03-06 19:59 [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Stanislav Fomichev
2019-03-06 19:59 ` [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build Stanislav Fomichev
2019-03-07 1:13 ` Y Song
2019-03-07 1:02 ` [PATCH bpf 1/2] selftests: bpf: fix compilation with out-of-tree $(OUTPUT) Y Song
2019-03-07 10:29 ` Daniel Borkmann
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).