[PATCH v1 00/14] Fix perf tools/lib includes

linux-perf-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [PATCH v1 00/14] Fix perf tools/lib includes
@ 2022-11-08  7:35 Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 01/14] tools lib api: Add install target Ian Rogers
                   ` (14 more replies)
  0 siblings, 15 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

The previous build would add -Itools/lib and get dependencies for
libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
overriding these libraries would change the link time dependency but
the headers would erroneously come from tools/lib. Fix the build to
install headers and then depend on these. To reduce exposing internal
headers/APIs some clean up is performed. tools/lib/symbol has a
Makefile added, while tools/lib/api and tools/lib/subcmd have install
targets added. The pattern used for the dependencies in Makefile.perf
is modelled on libbpf.

The problem and solution were motivated by this issue and discussion:
https://lore.kernel.org/lkml/CAEf4BzbbOHQZUAe6iWaehKCPQAf3VC=hq657buqe2_yRKxaK-A@mail.gmail.com/

Ian Rogers (14):
  tools lib api: Add install target
  tools lib subcmd: Add install target
  perf build: Install libsubcmd locally when building
  perf build: Install libapi locally when building
  perf build: Install libperf locally when building
  perf build: Install libtraceevent locally when building
  tools lib api: Add missing install headers
  tools lib perf: Add missing install headers
  tool lib symbol: Add Makefile/Build
  perf build: Install libsymbol locally when building
  perf expr: Tidy hashmap dependency
  perf thread_map: Reduce exposure of libperf internal API
  perf cpumap: Tidy libperf includes
  perf build: Use tools/lib headers from install path

 tools/lib/api/Makefile                        |  52 ++++++
 tools/lib/perf/Makefile                       |  10 +-
 tools/lib/subcmd/Makefile                     |  49 ++++++
 tools/lib/symbol/Build                        |   1 +
 tools/lib/symbol/Makefile                     | 115 +++++++++++++
 tools/perf/.gitignore                         |   7 +-
 tools/perf/Makefile.config                    |   2 -
 tools/perf/Makefile.perf                      | 152 ++++++++++++------
 tools/perf/builtin-stat.c                     |   1 +
 tools/perf/builtin-trace.c                    |   4 +-
 tools/perf/tests/cpumap.c                     |   2 +-
 tools/perf/tests/expr.c                       |   1 +
 tools/perf/tests/openat-syscall.c             |   1 +
 tools/perf/tests/pmu-events.c                 |   1 +
 tools/perf/tests/thread-map.c                 |   1 +
 tools/perf/util/Build                         |   5 -
 tools/perf/util/auxtrace.h                    |   2 +-
 tools/perf/util/bpf-loader.c                  |   4 -
 tools/perf/util/bpf_counter.c                 |   2 +-
 tools/perf/util/cpumap.c                      |   1 +
 tools/perf/util/cpumap.h                      |   2 +-
 tools/perf/util/evsel.c                       |   5 +-
 tools/perf/util/evsel.h                       |   2 -
 tools/perf/util/expr.c                        |   1 +
 tools/perf/util/expr.h                        |   7 +-
 tools/perf/util/metricgroup.c                 |   1 +
 tools/perf/util/python.c                      |   6 +-
 .../scripting-engines/trace-event-python.c    |   2 +-
 tools/perf/util/stat-shadow.c                 |   1 +
 tools/perf/util/stat.c                        |   4 -
 tools/perf/util/thread_map.c                  |   1 +
 tools/perf/util/thread_map.h                  |   2 -
 32 files changed, 361 insertions(+), 86 deletions(-)
 create mode 100644 tools/lib/symbol/Build
 create mode 100644 tools/lib/symbol/Makefile

-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v1 01/14] tools lib api: Add install target
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 02/14] tools lib subcmd: " Ian Rogers
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

This allows libapi to be installed as a dependency.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/api/Makefile | 49 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/tools/lib/api/Makefile b/tools/lib/api/Makefile
index e21e1b40b525..6629d0fd0130 100644
--- a/tools/lib/api/Makefile
+++ b/tools/lib/api/Makefile
@@ -15,6 +15,16 @@ LD ?= $(CROSS_COMPILE)ld
 
 MAKEFLAGS += --no-print-directory
 
+INSTALL = install
+
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
 LIBFILE = $(OUTPUT)libapi.a
 
 CFLAGS := $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
@@ -45,10 +55,23 @@ RM = rm -f
 
 API_IN := $(OUTPUT)libapi-in.o
 
+ifeq ($(LP64), 1)
+  libdir_relative = lib64
+else
+  libdir_relative = lib
+endif
+
+prefix ?=
+libdir = $(prefix)/$(libdir_relative)
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+
 all:
 
 export srctree OUTPUT CC LD CFLAGS V
 include $(srctree)/tools/build/Makefile.include
+include $(srctree)/tools/scripts/Makefile.include
 
 all: fixdep $(LIBFILE)
 
@@ -58,6 +81,32 @@ $(API_IN): FORCE
 $(LIBFILE): $(API_IN)
 	$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(API_IN)
 
+define do_install_mkdir
+	if [ ! -d '$(DESTDIR_SQ)$1' ]; then             \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
+	fi
+endef
+
+define do_install
+	if [ ! -d '$(DESTDIR_SQ)$2' ]; then             \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
+	fi;                                             \
+	$(INSTALL) $1 $(if $3,-m $3,) '$(DESTDIR_SQ)$2'
+endef
+
+install_lib: $(LIBFILE)
+	$(call QUIET_INSTALL, $(LIBFILE)) \
+		$(call do_install_mkdir,$(libdir_SQ)); \
+		cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ)
+
+install_headers:
+	$(call QUIET_INSTALL, headers) \
+		$(call do_install,cpu.h,$(prefix)/include/api,644); \
+		$(call do_install,debug.h,$(prefix)/include/api,644); \
+		$(call do_install,io.h,$(prefix)/include/api,644);
+
+install: install_lib install_headers
+
 clean:
 	$(call QUIET_CLEAN, libapi) $(RM) $(LIBFILE); \
 	find $(or $(OUTPUT),.) -name \*.o -or -name \*.o.cmd -or -name \*.o.d | xargs $(RM)
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 02/14] tools lib subcmd: Add install target
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 01/14] tools lib api: Add install target Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 03/14] perf build: Install libsubcmd locally when building Ian Rogers
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

This allows libsubcmd to be installed as a dependency.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/subcmd/Makefile | 49 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/tools/lib/subcmd/Makefile b/tools/lib/subcmd/Makefile
index 8f1a09cdfd17..e96566f8991c 100644
--- a/tools/lib/subcmd/Makefile
+++ b/tools/lib/subcmd/Makefile
@@ -17,6 +17,15 @@ RM = rm -f
 
 MAKEFLAGS += --no-print-directory
 
+INSTALL = install
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
 LIBFILE = $(OUTPUT)libsubcmd.a
 
 CFLAGS := -ggdb3 -Wall -Wextra -std=gnu99 -fPIC
@@ -48,6 +57,18 @@ CFLAGS += $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
 
 SUBCMD_IN := $(OUTPUT)libsubcmd-in.o
 
+ifeq ($(LP64), 1)
+  libdir_relative = lib64
+else
+  libdir_relative = lib
+endif
+
+prefix ?=
+libdir = $(prefix)/$(libdir_relative)
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+
 all:
 
 export srctree OUTPUT CC LD CFLAGS V
@@ -61,6 +82,34 @@ $(SUBCMD_IN): FORCE
 $(LIBFILE): $(SUBCMD_IN)
 	$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(SUBCMD_IN)
 
+define do_install_mkdir
+	if [ ! -d '$(DESTDIR_SQ)$1' ]; then             \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
+	fi
+endef
+
+define do_install
+	if [ ! -d '$(DESTDIR_SQ)$2' ]; then             \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
+	fi;                                             \
+	$(INSTALL) $1 $(if $3,-m $3,) '$(DESTDIR_SQ)$2'
+endef
+
+install_lib: $(LIBFILE)
+	$(call QUIET_INSTALL, $(LIBFILE)) \
+		$(call do_install_mkdir,$(libdir_SQ)); \
+		cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ)
+
+install_headers:
+	$(call QUIET_INSTALL, headers) \
+		$(call do_install,exec-cmd.h,$(prefix)/include/subcmd,644); \
+		$(call do_install,help.h,$(prefix)/include/subcmd,644); \
+		$(call do_install,pager.h,$(prefix)/include/subcmd,644); \
+		$(call do_install,parse-options.h,$(prefix)/include/subcmd,644); \
+		$(call do_install,run-command.h,$(prefix)/include/subcmd,644);
+
+install: install_lib install_headers
+
 clean:
 	$(call QUIET_CLEAN, libsubcmd) $(RM) $(LIBFILE); \
 	find $(or $(OUTPUT),.) -name \*.o -or -name \*.o.cmd -or -name \*.o.d | xargs $(RM)
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 03/14] perf build: Install libsubcmd locally when building
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 01/14] tools lib api: Add install target Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 02/14] tools lib subcmd: " Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 04/14] perf build: Install libapi " Ian Rogers
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

The perf build currently has a '-Itools/lib' on the CC command
line. This causes issues as the libapi, libsubcmd, libtraceevent,
libbpf headers are all found via this path, making it impossible to
override include behavior. Change the libsubcmd build mirroring the
libbpf build, so that it is installed in a directory along with its
headers. A later change will modify the include behavior.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/.gitignore    |  1 +
 tools/perf/Makefile.perf | 24 ++++++++++++++++--------
 2 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index a653311d9693..626f5dd9a037 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -39,6 +39,7 @@ pmu-events/pmu-events.c
 pmu-events/jevents
 feature/
 libbpf/
+libsubcmd/
 fixdep
 libtraceevent-dynamic-list
 Documentation/doc.dep
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index a432e59afc42..af62c6b9ed7a 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -244,7 +244,7 @@ else # force_fixdep
 LIB_DIR         = $(srctree)/tools/lib/api/
 TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
 LIBBPF_DIR      = $(srctree)/tools/lib/bpf/
-SUBCMD_DIR      = $(srctree)/tools/lib/subcmd/
+LIBSUBCMD_DIR   = $(srctree)/tools/lib/subcmd/
 LIBPERF_DIR     = $(srctree)/tools/lib/perf/
 DOC_DIR         = $(srctree)/tools/perf/Documentation/
 
@@ -294,7 +294,6 @@ strip-libs = $(filter-out -l%,$(1))
 ifneq ($(OUTPUT),)
   TE_PATH=$(OUTPUT)
   PLUGINS_PATH=$(OUTPUT)
-  SUBCMD_PATH=$(OUTPUT)
   LIBPERF_PATH=$(OUTPUT)
 ifneq ($(subdir),)
   API_PATH=$(OUTPUT)/../lib/api/
@@ -305,7 +304,6 @@ else
   TE_PATH=$(TRACE_EVENT_DIR)
   PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/
   API_PATH=$(LIB_DIR)
-  SUBCMD_PATH=$(SUBCMD_DIR)
   LIBPERF_PATH=$(LIBPERF_DIR)
 endif
 
@@ -332,7 +330,14 @@ LIBBPF_DESTDIR = $(LIBBPF_OUTPUT)
 LIBBPF_INCLUDE = $(LIBBPF_DESTDIR)/include
 LIBBPF = $(LIBBPF_OUTPUT)/libbpf.a
 
-LIBSUBCMD = $(SUBCMD_PATH)libsubcmd.a
+ifneq ($(OUTPUT),)
+  LIBSUBCMD_OUTPUT = $(abspath $(OUTPUT))/libsubcmd
+else
+  LIBSUBCMD_OUTPUT = $(CURDIR)/libsubcmd
+endif
+LIBSUBCMD_DESTDIR = $(LIBSUBCMD_OUTPUT)
+LIBSUBCMD_INCLUDE = $(LIBSUBCMD_DESTDIR)/include
+LIBSUBCMD = $(LIBSUBCMD_OUTPUT)/libsubcmd.a
 
 LIBPERF = $(LIBPERF_PATH)libperf.a
 export LIBPERF
@@ -849,11 +854,14 @@ $(LIBPERF)-clean:
 	$(call QUIET_CLEAN, libperf)
 	$(Q)$(MAKE) -C $(LIBPERF_DIR) O=$(OUTPUT) clean >/dev/null
 
-$(LIBSUBCMD): FORCE
-	$(Q)$(MAKE) -C $(SUBCMD_DIR) O=$(OUTPUT) $(OUTPUT)libsubcmd.a
+$(LIBSUBCMD): FORCE | $(LIBSUBCMD_OUTPUT)
+	$(Q)$(MAKE) -C $(LIBSUBCMD_DIR) O=$(LIBSUBCMD_OUTPUT) \
+		DESTDIR=$(LIBSUBCMD_DESTDIR) prefix= \
+		$@ install_headers
 
 $(LIBSUBCMD)-clean:
-	$(Q)$(MAKE) -C $(SUBCMD_DIR) O=$(OUTPUT) clean
+	$(call QUIET_CLEAN, libsubcmd)
+	$(Q)$(RM) -r -- $(LIBSUBCMD_OUTPUT)
 
 help:
 	@echo 'Perf make targets:'
@@ -1044,7 +1052,7 @@ SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h
 SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h
 SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h
 
-$(SKEL_TMP_OUT) $(LIBBPF_OUTPUT):
+$(SKEL_TMP_OUT) $(LIBBPF_OUTPUT) $(LIBSUBCMD_OUTPUT):
 	$(Q)$(MKDIR) -p $@
 
 ifdef BUILD_BPF_SKEL
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 04/14] perf build: Install libapi locally when building
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (2 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 03/14] perf build: Install libsubcmd locally when building Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 05/14] perf build: Install libperf " Ian Rogers
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

The perf build currently has a '-Itools/lib' on the CC command
line. This causes issues as the libapi, libsubcmd, libtraceevent,
libbpf headers are all found via this path, making it impossible to
override include behavior. Change the libapi build mirroring the
libbpf and libsubcmd build, so that it is installed in a directory
along with its headers. A later change will modify the include
behavior.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/.gitignore    |  1 +
 tools/perf/Makefile.perf | 27 +++++++++++++++------------
 2 files changed, 16 insertions(+), 12 deletions(-)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 626f5dd9a037..1932848343f3 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -38,6 +38,7 @@ trace/beauty/generated/
 pmu-events/pmu-events.c
 pmu-events/jevents
 feature/
+libapi/
 libbpf/
 libsubcmd/
 fixdep
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index af62c6b9ed7a..3874d88d72c9 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -241,7 +241,7 @@ sub-make: fixdep
 
 else # force_fixdep
 
-LIB_DIR         = $(srctree)/tools/lib/api/
+LIBAPI_DIR      = $(srctree)/tools/lib/api/
 TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
 LIBBPF_DIR      = $(srctree)/tools/lib/bpf/
 LIBSUBCMD_DIR   = $(srctree)/tools/lib/subcmd/
@@ -295,15 +295,9 @@ ifneq ($(OUTPUT),)
   TE_PATH=$(OUTPUT)
   PLUGINS_PATH=$(OUTPUT)
   LIBPERF_PATH=$(OUTPUT)
-ifneq ($(subdir),)
-  API_PATH=$(OUTPUT)/../lib/api/
-else
-  API_PATH=$(OUTPUT)
-endif
 else
   TE_PATH=$(TRACE_EVENT_DIR)
   PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/
-  API_PATH=$(LIB_DIR)
   LIBPERF_PATH=$(LIBPERF_DIR)
 endif
 
@@ -318,7 +312,14 @@ LIBTRACEEVENT_DYNAMIC_LIST = $(PLUGINS_PATH)libtraceevent-dynamic-list
 DYNAMIC_LIST_LDFLAGS               = -Xlinker --dynamic-list=$(LIBTRACEEVENT_DYNAMIC_LIST)
 LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS = $(if $(findstring -static,$(LDFLAGS)),,$(DYNAMIC_LIST_LDFLAGS))
 
-LIBAPI = $(API_PATH)libapi.a
+ifneq ($(OUTPUT),)
+  LIBAPI_OUTPUT = $(abspath $(OUTPUT))/libapi
+else
+  LIBAPI_OUTPUT = $(CURDIR)/libapi
+endif
+LIBAPI_DESTDIR = $(LIBAPI_OUTPUT)
+LIBAPI_INCLUDE = $(LIBAPI_DESTDIR)/include
+LIBAPI = $(LIBAPI_OUTPUT)/libapi.a
 export LIBAPI
 
 ifneq ($(OUTPUT),)
@@ -831,12 +832,14 @@ $(LIBTRACEEVENT)-clean:
 install-traceevent-plugins: libtraceevent_plugins
 	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) install_plugins
 
-$(LIBAPI): FORCE
-	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) $(OUTPUT)libapi.a
+$(LIBAPI): FORCE | $(LIBAPI_OUTPUT)
+	$(Q)$(MAKE) -C $(LIBAPI_DIR) O=$(LIBAPI_OUTPUT) \
+		DESTDIR=$(LIBAPI_DESTDIR) prefix= \
+		$@ install_headers
 
 $(LIBAPI)-clean:
 	$(call QUIET_CLEAN, libapi)
-	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
+	$(Q)$(RM) -r -- $(LIBAPI_OUTPUT)
 
 $(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
 	$(Q)$(MAKE) -C $(LIBBPF_DIR) FEATURES_DUMP=$(FEATURE_DUMP_EXPORT) \
@@ -1052,7 +1055,7 @@ SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h
 SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h
 SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h
 
-$(SKEL_TMP_OUT) $(LIBBPF_OUTPUT) $(LIBSUBCMD_OUTPUT):
+$(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBSUBCMD_OUTPUT):
 	$(Q)$(MKDIR) -p $@
 
 ifdef BUILD_BPF_SKEL
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 05/14] perf build: Install libperf locally when building
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (3 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 04/14] perf build: Install libapi " Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 06/14] perf build: Install libtraceevent " Ian Rogers
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

The perf build currently has a '-Itools/lib' on the CC command
line. This causes issues as the libapi, libsubcmd, libtraceevent,
libbpf headers are all found via this path, making it impossible to
override include behavior. Change the libperf build mirroring the
libbpf, libsubcmd and libapi build, so that it is installed in a
directory along with its headers. A later change will modify the
include behavior.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/.gitignore    |  1 +
 tools/perf/Makefile.perf | 21 ++++++++++++++-------
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 1932848343f3..43f6621ef05e 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -40,6 +40,7 @@ pmu-events/jevents
 feature/
 libapi/
 libbpf/
+libperf/
 libsubcmd/
 fixdep
 libtraceevent-dynamic-list
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 3874d88d72c9..5a2a3c4f045d 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -294,11 +294,9 @@ strip-libs = $(filter-out -l%,$(1))
 ifneq ($(OUTPUT),)
   TE_PATH=$(OUTPUT)
   PLUGINS_PATH=$(OUTPUT)
-  LIBPERF_PATH=$(OUTPUT)
 else
   TE_PATH=$(TRACE_EVENT_DIR)
   PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/
-  LIBPERF_PATH=$(LIBPERF_DIR)
 endif
 
 LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
@@ -340,7 +338,14 @@ LIBSUBCMD_DESTDIR = $(LIBSUBCMD_OUTPUT)
 LIBSUBCMD_INCLUDE = $(LIBSUBCMD_DESTDIR)/include
 LIBSUBCMD = $(LIBSUBCMD_OUTPUT)/libsubcmd.a
 
-LIBPERF = $(LIBPERF_PATH)libperf.a
+ifneq ($(OUTPUT),)
+  LIBPERF_OUTPUT = $(abspath $(OUTPUT))/libperf
+else
+  LIBPERF_OUTPUT = $(CURDIR)/libperf
+endif
+LIBPERF_DESTDIR = $(LIBPERF_OUTPUT)
+LIBPERF_INCLUDE = $(LIBPERF_DESTDIR)/include
+LIBPERF = $(LIBPERF_OUTPUT)/libperf.a
 export LIBPERF
 
 # python extension build directories
@@ -850,12 +855,14 @@ $(LIBBPF)-clean:
 	$(call QUIET_CLEAN, libbpf)
 	$(Q)$(RM) -r -- $(LIBBPF_OUTPUT)
 
-$(LIBPERF): FORCE
-	$(Q)$(MAKE) -C $(LIBPERF_DIR) EXTRA_CFLAGS="$(LIBPERF_CFLAGS)" O=$(OUTPUT) $(OUTPUT)libperf.a
+$(LIBPERF): FORCE | $(LIBPERF_OUTPUT)
+	$(Q)$(MAKE) -C $(LIBPERF_DIR) O=$(LIBPERF_OUTPUT) \
+		DESTDIR=$(LIBPERF_DESTDIR) prefix= \
+		$@ install_headers
 
 $(LIBPERF)-clean:
 	$(call QUIET_CLEAN, libperf)
-	$(Q)$(MAKE) -C $(LIBPERF_DIR) O=$(OUTPUT) clean >/dev/null
+	$(Q)$(RM) -r -- $(LIBPERF_OUTPUT)
 
 $(LIBSUBCMD): FORCE | $(LIBSUBCMD_OUTPUT)
 	$(Q)$(MAKE) -C $(LIBSUBCMD_DIR) O=$(LIBSUBCMD_OUTPUT) \
@@ -1055,7 +1062,7 @@ SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h
 SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h
 SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h
 
-$(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBSUBCMD_OUTPUT):
+$(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT):
 	$(Q)$(MKDIR) -p $@
 
 ifdef BUILD_BPF_SKEL
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 06/14] perf build: Install libtraceevent locally when building
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (4 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 05/14] perf build: Install libperf " Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 07/14] tools lib api: Add missing install headers Ian Rogers
                   ` (8 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

The perf build currently has a '-Itools/lib' on the CC command
line. This causes issues as the libapi, libsubcmd, libtraceevent,
libbpf headers are all found via this path, making it impossible to
override include behavior. Change the libtraceevent build mirroring
the libbpf, libsubcmd, libapi and libperf build, so that it is
installed in a directory along with its headers. A later change will
modify the include behavior.

Similarly, the plugins are now installed into libtraceevent_plugins
except they have no header files.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/.gitignore    |  3 ++-
 tools/perf/Makefile.perf | 57 ++++++++++++++++++++++++++--------------
 2 files changed, 40 insertions(+), 20 deletions(-)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 43f6621ef05e..65b995159cf1 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -42,6 +42,7 @@ libapi/
 libbpf/
 libperf/
 libsubcmd/
+libtraceevent/
+libtraceevent_plugins/
 fixdep
-libtraceevent-dynamic-list
 Documentation/doc.dep
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 5a2a3c4f045d..537ac7055a35 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -242,7 +242,8 @@ sub-make: fixdep
 else # force_fixdep
 
 LIBAPI_DIR      = $(srctree)/tools/lib/api/
-TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
+LIBTRACEEVENT_DIR = $(srctree)/tools/lib/traceevent/
+LIBTRACEEVENT_PLUGINS_DIR = $(LIBTRACEEVENT_DIR)/plugins
 LIBBPF_DIR      = $(srctree)/tools/lib/bpf/
 LIBSUBCMD_DIR   = $(srctree)/tools/lib/subcmd/
 LIBPERF_DIR     = $(srctree)/tools/lib/perf/
@@ -292,16 +293,17 @@ grep-libs = $(filter -l%,$(1))
 strip-libs = $(filter-out -l%,$(1))
 
 ifneq ($(OUTPUT),)
-  TE_PATH=$(OUTPUT)
-  PLUGINS_PATH=$(OUTPUT)
+  LIBTRACEEVENT_OUTPUT = $(abspath $(OUTPUT))/libtraceevent
 else
-  TE_PATH=$(TRACE_EVENT_DIR)
-  PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/
+  LIBTRACEEVENT_OUTPUT = $(CURDIR)/libtraceevent
 endif
-
-LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
+LIBTRACEEVENT_PLUGINS_OUTPUT = $(LIBTRACEEVENT_OUTPUT)_plugins
+LIBTRACEEVENT_DESTDIR = $(LIBTRACEEVENT_OUTPUT)
+LIBTRACEEVENT_PLUGINS_DESTDIR = $(LIBTRACEEVENT_PLUGINS_OUTPUT)
+LIBTRACEEVENT_INCLUDE = $(LIBTRACEEVENT_DESTDIR)/include
+LIBTRACEEVENT = $(LIBTRACEEVENT_OUTPUT)/libtraceevent.a
 export LIBTRACEEVENT
-LIBTRACEEVENT_DYNAMIC_LIST = $(PLUGINS_PATH)libtraceevent-dynamic-list
+LIBTRACEEVENT_DYNAMIC_LIST = $(LIBTRACEEVENT_PLUGINS_OUTPUT)/libtraceevent-dynamic-list
 
 #
 # The static build has no dynsym table, so this does not work for
@@ -821,21 +823,33 @@ $(patsubst perf-%,%.o,$(PROGRAMS)): $(wildcard */*.h)
 
 LIBTRACEEVENT_FLAGS += plugin_dir=$(plugindir_SQ) 'EXTRA_CFLAGS=$(EXTRA_CFLAGS)' 'LDFLAGS=$(filter-out -static,$(LDFLAGS))'
 
-$(LIBTRACEEVENT): FORCE
-	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent.a
-
-libtraceevent_plugins: FORCE
-	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins
-
-$(LIBTRACEEVENT_DYNAMIC_LIST): libtraceevent_plugins
-	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent-dynamic-list
+$(LIBTRACEEVENT): FORCE | $(LIBTRACEEVENT_OUTPUT)
+	$(Q)$(MAKE) -C $(LIBTRACEEVENT_DIR) O=$(LIBTRACEEVENT_OUTPUT) \
+		DESTDIR=$(LIBTRACEEVENT_DESTDIR) prefix= \
+		$@ install_headers
 
 $(LIBTRACEEVENT)-clean:
 	$(call QUIET_CLEAN, libtraceevent)
-	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) O=$(OUTPUT) clean >/dev/null
+	$(Q)$(RM) -r -- $(LIBTRACEEVENT_OUTPUT)
+
+libtraceevent_plugins: FORCE | $(LIBTRACEEVENT_PLUGINS_OUTPUT)
+	$(Q)$(MAKE) -C $(LIBTRACEEVENT_PLUGINS_DIR) O=$(LIBTRACEEVENT_PLUGINS_OUTPUT) \
+		DESTDIR=$(LIBTRACEEVENT_PLUGINS_DESTDIR) prefix= \
+		plugins
+
+libtraceevent_plugins-clean:
+	$(call QUIET_CLEAN, libtraceevent_plugins)
+	$(Q)$(RM) -r -- $(LIBTRACEEVENT_PLUGINS_OUTPUT)
+
+$(LIBTRACEEVENT_DYNAMIC_LIST): libtraceevent_plugins
+	$(Q)$(MAKE) -C $(LIBTRACEEVENT_PLUGINS_DIR) O=$(LIBTRACEEVENT_PLUGINS_OUTPUT) \
+		DESTDIR=$(LIBTRACEEVENT_PLUGINS_DESTDIR) prefix= \
+		$(LIBTRACEEVENT_FLAGS) $@
 
 install-traceevent-plugins: libtraceevent_plugins
-	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) install_plugins
+	$(Q)$(MAKE) -C $(LIBTRACEEVENT_PLUGINS_DIR) O=$(LIBTRACEEVENT_PLUGINS_OUTPUT) \
+		DESTDIR=$(LIBTRACEEVENT_PLUGINS_DESTDIR) prefix= \
+		$(LIBTRACEEVENT_FLAGS) install
 
 $(LIBAPI): FORCE | $(LIBAPI_OUTPUT)
 	$(Q)$(MAKE) -C $(LIBAPI_DIR) O=$(LIBAPI_OUTPUT) \
@@ -1065,6 +1079,11 @@ SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h
 $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT):
 	$(Q)$(MKDIR) -p $@
 
+ifndef LIBTRACEEVENT_DYNAMIC
+$(LIBTRACEEVENT_OUTPUT) $(LIBTRACEEVENT_PLUGINS_OUTPUT):
+	$(Q)$(MKDIR) -p $@
+endif
+
 ifdef BUILD_BPF_SKEL
 BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool
 BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE)
@@ -1107,7 +1126,7 @@ endif # BUILD_BPF_SKEL
 bpf-skel-clean:
 	$(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
 
-clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
+clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean libtraceevent_plugins-clean
 	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
 	$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT).config-detected
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 07/14] tools lib api: Add missing install headers
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (5 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 06/14] perf build: Install libtraceevent " Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 08/14] tools lib perf: " Ian Rogers
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

Headers necessary for the perf build.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/api/Makefile | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/lib/api/Makefile b/tools/lib/api/Makefile
index 6629d0fd0130..3e5ef1e0e890 100644
--- a/tools/lib/api/Makefile
+++ b/tools/lib/api/Makefile
@@ -103,7 +103,10 @@ install_headers:
 	$(call QUIET_INSTALL, headers) \
 		$(call do_install,cpu.h,$(prefix)/include/api,644); \
 		$(call do_install,debug.h,$(prefix)/include/api,644); \
-		$(call do_install,io.h,$(prefix)/include/api,644);
+		$(call do_install,io.h,$(prefix)/include/api,644); \
+		$(call do_install,fd/array.h,$(prefix)/include/api/fd,644); \
+		$(call do_install,fs/fs.h,$(prefix)/include/api/fs,644);
+		$(call do_install,fs/tracing_path.h,$(prefix)/include/api/fs,644);
 
 install: install_lib install_headers
 
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 08/14] tools lib perf: Add missing install headers
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (6 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 07/14] tools lib api: Add missing install headers Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 09/14] tool lib symbol: Add Makefile/Build Ian Rogers
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

Headers necessary for the perf build. Note, internal headers are also
installed as these are necessary for the build.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/perf/Makefile | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tools/lib/perf/Makefile b/tools/lib/perf/Makefile
index 21df023a2103..1badc0a04676 100644
--- a/tools/lib/perf/Makefile
+++ b/tools/lib/perf/Makefile
@@ -189,13 +189,21 @@ install_lib: libs
 
 install_headers:
 	$(call QUIET_INSTALL, headers) \
+		$(call do_install,include/perf/bpf_perf.h,$(prefix)/include/perf,644); \
 		$(call do_install,include/perf/core.h,$(prefix)/include/perf,644); \
 		$(call do_install,include/perf/cpumap.h,$(prefix)/include/perf,644); \
 		$(call do_install,include/perf/threadmap.h,$(prefix)/include/perf,644); \
 		$(call do_install,include/perf/evlist.h,$(prefix)/include/perf,644); \
 		$(call do_install,include/perf/evsel.h,$(prefix)/include/perf,644); \
 		$(call do_install,include/perf/event.h,$(prefix)/include/perf,644); \
-		$(call do_install,include/perf/mmap.h,$(prefix)/include/perf,644);
+		$(call do_install,include/perf/mmap.h,$(prefix)/include/perf,644); \
+		$(call do_install,include/internal/cpumap.h,$(prefix)/include/internal,644); \
+		$(call do_install,include/internal/evlist.h,$(prefix)/include/internal,644); \
+		$(call do_install,include/internal/evsel.h,$(prefix)/include/internal,644); \
+		$(call do_install,include/internal/lib.h,$(prefix)/include/internal,644); \
+		$(call do_install,include/internal/mmap.h,$(prefix)/include/internal,644); \
+		$(call do_install,include/internal/threadmap.h,$(prefix)/include/internal,644); \
+		$(call do_install,include/internal/xyarray.h,$(prefix)/include/internal,644);
 
 install_pkgconfig: $(LIBPERF_PC)
 	$(call QUIET_INSTALL, $(LIBPERF_PC)) \
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 09/14] tool lib symbol: Add Makefile/Build
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (7 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 08/14] tools lib perf: " Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 10/14] perf build: Install libsymbol locally when building Ian Rogers
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

Add sufficient Makefile for libsymbol to be built as a dependency and
header files installed.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/symbol/Build    |   1 +
 tools/lib/symbol/Makefile | 115 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 116 insertions(+)
 create mode 100644 tools/lib/symbol/Build
 create mode 100644 tools/lib/symbol/Makefile

diff --git a/tools/lib/symbol/Build b/tools/lib/symbol/Build
new file mode 100644
index 000000000000..9b9a9c78d3c9
--- /dev/null
+++ b/tools/lib/symbol/Build
@@ -0,0 +1 @@
+libsymbol-y += kallsyms.o
diff --git a/tools/lib/symbol/Makefile b/tools/lib/symbol/Makefile
new file mode 100644
index 000000000000..4c1d6b53032d
--- /dev/null
+++ b/tools/lib/symbol/Makefile
@@ -0,0 +1,115 @@
+# SPDX-License-Identifier: GPL-2.0
+include ../../scripts/Makefile.include
+include ../../scripts/utilities.mak		# QUIET_CLEAN
+
+ifeq ($(srctree),)
+srctree := $(patsubst %/,%,$(dir $(CURDIR)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+#$(info Determined 'srctree' to be $(srctree))
+endif
+
+CC ?= $(CROSS_COMPILE)gcc
+AR ?= $(CROSS_COMPILE)ar
+LD ?= $(CROSS_COMPILE)ld
+
+MAKEFLAGS += --no-print-directory
+
+INSTALL = install
+
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
+LIBFILE = $(OUTPUT)libsymbol.a
+
+CFLAGS := $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
+CFLAGS += -ggdb3 -Wall -Wextra -std=gnu11 -U_FORTIFY_SOURCE -fPIC
+
+ifeq ($(DEBUG),0)
+ifeq ($(CC_NO_CLANG), 0)
+  CFLAGS += -O3
+else
+  CFLAGS += -O6
+endif
+endif
+
+ifeq ($(DEBUG),0)
+  CFLAGS += -D_FORTIFY_SOURCE
+endif
+
+# Treat warnings as errors unless directed not to
+ifneq ($(WERROR),0)
+  CFLAGS += -Werror
+endif
+
+CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
+
+CFLAGS += -I$(srctree)/tools/lib
+CFLAGS += -I$(srctree)/tools/include
+
+RM = rm -f
+
+SYMBOL_IN := $(OUTPUT)libsymbol-in.o
+
+ifeq ($(LP64), 1)
+  libdir_relative = lib64
+else
+  libdir_relative = lib
+endif
+
+prefix ?=
+libdir = $(prefix)/$(libdir_relative)
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+
+all:
+
+export srctree OUTPUT CC LD CFLAGS V
+include $(srctree)/tools/build/Makefile.include
+include $(srctree)/tools/scripts/Makefile.include
+
+all: fixdep $(LIBFILE)
+
+$(SYMBOL_IN): FORCE
+	$(MAKE) $(build)=libsymbol V=1
+
+$(LIBFILE): $(SYMBOL_IN)
+	$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(SYMBOL_IN)
+
+define do_install_mkdir
+	if [ ! -d '$(DESTDIR_SQ)$1' ]; then             \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
+	fi
+endef
+
+define do_install
+	if [ ! -d '$(DESTDIR_SQ)$2' ]; then             \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
+	fi;                                             \
+	$(INSTALL) $1 $(if $3,-m $3,) '$(DESTDIR_SQ)$2'
+endef
+
+install_lib: $(LIBFILE)
+	$(call QUIET_INSTALL, $(LIBFILE)) \
+		$(call do_install_mkdir,$(libdir_SQ)); \
+		cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ)
+
+install_headers:
+	$(call QUIET_INSTALL, headers) \
+		$(call do_install,kallsyms.h,$(prefix)/include/symbol,644);
+
+install: install_lib install_headers
+
+clean:
+	$(call QUIET_CLEAN, libsymbol) $(RM) $(LIBFILE); \
+	find $(or $(OUTPUT),.) -name \*.o -or -name \*.o.cmd -or -name \*.o.d | xargs $(RM)
+
+FORCE:
+
+.PHONY: clean FORCE
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 10/14] perf build: Install libsymbol locally when building
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (8 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 09/14] tool lib symbol: Add Makefile/Build Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 11/14] perf expr: Tidy hashmap dependency Ian Rogers
                   ` (4 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

The perf build currently has a '-Itools/lib' on the CC command
line. This causes issues as the libapi, libsubcmd, libtraceevent,
libbpf and libsymbol headers are all found via this path, making it
impossible to override include behavior. Change the libsymbol build
mirroring the libbpf, libsubcmd, libapi, libperf and libtraceevent
build, so that it is installed in a directory along with its
headers. A later change will modify the include behavior. Don't build
kallsyms.o as part of util as this will lead to duplicate definitions.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/.gitignore    |  1 +
 tools/perf/Makefile.perf | 25 ++++++++++++++++++++++---
 tools/perf/util/Build    |  5 -----
 3 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 65b995159cf1..675c03ec1f1c 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -42,6 +42,7 @@ libapi/
 libbpf/
 libperf/
 libsubcmd/
+libsymbol/
 libtraceevent/
 libtraceevent_plugins/
 fixdep
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 537ac7055a35..1d90d78303b4 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -246,6 +246,7 @@ LIBTRACEEVENT_DIR = $(srctree)/tools/lib/traceevent/
 LIBTRACEEVENT_PLUGINS_DIR = $(LIBTRACEEVENT_DIR)/plugins
 LIBBPF_DIR      = $(srctree)/tools/lib/bpf/
 LIBSUBCMD_DIR   = $(srctree)/tools/lib/subcmd/
+LIBSYMBOL_DIR   = $(srctree)/tools/lib/symbol/
 LIBPERF_DIR     = $(srctree)/tools/lib/perf/
 DOC_DIR         = $(srctree)/tools/perf/Documentation/
 
@@ -340,6 +341,15 @@ LIBSUBCMD_DESTDIR = $(LIBSUBCMD_OUTPUT)
 LIBSUBCMD_INCLUDE = $(LIBSUBCMD_DESTDIR)/include
 LIBSUBCMD = $(LIBSUBCMD_OUTPUT)/libsubcmd.a
 
+ifneq ($(OUTPUT),)
+  LIBSYMBOL_OUTPUT = $(abspath $(OUTPUT))/libsymbol
+else
+  LIBSYMBOL_OUTPUT = $(CURDIR)/libsymbol
+endif
+LIBSYMBOL_DESTDIR = $(LIBSYMBOL_OUTPUT)
+LIBSYMBOL_INCLUDE = $(LIBSYMBOL_DESTDIR)/include
+LIBSYMBOL = $(LIBSYMBOL_OUTPUT)/libsymbol.a
+
 ifneq ($(OUTPUT),)
   LIBPERF_OUTPUT = $(abspath $(OUTPUT))/libperf
 else
@@ -398,7 +408,7 @@ endif
 
 export PERL_PATH
 
-PERFLIBS = $(LIBAPI) $(LIBSUBCMD) $(LIBPERF)
+PERFLIBS = $(LIBAPI) $(LIBPERF) $(LIBSUBCMD) $(LIBSYMBOL)
 ifndef NO_LIBBPF
   ifndef LIBBPF_DYNAMIC
     PERFLIBS += $(LIBBPF)
@@ -887,6 +897,15 @@ $(LIBSUBCMD)-clean:
 	$(call QUIET_CLEAN, libsubcmd)
 	$(Q)$(RM) -r -- $(LIBSUBCMD_OUTPUT)
 
+$(LIBSYMBOL): FORCE | $(LIBSYMBOL_OUTPUT)
+	$(Q)$(MAKE) -C $(LIBSYMBOL_DIR) O=$(LIBSYMBOL_OUTPUT) \
+		DESTDIR=$(LIBSYMBOL_DESTDIR) prefix= \
+		$@ install_headers
+
+$(LIBSYMBOL)-clean:
+	$(call QUIET_CLEAN, libsymbol)
+	$(Q)$(RM) -r -- $(LIBSYMBOL_OUTPUT)
+
 help:
 	@echo 'Perf make targets:'
 	@echo '  doc		- make *all* documentation (see below)'
@@ -1076,7 +1095,7 @@ SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h
 SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h
 SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h
 
-$(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT):
+$(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT):
 	$(Q)$(MKDIR) -p $@
 
 ifndef LIBTRACEEVENT_DYNAMIC
@@ -1126,7 +1145,7 @@ endif # BUILD_BPF_SKEL
 bpf-skel-clean:
 	$(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
 
-clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean libtraceevent_plugins-clean
+clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean libtraceevent_plugins-clean
 	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
 	$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT).config-detected
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index e315ecaec323..45f3d2723725 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -19,7 +19,6 @@ perf-y += perf_event_attr_fprintf.o
 perf-y += evswitch.o
 perf-y += find_bit.o
 perf-y += get_current_dir_name.o
-perf-y += kallsyms.o
 perf-y += levenshtein.o
 perf-y += llvm-utils.o
 perf-y += mmap.o
@@ -294,10 +293,6 @@ CFLAGS_expr.o          += -Wno-redundant-decls
 CFLAGS_header.o        += -include $(OUTPUT)PERF-VERSION-FILE
 CFLAGS_arm-spe.o       += -I$(srctree)/tools/arch/arm64/include/
 
-$(OUTPUT)util/kallsyms.o: ../lib/symbol/kallsyms.c FORCE
-	$(call rule_mkdir)
-	$(call if_changed_dep,cc_o_c)
-
 $(OUTPUT)util/argv_split.o: ../lib/argv_split.c FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,cc_o_c)
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 11/14] perf expr: Tidy hashmap dependency
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (9 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 10/14] perf build: Install libsymbol locally when building Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 12/14] perf thread_map: Reduce exposure of libperf internal API Ian Rogers
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

hashmap.h comes from libbpf but isn't installed with its
headers. Always use the header file of the code in util. Change the
hashmap.h dependency in expr.h to a forward declaration, add the
necessary header file includes in the C files.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/expr.c       | 1 +
 tools/perf/tests/pmu-events.c | 1 +
 tools/perf/util/bpf-loader.c  | 4 ----
 tools/perf/util/evsel.c       | 4 ----
 tools/perf/util/expr.c        | 1 +
 tools/perf/util/expr.h        | 7 +------
 tools/perf/util/metricgroup.c | 1 +
 tools/perf/util/stat-shadow.c | 1 +
 tools/perf/util/stat.c        | 4 ----
 9 files changed, 6 insertions(+), 18 deletions(-)

diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index 6512f5e22045..b6667501ebb4 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -2,6 +2,7 @@
 #include "util/cputopo.h"
 #include "util/debug.h"
 #include "util/expr.h"
+#include "util/hashmap.h"
 #include "util/header.h"
 #include "util/smt.h"
 #include "tests.h"
diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index 5d0d3b239a68..f7b9dbbad97f 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -12,6 +12,7 @@
 #include <perf/evlist.h>
 #include "util/evlist.h"
 #include "util/expr.h"
+#include "util/hashmap.h"
 #include "util/parse-events.h"
 #include "metricgroup.h"
 #include "stat.h"
diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index f4adeccdbbcb..b3c8174360bf 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -27,11 +27,7 @@
 #include "util.h"
 #include "llvm-utils.h"
 #include "c++/clang-c.h"
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
 #include "util/hashmap.h"
-#endif
 #include "asm/bug.h"
 
 #include <internal/xyarray.h>
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index cdde5b5f8ad2..2139e8b0e401 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -46,11 +46,7 @@
 #include "string2.h"
 #include "memswap.h"
 #include "util.h"
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
 #include "util/hashmap.h"
-#endif
 #include "pmu-hybrid.h"
 #include "off_cpu.h"
 #include "../perf-sys.h"
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index aaacf514dc09..140f2acdb325 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -11,6 +11,7 @@
 #include "expr.h"
 #include "expr-bison.h"
 #include "expr-flex.h"
+#include "util/hashmap.h"
 #include "smt.h"
 #include "tsc.h"
 #include <linux/err.h>
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index d6c1668dc1a0..029271540fb0 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -2,12 +2,7 @@
 #ifndef PARSE_CTX_H
 #define PARSE_CTX_H 1
 
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
-#include "util/hashmap.h"
-#endif
-
+struct hashmap;
 struct metric_ref;
 
 struct expr_scanner_ctx {
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 4c98ac29ee13..15441e9bfb73 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -28,6 +28,7 @@
 #include "util.h"
 #include <asm/bug.h>
 #include "cgroup.h"
+#include "util/hashmap.h"
 
 struct metric_event *metricgroup__lookup(struct rblist *metric_events,
 					 struct evsel *evsel,
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 07b29fe272c7..9bde9224a97c 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -14,6 +14,7 @@
 #include "units.h"
 #include <linux/zalloc.h>
 #include "iostat.h"
+#include "util/hashmap.h"
 
 /*
  * AGGR_GLOBAL: Use CPU 0
diff --git a/tools/perf/util/stat.c b/tools/perf/util/stat.c
index 3a432a949d46..273a5b32e815 100644
--- a/tools/perf/util/stat.c
+++ b/tools/perf/util/stat.c
@@ -14,11 +14,7 @@
 #include "evlist.h"
 #include "evsel.h"
 #include "thread_map.h"
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
 #include "util/hashmap.h"
-#endif
 #include <linux/zalloc.h>
 
 void update_stats(struct stats *stats, u64 val)
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 12/14] perf thread_map: Reduce exposure of libperf internal API
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (10 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 11/14] perf expr: Tidy hashmap dependency Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 13/14] perf cpumap: Tidy libperf includes Ian Rogers
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

Remove unnecessary include of internal threadmap.h and refcount.h in
thread_map.h. Switch to using public APIs when possible or including
the internal header file in the C file. Fix a transitive dependency in
openat-syscall.c broken by the clean up.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-stat.c                              | 1 +
 tools/perf/builtin-trace.c                             | 4 ++--
 tools/perf/tests/openat-syscall.c                      | 1 +
 tools/perf/tests/thread-map.c                          | 1 +
 tools/perf/util/bpf_counter.c                          | 2 +-
 tools/perf/util/evsel.c                                | 1 +
 tools/perf/util/python.c                               | 6 +++---
 tools/perf/util/scripting-engines/trace-event-python.c | 2 +-
 tools/perf/util/thread_map.c                           | 1 +
 tools/perf/util/thread_map.h                           | 2 --
 10 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index d5e1670bca20..fb2d13650e42 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -93,6 +93,7 @@
 
 #include <linux/ctype.h>
 #include <perf/evlist.h>
+#include <internal/threadmap.h>
 
 #define DEFAULT_SEPARATOR	" "
 #define FREEZE_ON_SMI_PATH	"devices/cpu/freeze_on_smi"
diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 72991528687e..b403f3fab2a4 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -4092,8 +4092,8 @@ static int trace__run(struct trace *trace, int argc, const char **argv)
 	}
 
 	trace->multiple_threads = perf_thread_map__pid(evlist->core.threads, 0) == -1 ||
-				  evlist->core.threads->nr > 1 ||
-				  evlist__first(evlist)->core.attr.inherit;
+		perf_thread_map__nr(evlist->core.threads) > 1 ||
+		evlist__first(evlist)->core.attr.inherit;
 
 	/*
 	 * Now that we already used evsel->core.attr to ask the kernel to setup the
diff --git a/tools/perf/tests/openat-syscall.c b/tools/perf/tests/openat-syscall.c
index 7e05b8b5cc95..131b62271bfa 100644
--- a/tools/perf/tests/openat-syscall.c
+++ b/tools/perf/tests/openat-syscall.c
@@ -7,6 +7,7 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <fcntl.h>
+#include <unistd.h>
 #include "thread_map.h"
 #include "evsel.h"
 #include "debug.h"
diff --git a/tools/perf/tests/thread-map.c b/tools/perf/tests/thread-map.c
index e413c1387fcb..74308c1368fe 100644
--- a/tools/perf/tests/thread-map.c
+++ b/tools/perf/tests/thread-map.c
@@ -11,6 +11,7 @@
 #include "util/synthetic-events.h"
 #include <linux/zalloc.h>
 #include <perf/event.h>
+#include <internal/threadmap.h>
 
 struct perf_sample;
 struct perf_tool;
diff --git a/tools/perf/util/bpf_counter.c b/tools/perf/util/bpf_counter.c
index ef1c15e4aeba..eeee899fcf34 100644
--- a/tools/perf/util/bpf_counter.c
+++ b/tools/perf/util/bpf_counter.c
@@ -561,7 +561,7 @@ static int bperf__load(struct evsel *evsel, struct target *target)
 
 		if (filter_type == BPERF_FILTER_PID ||
 		    filter_type == BPERF_FILTER_TGID)
-			key = evsel->core.threads->map[i].pid;
+			key = perf_thread_map__pid(evsel->core.threads, i);
 		else if (filter_type == BPERF_FILTER_CPU)
 			key = evsel->core.cpus->map[i].cpu;
 		else
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 2139e8b0e401..2e212ae73b9f 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -53,6 +53,7 @@
 #include "util/parse-branch-options.h"
 #include <internal/xyarray.h>
 #include <internal/lib.h>
+#include <internal/threadmap.h>
 
 #include <linux/ctype.h>
 
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 5be5fa2391de..b5941c74a0d6 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -718,17 +718,17 @@ static Py_ssize_t pyrf_thread_map__length(PyObject *obj)
 {
 	struct pyrf_thread_map *pthreads = (void *)obj;
 
-	return pthreads->threads->nr;
+	return perf_thread_map__nr(pthreads->threads);
 }
 
 static PyObject *pyrf_thread_map__item(PyObject *obj, Py_ssize_t i)
 {
 	struct pyrf_thread_map *pthreads = (void *)obj;
 
-	if (i >= pthreads->threads->nr)
+	if (i >= perf_thread_map__nr(pthreads->threads))
 		return NULL;
 
-	return Py_BuildValue("i", pthreads->threads->map[i]);
+	return Py_BuildValue("i", perf_thread_map__pid(pthreads->threads, i));
 }
 
 static PySequenceMethods pyrf_thread_map__sequence_methods = {
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 1985d1a42a22..1cf65db8f861 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -1654,7 +1654,7 @@ static void python_process_stat(struct perf_stat_config *config,
 	struct perf_cpu_map *cpus = counter->core.cpus;
 	int cpu, thread;
 
-	for (thread = 0; thread < threads->nr; thread++) {
+	for (thread = 0; thread < perf_thread_map__nr(threads); thread++) {
 		for (cpu = 0; cpu < perf_cpu_map__nr(cpus); cpu++) {
 			process_stat(counter, perf_cpu_map__cpu(cpus, cpu),
 				     perf_thread_map__pid(threads, thread), tstamp,
diff --git a/tools/perf/util/thread_map.c b/tools/perf/util/thread_map.c
index c9bfe4696943..e848579e61a8 100644
--- a/tools/perf/util/thread_map.c
+++ b/tools/perf/util/thread_map.c
@@ -18,6 +18,7 @@
 #include "thread_map.h"
 #include "debug.h"
 #include "event.h"
+#include <internal/threadmap.h>
 
 /* Skip "." and ".." directories */
 static int filter(const struct dirent *dir)
diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h
index 3bb860a32b8e..00ec05fc1656 100644
--- a/tools/perf/util/thread_map.h
+++ b/tools/perf/util/thread_map.h
@@ -4,8 +4,6 @@
 
 #include <sys/types.h>
 #include <stdio.h>
-#include <linux/refcount.h>
-#include <internal/threadmap.h>
 #include <perf/threadmap.h>
 
 struct perf_record_thread_map;
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 13/14] perf cpumap: Tidy libperf includes
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (11 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 12/14] perf thread_map: Reduce exposure of libperf internal API Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-08  7:35 ` [PATCH v1 14/14] perf build: Use tools/lib headers from install path Ian Rogers
  2022-11-09 15:27 ` [PATCH v1 00/14] Fix perf tools/lib includes Arnaldo Carvalho de Melo
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

Use public API when possible, don't include internal API in header
files in evsel.h. Fix any related breakages.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/cpumap.c  | 2 +-
 tools/perf/util/auxtrace.h | 2 +-
 tools/perf/util/cpumap.c   | 1 +
 tools/perf/util/cpumap.h   | 2 +-
 tools/perf/util/evsel.h    | 2 --
 5 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/perf/tests/cpumap.c b/tools/perf/tests/cpumap.c
index 7c873c6ae3eb..3150fc1fed6f 100644
--- a/tools/perf/tests/cpumap.c
+++ b/tools/perf/tests/cpumap.c
@@ -6,7 +6,7 @@
 #include "util/synthetic-events.h"
 #include <string.h>
 #include <linux/bitops.h>
-#include <perf/cpumap.h>
+#include <internal/cpumap.h>
 #include "debug.h"
 
 struct machine;
diff --git a/tools/perf/util/auxtrace.h b/tools/perf/util/auxtrace.h
index 6a0f9b98f059..2cf63d377831 100644
--- a/tools/perf/util/auxtrace.h
+++ b/tools/perf/util/auxtrace.h
@@ -15,7 +15,7 @@
 #include <linux/list.h>
 #include <linux/perf_event.h>
 #include <linux/types.h>
-#include <internal/cpumap.h>
+#include <perf/cpumap.h>
 #include <asm/bitsperlong.h>
 #include <asm/barrier.h>
 
diff --git a/tools/perf/util/cpumap.c b/tools/perf/util/cpumap.c
index 6e3fcf523de9..5e564974fba4 100644
--- a/tools/perf/util/cpumap.c
+++ b/tools/perf/util/cpumap.c
@@ -12,6 +12,7 @@
 
 #include <linux/ctype.h>
 #include <linux/zalloc.h>
+#include <internal/cpumap.h>
 
 static struct perf_cpu max_cpu_num;
 static struct perf_cpu max_present_cpu_num;
diff --git a/tools/perf/util/cpumap.h b/tools/perf/util/cpumap.h
index da28b3146ef9..c2f5824a3a22 100644
--- a/tools/perf/util/cpumap.h
+++ b/tools/perf/util/cpumap.h
@@ -4,8 +4,8 @@
 
 #include <stdbool.h>
 #include <stdio.h>
-#include <internal/cpumap.h>
 #include <perf/cpumap.h>
+#include <linux/refcount.h>
 
 /** Identify where counts are aggregated, -1 implies not to aggregate. */
 struct aggr_cpu_id {
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 989865e16aad..f5d9f6a351cd 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -10,8 +10,6 @@
 #include <internal/evsel.h>
 #include <perf/evsel.h>
 #include "symbol_conf.h"
-#include <internal/cpumap.h>
-#include <perf/cpumap.h>
 
 struct bpf_object;
 struct cgroup;
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH v1 14/14] perf build: Use tools/lib headers from install path
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (12 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 13/14] perf cpumap: Tidy libperf includes Ian Rogers
@ 2022-11-08  7:35 ` Ian Rogers
  2022-11-09 15:27 ` [PATCH v1 00/14] Fix perf tools/lib includes Arnaldo Carvalho de Melo
  14 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-08  7:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Masahiro Yamada, Nicolas Schier, linux-kernel, linux-perf-users,
	bpf, Nick Desaulniers, Andrii Nakryiko
  Cc: Stephane Eranian, Ian Rogers

Switch -I from tools/lib to the install path for the tools/lib
libraries that are depended upon.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.config | 2 --
 tools/perf/Makefile.perf   | 6 ++++++
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index d3d3c13a9f25..d7fcd1624cd7 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -349,7 +349,6 @@ ifeq ($(DEBUG),0)
   endif
 endif
 
-INC_FLAGS += -I$(srctree)/tools/lib/perf/include
 INC_FLAGS += -I$(src-perf)/util/include
 INC_FLAGS += -I$(src-perf)/arch/$(SRCARCH)/include
 INC_FLAGS += -I$(srctree)/tools/include/
@@ -367,7 +366,6 @@ endif
 
 INC_FLAGS += -I$(src-perf)/util
 INC_FLAGS += -I$(src-perf)
-INC_FLAGS += -I$(srctree)/tools/lib/
 
 CORE_CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
 
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 1d90d78303b4..851dcc0272e5 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -305,6 +305,7 @@ LIBTRACEEVENT_INCLUDE = $(LIBTRACEEVENT_DESTDIR)/include
 LIBTRACEEVENT = $(LIBTRACEEVENT_OUTPUT)/libtraceevent.a
 export LIBTRACEEVENT
 LIBTRACEEVENT_DYNAMIC_LIST = $(LIBTRACEEVENT_PLUGINS_OUTPUT)/libtraceevent-dynamic-list
+CFLAGS += -I$(LIBTRACEEVENT_OUTPUT)/include
 
 #
 # The static build has no dynsym table, so this does not work for
@@ -322,6 +323,7 @@ LIBAPI_DESTDIR = $(LIBAPI_OUTPUT)
 LIBAPI_INCLUDE = $(LIBAPI_DESTDIR)/include
 LIBAPI = $(LIBAPI_OUTPUT)/libapi.a
 export LIBAPI
+CFLAGS += -I$(LIBAPI_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBBPF_OUTPUT = $(abspath $(OUTPUT))/libbpf
@@ -331,6 +333,7 @@ endif
 LIBBPF_DESTDIR = $(LIBBPF_OUTPUT)
 LIBBPF_INCLUDE = $(LIBBPF_DESTDIR)/include
 LIBBPF = $(LIBBPF_OUTPUT)/libbpf.a
+CFLAGS += -I$(LIBBPF_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBSUBCMD_OUTPUT = $(abspath $(OUTPUT))/libsubcmd
@@ -340,6 +343,7 @@ endif
 LIBSUBCMD_DESTDIR = $(LIBSUBCMD_OUTPUT)
 LIBSUBCMD_INCLUDE = $(LIBSUBCMD_DESTDIR)/include
 LIBSUBCMD = $(LIBSUBCMD_OUTPUT)/libsubcmd.a
+CFLAGS += -I$(LIBSUBCMD_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBSYMBOL_OUTPUT = $(abspath $(OUTPUT))/libsymbol
@@ -349,6 +353,7 @@ endif
 LIBSYMBOL_DESTDIR = $(LIBSYMBOL_OUTPUT)
 LIBSYMBOL_INCLUDE = $(LIBSYMBOL_DESTDIR)/include
 LIBSYMBOL = $(LIBSYMBOL_OUTPUT)/libsymbol.a
+CFLAGS += -I$(LIBSYMBOL_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBPERF_OUTPUT = $(abspath $(OUTPUT))/libperf
@@ -359,6 +364,7 @@ LIBPERF_DESTDIR = $(LIBPERF_OUTPUT)
 LIBPERF_INCLUDE = $(LIBPERF_DESTDIR)/include
 LIBPERF = $(LIBPERF_OUTPUT)/libperf.a
 export LIBPERF
+CFLAGS += -I$(LIBPERF_OUTPUT)/include
 
 # python extension build directories
 PYTHON_EXTBUILD     := $(OUTPUT)python_ext_build/
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* Re: [PATCH v1 00/14] Fix perf tools/lib includes
  2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
                   ` (13 preceding siblings ...)
  2022-11-08  7:35 ` [PATCH v1 14/14] perf build: Use tools/lib headers from install path Ian Rogers
@ 2022-11-09 15:27 ` Arnaldo Carvalho de Melo
  2022-11-09 16:52   ` Ian Rogers
  2022-11-09 18:00   ` Arnaldo Carvalho de Melo
  14 siblings, 2 replies; 21+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-11-09 15:27 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Masahiro Yamada, Nicolas Schier,
	linux-kernel, linux-perf-users, bpf, Nick Desaulniers,
	Andrii Nakryiko, Stephane Eranian

Em Mon, Nov 07, 2022 at 11:35:04PM -0800, Ian Rogers escreveu:
> The previous build would add -Itools/lib and get dependencies for
> libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
> overriding these libraries would change the link time dependency but
> the headers would erroneously come from tools/lib. Fix the build to
> install headers and then depend on these. To reduce exposing internal
> headers/APIs some clean up is performed. tools/lib/symbol has a
> Makefile added, while tools/lib/api and tools/lib/subcmd have install
> targets added. The pattern used for the dependencies in Makefile.perf
> is modelled on libbpf.

It builds with O=, I tried it one by one, but  there are problems with
make from a detached tarball, that is how I do the container builds, see
below, I'm trying to figure this out...

⬢[acme@toolbox perf]$ make perf-tar-src-pkg
  TAR
  PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
⬢[acme@toolbox perf]$ mv perf-6.1.0-rc3.tar /tmp
⬢[acme@toolbox perf]$ cd /tmp
⬢[acme@toolbox tmp]$ tar xf perf-6.1.0-rc3.tar
⬢[acme@toolbox tmp]$ cd perf-6.1.0-rc3/
⬢[acme@toolbox perf-6.1.0-rc3]$ make -C tools/perf
make: Entering directory '/tmp/perf-6.1.0-rc3/tools/perf'
  BUILD:   Doing 'make -j32' parallel build
  HOSTCC  fixdep.o
  HOSTLD  fixdep-in.o
  LINK    fixdep

Auto-detecting system features:
...                                   dwarf: [ on  ]
...                      dwarf_getlocations: [ on  ]
...                                   glibc: [ on  ]
...                                  libbfd: [ on  ]
...                          libbfd-buildid: [ on  ]
...                                  libcap: [ on  ]
...                                  libelf: [ on  ]
...                                 libnuma: [ on  ]
...                  numa_num_possible_cpus: [ on  ]
...                                 libperl: [ on  ]
...                               libpython: [ on  ]
...                               libcrypto: [ on  ]
...                               libunwind: [ on  ]
...                      libdw-dwarf-unwind: [ on  ]
...                                    zlib: [ on  ]
...                                    lzma: [ on  ]
...                               get_cpuid: [ on  ]
...                                     bpf: [ on  ]
...                                  libaio: [ on  ]
...                                 libzstd: [ on  ]

  GEN     common-cmds.h
  PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
  CC      perf-read-vdso32
  GEN     perf-archive
  GEN     perf-iostat
  CC      dlfilters/dlfilter-test-api-v0.o
  CC      dlfilters/dlfilter-show-cycles.o
  CC      jvmti/libjvmti.o
make[3]: *** No rule to make target '/tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a'.  Stop.
make[2]: *** [Makefile.perf:907: /tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a] Error 2
make[2]: *** Waiting for unfinished jobs....
  CC      jvmti/jvmti_agent.o
  CC      jvmti/libstring.o
  CC      jvmti/libctype.o
  GEN     pmu-events/pmu-events.c
  INSTALL headers
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/exec-cmd.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/help.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/pager.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/parse-options.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/run-command.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/sigchain.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/subcmd-config.o
  INSTALL headers
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.o
  INSTALL headers
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/core.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/cpu.o
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/array.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/fs.o
  GEN     /tmp/perf-6.1.0-rc3/tools/perf/libbpf/bpf_helper_defs.h
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/tracing_path.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/cgroup.o
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf.h
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/debug.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2-in.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer-in.o
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf.h
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem-in.o
  INSTALL headers
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/btf.h
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211-in.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch-in.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/cpumap.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/threadmap.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/str_error_r.o
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_common.h
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.so
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.so
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_legacy.h
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function-in.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evsel.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm-in.o
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.so
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.so
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.so
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helpers.h
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_tracing.h
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-plugin.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evlist.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/mmap.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/trace-seq.o
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_endian.h
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_core_read.h
if [ ! -d ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs' ]; then install -d -m 755 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs'; fi; install fs/tracing_path.h -m 644 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs';
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/libapi-in.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/zalloc.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-filter.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-utils.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/kbuffer-parse.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen-in.o
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/skel_internal.h
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_version.h
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/tep_strerror.o
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/usdt.bpf.h
  INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helper_defs.h
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse-api.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/xyarray.o
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.so
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.so
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex-in.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/lib.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi-in.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211-in.o
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.so
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf.o
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.so
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.so
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/libapi-in.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/nlattr.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf.o
  MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb-in.o
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.so
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/str_error.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/netlink.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi-in.o
  LD      jvmti/jvmti-in.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_errno.o
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.so
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf_prog_linfo.o
  AR      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi.a
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_probes.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/hashmap.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf_dump.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/ringbuf.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/strset.o
  GEN     /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/libtraceevent-dynamic-list
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/linker.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/gen_loader.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/relo_core.o
  CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/usdt.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd-in.o
  AR      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd.a
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf-in.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent-in.o
  AR      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf.a
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent.a
  CC      pmu-events/pmu-events.o
  LD      pmu-events/pmu-events-in.o
  LD      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf-in.o
  LINK    /tmp/perf-6.1.0-rc3/tools/perf/libbpf/libbpf.a
make[1]: *** [Makefile.perf:240: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
make: Leaving directory '/tmp/perf-6.1.0-rc3/tools/perf'
⬢[acme@toolbox perf-6.1.0-rc3]$

> The problem and solution were motivated by this issue and discussion:
> https://lore.kernel.org/lkml/CAEf4BzbbOHQZUAe6iWaehKCPQAf3VC=hq657buqe2_yRKxaK-A@mail.gmail.com/
> 
> Ian Rogers (14):
>   tools lib api: Add install target
>   tools lib subcmd: Add install target
>   perf build: Install libsubcmd locally when building
>   perf build: Install libapi locally when building
>   perf build: Install libperf locally when building
>   perf build: Install libtraceevent locally when building
>   tools lib api: Add missing install headers
>   tools lib perf: Add missing install headers
>   tool lib symbol: Add Makefile/Build
>   perf build: Install libsymbol locally when building
>   perf expr: Tidy hashmap dependency
>   perf thread_map: Reduce exposure of libperf internal API
>   perf cpumap: Tidy libperf includes
>   perf build: Use tools/lib headers from install path
> 
>  tools/lib/api/Makefile                        |  52 ++++++
>  tools/lib/perf/Makefile                       |  10 +-
>  tools/lib/subcmd/Makefile                     |  49 ++++++
>  tools/lib/symbol/Build                        |   1 +
>  tools/lib/symbol/Makefile                     | 115 +++++++++++++
>  tools/perf/.gitignore                         |   7 +-
>  tools/perf/Makefile.config                    |   2 -
>  tools/perf/Makefile.perf                      | 152 ++++++++++++------
>  tools/perf/builtin-stat.c                     |   1 +
>  tools/perf/builtin-trace.c                    |   4 +-
>  tools/perf/tests/cpumap.c                     |   2 +-
>  tools/perf/tests/expr.c                       |   1 +
>  tools/perf/tests/openat-syscall.c             |   1 +
>  tools/perf/tests/pmu-events.c                 |   1 +
>  tools/perf/tests/thread-map.c                 |   1 +
>  tools/perf/util/Build                         |   5 -
>  tools/perf/util/auxtrace.h                    |   2 +-
>  tools/perf/util/bpf-loader.c                  |   4 -
>  tools/perf/util/bpf_counter.c                 |   2 +-
>  tools/perf/util/cpumap.c                      |   1 +
>  tools/perf/util/cpumap.h                      |   2 +-
>  tools/perf/util/evsel.c                       |   5 +-
>  tools/perf/util/evsel.h                       |   2 -
>  tools/perf/util/expr.c                        |   1 +
>  tools/perf/util/expr.h                        |   7 +-
>  tools/perf/util/metricgroup.c                 |   1 +
>  tools/perf/util/python.c                      |   6 +-
>  .../scripting-engines/trace-event-python.c    |   2 +-
>  tools/perf/util/stat-shadow.c                 |   1 +
>  tools/perf/util/stat.c                        |   4 -
>  tools/perf/util/thread_map.c                  |   1 +
>  tools/perf/util/thread_map.h                  |   2 -
>  32 files changed, 361 insertions(+), 86 deletions(-)
>  create mode 100644 tools/lib/symbol/Build
>  create mode 100644 tools/lib/symbol/Makefile
> 
> -- 
> 2.38.1.431.g37b22c650d-goog

-- 

- Arnaldo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v1 00/14] Fix perf tools/lib includes
  2022-11-09 15:27 ` [PATCH v1 00/14] Fix perf tools/lib includes Arnaldo Carvalho de Melo
@ 2022-11-09 16:52   ` Ian Rogers
  2022-11-09 16:57     ` Ian Rogers
  2022-11-09 18:00   ` Arnaldo Carvalho de Melo
  1 sibling, 1 reply; 21+ messages in thread
From: Ian Rogers @ 2022-11-09 16:52 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Masahiro Yamada, Nicolas Schier,
	linux-kernel, linux-perf-users, bpf, Nick Desaulniers,
	Andrii Nakryiko, Stephane Eranian

On Wed, Nov 9, 2022 at 7:27 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
>
> Em Mon, Nov 07, 2022 at 11:35:04PM -0800, Ian Rogers escreveu:
> > The previous build would add -Itools/lib and get dependencies for
> > libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
> > overriding these libraries would change the link time dependency but
> > the headers would erroneously come from tools/lib. Fix the build to
> > install headers and then depend on these. To reduce exposing internal
> > headers/APIs some clean up is performed. tools/lib/symbol has a
> > Makefile added, while tools/lib/api and tools/lib/subcmd have install
> > targets added. The pattern used for the dependencies in Makefile.perf
> > is modelled on libbpf.
>
> It builds with O=, I tried it one by one, but  there are problems with
> make from a detached tarball, that is how I do the container builds, see
> below, I'm trying to figure this out...

Sorry, I didn't know to test this. The added Build and Makefile in
libsymbol are missing, so I'd guess that's the major issue.

Thanks,
Ian

> ⬢[acme@toolbox perf]$ make perf-tar-src-pkg
>   TAR
>   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> ⬢[acme@toolbox perf]$ mv perf-6.1.0-rc3.tar /tmp
> ⬢[acme@toolbox perf]$ cd /tmp
> ⬢[acme@toolbox tmp]$ tar xf perf-6.1.0-rc3.tar
> ⬢[acme@toolbox tmp]$ cd perf-6.1.0-rc3/
> ⬢[acme@toolbox perf-6.1.0-rc3]$ make -C tools/perf
> make: Entering directory '/tmp/perf-6.1.0-rc3/tools/perf'
>   BUILD:   Doing 'make -j32' parallel build
>   HOSTCC  fixdep.o
>   HOSTLD  fixdep-in.o
>   LINK    fixdep
>
> Auto-detecting system features:
> ...                                   dwarf: [ on  ]
> ...                      dwarf_getlocations: [ on  ]
> ...                                   glibc: [ on  ]
> ...                                  libbfd: [ on  ]
> ...                          libbfd-buildid: [ on  ]
> ...                                  libcap: [ on  ]
> ...                                  libelf: [ on  ]
> ...                                 libnuma: [ on  ]
> ...                  numa_num_possible_cpus: [ on  ]
> ...                                 libperl: [ on  ]
> ...                               libpython: [ on  ]
> ...                               libcrypto: [ on  ]
> ...                               libunwind: [ on  ]
> ...                      libdw-dwarf-unwind: [ on  ]
> ...                                    zlib: [ on  ]
> ...                                    lzma: [ on  ]
> ...                               get_cpuid: [ on  ]
> ...                                     bpf: [ on  ]
> ...                                  libaio: [ on  ]
> ...                                 libzstd: [ on  ]
>
>   GEN     common-cmds.h
>   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
>   CC      perf-read-vdso32
>   GEN     perf-archive
>   GEN     perf-iostat
>   CC      dlfilters/dlfilter-test-api-v0.o
>   CC      dlfilters/dlfilter-show-cycles.o
>   CC      jvmti/libjvmti.o
> make[3]: *** No rule to make target '/tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a'.  Stop.
> make[2]: *** [Makefile.perf:907: /tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a] Error 2
> make[2]: *** Waiting for unfinished jobs....
>   CC      jvmti/jvmti_agent.o
>   CC      jvmti/libstring.o
>   CC      jvmti/libctype.o
>   GEN     pmu-events/pmu-events.c
>   INSTALL headers
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/exec-cmd.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/help.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/pager.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/parse-options.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/run-command.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/sigchain.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/subcmd-config.o
>   INSTALL headers
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.o
>   INSTALL headers
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/core.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/cpu.o
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/array.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/fs.o
>   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libbpf/bpf_helper_defs.h
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/tracing_path.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/cgroup.o
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf.h
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/debug.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2-in.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer-in.o
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf.h
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem-in.o
>   INSTALL headers
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/btf.h
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211-in.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch-in.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/cpumap.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/threadmap.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/str_error_r.o
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_common.h
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.so
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.so
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_legacy.h
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function-in.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evsel.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm-in.o
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.so
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.so
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.so
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helpers.h
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_tracing.h
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-plugin.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evlist.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/mmap.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/trace-seq.o
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_endian.h
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_core_read.h
> if [ ! -d ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs' ]; then install -d -m 755 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs'; fi; install fs/tracing_path.h -m 644 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs';
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/libapi-in.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/zalloc.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-filter.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-utils.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/kbuffer-parse.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen-in.o
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/skel_internal.h
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_version.h
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/tep_strerror.o
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/usdt.bpf.h
>   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helper_defs.h
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse-api.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/xyarray.o
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.so
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.so
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex-in.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/lib.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi-in.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211-in.o
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.so
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf.o
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.so
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.so
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/libapi-in.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/nlattr.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf.o
>   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb-in.o
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.so
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/str_error.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/netlink.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi-in.o
>   LD      jvmti/jvmti-in.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_errno.o
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.so
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf_prog_linfo.o
>   AR      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi.a
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_probes.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/hashmap.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf_dump.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/ringbuf.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/strset.o
>   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/libtraceevent-dynamic-list
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/linker.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/gen_loader.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/relo_core.o
>   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/usdt.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd-in.o
>   AR      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd.a
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf-in.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent-in.o
>   AR      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf.a
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent.a
>   CC      pmu-events/pmu-events.o
>   LD      pmu-events/pmu-events-in.o
>   LD      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf-in.o
>   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libbpf/libbpf.a
> make[1]: *** [Makefile.perf:240: sub-make] Error 2
> make: *** [Makefile:70: all] Error 2
> make: Leaving directory '/tmp/perf-6.1.0-rc3/tools/perf'
> ⬢[acme@toolbox perf-6.1.0-rc3]$
>
>
>
> > The problem and solution were motivated by this issue and discussion:
> > https://lore.kernel.org/lkml/CAEf4BzbbOHQZUAe6iWaehKCPQAf3VC=hq657buqe2_yRKxaK-A@mail.gmail.com/
> >
> > Ian Rogers (14):
> >   tools lib api: Add install target
> >   tools lib subcmd: Add install target
> >   perf build: Install libsubcmd locally when building
> >   perf build: Install libapi locally when building
> >   perf build: Install libperf locally when building
> >   perf build: Install libtraceevent locally when building
> >   tools lib api: Add missing install headers
> >   tools lib perf: Add missing install headers
> >   tool lib symbol: Add Makefile/Build
> >   perf build: Install libsymbol locally when building
> >   perf expr: Tidy hashmap dependency
> >   perf thread_map: Reduce exposure of libperf internal API
> >   perf cpumap: Tidy libperf includes
> >   perf build: Use tools/lib headers from install path
> >
> >  tools/lib/api/Makefile                        |  52 ++++++
> >  tools/lib/perf/Makefile                       |  10 +-
> >  tools/lib/subcmd/Makefile                     |  49 ++++++
> >  tools/lib/symbol/Build                        |   1 +
> >  tools/lib/symbol/Makefile                     | 115 +++++++++++++
> >  tools/perf/.gitignore                         |   7 +-
> >  tools/perf/Makefile.config                    |   2 -
> >  tools/perf/Makefile.perf                      | 152 ++++++++++++------
> >  tools/perf/builtin-stat.c                     |   1 +
> >  tools/perf/builtin-trace.c                    |   4 +-
> >  tools/perf/tests/cpumap.c                     |   2 +-
> >  tools/perf/tests/expr.c                       |   1 +
> >  tools/perf/tests/openat-syscall.c             |   1 +
> >  tools/perf/tests/pmu-events.c                 |   1 +
> >  tools/perf/tests/thread-map.c                 |   1 +
> >  tools/perf/util/Build                         |   5 -
> >  tools/perf/util/auxtrace.h                    |   2 +-
> >  tools/perf/util/bpf-loader.c                  |   4 -
> >  tools/perf/util/bpf_counter.c                 |   2 +-
> >  tools/perf/util/cpumap.c                      |   1 +
> >  tools/perf/util/cpumap.h                      |   2 +-
> >  tools/perf/util/evsel.c                       |   5 +-
> >  tools/perf/util/evsel.h                       |   2 -
> >  tools/perf/util/expr.c                        |   1 +
> >  tools/perf/util/expr.h                        |   7 +-
> >  tools/perf/util/metricgroup.c                 |   1 +
> >  tools/perf/util/python.c                      |   6 +-
> >  .../scripting-engines/trace-event-python.c    |   2 +-
> >  tools/perf/util/stat-shadow.c                 |   1 +
> >  tools/perf/util/stat.c                        |   4 -
> >  tools/perf/util/thread_map.c                  |   1 +
> >  tools/perf/util/thread_map.h                  |   2 -
> >  32 files changed, 361 insertions(+), 86 deletions(-)
> >  create mode 100644 tools/lib/symbol/Build
> >  create mode 100644 tools/lib/symbol/Makefile
> >
> > --
> > 2.38.1.431.g37b22c650d-goog
>
> --
>
> - Arnaldo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v1 00/14] Fix perf tools/lib includes
  2022-11-09 16:52   ` Ian Rogers
@ 2022-11-09 16:57     ` Ian Rogers
  2022-11-09 18:27       ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 21+ messages in thread
From: Ian Rogers @ 2022-11-09 16:57 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Masahiro Yamada, Nicolas Schier,
	linux-kernel, linux-perf-users, bpf, Nick Desaulniers,
	Andrii Nakryiko, Stephane Eranian

On Wed, Nov 9, 2022 at 8:52 AM Ian Rogers <irogers@google.com> wrote:
>
> On Wed, Nov 9, 2022 at 7:27 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> >
> > Em Mon, Nov 07, 2022 at 11:35:04PM -0800, Ian Rogers escreveu:
> > > The previous build would add -Itools/lib and get dependencies for
> > > libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
> > > overriding these libraries would change the link time dependency but
> > > the headers would erroneously come from tools/lib. Fix the build to
> > > install headers and then depend on these. To reduce exposing internal
> > > headers/APIs some clean up is performed. tools/lib/symbol has a
> > > Makefile added, while tools/lib/api and tools/lib/subcmd have install
> > > targets added. The pattern used for the dependencies in Makefile.perf
> > > is modelled on libbpf.
> >
> > It builds with O=, I tried it one by one, but  there are problems with
> > make from a detached tarball, that is how I do the container builds, see
> > below, I'm trying to figure this out...
>
> Sorry, I didn't know to test this. The added Build and Makefile in
> libsymbol are missing, so I'd guess that's the major issue.
>
> Thanks,
> Ian

The following fixes this for me:

--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -13,8 +13,7 @@ tools/lib/ctype.c
tools/lib/hweight.c
tools/lib/rbtree.c
tools/lib/string.c
-tools/lib/symbol/kallsyms.c
-tools/lib/symbol/kallsyms.h
+tools/lib/symbol
tools/lib/find_bit.c
tools/lib/bitmap.c
tools/lib/list_sort.c

Thanks,
Ian

> > ⬢[acme@toolbox perf]$ make perf-tar-src-pkg
> >   TAR
> >   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> > ⬢[acme@toolbox perf]$ mv perf-6.1.0-rc3.tar /tmp
> > ⬢[acme@toolbox perf]$ cd /tmp
> > ⬢[acme@toolbox tmp]$ tar xf perf-6.1.0-rc3.tar
> > ⬢[acme@toolbox tmp]$ cd perf-6.1.0-rc3/
> > ⬢[acme@toolbox perf-6.1.0-rc3]$ make -C tools/perf
> > make: Entering directory '/tmp/perf-6.1.0-rc3/tools/perf'
> >   BUILD:   Doing 'make -j32' parallel build
> >   HOSTCC  fixdep.o
> >   HOSTLD  fixdep-in.o
> >   LINK    fixdep
> >
> > Auto-detecting system features:
> > ...                                   dwarf: [ on  ]
> > ...                      dwarf_getlocations: [ on  ]
> > ...                                   glibc: [ on  ]
> > ...                                  libbfd: [ on  ]
> > ...                          libbfd-buildid: [ on  ]
> > ...                                  libcap: [ on  ]
> > ...                                  libelf: [ on  ]
> > ...                                 libnuma: [ on  ]
> > ...                  numa_num_possible_cpus: [ on  ]
> > ...                                 libperl: [ on  ]
> > ...                               libpython: [ on  ]
> > ...                               libcrypto: [ on  ]
> > ...                               libunwind: [ on  ]
> > ...                      libdw-dwarf-unwind: [ on  ]
> > ...                                    zlib: [ on  ]
> > ...                                    lzma: [ on  ]
> > ...                               get_cpuid: [ on  ]
> > ...                                     bpf: [ on  ]
> > ...                                  libaio: [ on  ]
> > ...                                 libzstd: [ on  ]
> >
> >   GEN     common-cmds.h
> >   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> >   CC      perf-read-vdso32
> >   GEN     perf-archive
> >   GEN     perf-iostat
> >   CC      dlfilters/dlfilter-test-api-v0.o
> >   CC      dlfilters/dlfilter-show-cycles.o
> >   CC      jvmti/libjvmti.o
> > make[3]: *** No rule to make target '/tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a'.  Stop.
> > make[2]: *** [Makefile.perf:907: /tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a] Error 2
> > make[2]: *** Waiting for unfinished jobs....
> >   CC      jvmti/jvmti_agent.o
> >   CC      jvmti/libstring.o
> >   CC      jvmti/libctype.o
> >   GEN     pmu-events/pmu-events.c
> >   INSTALL headers
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/exec-cmd.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/help.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/pager.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/parse-options.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/run-command.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/sigchain.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/subcmd-config.o
> >   INSTALL headers
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.o
> >   INSTALL headers
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/core.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/cpu.o
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/array.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/fs.o
> >   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libbpf/bpf_helper_defs.h
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/tracing_path.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/cgroup.o
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf.h
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/debug.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2-in.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer-in.o
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf.h
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem-in.o
> >   INSTALL headers
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/btf.h
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211-in.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch-in.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/cpumap.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/threadmap.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/str_error_r.o
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_common.h
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.so
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.so
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_legacy.h
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function-in.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evsel.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm-in.o
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.so
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.so
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.so
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helpers.h
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_tracing.h
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-plugin.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evlist.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/mmap.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/trace-seq.o
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_endian.h
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_core_read.h
> > if [ ! -d ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs' ]; then install -d -m 755 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs'; fi; install fs/tracing_path.h -m 644 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs';
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/libapi-in.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/zalloc.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-filter.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-utils.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/kbuffer-parse.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen-in.o
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/skel_internal.h
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_version.h
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/tep_strerror.o
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/usdt.bpf.h
> >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helper_defs.h
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse-api.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/xyarray.o
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.so
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.so
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex-in.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/lib.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi-in.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211-in.o
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.so
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf.o
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.so
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.so
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/libapi-in.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/nlattr.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf.o
> >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb-in.o
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.so
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/str_error.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/netlink.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi-in.o
> >   LD      jvmti/jvmti-in.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_errno.o
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.so
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf_prog_linfo.o
> >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi.a
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_probes.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/hashmap.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf_dump.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/ringbuf.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/strset.o
> >   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/libtraceevent-dynamic-list
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/linker.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/gen_loader.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/relo_core.o
> >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/usdt.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd-in.o
> >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd.a
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf-in.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent-in.o
> >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf.a
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent.a
> >   CC      pmu-events/pmu-events.o
> >   LD      pmu-events/pmu-events-in.o
> >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf-in.o
> >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libbpf/libbpf.a
> > make[1]: *** [Makefile.perf:240: sub-make] Error 2
> > make: *** [Makefile:70: all] Error 2
> > make: Leaving directory '/tmp/perf-6.1.0-rc3/tools/perf'
> > ⬢[acme@toolbox perf-6.1.0-rc3]$
> >
> >
> >
> > > The problem and solution were motivated by this issue and discussion:
> > > https://lore.kernel.org/lkml/CAEf4BzbbOHQZUAe6iWaehKCPQAf3VC=hq657buqe2_yRKxaK-A@mail.gmail.com/
> > >
> > > Ian Rogers (14):
> > >   tools lib api: Add install target
> > >   tools lib subcmd: Add install target
> > >   perf build: Install libsubcmd locally when building
> > >   perf build: Install libapi locally when building
> > >   perf build: Install libperf locally when building
> > >   perf build: Install libtraceevent locally when building
> > >   tools lib api: Add missing install headers
> > >   tools lib perf: Add missing install headers
> > >   tool lib symbol: Add Makefile/Build
> > >   perf build: Install libsymbol locally when building
> > >   perf expr: Tidy hashmap dependency
> > >   perf thread_map: Reduce exposure of libperf internal API
> > >   perf cpumap: Tidy libperf includes
> > >   perf build: Use tools/lib headers from install path
> > >
> > >  tools/lib/api/Makefile                        |  52 ++++++
> > >  tools/lib/perf/Makefile                       |  10 +-
> > >  tools/lib/subcmd/Makefile                     |  49 ++++++
> > >  tools/lib/symbol/Build                        |   1 +
> > >  tools/lib/symbol/Makefile                     | 115 +++++++++++++
> > >  tools/perf/.gitignore                         |   7 +-
> > >  tools/perf/Makefile.config                    |   2 -
> > >  tools/perf/Makefile.perf                      | 152 ++++++++++++------
> > >  tools/perf/builtin-stat.c                     |   1 +
> > >  tools/perf/builtin-trace.c                    |   4 +-
> > >  tools/perf/tests/cpumap.c                     |   2 +-
> > >  tools/perf/tests/expr.c                       |   1 +
> > >  tools/perf/tests/openat-syscall.c             |   1 +
> > >  tools/perf/tests/pmu-events.c                 |   1 +
> > >  tools/perf/tests/thread-map.c                 |   1 +
> > >  tools/perf/util/Build                         |   5 -
> > >  tools/perf/util/auxtrace.h                    |   2 +-
> > >  tools/perf/util/bpf-loader.c                  |   4 -
> > >  tools/perf/util/bpf_counter.c                 |   2 +-
> > >  tools/perf/util/cpumap.c                      |   1 +
> > >  tools/perf/util/cpumap.h                      |   2 +-
> > >  tools/perf/util/evsel.c                       |   5 +-
> > >  tools/perf/util/evsel.h                       |   2 -
> > >  tools/perf/util/expr.c                        |   1 +
> > >  tools/perf/util/expr.h                        |   7 +-
> > >  tools/perf/util/metricgroup.c                 |   1 +
> > >  tools/perf/util/python.c                      |   6 +-
> > >  .../scripting-engines/trace-event-python.c    |   2 +-
> > >  tools/perf/util/stat-shadow.c                 |   1 +
> > >  tools/perf/util/stat.c                        |   4 -
> > >  tools/perf/util/thread_map.c                  |   1 +
> > >  tools/perf/util/thread_map.h                  |   2 -
> > >  32 files changed, 361 insertions(+), 86 deletions(-)
> > >  create mode 100644 tools/lib/symbol/Build
> > >  create mode 100644 tools/lib/symbol/Makefile
> > >
> > > --
> > > 2.38.1.431.g37b22c650d-goog
> >
> > --
> >
> > - Arnaldo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v1 00/14] Fix perf tools/lib includes
  2022-11-09 16:57     ` Ian Rogers
@ 2022-11-09 18:27       ` Arnaldo Carvalho de Melo
  2022-11-15 21:28         ` Ian Rogers
  0 siblings, 1 reply; 21+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-11-09 18:27 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Masahiro Yamada, Nicolas Schier,
	linux-kernel, linux-perf-users, bpf, Nick Desaulniers,
	Andrii Nakryiko, Stephane Eranian

Em Wed, Nov 09, 2022 at 08:57:01AM -0800, Ian Rogers escreveu:
> On Wed, Nov 9, 2022 at 8:52 AM Ian Rogers <irogers@google.com> wrote:
> >
> > On Wed, Nov 9, 2022 at 7:27 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > >
> > > Em Mon, Nov 07, 2022 at 11:35:04PM -0800, Ian Rogers escreveu:
> > > > The previous build would add -Itools/lib and get dependencies for
> > > > libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
> > > > overriding these libraries would change the link time dependency but
> > > > the headers would erroneously come from tools/lib. Fix the build to
> > > > install headers and then depend on these. To reduce exposing internal
> > > > headers/APIs some clean up is performed. tools/lib/symbol has a
> > > > Makefile added, while tools/lib/api and tools/lib/subcmd have install
> > > > targets added. The pattern used for the dependencies in Makefile.perf
> > > > is modelled on libbpf.
> > >
> > > It builds with O=, I tried it one by one, but  there are problems with
> > > make from a detached tarball, that is how I do the container builds, see
> > > below, I'm trying to figure this out...
> >
> > Sorry, I didn't know to test this. The added Build and Makefile in
> > libsymbol are missing, so I'd guess that's the major issue.
> >
> > Thanks,
> > Ian
> 
> The following fixes this for me:

But then there are some other problems related to making sure those
libraries are built and installed _before_ perf proper starts building
when it expects to find the headers for those libraries.

So perhaps we need to first run the 'install' target for those libraries
before build perf proper?

I'm pushing what I have to perf/tools-libs-includes branch.

- Arnaldo
 
> --- a/tools/perf/MANIFEST
> +++ b/tools/perf/MANIFEST
> @@ -13,8 +13,7 @@ tools/lib/ctype.c
> tools/lib/hweight.c
> tools/lib/rbtree.c
> tools/lib/string.c
> -tools/lib/symbol/kallsyms.c
> -tools/lib/symbol/kallsyms.h
> +tools/lib/symbol
> tools/lib/find_bit.c
> tools/lib/bitmap.c
> tools/lib/list_sort.c
> 
> Thanks,
> Ian
> 
> > > ⬢[acme@toolbox perf]$ make perf-tar-src-pkg
> > >   TAR
> > >   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> > > ⬢[acme@toolbox perf]$ mv perf-6.1.0-rc3.tar /tmp
> > > ⬢[acme@toolbox perf]$ cd /tmp
> > > ⬢[acme@toolbox tmp]$ tar xf perf-6.1.0-rc3.tar
> > > ⬢[acme@toolbox tmp]$ cd perf-6.1.0-rc3/
> > > ⬢[acme@toolbox perf-6.1.0-rc3]$ make -C tools/perf
> > > make: Entering directory '/tmp/perf-6.1.0-rc3/tools/perf'
> > >   BUILD:   Doing 'make -j32' parallel build
> > >   HOSTCC  fixdep.o
> > >   HOSTLD  fixdep-in.o
> > >   LINK    fixdep
> > >
> > > Auto-detecting system features:
> > > ...                                   dwarf: [ on  ]
> > > ...                      dwarf_getlocations: [ on  ]
> > > ...                                   glibc: [ on  ]
> > > ...                                  libbfd: [ on  ]
> > > ...                          libbfd-buildid: [ on  ]
> > > ...                                  libcap: [ on  ]
> > > ...                                  libelf: [ on  ]
> > > ...                                 libnuma: [ on  ]
> > > ...                  numa_num_possible_cpus: [ on  ]
> > > ...                                 libperl: [ on  ]
> > > ...                               libpython: [ on  ]
> > > ...                               libcrypto: [ on  ]
> > > ...                               libunwind: [ on  ]
> > > ...                      libdw-dwarf-unwind: [ on  ]
> > > ...                                    zlib: [ on  ]
> > > ...                                    lzma: [ on  ]
> > > ...                               get_cpuid: [ on  ]
> > > ...                                     bpf: [ on  ]
> > > ...                                  libaio: [ on  ]
> > > ...                                 libzstd: [ on  ]
> > >
> > >   GEN     common-cmds.h
> > >   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> > >   CC      perf-read-vdso32
> > >   GEN     perf-archive
> > >   GEN     perf-iostat
> > >   CC      dlfilters/dlfilter-test-api-v0.o
> > >   CC      dlfilters/dlfilter-show-cycles.o
> > >   CC      jvmti/libjvmti.o
> > > make[3]: *** No rule to make target '/tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a'.  Stop.
> > > make[2]: *** [Makefile.perf:907: /tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a] Error 2
> > > make[2]: *** Waiting for unfinished jobs....
> > >   CC      jvmti/jvmti_agent.o
> > >   CC      jvmti/libstring.o
> > >   CC      jvmti/libctype.o
> > >   GEN     pmu-events/pmu-events.c
> > >   INSTALL headers
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/exec-cmd.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/help.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/pager.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/parse-options.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/run-command.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/sigchain.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/subcmd-config.o
> > >   INSTALL headers
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.o
> > >   INSTALL headers
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/core.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/cpu.o
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/array.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/fs.o
> > >   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libbpf/bpf_helper_defs.h
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/tracing_path.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/cgroup.o
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf.h
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/debug.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2-in.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer-in.o
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf.h
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem-in.o
> > >   INSTALL headers
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/btf.h
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211-in.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch-in.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/cpumap.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/threadmap.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/str_error_r.o
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_common.h
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.so
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.so
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_legacy.h
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function-in.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evsel.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm-in.o
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.so
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.so
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.so
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helpers.h
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_tracing.h
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-plugin.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evlist.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/mmap.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/trace-seq.o
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_endian.h
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_core_read.h
> > > if [ ! -d ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs' ]; then install -d -m 755 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs'; fi; install fs/tracing_path.h -m 644 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs';
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/libapi-in.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/zalloc.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-filter.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-utils.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/kbuffer-parse.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen-in.o
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/skel_internal.h
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_version.h
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/tep_strerror.o
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/usdt.bpf.h
> > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helper_defs.h
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse-api.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/xyarray.o
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.so
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.so
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex-in.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/lib.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi-in.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211-in.o
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.so
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf.o
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.so
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.so
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/libapi-in.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/nlattr.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf.o
> > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb-in.o
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.so
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/str_error.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/netlink.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi-in.o
> > >   LD      jvmti/jvmti-in.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_errno.o
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.so
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf_prog_linfo.o
> > >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi.a
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_probes.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/hashmap.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf_dump.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/ringbuf.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/strset.o
> > >   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/libtraceevent-dynamic-list
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/linker.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/gen_loader.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/relo_core.o
> > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/usdt.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd-in.o
> > >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd.a
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf-in.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent-in.o
> > >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf.a
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent.a
> > >   CC      pmu-events/pmu-events.o
> > >   LD      pmu-events/pmu-events-in.o
> > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf-in.o
> > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libbpf/libbpf.a
> > > make[1]: *** [Makefile.perf:240: sub-make] Error 2
> > > make: *** [Makefile:70: all] Error 2
> > > make: Leaving directory '/tmp/perf-6.1.0-rc3/tools/perf'
> > > ⬢[acme@toolbox perf-6.1.0-rc3]$
> > >
> > >
> > >
> > > > The problem and solution were motivated by this issue and discussion:
> > > > https://lore.kernel.org/lkml/CAEf4BzbbOHQZUAe6iWaehKCPQAf3VC=hq657buqe2_yRKxaK-A@mail.gmail.com/
> > > >
> > > > Ian Rogers (14):
> > > >   tools lib api: Add install target
> > > >   tools lib subcmd: Add install target
> > > >   perf build: Install libsubcmd locally when building
> > > >   perf build: Install libapi locally when building
> > > >   perf build: Install libperf locally when building
> > > >   perf build: Install libtraceevent locally when building
> > > >   tools lib api: Add missing install headers
> > > >   tools lib perf: Add missing install headers
> > > >   tool lib symbol: Add Makefile/Build
> > > >   perf build: Install libsymbol locally when building
> > > >   perf expr: Tidy hashmap dependency
> > > >   perf thread_map: Reduce exposure of libperf internal API
> > > >   perf cpumap: Tidy libperf includes
> > > >   perf build: Use tools/lib headers from install path
> > > >
> > > >  tools/lib/api/Makefile                        |  52 ++++++
> > > >  tools/lib/perf/Makefile                       |  10 +-
> > > >  tools/lib/subcmd/Makefile                     |  49 ++++++
> > > >  tools/lib/symbol/Build                        |   1 +
> > > >  tools/lib/symbol/Makefile                     | 115 +++++++++++++
> > > >  tools/perf/.gitignore                         |   7 +-
> > > >  tools/perf/Makefile.config                    |   2 -
> > > >  tools/perf/Makefile.perf                      | 152 ++++++++++++------
> > > >  tools/perf/builtin-stat.c                     |   1 +
> > > >  tools/perf/builtin-trace.c                    |   4 +-
> > > >  tools/perf/tests/cpumap.c                     |   2 +-
> > > >  tools/perf/tests/expr.c                       |   1 +
> > > >  tools/perf/tests/openat-syscall.c             |   1 +
> > > >  tools/perf/tests/pmu-events.c                 |   1 +
> > > >  tools/perf/tests/thread-map.c                 |   1 +
> > > >  tools/perf/util/Build                         |   5 -
> > > >  tools/perf/util/auxtrace.h                    |   2 +-
> > > >  tools/perf/util/bpf-loader.c                  |   4 -
> > > >  tools/perf/util/bpf_counter.c                 |   2 +-
> > > >  tools/perf/util/cpumap.c                      |   1 +
> > > >  tools/perf/util/cpumap.h                      |   2 +-
> > > >  tools/perf/util/evsel.c                       |   5 +-
> > > >  tools/perf/util/evsel.h                       |   2 -
> > > >  tools/perf/util/expr.c                        |   1 +
> > > >  tools/perf/util/expr.h                        |   7 +-
> > > >  tools/perf/util/metricgroup.c                 |   1 +
> > > >  tools/perf/util/python.c                      |   6 +-
> > > >  .../scripting-engines/trace-event-python.c    |   2 +-
> > > >  tools/perf/util/stat-shadow.c                 |   1 +
> > > >  tools/perf/util/stat.c                        |   4 -
> > > >  tools/perf/util/thread_map.c                  |   1 +
> > > >  tools/perf/util/thread_map.h                  |   2 -
> > > >  32 files changed, 361 insertions(+), 86 deletions(-)
> > > >  create mode 100644 tools/lib/symbol/Build
> > > >  create mode 100644 tools/lib/symbol/Makefile
> > > >
> > > > --
> > > > 2.38.1.431.g37b22c650d-goog
> > >
> > > --
> > >
> > > - Arnaldo

-- 

- Arnaldo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v1 00/14] Fix perf tools/lib includes
  2022-11-09 18:27       ` Arnaldo Carvalho de Melo
@ 2022-11-15 21:28         ` Ian Rogers
  0 siblings, 0 replies; 21+ messages in thread
From: Ian Rogers @ 2022-11-15 21:28 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Masahiro Yamada, Nicolas Schier,
	linux-kernel, linux-perf-users, bpf, Nick Desaulniers,
	Andrii Nakryiko, Stephane Eranian

[-- Attachment #1: Type: text/plain, Size: 18858 bytes --]

On Wed, Nov 9, 2022 at 10:27 AM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Wed, Nov 09, 2022 at 08:57:01AM -0800, Ian Rogers escreveu:
> > On Wed, Nov 9, 2022 at 8:52 AM Ian Rogers <irogers@google.com> wrote:
> > >
> > > On Wed, Nov 9, 2022 at 7:27 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > >
> > > > Em Mon, Nov 07, 2022 at 11:35:04PM -0800, Ian Rogers escreveu:
> > > > > The previous build would add -Itools/lib and get dependencies for
> > > > > libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
> > > > > overriding these libraries would change the link time dependency but
> > > > > the headers would erroneously come from tools/lib. Fix the build to
> > > > > install headers and then depend on these. To reduce exposing internal
> > > > > headers/APIs some clean up is performed. tools/lib/symbol has a
> > > > > Makefile added, while tools/lib/api and tools/lib/subcmd have install
> > > > > targets added. The pattern used for the dependencies in Makefile.perf
> > > > > is modelled on libbpf.
> > > >
> > > > It builds with O=, I tried it one by one, but  there are problems with
> > > > make from a detached tarball, that is how I do the container builds, see
> > > > below, I'm trying to figure this out...
> > >
> > > Sorry, I didn't know to test this. The added Build and Makefile in
> > > libsymbol are missing, so I'd guess that's the major issue.
> > >
> > > Thanks,
> > > Ian
> >
> > The following fixes this for me:
>
> But then there are some other problems related to making sure those
> libraries are built and installed _before_ perf proper starts building
> when it expects to find the headers for those libraries.
>
> So perhaps we need to first run the 'install' target for those libraries
> before build perf proper?
>
> I'm pushing what I have to perf/tools-libs-includes branch.

Attached is the last patch with changes to the dependencies. Moving
the dependencies to prepare addresses the issue. I tested by adding
sleeps to the install_headers targets to make them slow.

Thanks,
Ian

> - Arnaldo
>
> > --- a/tools/perf/MANIFEST
> > +++ b/tools/perf/MANIFEST
> > @@ -13,8 +13,7 @@ tools/lib/ctype.c
> > tools/lib/hweight.c
> > tools/lib/rbtree.c
> > tools/lib/string.c
> > -tools/lib/symbol/kallsyms.c
> > -tools/lib/symbol/kallsyms.h
> > +tools/lib/symbol
> > tools/lib/find_bit.c
> > tools/lib/bitmap.c
> > tools/lib/list_sort.c
> >
> > Thanks,
> > Ian
> >
> > > > ⬢[acme@toolbox perf]$ make perf-tar-src-pkg
> > > >   TAR
> > > >   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> > > > ⬢[acme@toolbox perf]$ mv perf-6.1.0-rc3.tar /tmp
> > > > ⬢[acme@toolbox perf]$ cd /tmp
> > > > ⬢[acme@toolbox tmp]$ tar xf perf-6.1.0-rc3.tar
> > > > ⬢[acme@toolbox tmp]$ cd perf-6.1.0-rc3/
> > > > ⬢[acme@toolbox perf-6.1.0-rc3]$ make -C tools/perf
> > > > make: Entering directory '/tmp/perf-6.1.0-rc3/tools/perf'
> > > >   BUILD:   Doing 'make -j32' parallel build
> > > >   HOSTCC  fixdep.o
> > > >   HOSTLD  fixdep-in.o
> > > >   LINK    fixdep
> > > >
> > > > Auto-detecting system features:
> > > > ...                                   dwarf: [ on  ]
> > > > ...                      dwarf_getlocations: [ on  ]
> > > > ...                                   glibc: [ on  ]
> > > > ...                                  libbfd: [ on  ]
> > > > ...                          libbfd-buildid: [ on  ]
> > > > ...                                  libcap: [ on  ]
> > > > ...                                  libelf: [ on  ]
> > > > ...                                 libnuma: [ on  ]
> > > > ...                  numa_num_possible_cpus: [ on  ]
> > > > ...                                 libperl: [ on  ]
> > > > ...                               libpython: [ on  ]
> > > > ...                               libcrypto: [ on  ]
> > > > ...                               libunwind: [ on  ]
> > > > ...                      libdw-dwarf-unwind: [ on  ]
> > > > ...                                    zlib: [ on  ]
> > > > ...                                    lzma: [ on  ]
> > > > ...                               get_cpuid: [ on  ]
> > > > ...                                     bpf: [ on  ]
> > > > ...                                  libaio: [ on  ]
> > > > ...                                 libzstd: [ on  ]
> > > >
> > > >   GEN     common-cmds.h
> > > >   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> > > >   CC      perf-read-vdso32
> > > >   GEN     perf-archive
> > > >   GEN     perf-iostat
> > > >   CC      dlfilters/dlfilter-test-api-v0.o
> > > >   CC      dlfilters/dlfilter-show-cycles.o
> > > >   CC      jvmti/libjvmti.o
> > > > make[3]: *** No rule to make target '/tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a'.  Stop.
> > > > make[2]: *** [Makefile.perf:907: /tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a] Error 2
> > > > make[2]: *** Waiting for unfinished jobs....
> > > >   CC      jvmti/jvmti_agent.o
> > > >   CC      jvmti/libstring.o
> > > >   CC      jvmti/libctype.o
> > > >   GEN     pmu-events/pmu-events.c
> > > >   INSTALL headers
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/exec-cmd.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/help.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/pager.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/parse-options.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/run-command.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/sigchain.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/subcmd-config.o
> > > >   INSTALL headers
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.o
> > > >   INSTALL headers
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/core.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/cpu.o
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/array.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/fs.o
> > > >   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libbpf/bpf_helper_defs.h
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/tracing_path.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/cgroup.o
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf.h
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/debug.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2-in.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer-in.o
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf.h
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem-in.o
> > > >   INSTALL headers
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/btf.h
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211-in.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch-in.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/cpumap.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/threadmap.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libapi/str_error_r.o
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_common.h
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_jbd2.so
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_hrtimer.so
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_legacy.h
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function-in.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evsel.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm-in.o
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kmem.so
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_mac80211.so
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_sched_switch.so
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helpers.h
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_tracing.h
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-plugin.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/evlist.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/mmap.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/trace-seq.o
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_endian.h
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_core_read.h
> > > > if [ ! -d ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs' ]; then install -d -m 755 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs'; fi; install fs/tracing_path.h -m 644 ''/tmp/perf-6.1.0-rc3/tools/perf/libapi'/include/api/fs';
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fd/libapi-in.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/zalloc.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-filter.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/parse-utils.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/kbuffer-parse.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen-in.o
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/skel_internal.h
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/libbpf_version.h
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/tep_strerror.o
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/usdt.bpf.h
> > > >   INSTALL /tmp/perf-6.1.0-rc3/tools/perf/libbpf/include/bpf/bpf_helper_defs.h
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/event-parse-api.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/xyarray.o
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_function.so
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_xen.so
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex-in.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libperf/lib.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi-in.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211-in.o
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_kvm.so
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf.o
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_futex.so
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_scsi.so
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/fs/libapi-in.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/nlattr.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf.o
> > > >   MKDIR   /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb-in.o
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_cfg80211.so
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/str_error.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/netlink.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi-in.o
> > > >   LD      jvmti/jvmti-in.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_errno.o
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/plugin_tlb.so
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/bpf_prog_linfo.o
> > > >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libapi/libapi.a
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf_probes.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/hashmap.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/btf_dump.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/ringbuf.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/strset.o
> > > >   GEN     /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent_plugins/libtraceevent-dynamic-list
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/linker.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/gen_loader.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/relo_core.o
> > > >   CC      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/usdt.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd-in.o
> > > >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libsubcmd/libsubcmd.a
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf-in.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent-in.o
> > > >   AR      /tmp/perf-6.1.0-rc3/tools/perf/libperf/libperf.a
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libtraceevent/libtraceevent.a
> > > >   CC      pmu-events/pmu-events.o
> > > >   LD      pmu-events/pmu-events-in.o
> > > >   LD      /tmp/perf-6.1.0-rc3/tools/perf/libbpf/staticobjs/libbpf-in.o
> > > >   LINK    /tmp/perf-6.1.0-rc3/tools/perf/libbpf/libbpf.a
> > > > make[1]: *** [Makefile.perf:240: sub-make] Error 2
> > > > make: *** [Makefile:70: all] Error 2
> > > > make: Leaving directory '/tmp/perf-6.1.0-rc3/tools/perf'
> > > > ⬢[acme@toolbox perf-6.1.0-rc3]$
> > > >
> > > >
> > > >
> > > > > The problem and solution were motivated by this issue and discussion:
> > > > > https://lore.kernel.org/lkml/CAEf4BzbbOHQZUAe6iWaehKCPQAf3VC=hq657buqe2_yRKxaK-A@mail.gmail.com/
> > > > >
> > > > > Ian Rogers (14):
> > > > >   tools lib api: Add install target
> > > > >   tools lib subcmd: Add install target
> > > > >   perf build: Install libsubcmd locally when building
> > > > >   perf build: Install libapi locally when building
> > > > >   perf build: Install libperf locally when building
> > > > >   perf build: Install libtraceevent locally when building
> > > > >   tools lib api: Add missing install headers
> > > > >   tools lib perf: Add missing install headers
> > > > >   tool lib symbol: Add Makefile/Build
> > > > >   perf build: Install libsymbol locally when building
> > > > >   perf expr: Tidy hashmap dependency
> > > > >   perf thread_map: Reduce exposure of libperf internal API
> > > > >   perf cpumap: Tidy libperf includes
> > > > >   perf build: Use tools/lib headers from install path
> > > > >
> > > > >  tools/lib/api/Makefile                        |  52 ++++++
> > > > >  tools/lib/perf/Makefile                       |  10 +-
> > > > >  tools/lib/subcmd/Makefile                     |  49 ++++++
> > > > >  tools/lib/symbol/Build                        |   1 +
> > > > >  tools/lib/symbol/Makefile                     | 115 +++++++++++++
> > > > >  tools/perf/.gitignore                         |   7 +-
> > > > >  tools/perf/Makefile.config                    |   2 -
> > > > >  tools/perf/Makefile.perf                      | 152 ++++++++++++------
> > > > >  tools/perf/builtin-stat.c                     |   1 +
> > > > >  tools/perf/builtin-trace.c                    |   4 +-
> > > > >  tools/perf/tests/cpumap.c                     |   2 +-
> > > > >  tools/perf/tests/expr.c                       |   1 +
> > > > >  tools/perf/tests/openat-syscall.c             |   1 +
> > > > >  tools/perf/tests/pmu-events.c                 |   1 +
> > > > >  tools/perf/tests/thread-map.c                 |   1 +
> > > > >  tools/perf/util/Build                         |   5 -
> > > > >  tools/perf/util/auxtrace.h                    |   2 +-
> > > > >  tools/perf/util/bpf-loader.c                  |   4 -
> > > > >  tools/perf/util/bpf_counter.c                 |   2 +-
> > > > >  tools/perf/util/cpumap.c                      |   1 +
> > > > >  tools/perf/util/cpumap.h                      |   2 +-
> > > > >  tools/perf/util/evsel.c                       |   5 +-
> > > > >  tools/perf/util/evsel.h                       |   2 -
> > > > >  tools/perf/util/expr.c                        |   1 +
> > > > >  tools/perf/util/expr.h                        |   7 +-
> > > > >  tools/perf/util/metricgroup.c                 |   1 +
> > > > >  tools/perf/util/python.c                      |   6 +-
> > > > >  .../scripting-engines/trace-event-python.c    |   2 +-
> > > > >  tools/perf/util/stat-shadow.c                 |   1 +
> > > > >  tools/perf/util/stat.c                        |   4 -
> > > > >  tools/perf/util/thread_map.c                  |   1 +
> > > > >  tools/perf/util/thread_map.h                  |   2 -
> > > > >  32 files changed, 361 insertions(+), 86 deletions(-)
> > > > >  create mode 100644 tools/lib/symbol/Build
> > > > >  create mode 100644 tools/lib/symbol/Makefile
> > > > >
> > > > > --
> > > > > 2.38.1.431.g37b22c650d-goog
> > > >
> > > > --
> > > >
> > > > - Arnaldo
>
> --
>
> - Arnaldo

[-- Attachment #2: 0001-perf-build-Use-tools-lib-headers-from-install-path.patch --]
[-- Type: text/x-patch, Size: 4432 bytes --]

From 8cdca3f85cd4398ceeddea5bbc9d421f0bfdc76a Mon Sep 17 00:00:00 2001
From: Ian Rogers <irogers@google.com>
Date: Wed, 9 Nov 2022 10:49:14 -0800
Subject: [PATCH] perf build: Use tools/lib headers from install path

Switch -I from tools/lib to the install path for the tools/lib
libraries. Add the include_headers build targets to prepare target, as
well as pmu-events.c compilation that dependes on libperf.

Signed-off-by: Ian Rogers <irogers@google.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Nicolas Schier <nicolas@fjasle.eu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: bpf@vger.kernel.org
Link: http://lore.kernel.org/lkml/20221109184914.1357295-15-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Makefile.config |  2 --
 tools/perf/Makefile.perf   | 14 +++++++++++++-
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index a7f6c0669fae..9cc3c48f3288 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -349,7 +349,6 @@ ifeq ($(DEBUG),0)
   endif
 endif
 
-INC_FLAGS += -I$(srctree)/tools/lib/perf/include
 INC_FLAGS += -I$(src-perf)/util/include
 INC_FLAGS += -I$(src-perf)/arch/$(SRCARCH)/include
 INC_FLAGS += -I$(srctree)/tools/include/
@@ -367,7 +366,6 @@ endif
 
 INC_FLAGS += -I$(src-perf)/util
 INC_FLAGS += -I$(src-perf)
-INC_FLAGS += -I$(srctree)/tools/lib/
 
 CORE_CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
 
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 6c1a2a3ccc38..766ec9471183 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -305,6 +305,7 @@ LIBTRACEEVENT_INCLUDE = $(LIBTRACEEVENT_DESTDIR)/include
 LIBTRACEEVENT = $(LIBTRACEEVENT_OUTPUT)/libtraceevent.a
 export LIBTRACEEVENT
 LIBTRACEEVENT_DYNAMIC_LIST = $(LIBTRACEEVENT_PLUGINS_OUTPUT)/libtraceevent-dynamic-list
+CFLAGS += -I$(LIBTRACEEVENT_OUTPUT)/include
 
 #
 # The static build has no dynsym table, so this does not work for
@@ -322,6 +323,7 @@ LIBAPI_DESTDIR = $(LIBAPI_OUTPUT)
 LIBAPI_INCLUDE = $(LIBAPI_DESTDIR)/include
 LIBAPI = $(LIBAPI_OUTPUT)/libapi.a
 export LIBAPI
+CFLAGS += -I$(LIBAPI_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBBPF_OUTPUT = $(abspath $(OUTPUT))/libbpf
@@ -331,6 +333,7 @@ endif
 LIBBPF_DESTDIR = $(LIBBPF_OUTPUT)
 LIBBPF_INCLUDE = $(LIBBPF_DESTDIR)/include
 LIBBPF = $(LIBBPF_OUTPUT)/libbpf.a
+CFLAGS += -I$(LIBBPF_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBSUBCMD_OUTPUT = $(abspath $(OUTPUT))/libsubcmd
@@ -340,6 +343,7 @@ endif
 LIBSUBCMD_DESTDIR = $(LIBSUBCMD_OUTPUT)
 LIBSUBCMD_INCLUDE = $(LIBSUBCMD_DESTDIR)/include
 LIBSUBCMD = $(LIBSUBCMD_OUTPUT)/libsubcmd.a
+CFLAGS += -I$(LIBSUBCMD_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBSYMBOL_OUTPUT = $(abspath $(OUTPUT))/libsymbol
@@ -349,6 +353,7 @@ endif
 LIBSYMBOL_DESTDIR = $(LIBSYMBOL_OUTPUT)
 LIBSYMBOL_INCLUDE = $(LIBSYMBOL_DESTDIR)/include
 LIBSYMBOL = $(LIBSYMBOL_OUTPUT)/libsymbol.a
+CFLAGS += -I$(LIBSYMBOL_OUTPUT)/include
 
 ifneq ($(OUTPUT),)
   LIBPERF_OUTPUT = $(abspath $(OUTPUT))/libperf
@@ -359,6 +364,7 @@ LIBPERF_DESTDIR = $(LIBPERF_OUTPUT)
 LIBPERF_INCLUDE = $(LIBPERF_DESTDIR)/include
 LIBPERF = $(LIBPERF_OUTPUT)/libperf.a
 export LIBPERF
+CFLAGS += -I$(LIBPERF_OUTPUT)/include
 
 # python extension build directories
 PYTHON_EXTBUILD     := $(OUTPUT)python_ext_build/
@@ -691,7 +697,7 @@ build := -f $(srctree)/tools/build/Makefile.build dir=. obj
 $(PERF_IN): prepare FORCE
 	$(Q)$(MAKE) $(build)=perf
 
-$(PMU_EVENTS_IN): FORCE
+$(PMU_EVENTS_IN): FORCE $(LIBPERF)
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events
 
 $(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) $(LIBTRACEEVENT_DYNAMIC_LIST)
@@ -774,6 +780,12 @@ prepare: $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)common-cmds.h archheaders $(drm_ioc
 	$(rename_flags_array) \
 	$(arch_errno_name_array) \
 	$(sync_file_range_arrays) \
+	$(LIBAPI) \
+	$(LIBBPF) \
+	$(LIBPERF) \
+	$(LIBSUBCMD) \
+	$(LIBSYMBOL) \
+	$(LIBTRACEEVENT) \
 	bpf-skel
 
 $(OUTPUT)%.o: %.c prepare FORCE
-- 
2.38.1.431.g37b22c650d-goog


^ permalink raw reply related	[flat|nested] 21+ messages in thread

* Re: [PATCH v1 00/14] Fix perf tools/lib includes
  2022-11-09 15:27 ` [PATCH v1 00/14] Fix perf tools/lib includes Arnaldo Carvalho de Melo
  2022-11-09 16:52   ` Ian Rogers
@ 2022-11-09 18:00   ` Arnaldo Carvalho de Melo
  1 sibling, 0 replies; 21+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-11-09 18:00 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Masahiro Yamada, Nicolas Schier,
	linux-kernel, linux-perf-users, bpf, Nick Desaulniers,
	Andrii Nakryiko, Stephane Eranian

Em Wed, Nov 09, 2022 at 12:27:19PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Nov 07, 2022 at 11:35:04PM -0800, Ian Rogers escreveu:
> > The previous build would add -Itools/lib and get dependencies for
> > libtraceevent, libsubcmd, libsymbol, libapi and libbpf meaning that
> > overriding these libraries would change the link time dependency but
> > the headers would erroneously come from tools/lib. Fix the build to
> > install headers and then depend on these. To reduce exposing internal
> > headers/APIs some clean up is performed. tools/lib/symbol has a
> > Makefile added, while tools/lib/api and tools/lib/subcmd have install
> > targets added. The pattern used for the dependencies in Makefile.perf
> > is modelled on libbpf.
> 
> It builds with O=, I tried it one by one, but  there are problems with
> make from a detached tarball, that is how I do the container builds, see
> below, I'm trying to figure this out...
> 
> ⬢[acme@toolbox perf]$ make perf-tar-src-pkg
>   TAR
>   PERF_VERSION = 6.1.rc3.g7e5d8b7a1fbd
> ⬢[acme@toolbox perf]$ mv perf-6.1.0-rc3.tar /tmp
> ⬢[acme@toolbox perf]$ cd /tmp
> ⬢[acme@toolbox tmp]$ tar xf perf-6.1.0-rc3.tar
> ⬢[acme@toolbox tmp]$ cd perf-6.1.0-rc3/
> ⬢[acme@toolbox perf-6.1.0-rc3]$ make -C tools/perf
> make: Entering directory '/tmp/perf-6.1.0-rc3/tools/perf'
>   BUILD:   Doing 'make -j32' parallel build
>   HOSTCC  fixdep.o
>   HOSTLD  fixdep-in.o
>   LINK    fixdep
> 
> Auto-detecting system features:
<SNIP>
> ...                                   dwarf: [ on  ]
>   CC      jvmti/libjvmti.o
> make[3]: *** No rule to make target '/tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a'.  Stop.
> make[2]: *** [Makefile.perf:907: /tmp/perf-6.1.0-rc3/tools/perf/libsymbol/libsymbol.a] Error 2
> make[2]: *** Waiting for unfinished jobs....

So this cures it:

diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index f5d72f936a6bad2c..c8e8e05b4ff1f56f 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -13,8 +13,7 @@ tools/lib/ctype.c
 tools/lib/hweight.c
 tools/lib/rbtree.c
 tools/lib/string.c
-tools/lib/symbol/kallsyms.c
-tools/lib/symbol/kallsyms.h
+tools/lib/symbol
 tools/lib/find_bit.c
 tools/lib/bitmap.c
 tools/lib/list_sort.c

^ permalink raw reply related	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2022-11-15 21:29 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-11-08  7:35 [PATCH v1 00/14] Fix perf tools/lib includes Ian Rogers
2022-11-08  7:35 ` [PATCH v1 01/14] tools lib api: Add install target Ian Rogers
2022-11-08  7:35 ` [PATCH v1 02/14] tools lib subcmd: " Ian Rogers
2022-11-08  7:35 ` [PATCH v1 03/14] perf build: Install libsubcmd locally when building Ian Rogers
2022-11-08  7:35 ` [PATCH v1 04/14] perf build: Install libapi " Ian Rogers
2022-11-08  7:35 ` [PATCH v1 05/14] perf build: Install libperf " Ian Rogers
2022-11-08  7:35 ` [PATCH v1 06/14] perf build: Install libtraceevent " Ian Rogers
2022-11-08  7:35 ` [PATCH v1 07/14] tools lib api: Add missing install headers Ian Rogers
2022-11-08  7:35 ` [PATCH v1 08/14] tools lib perf: " Ian Rogers
2022-11-08  7:35 ` [PATCH v1 09/14] tool lib symbol: Add Makefile/Build Ian Rogers
2022-11-08  7:35 ` [PATCH v1 10/14] perf build: Install libsymbol locally when building Ian Rogers
2022-11-08  7:35 ` [PATCH v1 11/14] perf expr: Tidy hashmap dependency Ian Rogers
2022-11-08  7:35 ` [PATCH v1 12/14] perf thread_map: Reduce exposure of libperf internal API Ian Rogers
2022-11-08  7:35 ` [PATCH v1 13/14] perf cpumap: Tidy libperf includes Ian Rogers
2022-11-08  7:35 ` [PATCH v1 14/14] perf build: Use tools/lib headers from install path Ian Rogers
2022-11-09 15:27 ` [PATCH v1 00/14] Fix perf tools/lib includes Arnaldo Carvalho de Melo
2022-11-09 16:52   ` Ian Rogers
2022-11-09 16:57     ` Ian Rogers
2022-11-09 18:27       ` Arnaldo Carvalho de Melo
2022-11-15 21:28         ` Ian Rogers
2022-11-09 18:00   ` Arnaldo Carvalho de Melo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).