Linux Trace Kernel
 help / color / mirror / Atom feed
* [PATCH 0/3] Fix out-of-tree build of some tools
@ 2026-05-21 14:34 Ben Hutchings
  2026-05-21 14:35 ` [PATCH 1/3] rtla: Fix output files in source tree Ben Hutchings
                   ` (4 more replies)
  0 siblings, 5 replies; 14+ messages in thread
From: Ben Hutchings @ 2026-05-21 14:34 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Steven Rostedt, Tomas Glozar
  Cc: linux-perf-users, linux-trace-kernel

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

perf and rtla currently don't fully support out-of-tree builds, as
they may still create files in their source directory.  This series
fixes all the instances of this problem that I have found.

Ben.

Ben Hutchings (3):
  rtla: Fix output files in source tree
  perf tools: Put Python egg info in output directory
  perf tools: Put Python bytecode in output directory

 tools/perf/Makefile.perf            |  9 ++++++++-
 tools/tracing/rtla/Makefile         | 31 ++++++++++++++++++-----------
 tools/tracing/rtla/tests/timerlat.t |  4 ++--
 3 files changed, 29 insertions(+), 15 deletions(-)


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 1/3] rtla: Fix output files in source tree
  2026-05-21 14:34 [PATCH 0/3] Fix out-of-tree build of some tools Ben Hutchings
@ 2026-05-21 14:35 ` Ben Hutchings
  2026-05-21 14:48   ` Steven Rostedt
  2026-05-21 14:35 ` [PATCH 2/3] perf tools: Put Python egg info in output directory Ben Hutchings
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 14+ messages in thread
From: Ben Hutchings @ 2026-05-21 14:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Steven Rostedt, Tomas Glozar
  Cc: linux-perf-users, linux-trace-kernel

[-- Attachment #1: Type: text/plain, Size: 4742 bytes --]

Some output files (src/timerlat.bpf.o, src/timerlat.skel.h,
example/timerlat_bpf_action.o, tests/bpf/bpf_action_map.o) are
currently generated in the source tree, preventing a fully out-of-tree
build.  To fix this:

- Add $(OUTPUT) to their filenames in the relevant Makefile rules, and
  create subdirectories as needed
- Add $(OUTPUT)src to the include path
- Add ${OUTPUT} to the BPF object filename in tests/timerlat.t

Fixes: e34293ddcebd ("rtla/timerlat: Add BPF skeleton to collect samples")
Fixes: 0304a3b7ec9a ("rtla/timerlat: Add example for BPF action program")
Fixes: 5525aebd4e0c ("rtla/tests: Test BPF action program")
Signed-off-by: Ben Hutchings <benh@debian.org>
---
 tools/tracing/rtla/Makefile         | 31 ++++++++++++++++++-----------
 tools/tracing/rtla/tests/timerlat.t |  4 ++--
 2 files changed, 21 insertions(+), 14 deletions(-)

diff --git a/tools/tracing/rtla/Makefile b/tools/tracing/rtla/Makefile
index 45690ee14544..f54da7be735d 100644
--- a/tools/tracing/rtla/Makefile
+++ b/tools/tracing/rtla/Makefile
@@ -66,30 +66,37 @@ ifeq ($(config),1)
   include Makefile.config
 endif
 
+INCLUDES	= -I$(OUTPUT)src
+
 CFLAGS		+= $(INCLUDES) $(LIB_INCLUDES)
 
 export CFLAGS OUTPUT srctree
 
 ifeq ($(BUILD_BPF_SKEL),1)
-src/timerlat.bpf.o: src/timerlat.bpf.c
+$(OUTPUT)src/timerlat.bpf.o: src/timerlat.bpf.c
+	mkdir -p $(@D)
 	$(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -c $(filter %.c,$^) -o $@
 
-src/timerlat.skel.h: src/timerlat.bpf.o
+$(OUTPUT)src/timerlat.skel.h: $(OUTPUT)src/timerlat.bpf.o
+	mkdir -p $(@D)
 	$(QUIET_GENSKEL)$(SYSTEM_BPFTOOL) gen skeleton $< > $@
 
-example/timerlat_bpf_action.o: example/timerlat_bpf_action.c
+$(OUTPUT)example/timerlat_bpf_action.o: example/timerlat_bpf_action.c
+	mkdir -p $(@D)
 	$(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -c $(filter %.c,$^) -o $@
 
-tests/bpf/bpf_action_map.o: tests/bpf/bpf_action_map.c
+$(OUTPUT)tests/bpf/bpf_action_map.o: tests/bpf/bpf_action_map.c
+	mkdir -p $(@D)
 	$(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -c $(filter %.c,$^) -o $@
 else
-src/timerlat.skel.h:
-	$(Q)echo '/* BPF skeleton is disabled */' > src/timerlat.skel.h
+$(OUTPUT)src/timerlat.skel.h:
+	mkdir -p $(@D)
+	$(Q)echo '/* BPF skeleton is disabled */' > $@
 
-example/timerlat_bpf_action.o: example/timerlat_bpf_action.c
+$(OUTPUT)example/timerlat_bpf_action.o: example/timerlat_bpf_action.c
 	$(Q)echo "BPF skeleton support is disabled, skipping example/timerlat_bpf_action.o"
 
-tests/bpf/bpf_action_map.o: tests/bpf/bpf_action_map.c
+$(OUTPUT)tests/bpf/bpf_action_map.o: tests/bpf/bpf_action_map.c
 	$(Q)echo "BPF skeleton support is disabled, skipping tests/bpf/bpf_action_map.o"
 endif
 
@@ -103,7 +110,7 @@ static: $(RTLA_IN)
 rtla.%: fixdep FORCE
 	make -f $(srctree)/tools/build/Makefile.build dir=. $@
 
-$(RTLA_IN): fixdep FORCE src/timerlat.skel.h
+$(RTLA_IN): fixdep FORCE $(OUTPUT)src/timerlat.skel.h
 	make $(build)=rtla
 
 clean: doc_clean fixdep-clean
@@ -111,10 +118,10 @@ clean: doc_clean fixdep-clean
 	$(Q)find . -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)rm -f rtla rtla-static fixdep FEATURE-DUMP rtla-*
 	$(Q)rm -rf feature
-	$(Q)rm -f src/timerlat.bpf.o src/timerlat.skel.h example/timerlat_bpf_action.o
+	$(Q)rm -f $(OUTPUT)src/timerlat.bpf.o $(OUTPUT)src/timerlat.skel.h $(OUTPUT)example/timerlat_bpf_action.o
 	$(Q)rm -f $(UNIT_TESTS)
 
-check: $(RTLA) tests/bpf/bpf_action_map.o
+check: $(RTLA) $(OUTPUT)tests/bpf/bpf_action_map.o
 	RTLA=$(RTLA) BPFTOOL=$(SYSTEM_BPFTOOL) prove -o -f -v tests/
-examples: example/timerlat_bpf_action.o
+examples: $(OUTPUT)example/timerlat_bpf_action.o
 .PHONY: FORCE clean check
diff --git a/tools/tracing/rtla/tests/timerlat.t b/tools/tracing/rtla/tests/timerlat.t
index fd4935fd7b49..e0f3fc4df655 100644
--- a/tools/tracing/rtla/tests/timerlat.t
+++ b/tools/tracing/rtla/tests/timerlat.t
@@ -74,12 +74,12 @@ then
 	# Test BPF action program properly in BPF mode
 	[ -z "$BPFTOOL" ] && BPFTOOL=bpftool
 	check "hist with BPF action program (BPF mode)" \
-		"timerlat hist -T 2 --bpf-action tests/bpf/bpf_action_map.o --on-threshold shell,command='$BPFTOOL map dump name rtla_test_map'" \
+		"timerlat hist -T 2 --bpf-action ${OUTPUT}tests/bpf/bpf_action_map.o --on-threshold shell,command='$BPFTOOL map dump name rtla_test_map'" \
 		2 '"value": 42'
 else
 	# Test BPF action program failure in non-BPF mode
 	check "hist with BPF action program (non-BPF mode)" \
-		"timerlat hist -T 2 --bpf-action tests/bpf/bpf_action_map.o" \
+		"timerlat hist -T 2 --bpf-action ${OUTPUT}tests/bpf/bpf_action_map.o" \
 		1 "BPF actions are not supported in tracefs-only mode"
 fi
 done


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH 2/3] perf tools: Put Python egg info in output directory
  2026-05-21 14:34 [PATCH 0/3] Fix out-of-tree build of some tools Ben Hutchings
  2026-05-21 14:35 ` [PATCH 1/3] rtla: Fix output files in source tree Ben Hutchings
@ 2026-05-21 14:35 ` Ben Hutchings
  2026-05-21 14:35 ` [PATCH 3/3] perf tools: Put Python bytecode " Ben Hutchings
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 14+ messages in thread
From: Ben Hutchings @ 2026-05-21 14:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Steven Rostedt, Tomas Glozar
  Cc: linux-perf-users, linux-trace-kernel

[-- Attachment #1: Type: text/plain, Size: 990 bytes --]

Installing the Python extension currently creates an egg-info
directory in the source tree, preventing a fully out-of-tree build.

Add the necessary runes to the setup.py comamnd line to relocate the
egg-info directory in an out-of-tree build.

Signed-off-by: Ben Hutchings <benh@debian.org>
---
 tools/perf/Makefile.perf | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index cee19c923c06..899a4249a42f 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1152,7 +1152,9 @@ install-bin: install-tools install-tests
 install: install-bin try-install-man
 
 install-python_ext:
-	$(PYTHON_WORD) util/setup.py --quiet install --root='/$(DESTDIR_SQ)'
+	$(PYTHON_WORD) util/setup.py --quiet \
+		$(if $(OUTPUT),egg_info --egg-base $(OUTPUT)) \
+		install --root='/$(DESTDIR_SQ)'
 
 # 'make install-doc' should call 'make -C Documentation install'
 $(INSTALL_DOC_TARGETS):


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH 3/3] perf tools: Put Python bytecode in output directory
  2026-05-21 14:34 [PATCH 0/3] Fix out-of-tree build of some tools Ben Hutchings
  2026-05-21 14:35 ` [PATCH 1/3] rtla: Fix output files in source tree Ben Hutchings
  2026-05-21 14:35 ` [PATCH 2/3] perf tools: Put Python egg info in output directory Ben Hutchings
@ 2026-05-21 14:35 ` Ben Hutchings
  2026-05-21 17:50 ` [PATCH 0/3] Fix out-of-tree build of some tools Ian Rogers
  2026-09-21 17:57 ` [PATCH v2 1/2] perf tools: Put Python egg info in output directory Ian Rogers
  4 siblings, 0 replies; 14+ messages in thread
From: Ben Hutchings @ 2026-05-21 14:35 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Steven Rostedt, Tomas Glozar
  Cc: linux-perf-users, linux-trace-kernel

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

The PMU events are processed into C sources by Python scripts, which
normally results in writing bytecode for each module into the source
tree.  This prevents a fully out-of-tree build.

To fix this, set $PYTHONPYCACHEPREFIX to relocate the bytecode cache
directory in an out-of-tree build.

Signed-off-by: Ben Hutchings <benh@debian.org>
---
 tools/perf/Makefile.perf | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 899a4249a42f..c35b65f9fdda 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -400,6 +400,11 @@ PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
 PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
 export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
 
+# Put Python bytecode in output directory
+ifdef OUTPUT
+export PYTHONPYCACHEPREFIX := $(OUTPUT)/__pycache__
+endif
+
 python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
 
 # Use the detected configuration

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/3] rtla: Fix output files in source tree
  2026-05-21 14:35 ` [PATCH 1/3] rtla: Fix output files in source tree Ben Hutchings
@ 2026-05-21 14:48   ` Steven Rostedt
  0 siblings, 0 replies; 14+ messages in thread
From: Steven Rostedt @ 2026-05-21 14:48 UTC (permalink / raw)
  To: Ben Hutchings
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Tomas Glozar, linux-perf-users, linux-trace-kernel

On Thu, 21 May 2026 16:35:25 +0200
Ben Hutchings <benh@debian.org> wrote:

> Some output files (src/timerlat.bpf.o, src/timerlat.skel.h,
> example/timerlat_bpf_action.o, tests/bpf/bpf_action_map.o) are
> currently generated in the source tree, preventing a fully out-of-tree
> build.  To fix this:
> 
> - Add $(OUTPUT) to their filenames in the relevant Makefile rules, and
>   create subdirectories as needed
> - Add $(OUTPUT)src to the include path
> - Add ${OUTPUT} to the BPF object filename in tests/timerlat.t
> 
> Fixes: e34293ddcebd ("rtla/timerlat: Add BPF skeleton to collect samples")
> Fixes: 0304a3b7ec9a ("rtla/timerlat: Add example for BPF action program")
> Fixes: 5525aebd4e0c ("rtla/tests: Test BPF action program")
> Signed-off-by: Ben Hutchings <benh@debian.org>

Hi Ben,

Can you send this as a separate patch. The rtla code is handled via a
different tree than the perf code. So these patches will not be going
together.

Thanks,

-- Steve

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 0/3] Fix out-of-tree build of some tools
  2026-05-21 14:34 [PATCH 0/3] Fix out-of-tree build of some tools Ben Hutchings
                   ` (2 preceding siblings ...)
  2026-05-21 14:35 ` [PATCH 3/3] perf tools: Put Python bytecode " Ben Hutchings
@ 2026-05-21 17:50 ` Ian Rogers
  2026-09-15 23:08   ` Ian Rogers
  2026-09-21 17:57 ` [PATCH v2 1/2] perf tools: Put Python egg info in output directory Ian Rogers
  4 siblings, 1 reply; 14+ messages in thread
From: Ian Rogers @ 2026-05-21 17:50 UTC (permalink / raw)
  To: Ben Hutchings
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Steven Rostedt, Tomas Glozar, linux-perf-users,
	linux-trace-kernel

On Thu, May 21, 2026 at 7:57 AM Ben Hutchings <benh@debian.org> wrote:
>
> perf and rtla currently don't fully support out-of-tree builds, as
> they may still create files in their source directory.  This series
> fixes all the instances of this problem that I have found.
>
> Ben.

Thanks for the cleanup! I ran Sashiko locally and it noted that we
should also fix the clean steps to remove these files (I'll copy the
full text below).
Reviewed-by: Ian Rogers <irogers@google.com>

```
commit cf212be834ce6b0df5cccdbc4ad977a917ff8e48
   Author: Ben Hutchings <benh@debian.org>

   perf tools: Put Python egg info in output directory

   Installing the Python extension currently creates an egg-info
   directory in the source tree, preventing a fully out-of-tree build.

   Add the necessary runes to the setup.py comamnd line to relocate the
   egg-info directory in an out-of-tree build.

   > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
   > index fc92d6ceac5b..b9a9bdddb89a 100644
   > --- a/tools/perf/Makefile.perf
   > +++ b/tools/perf/Makefile.perf
   > @@ -905,7 +910,9 @@ install-bin: install-tools install-tests
   >  install: install-bin try-install-man
   >
   >  install-python_ext:
   > -       $(PYTHON_WORD) util/setup.py --quiet install
--root='/$(DESTDIR_SQ)'
   > +       $(PYTHON_WORD) util/setup.py --quiet \
   > +               $(if $(OUTPUT),egg_info --egg-base $(OUTPUT)) \
   > +               install --root='/$(DESTDIR_SQ)'

   This relocates the egg-info directory to the output directory, which
   helps keep the source tree clean. However, does the `python-clean`
   target need to be updated to clean up this directory?

   Currently, `python-clean` is defined as:

   python-clean := $(call QUIET_CLEAN, python) $(RM) -r
$(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so

   Should we also clean `$(OUTPUT)/perf.egg-info` (or `perf.egg-info` for
   in-tree builds) to ensure `make clean` removes all generated python
   metadata? For example:

   python-clean := $(call QUIET_CLEAN, python) \
                   $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so \
                   $(OUTPUT)perf.egg-info perf.egg-info

   commit dd754ea75966ec7ade41500afeb32d944f20a8f9
   Author: Ben Hutchings <benh@debian.org>

   perf tools: Put Python bytecode in output directory

   The PMU events are processed into C sources by Python scripts,
which
   normally results in writing bytecode for each module into the
source
   tree.  This prevents a fully out-of-tree build.

   To fix this, set $PYTHONPYCACHEPREFIX to relocate the bytecode cache
   directory in an out-of-tree build.

   > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
   > index fc92d6ceac5b..b9a9bdddb89a 100644
   > --- a/tools/perf/Makefile.perf
   > +++ b/tools/perf/Makefile.perf
   > @@ -400,6 +400,11 @@ PYTHON_EXTBUILD_LIB :=
$(PYTHON_EXTBUILD)lib/
   >  PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
   >  export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
   >
   > +# Put Python bytecode in output directory
   > +ifdef OUTPUT
   > +export PYTHONPYCACHEPREFIX := $(OUTPUT)/__pycache__
   > +endif
   > +
   >  python-clean := $(call QUIET_CLEAN, python) $(RM) -r
$(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so

   Similar to the egg-info relocation, does `python-clean` need to
clean up
   the relocated `__pycache__` directory under `$(OUTPUT)`?

   If `PYTHONPYCACHEPREFIX` is exported to `$(OUTPUT)/__pycache__`, running
   `make clean` currently leaves this directory behind. Should we add it to
   the `RM` list in `python-clean`? For example:

   python-clean := $(call QUIET_CLEAN, python) \
                   $(RM) -r $(PYTHON_EXTBUILD)
$(OUTPUT)python/perf*.so \
                   $(if $(OUTPUT),$(OUTPUT)/__pycache__)
```

Thanks,
Ian

> Ben Hutchings (3):
>   rtla: Fix output files in source tree
>   perf tools: Put Python egg info in output directory
>   perf tools: Put Python bytecode in output directory
>
>  tools/perf/Makefile.perf            |  9 ++++++++-
>  tools/tracing/rtla/Makefile         | 31 ++++++++++++++++++-----------
>  tools/tracing/rtla/tests/timerlat.t |  4 ++--
>  3 files changed, 29 insertions(+), 15 deletions(-)
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 0/3] Fix out-of-tree build of some tools
  2026-05-21 17:50 ` [PATCH 0/3] Fix out-of-tree build of some tools Ian Rogers
@ 2026-09-15 23:08   ` Ian Rogers
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Rogers @ 2026-09-15 23:08 UTC (permalink / raw)
  To: Ben Hutchings
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Steven Rostedt, Tomas Glozar, linux-perf-users,
	linux-trace-kernel

On Thu, May 21, 2026 at 10:50 AM Ian Rogers <irogers@google.com> wrote:
>
> On Thu, May 21, 2026 at 7:57 AM Ben Hutchings <benh@debian.org> wrote:
> >
> > perf and rtla currently don't fully support out-of-tree builds, as
> > they may still create files in their source directory.  This series
> > fixes all the instances of this problem that I have found.
> >
> > Ben.
>
> Thanks for the cleanup! I ran Sashiko locally and it noted that we
> should also fix the clean steps to remove these files (I'll copy the
> full text below).
> Reviewed-by: Ian Rogers <irogers@google.com>

Hi Ben,

Did you make any progress on this? Checking upstream I don't see the
patches as landed and they look useful.

Thanks,
Ian


> ```
> commit cf212be834ce6b0df5cccdbc4ad977a917ff8e48
>    Author: Ben Hutchings <benh@debian.org>
>
>    perf tools: Put Python egg info in output directory
>
>    Installing the Python extension currently creates an egg-info
>    directory in the source tree, preventing a fully out-of-tree build.
>
>    Add the necessary runes to the setup.py comamnd line to relocate the
>    egg-info directory in an out-of-tree build.
>
>    > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
>    > index fc92d6ceac5b..b9a9bdddb89a 100644
>    > --- a/tools/perf/Makefile.perf
>    > +++ b/tools/perf/Makefile.perf
>    > @@ -905,7 +910,9 @@ install-bin: install-tools install-tests
>    >  install: install-bin try-install-man
>    >
>    >  install-python_ext:
>    > -       $(PYTHON_WORD) util/setup.py --quiet install
> --root='/$(DESTDIR_SQ)'
>    > +       $(PYTHON_WORD) util/setup.py --quiet \
>    > +               $(if $(OUTPUT),egg_info --egg-base $(OUTPUT)) \
>    > +               install --root='/$(DESTDIR_SQ)'
>
>    This relocates the egg-info directory to the output directory, which
>    helps keep the source tree clean. However, does the `python-clean`
>    target need to be updated to clean up this directory?
>
>    Currently, `python-clean` is defined as:
>
>    python-clean := $(call QUIET_CLEAN, python) $(RM) -r
> $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
>
>    Should we also clean `$(OUTPUT)/perf.egg-info` (or `perf.egg-info` for
>    in-tree builds) to ensure `make clean` removes all generated python
>    metadata? For example:
>
>    python-clean := $(call QUIET_CLEAN, python) \
>                    $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so \
>                    $(OUTPUT)perf.egg-info perf.egg-info
>
>    commit dd754ea75966ec7ade41500afeb32d944f20a8f9
>    Author: Ben Hutchings <benh@debian.org>
>
>    perf tools: Put Python bytecode in output directory
>
>    The PMU events are processed into C sources by Python scripts,
> which
>    normally results in writing bytecode for each module into the
> source
>    tree.  This prevents a fully out-of-tree build.
>
>    To fix this, set $PYTHONPYCACHEPREFIX to relocate the bytecode cache
>    directory in an out-of-tree build.
>
>    > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
>    > index fc92d6ceac5b..b9a9bdddb89a 100644
>    > --- a/tools/perf/Makefile.perf
>    > +++ b/tools/perf/Makefile.perf
>    > @@ -400,6 +400,11 @@ PYTHON_EXTBUILD_LIB :=
> $(PYTHON_EXTBUILD)lib/
>    >  PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
>    >  export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
>    >
>    > +# Put Python bytecode in output directory
>    > +ifdef OUTPUT
>    > +export PYTHONPYCACHEPREFIX := $(OUTPUT)/__pycache__
>    > +endif
>    > +
>    >  python-clean := $(call QUIET_CLEAN, python) $(RM) -r
> $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
>
>    Similar to the egg-info relocation, does `python-clean` need to
> clean up
>    the relocated `__pycache__` directory under `$(OUTPUT)`?
>
>    If `PYTHONPYCACHEPREFIX` is exported to `$(OUTPUT)/__pycache__`, running
>    `make clean` currently leaves this directory behind. Should we add it to
>    the `RM` list in `python-clean`? For example:
>
>    python-clean := $(call QUIET_CLEAN, python) \
>                    $(RM) -r $(PYTHON_EXTBUILD)
> $(OUTPUT)python/perf*.so \
>                    $(if $(OUTPUT),$(OUTPUT)/__pycache__)
> ```
>
> Thanks,
> Ian
>
> > Ben Hutchings (3):
> >   rtla: Fix output files in source tree
> >   perf tools: Put Python egg info in output directory
> >   perf tools: Put Python bytecode in output directory
> >
> >  tools/perf/Makefile.perf            |  9 ++++++++-
> >  tools/tracing/rtla/Makefile         | 31 ++++++++++++++++++-----------
> >  tools/tracing/rtla/tests/timerlat.t |  4 ++--
> >  3 files changed, 29 insertions(+), 15 deletions(-)
> >

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v2 1/2] perf tools: Put Python egg info in output directory
  2026-05-21 14:34 [PATCH 0/3] Fix out-of-tree build of some tools Ben Hutchings
                   ` (3 preceding siblings ...)
  2026-05-21 17:50 ` [PATCH 0/3] Fix out-of-tree build of some tools Ian Rogers
@ 2026-09-21 17:57 ` Ian Rogers
  2026-09-21 17:57   ` [PATCH v2 2/2] perf tools: Put Python bytecode " Ian Rogers
                     ` (2 more replies)
  4 siblings, 3 replies; 14+ messages in thread
From: Ian Rogers @ 2026-09-21 17:57 UTC (permalink / raw)
  To: benh, acme, namhyung
  Cc: linux-perf-users, linux-trace-kernel, mingo, peterz, rostedt,
	tglozar, Ian Rogers

From: Ben Hutchings <benh@debian.org>

Installing the Python extension currently creates an egg-info
directory in the source tree, preventing a fully out-of-tree build.

Add the necessary runes to the setup.py comamnd line to relocate the
egg-info directory in an out-of-tree build.

Signed-off-by: Ben Hutchings <benh@debian.org>
Reviewed-by: Ian Rogers <irogers@google.com>
---
v2: Rebase
---
 tools/perf/Makefile.perf | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 2438b40eaaec..438102bcae6c 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -908,7 +908,9 @@ install-build-deps:
 	$(Q)$(SHELL) $(srctree)/tools/perf/scripts/install-build-deps.sh $(INSTALL_BUILD_DEPS_ARGS)
 
 install-python_ext:
-	$(PYTHON_WORD) util/setup.py $(python_setup_quiet) install --root='/$(DESTDIR_SQ)'
+	$(PYTHON_WORD) util/setup.py $(python_setup_quiet) \
+		$(if $(OUTPUT),egg_info --egg-base $(OUTPUT)) \
+		install --root='/$(DESTDIR_SQ)'
 
 # 'make install-doc' should call 'make -C Documentation install'
 $(INSTALL_DOC_TARGETS):
-- 
2.55.0.1082.g2b9226bbc0-goog


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH v2 2/2] perf tools: Put Python bytecode in output directory
  2026-09-21 17:57 ` [PATCH v2 1/2] perf tools: Put Python egg info in output directory Ian Rogers
@ 2026-09-21 17:57   ` Ian Rogers
  2026-09-21 18:05     ` sashiko-bot
  2026-09-21 18:03   ` [PATCH v2 1/2] perf tools: Put Python egg info " sashiko-bot
  2026-09-28 16:12   ` Ian Rogers
  2 siblings, 1 reply; 14+ messages in thread
From: Ian Rogers @ 2026-09-21 17:57 UTC (permalink / raw)
  To: benh, acme, namhyung
  Cc: linux-perf-users, linux-trace-kernel, mingo, peterz, rostedt,
	tglozar, Ian Rogers

From: Ben Hutchings <benh@debian.org>

The PMU events are processed into C sources by Python scripts, which
normally results in writing bytecode for each module into the source
tree.  This prevents a fully out-of-tree build.

To fix this, set $PYTHONPYCACHEPREFIX to relocate the bytecode cache
directory in an out-of-tree build. Note, this environment variable is
only supported by python 3.8+, but the file Documentation/Changes
states the minimum supported python for builds is 3.9.x. In an in-tree
build in tools/perf setting the variable causes a single __pycache__
directory in the source tree.

Expand python-clean to also clean the __pycache__ and perf.pyi.

Signed-off-by: Ben Hutchings <benh@debian.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
v2: Rebase. Add clean step and commit message comment wrt older python
    versions (Sashiko). Make setting the environment variable true for
    in-tree builds, that simplifies the clean.
---
 tools/perf/Makefile.perf | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 438102bcae6c..715642630067 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -397,7 +397,8 @@ PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
 PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
 export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
 
-python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
+# Put Python bytecode in output directory
+export PYTHONPYCACHEPREFIX := $(OUTPUT)/__pycache__
 
 ifneq ($(quiet),)
 python_setup_quiet=--quiet
@@ -919,7 +920,12 @@ $(INSTALL_DOC_TARGETS):
 ### Cleaning rules
 
 python-clean:
-	$(python-clean)
+	$(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)__pycache__ \
+		$(OUTPUT)python/perf*.so
+	@if [ "$(abspath python/perf.pyi)" != "$(abspath $(OUTPUT)python/perf.pyi)" ]; \
+	then \
+		$(RM) $(OUTPUT)python/perf.pyi; \
+	fi
 
 $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT):
 	$(Q)$(MKDIR) -p $@
-- 
2.55.0.1082.g2b9226bbc0-goog


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 1/2] perf tools: Put Python egg info in output directory
  2026-09-21 17:57 ` [PATCH v2 1/2] perf tools: Put Python egg info in output directory Ian Rogers
  2026-09-21 17:57   ` [PATCH v2 2/2] perf tools: Put Python bytecode " Ian Rogers
@ 2026-09-21 18:03   ` sashiko-bot
  2026-09-28 16:12   ` Ian Rogers
  2 siblings, 0 replies; 14+ messages in thread
From: sashiko-bot @ 2026-09-21 18:03 UTC (permalink / raw)
  To: Ian Rogers; +Cc: linux-perf-users, linux-trace-kernel

> From: Ben Hutchings <benh@debian.org>
> 
> Installing the Python extension currently creates an egg-info
> directory in the source tree, preventing a fully out-of-tree build.
> 
> Add the necessary runes to the setup.py comamnd line to relocate the
> egg-info directory in an out-of-tree build.
> 
> Signed-off-by: Ben Hutchings <benh@debian.org>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260921175719.846883-1-irogers@google.com?part=1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 2/2] perf tools: Put Python bytecode in output directory
  2026-09-21 17:57   ` [PATCH v2 2/2] perf tools: Put Python bytecode " Ian Rogers
@ 2026-09-21 18:05     ` sashiko-bot
  2026-09-21 18:12       ` Ian Rogers
  0 siblings, 1 reply; 14+ messages in thread
From: sashiko-bot @ 2026-09-21 18:05 UTC (permalink / raw)
  To: Ian Rogers; +Cc: linux-perf-users, linux-trace-kernel

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf: root filesystem pollution or build failure due to stray slash in PYTHONPYCACHEPREFIX
--

commit 3de8bc01d44a9b11e10c327ec26bbff8605926df
Author: Ben Hutchings <benh@debian.org>

perf tools: Put Python bytecode in output directory

This commit sets the PYTHONPYCACHEPREFIX environment variable to relocate
the Python bytecode cache directory to the output directory during out-of-tree
builds. It also updates the python-clean rule to remove the new cache directory
and perf.pyi.

> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index a0b23d20d005c..50bde95c78a54 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -397,7 +397,8 @@ PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
>  PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
>  export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
>  
> -python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
> +# Put Python bytecode in output directory
> +export PYTHONPYCACHEPREFIX := $(OUTPUT)/__pycache__

[Severity: Medium]
For in-tree builds where $(OUTPUT) is empty, does the unconditional inclusion
of the slash cause this to evaluate to the absolute path /__pycache__?

When Python 3.8+ runs during the build, it appears it would attempt to write
bytecode files to the root of the filesystem. Could this cause a regression
by breaking the build with permission errors for unprivileged users, or
silently pollute the root filesystem if executed as root?

[ ... ]

> @@ -919,7 +920,12 @@ $(INSTALL_DOC_TARGETS):
>  ### Cleaning rules
>  
>  python-clean:
> -	$(python-clean)
> +	$(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)__pycache__ \

[Severity: Medium]
Since $(OUTPUT)__pycache__ would evaluate to __pycache__ in the current
directory during in-tree builds, would this fail to clean the incorrectly
placed bytecode directory at the root of the filesystem?

> +		$(OUTPUT)python/perf*.so
> +	@if [ "$(abspath python/perf.pyi)" != "$(abspath $(OUTPUT)python/perf.pyi)" ]; \
> +	then \
> +		$(RM) $(OUTPUT)python/perf.pyi; \
> +	fi

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260921175719.846883-1-irogers@google.com?part=2

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 2/2] perf tools: Put Python bytecode in output directory
  2026-09-21 18:05     ` sashiko-bot
@ 2026-09-21 18:12       ` Ian Rogers
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Rogers @ 2026-09-21 18:12 UTC (permalink / raw)
  To: sashiko-reviews; +Cc: linux-perf-users, linux-trace-kernel

On Mon, Sep 21, 2026 at 11:05 AM <sashiko-bot@kernel.org> wrote:
>
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] perf: root filesystem pollution or build failure due to stray slash in PYTHONPYCACHEPREFIX
> --
>
> commit 3de8bc01d44a9b11e10c327ec26bbff8605926df
> Author: Ben Hutchings <benh@debian.org>
>
> perf tools: Put Python bytecode in output directory
>
> This commit sets the PYTHONPYCACHEPREFIX environment variable to relocate
> the Python bytecode cache directory to the output directory during out-of-tree
> builds. It also updates the python-clean rule to remove the new cache directory
> and perf.pyi.
>
> > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> > index a0b23d20d005c..50bde95c78a54 100644
> > --- a/tools/perf/Makefile.perf
> > +++ b/tools/perf/Makefile.perf
> > @@ -397,7 +397,8 @@ PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
> >  PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
> >  export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
> >
> > -python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
> > +# Put Python bytecode in output directory
> > +export PYTHONPYCACHEPREFIX := $(OUTPUT)/__pycache__
>
> [Severity: Medium]
> For in-tree builds where $(OUTPUT) is empty, does the unconditional inclusion
> of the slash cause this to evaluate to the absolute path /__pycache__?
>
> When Python 3.8+ runs during the build, it appears it would attempt to write
> bytecode files to the root of the filesystem. Could this cause a regression
> by breaking the build with permission errors for unprivileged users, or
> silently pollute the root filesystem if executed as root?

This is already the behavior. The difference now is that instead of
multiple __pycache__ directories for the directory the python script
is within, there is a single top-level __pycache__ directory that is
handled by the clean step.

> [ ... ]
>
> > @@ -919,7 +920,12 @@ $(INSTALL_DOC_TARGETS):
> >  ### Cleaning rules
> >
> >  python-clean:
> > -     $(python-clean)
> > +     $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)__pycache__ \
>
> [Severity: Medium]
> Since $(OUTPUT)__pycache__ would evaluate to __pycache__ in the current
> directory during in-tree builds, would this fail to clean the incorrectly
> placed bytecode directory at the root of the filesystem?

Same issue; yes, the build would fail but the build would already be
failing if this were a problem.

Thanks,
Ian

> > +             $(OUTPUT)python/perf*.so
> > +     @if [ "$(abspath python/perf.pyi)" != "$(abspath $(OUTPUT)python/perf.pyi)" ]; \
> > +     then \
> > +             $(RM) $(OUTPUT)python/perf.pyi; \
> > +     fi
>
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20260921175719.846883-1-irogers@google.com?part=2

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 1/2] perf tools: Put Python egg info in output directory
  2026-09-21 17:57 ` [PATCH v2 1/2] perf tools: Put Python egg info in output directory Ian Rogers
  2026-09-21 17:57   ` [PATCH v2 2/2] perf tools: Put Python bytecode " Ian Rogers
  2026-09-21 18:03   ` [PATCH v2 1/2] perf tools: Put Python egg info " sashiko-bot
@ 2026-09-28 16:12   ` Ian Rogers
  2026-09-28 16:26     ` Arnaldo Carvalho de Melo
  2 siblings, 1 reply; 14+ messages in thread
From: Ian Rogers @ 2026-09-28 16:12 UTC (permalink / raw)
  To: benh, acme, namhyung
  Cc: linux-perf-users, linux-trace-kernel, mingo, peterz, rostedt,
	tglozar

On Mon, Sep 21, 2026 at 10:57 AM Ian Rogers <irogers@google.com> wrote:
>
> From: Ben Hutchings <benh@debian.org>
>
> Installing the Python extension currently creates an egg-info
> directory in the source tree, preventing a fully out-of-tree build.
>
> Add the necessary runes to the setup.py comamnd line to relocate the
> egg-info directory in an out-of-tree build.
>
> Signed-off-by: Ben Hutchings <benh@debian.org>
> Reviewed-by: Ian Rogers <irogers@google.com>
> ---
> v2: Rebase

Ben's build clean up that I've then rebased and addressed an issue
with building in tools/perf seems like a nice to land thing. I'm not
sure what more I can do to push this forward, and I replied on the
list about the Sashiko feedback, which was a little wide of the mark.
Let me know what you think; perhaps the tags are not done properly.

Thanks,
Ian


> ---
>  tools/perf/Makefile.perf | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 2438b40eaaec..438102bcae6c 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -908,7 +908,9 @@ install-build-deps:
>         $(Q)$(SHELL) $(srctree)/tools/perf/scripts/install-build-deps.sh $(INSTALL_BUILD_DEPS_ARGS)
>
>  install-python_ext:
> -       $(PYTHON_WORD) util/setup.py $(python_setup_quiet) install --root='/$(DESTDIR_SQ)'
> +       $(PYTHON_WORD) util/setup.py $(python_setup_quiet) \
> +               $(if $(OUTPUT),egg_info --egg-base $(OUTPUT)) \
> +               install --root='/$(DESTDIR_SQ)'
>
>  # 'make install-doc' should call 'make -C Documentation install'
>  $(INSTALL_DOC_TARGETS):
> --
> 2.55.0.1082.g2b9226bbc0-goog
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 1/2] perf tools: Put Python egg info in output directory
  2026-09-28 16:12   ` Ian Rogers
@ 2026-09-28 16:26     ` Arnaldo Carvalho de Melo
  0 siblings, 0 replies; 14+ messages in thread
From: Arnaldo Carvalho de Melo @ 2026-09-28 16:26 UTC (permalink / raw)
  To: Ian Rogers
  Cc: benh, namhyung, linux-perf-users, linux-trace-kernel, mingo,
	peterz, rostedt, tglozar

On Mon, Sep 28, 2026 at 09:12:32AM -0700, Ian Rogers wrote:
> On Mon, Sep 21, 2026 at 10:57 AM Ian Rogers <irogers@google.com> wrote:
> >
> > From: Ben Hutchings <benh@debian.org>
> >
> > Installing the Python extension currently creates an egg-info
> > directory in the source tree, preventing a fully out-of-tree build.
> >
> > Add the necessary runes to the setup.py comamnd line to relocate the
> > egg-info directory in an out-of-tree build.
> >
> > Signed-off-by: Ben Hutchings <benh@debian.org>
> > Reviewed-by: Ian Rogers <irogers@google.com>
> > ---
> > v2: Rebase
> 
> Ben's build clean up that I've then rebased and addressed an issue
> with building in tools/perf seems like a nice to land thing. I'm not
> sure what more I can do to push this forward, and I replied on the
> list about the Sashiko feedback, which was a little wide of the mark.
> Let me know what you think; perhaps the tags are not done properly.

I guess they are, you carried it forward and then pushed upstream, so
merits your Signed-off-by, I think.

Thanks, applied to perf-tools-next, for v7.4.

- Arnaldo

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2026-09-28 16:27 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-21 14:34 [PATCH 0/3] Fix out-of-tree build of some tools Ben Hutchings
2026-05-21 14:35 ` [PATCH 1/3] rtla: Fix output files in source tree Ben Hutchings
2026-05-21 14:48   ` Steven Rostedt
2026-05-21 14:35 ` [PATCH 2/3] perf tools: Put Python egg info in output directory Ben Hutchings
2026-05-21 14:35 ` [PATCH 3/3] perf tools: Put Python bytecode " Ben Hutchings
2026-05-21 17:50 ` [PATCH 0/3] Fix out-of-tree build of some tools Ian Rogers
2026-09-15 23:08   ` Ian Rogers
2026-09-21 17:57 ` [PATCH v2 1/2] perf tools: Put Python egg info in output directory Ian Rogers
2026-09-21 17:57   ` [PATCH v2 2/2] perf tools: Put Python bytecode " Ian Rogers
2026-09-21 18:05     ` sashiko-bot
2026-09-21 18:12       ` Ian Rogers
2026-09-21 18:03   ` [PATCH v2 1/2] perf tools: Put Python egg info " sashiko-bot
2026-09-28 16:12   ` Ian Rogers
2026-09-28 16:26     ` Arnaldo Carvalho de Melo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox