[PATCH 0/2] perf build: Fixes for arm64 sysreg-defs build rules

[PATCH 0/2] perf build: Fixes for arm64 sysreg-defs build rules

[Oliver Upton]

Small series to address the issues Ian raised in [*]. Tested with
perf-targz-src-pkg and building arm64 KVM selftests.

[*] https://lore.kernel.org/linux-perf-users/CAP-5=fXxiOJMVqaH1JPU_K6+1GhaEgVeR7YyN+F0r7qTE8DGEw@mail.gmail.com/

Oliver Upton (2):
  tools perf: Add arm64 sysreg files to MANIFEST
  perf build: Ensure sysreg-defs Makefile respects output dir

 tools/arch/arm64/tools/Makefile      |  2 +-
 tools/perf/MANIFEST                  |  2 ++
 tools/perf/Makefile.perf             | 24 +++++++++++++++---------
 tools/perf/util/Build                |  2 +-
 tools/testing/selftests/kvm/Makefile |  5 +++--
 5 files changed, 22 insertions(+), 13 deletions(-)


base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86
-- 
2.43.0.rc1.413.gea7ed67945-goog

[PATCH 1/2] tools perf: Add arm64 sysreg files to MANIFEST

[Oliver Upton]

Ian pointed out that source tarballs are incomplete as of commit
e2bdd172e665 ("perf build: Generate arm64's sysreg-defs.h and add to
include path"), since the source files needed from the kernel tree do
not appear in the manifest. Add them.

Reported-by: Ian Rogers <irogers@google.com>
Fixes: e2bdd172e665 ("perf build: Generate arm64's sysreg-defs.h and add to include path")
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 tools/perf/MANIFEST | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index 1da7f4b91b4f..dc42de1785ce 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -1,3 +1,5 @@
+arch/arm64/tools/gen-sysreg.awk
+arch/arm64/tools/sysreg
 tools/perf
 tools/arch
 tools/scripts
-- 
2.43.0.rc1.413.gea7ed67945-goog

[PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Oliver Upton]

Currently the sysreg-defs are written out to the source tree
unconditionally, ignoring the specified output directory. Correct the
build rule to emit the header to the output directory. Opportunistically
reorganize the rules to avoid interleaving with the set of beauty make
rules.

Reported-by: Ian Rogers <irogers@google.com>
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 tools/arch/arm64/tools/Makefile      |  2 +-
 tools/perf/Makefile.perf             | 24 +++++++++++++++---------
 tools/perf/util/Build                |  2 +-
 tools/testing/selftests/kvm/Makefile |  5 +++--
 4 files changed, 20 insertions(+), 13 deletions(-)

diff --git a/tools/arch/arm64/tools/Makefile b/tools/arch/arm64/tools/Makefile
index 7f64b8bb5107..7b42feedf647 100644
--- a/tools/arch/arm64/tools/Makefile
+++ b/tools/arch/arm64/tools/Makefile
@@ -22,7 +22,7 @@ endif
 arm64_tools_dir = $(top_srcdir)/arch/arm64/tools
 arm64_sysreg_tbl = $(arm64_tools_dir)/sysreg
 arm64_gen_sysreg = $(arm64_tools_dir)/gen-sysreg.awk
-arm64_generated_dir = $(top_srcdir)/tools/arch/arm64/include/generated
+arm64_generated_dir = $(OUTPUT)arch/arm64/include/generated
 arm64_sysreg_defs = $(arm64_generated_dir)/asm/sysreg-defs.h
 
 all: $(arm64_sysreg_defs)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d88da787e815..058c9aecf608 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -434,6 +434,21 @@ export INSTALL SHELL_PATH
 
 SHELL = $(SHELL_PATH)
 
+arm64_gen_sysreg_dir := $(srctree)/tools/arch/arm64/tools
+ifneq ($(OUTPUT),)
+  arm64_gen_sysreg_outdir := $(OUTPUT)
+else
+  arm64_gen_sysreg_outdir := $(CURDIR)
+endif
+
+arm64-sysreg-defs: FORCE
+	$(Q)$(MAKE) -C $(arm64_gen_sysreg_dir) O=$(arm64_gen_sysreg_outdir)
+
+arm64-sysreg-defs-clean:
+	$(call QUIET_CLEAN,arm64-sysreg-defs)
+	$(Q)$(MAKE) -C $(arm64_gen_sysreg_dir) O=$(arm64_gen_sysreg_outdir) \
+		clean > /dev/null
+
 beauty_linux_dir := $(srctree)/tools/perf/trace/beauty/include/linux/
 linux_uapi_dir := $(srctree)/tools/include/uapi/linux
 asm_generic_uapi_dir := $(srctree)/tools/include/uapi/asm-generic
@@ -450,15 +465,6 @@ drm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/drm_ioctl.sh
 # Create output directory if not already present
 _dummy := $(shell [ -d '$(beauty_ioctl_outdir)' ] || mkdir -p '$(beauty_ioctl_outdir)')
 
-arm64_gen_sysreg_dir := $(srctree)/tools/arch/arm64/tools
-
-arm64-sysreg-defs: FORCE
-	$(Q)$(MAKE) -C $(arm64_gen_sysreg_dir)
-
-arm64-sysreg-defs-clean:
-	$(call QUIET_CLEAN,arm64-sysreg-defs)
-	$(Q)$(MAKE) -C $(arm64_gen_sysreg_dir) clean > /dev/null
-
 $(drm_ioctl_array): $(drm_hdr_dir)/drm.h $(drm_hdr_dir)/i915_drm.h $(drm_ioctl_tbl)
 	$(Q)$(SHELL) '$(drm_ioctl_tbl)' $(drm_hdr_dir) > $@
 
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index fb661c48992f..988473bf907a 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -347,7 +347,7 @@ CFLAGS_rbtree.o        += -Wno-unused-parameter -DETC_PERFCONFIG="BUILD_STR($(ET
 CFLAGS_libstring.o     += -Wno-unused-parameter -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 CFLAGS_hweight.o       += -Wno-unused-parameter -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 CFLAGS_header.o        += -include $(OUTPUT)PERF-VERSION-FILE
-CFLAGS_arm-spe.o       += -I$(srctree)/tools/arch/arm64/include/ -I$(srctree)/tools/arch/arm64/include/generated/
+CFLAGS_arm-spe.o       += -I$(srctree)/tools/arch/arm64/include/ -I$(OUTPUT)arch/arm64/include/generated/
 
 $(OUTPUT)util/argv_split.o: ../lib/argv_split.c FORCE
 	$(call rule_mkdir)
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index a5963ab9215b..52c59bad7213 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -18,12 +18,13 @@ else
 endif
 
 ifeq ($(ARCH),arm64)
-arm64_tools_dir := $(top_srcdir)/tools/arch/arm64/tools/
+tools_dir := $(top_srcdir)/tools
+arm64_tools_dir := $(tools_dir)/arch/arm64/tools/
 GEN_HDRS := $(top_srcdir)/tools/arch/arm64/include/generated/
 CFLAGS += -I$(GEN_HDRS)
 
 $(GEN_HDRS): $(wildcard $(arm64_tools_dir)/*)
-	$(MAKE) -C $(arm64_tools_dir)
+	$(MAKE) -C $(arm64_tools_dir) O=$(tools_dir)
 endif
 
 LIBKVM += lib/assert.c
-- 
2.43.0.rc1.413.gea7ed67945-goog

Re: [PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 3785 bytes --]

On Tue, Nov 21, 2023 at 07:29:56PM +0000, Oliver Upton wrote:

> Currently the sysreg-defs are written out to the source tree
> unconditionally, ignoring the specified output directory. Correct the
> build rule to emit the header to the output directory. Opportunistically
> reorganize the rules to avoid interleaving with the set of beauty make
> rules.

This breaks some build configurations:

Running:

make --silent --keep-going --jobs=56 O=/home/broonie/.cache/tuxmake/builds/16/build ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LLVM=1 defconfig
make --silent --keep-going --jobs=56 O=/home/broonie/.cache/tuxmake/builds/16/bu
ild INSTALL_PATH=/home/broonie/.cache/tuxmake/builds/16/build/kselftest_install 
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LLVM=1 kselftest-install

gives:

make[4]: Leaving directory '/home/broonie/git/bisect/tools/testing/selftests/kex
ec'
make[4]: Entering directory '/home/broonie/git/bisect/tools/testing/selftests/kvm'
Makefile:270: warning: overriding recipe for target '/home/broonie/.cache/tuxmake/builds/16/build/kselftest/kvm/get-reg-list'
Makefile:265: warning: ignoring old recipe for target '/home/broonie/.cache/tuxmake/builds/16/build/kselftest/kvm/get-reg-list'
make -C ../../../../tools/arch/arm64/tools/ O=../../../../tools
make[5]: Entering directory '/home/broonie/git/bisect/tools/arch/arm64/tools'
/home/broonie/git/bisect/tools/scripts/Makefile.include:4: *** O=../../../../tools does not exist.  Stop.

I'm using tuxmake with:

    tuxmake -a arm64 -k defconfig -r docker -t llvm-17 kselftest

to invoke the build.

Bisection log identifying the commit that failed:

git bisect start
# bad: [1f5c003694fab4b1ba6cbdcc417488b975c088d0] Add linux-next specific files for 20231129
git bisect bad 1f5c003694fab4b1ba6cbdcc417488b975c088d0
# bad: [1618cb8f574175a196b0e8926c62efc45760e856] Merge branch 'for-linux-next-fixes' of git://anongit.freedesktop.org/drm/drm-misc
git bisect bad 1618cb8f574175a196b0e8926c62efc45760e856
# good: [18d46e76d7c2eedd8577fae67e3f1d4db25018b0] Merge tag 'for-6.7-rc3-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
git bisect good 18d46e76d7c2eedd8577fae67e3f1d4db25018b0
# good: [c33497720d50d39a9f3918975ee691ad570b77a6] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git
git bisect good c33497720d50d39a9f3918975ee691ad570b77a6
# good: [35fb33a953396cae852e18a316173f5a2887bb0a] Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86.git
git bisect good 35fb33a953396cae852e18a316173f5a2887bb0a
# good: [09a2ad2501da54228679dd21aaef72e783034a4e] Merge branch 'kunit-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git
git bisect good 09a2ad2501da54228679dd21aaef72e783034a4e
# bad: [a29ee6aea7030786a63fde0d6d83a8f477b060fb] perf build: Ensure sysreg-defs Makefile respects output dir
git bisect bad a29ee6aea7030786a63fde0d6d83a8f477b060fb
# good: [fd2ddee727d1ae9296b3875087410cc3698885f0] tools headers: Update tools's copy of socket.h header
git bisect good fd2ddee727d1ae9296b3875087410cc3698885f0
# good: [b3b11aed147af8f7c3d79c5b0b7474505d1dde7b] tools/perf: Update tools's copy of x86 syscall table
git bisect good b3b11aed147af8f7c3d79c5b0b7474505d1dde7b
# good: [d3968c974a2453de9289bdb9d34130b2ce323628] tools/perf: Update tools's copy of s390 syscall table
git bisect good d3968c974a2453de9289bdb9d34130b2ce323628
# good: [ef5c958090a909c9f2ab717ba6abb86869e42da7] tools perf: Add arm64 sysreg files to MANIFEST
git bisect good ef5c958090a909c9f2ab717ba6abb86869e42da7
# first bad commit: [a29ee6aea7030786a63fde0d6d83a8f477b060fb] perf build: Ensure sysreg-defs Makefile respects output dir

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Oliver Upton]

Hey Mark,

Thanks for the bug report. Apologies for the latency on my end, I've had
a mountain of non-upstream stuff to wade through.

On Wed, Nov 29, 2023 at 04:27:01PM +0000, Mark Brown wrote:
> On Tue, Nov 21, 2023 at 07:29:56PM +0000, Oliver Upton wrote:
> 
> > Currently the sysreg-defs are written out to the source tree
> > unconditionally, ignoring the specified output directory. Correct the
> > build rule to emit the header to the output directory. Opportunistically
> > reorganize the rules to avoid interleaving with the set of beauty make
> > rules.
> 
> This breaks some build configurations:
> 
> Running:
> 
> make --silent --keep-going --jobs=56 O=/home/broonie/.cache/tuxmake/builds/16/build ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LLVM=1 defconfig
> make --silent --keep-going --jobs=56 O=/home/broonie/.cache/tuxmake/builds/16/bu
> ild INSTALL_PATH=/home/broonie/.cache/tuxmake/builds/16/build/kselftest_install 
> ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LLVM=1 kselftest-install
> 
> gives:
> 
> make[4]: Leaving directory '/home/broonie/git/bisect/tools/testing/selftests/kex
> ec'
> make[4]: Entering directory '/home/broonie/git/bisect/tools/testing/selftests/kvm'
> Makefile:270: warning: overriding recipe for target '/home/broonie/.cache/tuxmake/builds/16/build/kselftest/kvm/get-reg-list'
> Makefile:265: warning: ignoring old recipe for target '/home/broonie/.cache/tuxmake/builds/16/build/kselftest/kvm/get-reg-list'
> make -C ../../../../tools/arch/arm64/tools/ O=../../../../tools
> make[5]: Entering directory '/home/broonie/git/bisect/tools/arch/arm64/tools'
> /home/broonie/git/bisect/tools/scripts/Makefile.include:4: *** O=../../../../tools does not exist.  Stop.

Looks like its because $(PWD) is tools/testing/selftests/ when building
from the top level makefile. I'll get a fix out in the next couple days
after I find my Make dunce cap :)

--
Thanks,
Oliver

Re: [PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1505 bytes --]

On Sat, Dec 02, 2023 at 01:34:20PM -0800, Oliver Upton wrote:
> Thanks for the bug report. Apologies for the latency on my end, I've had
> a mountain of non-upstream stuff to wade through.

> On Wed, Nov 29, 2023 at 04:27:01PM +0000, Mark Brown wrote:
> > On Tue, Nov 21, 2023 at 07:29:56PM +0000, Oliver Upton wrote:

> > > Currently the sysreg-defs are written out to the source tree
> > > unconditionally, ignoring the specified output directory. Correct the
> > > build rule to emit the header to the output directory. Opportunistically
> > > reorganize the rules to avoid interleaving with the set of beauty make
> > > rules.

> > This breaks some build configurations:

> > make[4]: Entering directory '/home/broonie/git/bisect/tools/testing/selftests/kvm'
> > Makefile:270: warning: overriding recipe for target '/home/broonie/.cache/tuxmake/builds/16/build/kselftest/kvm/get-reg-list'
> > Makefile:265: warning: ignoring old recipe for target '/home/broonie/.cache/tuxmake/builds/16/build/kselftest/kvm/get-reg-list'
> > make -C ../../../../tools/arch/arm64/tools/ O=../../../../tools
> > make[5]: Entering directory '/home/broonie/git/bisect/tools/arch/arm64/tools'
> > /home/broonie/git/bisect/tools/scripts/Makefile.include:4: *** O=../../../../tools does not exist.  Stop.

> Looks like its because $(PWD) is tools/testing/selftests/ when building
> from the top level makefile. I'll get a fix out in the next couple days
> after I find my Make dunce cap :)

This issue is now in Linus' tree.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

On Wed, Dec 06, 2023 at 01:45:56PM +0000, Mark Brown wrote:
> On Sat, Dec 02, 2023 at 01:34:20PM -0800, Oliver Upton wrote:
> > Thanks for the bug report. Apologies for the latency on my end, I've had
> > a mountain of non-upstream stuff to wade through.
> > On Wed, Nov 29, 2023 at 04:27:01PM +0000, Mark Brown wrote:

> > > This breaks some build configurations:

> > > make[5]: Entering directory '/home/broonie/git/bisect/tools/arch/arm64/tools'
> > > /home/broonie/git/bisect/tools/scripts/Makefile.include:4: *** O=../../../../tools does not exist.  Stop.

> > Looks like its because $(PWD) is tools/testing/selftests/ when building
> > from the top level makefile. I'll get a fix out in the next couple days
> > after I find my Make dunce cap :)

> This issue is now in Linus' tree.

Any news on this, the problem remains in -rc5?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Namhyung Kim]

Hello Oliver,

On Mon, Dec 11, 2023 at 5:19 AM Mark Brown <broonie@kernel.org> wrote:
>
> On Wed, Dec 06, 2023 at 01:45:56PM +0000, Mark Brown wrote:
> > On Sat, Dec 02, 2023 at 01:34:20PM -0800, Oliver Upton wrote:
> > > Thanks for the bug report. Apologies for the latency on my end, I've had
> > > a mountain of non-upstream stuff to wade through.
> > > On Wed, Nov 29, 2023 at 04:27:01PM +0000, Mark Brown wrote:
>
> > > > This breaks some build configurations:
>
> > > > make[5]: Entering directory '/home/broonie/git/bisect/tools/arch/arm64/tools'
> > > > /home/broonie/git/bisect/tools/scripts/Makefile.include:4: *** O=../../../../tools does not exist.  Stop.
>
> > > Looks like its because $(PWD) is tools/testing/selftests/ when building
> > > from the top level makefile. I'll get a fix out in the next couple days
> > > after I find my Make dunce cap :)
>
> > This issue is now in Linus' tree.
>
> Any news on this, the problem remains in -rc5?

Have you found some time to work on it?

Thanks,
Namhyung

Re: [PATCH 2/2] perf build: Ensure sysreg-defs Makefile respects output dir

[Oliver Upton]

On Mon, Dec 11, 2023 at 02:47:48PM -0800, Namhyung Kim wrote:
> Hello Oliver,
> 
> On Mon, Dec 11, 2023 at 5:19 AM Mark Brown <broonie@kernel.org> wrote:
> >
> > On Wed, Dec 06, 2023 at 01:45:56PM +0000, Mark Brown wrote:
> > > On Sat, Dec 02, 2023 at 01:34:20PM -0800, Oliver Upton wrote:
> > > > Thanks for the bug report. Apologies for the latency on my end, I've had
> > > > a mountain of non-upstream stuff to wade through.
> > > > On Wed, Nov 29, 2023 at 04:27:01PM +0000, Mark Brown wrote:
> >
> > > > > This breaks some build configurations:
> >
> > > > > make[5]: Entering directory '/home/broonie/git/bisect/tools/arch/arm64/tools'
> > > > > /home/broonie/git/bisect/tools/scripts/Makefile.include:4: *** O=../../../../tools does not exist.  Stop.
> >
> > > > Looks like its because $(PWD) is tools/testing/selftests/ when building
> > > > from the top level makefile. I'll get a fix out in the next couple days
> > > > after I find my Make dunce cap :)
> >
> > > This issue is now in Linus' tree.
> >
> > Any news on this, the problem remains in -rc5?
> 
> Have you found some time to work on it?

Thanks for the double-nudge, just sent the fix out. I'll take it
through the kvmarm tree as it only affects KVM selftests at this point.

https://lore.kernel.org/kvmarm/20231212070431.145544-2-oliver.upton@linux.dev/

-- 
Thanks,
Oliver

Re: [PATCH 0/2] perf build: Fixes for arm64 sysreg-defs build rules

[Ian Rogers]

On Tue, Nov 21, 2023 at 11:30 AM Oliver Upton <oliver.upton@linux.dev> wrote:
>
> Small series to address the issues Ian raised in [*]. Tested with
> perf-targz-src-pkg and building arm64 KVM selftests.
>
> [*] https://lore.kernel.org/linux-perf-users/CAP-5=fXxiOJMVqaH1JPU_K6+1GhaEgVeR7YyN+F0r7qTE8DGEw@mail.gmail.com/
>
> Oliver Upton (2):
>   tools perf: Add arm64 sysreg files to MANIFEST
>   perf build: Ensure sysreg-defs Makefile respects output dir

Sorry for missing these, scripts/get_maintainer.pl will recommend
linux-kernel@vger.kernel.org so this didn't show in my review pile due
to my email filters.

Tested-by: Ian Rogers <irogers@google.com>

Thanks,
Ian

>  tools/arch/arm64/tools/Makefile      |  2 +-
>  tools/perf/MANIFEST                  |  2 ++
>  tools/perf/Makefile.perf             | 24 +++++++++++++++---------
>  tools/perf/util/Build                |  2 +-
>  tools/testing/selftests/kvm/Makefile |  5 +++--
>  5 files changed, 22 insertions(+), 13 deletions(-)
>
>
> base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86
> --
> 2.43.0.rc1.413.gea7ed67945-goog
>

Re: [PATCH 0/2] perf build: Fixes for arm64 sysreg-defs build rules

[Oliver Upton]

On Wed, Nov 22, 2023 at 12:36:12PM -0800, Ian Rogers wrote:
> On Tue, Nov 21, 2023 at 11:30 AM Oliver Upton <oliver.upton@linux.dev> wrote:
> >
> > Small series to address the issues Ian raised in [*]. Tested with
> > perf-targz-src-pkg and building arm64 KVM selftests.
> >
> > [*] https://lore.kernel.org/linux-perf-users/CAP-5=fXxiOJMVqaH1JPU_K6+1GhaEgVeR7YyN+F0r7qTE8DGEw@mail.gmail.com/
> >
> > Oliver Upton (2):
> >   tools perf: Add arm64 sysreg files to MANIFEST
> >   perf build: Ensure sysreg-defs Makefile respects output dir
> 
> Sorry for missing these, scripts/get_maintainer.pl will recommend
> linux-kernel@vger.kernel.org so this didn't show in my review pile due
> to my email filters.

Yeah, I'd been in the (meaningless) habit of dropping the LKML
firehose for KVM changes, though I see it actually has an L: entry
for perf. My bad.

> Tested-by: Ian Rogers <irogers@google.com>

Appreciated!

-- 
Thanks,
Oliver