* [PATCH v2 1/6] tools/build: Don't pass test log files to linker
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
@ 2025-03-11 21:36 ` Ian Rogers
2025-03-11 21:36 ` [PATCH v2 2/6] perf build: Rename TEST_LOGS to SHELL_TEST_LOGS Ian Rogers
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-11 21:36 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
Separate test log files from object files. Depend on test log output
but don't pass to the linker.
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/build/Makefile.build | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
index e710ed67a1b4..3584ff308607 100644
--- a/tools/build/Makefile.build
+++ b/tools/build/Makefile.build
@@ -129,6 +129,10 @@ objprefix := $(subst ./,,$(OUTPUT)$(dir)/)
obj-y := $(addprefix $(objprefix),$(obj-y))
subdir-obj-y := $(addprefix $(objprefix),$(subdir-obj-y))
+# Separate out test log files from real build objects.
+test-y := $(filter %_log, $(obj-y))
+obj-y := $(filter-out %_log, $(obj-y))
+
# Final '$(obj)-in.o' object
in-target := $(objprefix)$(obj)-in.o
@@ -139,7 +143,7 @@ $(subdir-y):
$(sort $(subdir-obj-y)): $(subdir-y) ;
-$(in-target): $(obj-y) FORCE
+$(in-target): $(obj-y) $(test-y) FORCE
$(call rule_mkdir)
$(call if_changed,$(host)ld_multi)
--
2.49.0.rc0.332.g42c0ae87b1-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v2 2/6] perf build: Rename TEST_LOGS to SHELL_TEST_LOGS
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
2025-03-11 21:36 ` [PATCH v2 1/6] tools/build: Don't pass test log files to linker Ian Rogers
@ 2025-03-11 21:36 ` Ian Rogers
2025-03-11 21:36 ` [PATCH v2 3/6] perf build: Add mypy build tests Ian Rogers
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-11 21:36 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
Rename TEST_LOGS to SHELL_TEST_LOGS as later changes will add more
kinds of test logs.
Minor comment tweak in Makefile.perf as more than just test shell
tests are checked.
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Build | 6 +++---
tools/perf/Makefile.perf | 2 +-
tools/perf/arch/x86/Build | 6 +++---
tools/perf/arch/x86/tests/Build | 6 +++---
tools/perf/tests/Build | 6 +++---
tools/perf/trace/beauty/Build | 6 +++---
tools/perf/util/Build | 6 +++---
7 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/tools/perf/Build b/tools/perf/Build
index 3e486f7df94b..5e385f370dd7 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -65,14 +65,14 @@ gtk-y += ui/gtk/
ifdef SHELLCHECK
SHELL_TESTS := $(wildcard *.sh)
- TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
+ SHELL_TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
else
SHELL_TESTS :=
- TEST_LOGS :=
+ SHELL_TEST_LOGS :=
endif
$(OUTPUT)%.shellcheck_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)shellcheck -s bash -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-perf-y += $(TEST_LOGS)
+perf-y += $(SHELL_TEST_LOGS)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d0b50ccc9d7b..aebdf317afcc 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -289,7 +289,7 @@ else
force_fixdep := $(config)
endif
-# Runs shellcheck on perf test shell scripts
+# Runs shellcheck on perf shell scripts
ifeq ($(NO_SHELLCHECK),1)
SHELLCHECK :=
else
diff --git a/tools/perf/arch/x86/Build b/tools/perf/arch/x86/Build
index 02a1ca780a20..afae7b8f6bd6 100644
--- a/tools/perf/arch/x86/Build
+++ b/tools/perf/arch/x86/Build
@@ -2,14 +2,14 @@ perf-util-y += util/
perf-test-y += tests/
ifdef SHELLCHECK
- TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
+ SHELL_TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
else
SHELL_TESTS :=
- TEST_LOGS :=
+ SHELL_TEST_LOGS :=
endif
$(OUTPUT)%.shellcheck_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-perf-test-y += $(TEST_LOGS)
+perf-test-y += $(SHELL_TEST_LOGS)
diff --git a/tools/perf/arch/x86/tests/Build b/tools/perf/arch/x86/tests/Build
index 3227053f3355..86262c720857 100644
--- a/tools/perf/arch/x86/tests/Build
+++ b/tools/perf/arch/x86/tests/Build
@@ -13,14 +13,14 @@ perf-test-y += amd-ibs-via-core-pmu.o
ifdef SHELLCHECK
SHELL_TESTS := gen-insn-x86-dat.sh
- TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
+ SHELL_TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
else
SHELL_TESTS :=
- TEST_LOGS :=
+ SHELL_TEST_LOGS :=
endif
$(OUTPUT)%.shellcheck_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-perf-test-y += $(TEST_LOGS)
+perf-test-y += $(SHELL_TEST_LOGS)
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 4bf8d3f5eae7..5b4b7a3825fd 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -80,14 +80,14 @@ perf-test-y += workloads/
ifdef SHELLCHECK
SHELL_TESTS := $(shell find tests/shell -executable -type f -name '*.sh')
- TEST_LOGS := $(SHELL_TESTS:tests/shell/%=shell/%.shellcheck_log)
+ SHELL_TEST_LOGS := $(SHELL_TESTS:tests/shell/%=shell/%.shellcheck_log)
else
SHELL_TESTS :=
- TEST_LOGS :=
+ SHELL_TEST_LOGS :=
endif
$(OUTPUT)%.shellcheck_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-perf-test-y += $(TEST_LOGS)
+perf-test-y += $(SHELL_TEST_LOGS)
diff --git a/tools/perf/trace/beauty/Build b/tools/perf/trace/beauty/Build
index cb3c1399ff40..f50ebdc445b8 100644
--- a/tools/perf/trace/beauty/Build
+++ b/tools/perf/trace/beauty/Build
@@ -23,14 +23,14 @@ perf-y += tracepoints/
ifdef SHELLCHECK
SHELL_TESTS := $(wildcard trace/beauty/*.sh)
- TEST_LOGS := $(SHELL_TESTS:trace/beauty/%=%.shellcheck_log)
+ SHELL_TEST_LOGS := $(SHELL_TESTS:trace/beauty/%=%.shellcheck_log)
else
SHELL_TESTS :=
- TEST_LOGS :=
+ SHELL_TEST_LOGS :=
endif
$(OUTPUT)%.shellcheck_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)shellcheck -s bash -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-perf-y += $(TEST_LOGS)
+perf-y += $(SHELL_TEST_LOGS)
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 034a6603d5a8..fd595454766e 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -406,14 +406,14 @@ $(OUTPUT)util/list_sort.o: ../lib/list_sort.c FORCE
ifdef SHELLCHECK
SHELL_TESTS := generate-cmdlist.sh
- TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
+ SHELL_TEST_LOGS := $(SHELL_TESTS:%=%.shellcheck_log)
else
SHELL_TESTS :=
- TEST_LOGS :=
+ SHELL_TEST_LOGS :=
endif
$(OUTPUT)%.shellcheck_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-perf-util-y += $(TEST_LOGS)
+perf-util-y += $(SHELL_TEST_LOGS)
--
2.49.0.rc0.332.g42c0ae87b1-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v2 3/6] perf build: Add mypy build tests
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
2025-03-11 21:36 ` [PATCH v2 1/6] tools/build: Don't pass test log files to linker Ian Rogers
2025-03-11 21:36 ` [PATCH v2 2/6] perf build: Rename TEST_LOGS to SHELL_TEST_LOGS Ian Rogers
@ 2025-03-11 21:36 ` Ian Rogers
2025-03-11 21:36 ` [PATCH v2 4/6] perf build: Add pylint " Ian Rogers
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-11 21:36 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
If MYPY=1 is passed to the build then run mypy over python code in
perf. Unlike shellcheck this isn't default on as there are currently
too many errors.
An example of an error:
```
util/setup.py:8: error: Item "None" of "str | None" has no attribute "split" [union-attr]
util/setup.py:15: error: Item "None" of "IO[bytes] | None" has no attribute "readline" [union-attr]
util/setup.py:15: error: List item 0 has incompatible type "str | None"; expected "str | bytes | PathLike[str] | PathLike[bytes]" [list-item]
util/setup.py:16: error: Unsupported left operand type for + ("None") [operator]
util/setup.py:16: note: Left operand is of type "str | None"
util/setup.py:74: error: Unsupported left operand type for + ("None") [operator]
util/setup.py:74: note: Left operand is of type "str | None"
Found 5 errors in 1 file (checked 1 source file)
make[4]: *** [util/Build:430: util/setup.py.mypy_log] Error 1
```
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Build | 13 +++++++++++++
tools/perf/Makefile.perf | 7 ++++++-
tools/perf/pmu-events/Build | 14 +++++++++++++-
tools/perf/scripts/Build | 13 +++++++++++++
tools/perf/tests/Build | 13 +++++++++++++
tools/perf/util/Build | 13 +++++++++++++
6 files changed, 71 insertions(+), 2 deletions(-)
diff --git a/tools/perf/Build b/tools/perf/Build
index 5e385f370dd7..312914994c89 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -76,3 +76,16 @@ $(OUTPUT)%.shellcheck_log: %
$(Q)$(call echo-cmd,test)shellcheck -s bash -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
perf-y += $(SHELL_TEST_LOGS)
+
+ifdef MYPY
+ PY_TESTS := $(shell find python -type f -name '*.py')
+ MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
+else
+ MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index aebdf317afcc..c773f0dd8226 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -306,8 +306,13 @@ ifneq ($(SHELLCHECK),)
endif
endif
+# Runs mypy on perf python files
+ifeq ($(MYPY),1)
+ MYPY := $(shell which mypy 2> /dev/null)
+endif
+
export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
-export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK
+export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY
include $(srctree)/tools/build/Makefile.include
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index d941bc9d16e9..fc1dc810ffb6 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -41,7 +41,19 @@ $(EMPTY_PMU_EVENTS_TEST_LOG): $(EMPTY_PMU_EVENTS_C) $(TEST_EMPTY_PMU_EVENTS_C)
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)diff -u $^ 2> $@ || (cat $@ && false)
-$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) $(EMPTY_PMU_EVENTS_TEST_LOG)
+ifdef MYPY
+ PMU_EVENTS_PY_TESTS := $(wildcard *.py)
+ PMU_EVENTS_MYPY_TEST_LOGS := $(JEVENTS_PY_TESTS:%=%.mypy_log)
+else
+ PMU_EVENTS_MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) \
+ $(EMPTY_PMU_EVENTS_TEST_LOG) $(PMU_EVENTS_MYPY_TEST_LOGS)
$(call rule_mkdir)
$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) pmu-events/arch $@
endif
diff --git a/tools/perf/scripts/Build b/tools/perf/scripts/Build
index 46f0c6f76dbf..a5350dc6ac50 100644
--- a/tools/perf/scripts/Build
+++ b/tools/perf/scripts/Build
@@ -2,3 +2,16 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y)
perf-util-$(CONFIG_LIBPERL) += perl/Perf-Trace-Util/
endif
perf-util-$(CONFIG_LIBPYTHON) += python/Perf-Trace-Util/
+
+ifdef MYPY
+ PY_TESTS := $(shell find python -type f -name '*.py')
+ MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
+else
+ MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 5b4b7a3825fd..0d4af485bf55 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -91,3 +91,16 @@ $(OUTPUT)%.shellcheck_log: %
$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
perf-test-y += $(SHELL_TEST_LOGS)
+
+ifdef MYPY
+ PY_TESTS := $(shell find tests/shell -type f -name '*.py')
+ MYPY_TEST_LOGS := $(PY_TESTS:tests/shell/%=shell/%.mypy_log)
+else
+ MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-test-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index fd595454766e..583f425713eb 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -417,3 +417,16 @@ $(OUTPUT)%.shellcheck_log: %
$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
perf-util-y += $(SHELL_TEST_LOGS)
+
+PY_TESTS := setup.py
+ifdef MYPY
+ MYPY_TEST_LOGS := $(PY_TESTS:%=%.mypy_log)
+else
+ MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-util-y += $(MYPY_TEST_LOGS)
--
2.49.0.rc0.332.g42c0ae87b1-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v2 4/6] perf build: Add pylint build tests
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
` (2 preceding siblings ...)
2025-03-11 21:36 ` [PATCH v2 3/6] perf build: Add mypy build tests Ian Rogers
@ 2025-03-11 21:36 ` Ian Rogers
2025-03-11 21:36 ` [PATCH v2 5/6] perf test: Address attr.py mypy error Ian Rogers
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-11 21:36 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
If PYLINT=1 is passed to the build then run pylint over python code in
perf. Unlike shellcheck this isn't default on as there are currently
too many errors.
An example of an error:
```
************* Module setup
util/setup.py:19:0: C0301: Line too long (127/100) (line-too-long)
util/setup.py:20:0: C0301: Line too long (138/100) (line-too-long)
util/setup.py:63:0: C0301: Line too long (106/100) (line-too-long)
util/setup.py:1:0: C0114: Missing module docstring (missing-module-docstring)
util/setup.py:24:4: W0622: Redefining built-in 'vars' (redefined-builtin)
util/setup.py:11:4: C0103: Constant name "cc_options" doesn't conform to UPPER_CASE naming style (invalid-name)
util/setup.py:13:4: C0103: Constant name "cc_options" doesn't conform to UPPER_CASE naming style (invalid-name)
util/setup.py:15:34: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
util/setup.py:18:0: C0116: Missing function or method docstring (missing-function-docstring)
util/setup.py:19:16: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
util/setup.py:44:0: C0413: Import "from setuptools import setup, Extension" should be placed at the top of the module (wrong-import-position)
util/setup.py:46:0: C0413: Import "from setuptools.command.build_ext import build_ext as _build_ext" should be placed at the top of the module (wrong-import-position)
util/setup.py:47:0: C0413: Import "from setuptools.command.install_lib import install_lib as _install_lib" should be placed at the top of the module (wrong-import-position)
util/setup.py:49:0: C0115: Missing class docstring (missing-class-docstring)
util/setup.py:49:0: C0103: Class name "build_ext" doesn't conform to PascalCase naming style (invalid-name)
util/setup.py:52:8: W0201: Attribute 'build_lib' defined outside __init__ (attribute-defined-outside-init)
util/setup.py:53:8: W0201: Attribute 'build_temp' defined outside __init__ (attribute-defined-outside-init)
util/setup.py:55:0: C0115: Missing class docstring (missing-class-docstring)
util/setup.py:55:0: C0103: Class name "install_lib" doesn't conform to PascalCase naming style (invalid-name)
util/setup.py:58:8: W0201: Attribute 'build_dir' defined outside __init__ (attribute-defined-outside-init)
*-----------------------------------------------------------------
Your code has been rated at 6.67/10 (previous run: 6.51/10, +0.16)
make[4]: *** [util/Build:442: util/setup.py.pylint_log] Error 1
```
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Build | 13 +++++++++++++
tools/perf/Makefile.perf | 7 ++++++-
tools/perf/pmu-events/Build | 13 ++++++++++++-
tools/perf/scripts/Build | 13 +++++++++++++
tools/perf/tests/Build | 13 +++++++++++++
tools/perf/util/Build | 12 ++++++++++++
6 files changed, 69 insertions(+), 2 deletions(-)
diff --git a/tools/perf/Build b/tools/perf/Build
index 312914994c89..06107f1e1d42 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -89,3 +89,16 @@ $(OUTPUT)%.mypy_log: %
$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
perf-y += $(MYPY_TEST_LOGS)
+
+ifdef PYLINT
+ PY_TESTS := $(shell find python -type f -name '*.py')
+ PYLINT_TEST_LOGS := $(PY_TESTS:python/%=python/%.pylint_log)
+else
+ PYLINT_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.pylint_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(PYLINT_TEST_LOGS)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index c773f0dd8226..367746e3c9b0 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -311,8 +311,13 @@ ifeq ($(MYPY),1)
MYPY := $(shell which mypy 2> /dev/null)
endif
+# Runs pylint on perf python files
+ifeq ($(PYLINT),1)
+ PYLINT := $(shell which pylint 2> /dev/null)
+endif
+
export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
-export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY
+export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT
include $(srctree)/tools/build/Makefile.include
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index fc1dc810ffb6..32f387d48908 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -52,8 +52,19 @@ $(OUTPUT)%.mypy_log: %
$(call rule_mkdir)
$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+ifdef PYLINT
+ PMU_EVENTS_PY_TESTS := $(wildcard *.py)
+ PMU_EVENTS_PYLINT_TEST_LOGS := $(JEVENTS_PY_TESTS:%=%.pylint_log)
+else
+ PMU_EVENTS_PYLINT_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.pylint_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+
$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) \
- $(EMPTY_PMU_EVENTS_TEST_LOG) $(PMU_EVENTS_MYPY_TEST_LOGS)
+ $(EMPTY_PMU_EVENTS_TEST_LOG) $(PMU_EVENTS_MYPY_TEST_LOGS) $(PMU_EVENTS_PYLINT_TEST_LOGS)
$(call rule_mkdir)
$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) pmu-events/arch $@
endif
diff --git a/tools/perf/scripts/Build b/tools/perf/scripts/Build
index a5350dc6ac50..91229a1fe3ff 100644
--- a/tools/perf/scripts/Build
+++ b/tools/perf/scripts/Build
@@ -15,3 +15,16 @@ $(OUTPUT)%.mypy_log: %
$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
perf-y += $(MYPY_TEST_LOGS)
+
+ifdef PYLINT
+ PY_TESTS := $(shell find python -type f -name '*.py')
+ PYLINT_TEST_LOGS := $(PY_TESTS:python/%=python/%.pylint_log)
+else
+ PYLINT_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.pylint_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(PYLINT_TEST_LOGS)
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 0d4af485bf55..934f32090553 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -104,3 +104,16 @@ $(OUTPUT)%.mypy_log: %
$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
perf-test-y += $(MYPY_TEST_LOGS)
+
+ifdef PYLINT
+ PY_TESTS := $(shell find tests/shell -type f -name '*.py')
+ PYLINT_TEST_LOGS := $(PY_TESTS:tests/shell/%=shell/%.pylint_log)
+else
+ PYLINT_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.pylint_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-test-y += $(PYLINT_TEST_LOGS)
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 583f425713eb..946bce6628f3 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -430,3 +430,15 @@ $(OUTPUT)%.mypy_log: %
$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
perf-util-y += $(MYPY_TEST_LOGS)
+
+ifdef PYLINT
+ PYLINT_TEST_LOGS := $(PY_TESTS:%=%.pylint_log)
+else
+ PYLINT_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.pylint_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-util-y += $(PYLINT_TEST_LOGS)
--
2.49.0.rc0.332.g42c0ae87b1-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v2 5/6] perf test: Address attr.py mypy error
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
` (3 preceding siblings ...)
2025-03-11 21:36 ` [PATCH v2 4/6] perf build: Add pylint " Ian Rogers
@ 2025-03-11 21:36 ` Ian Rogers
2025-03-11 21:36 ` [PATCH v2 6/6] perf python: Fix setup.py mypy errors Ian Rogers
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-11 21:36 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
ConfigParser existed in python2 but not in python3 causing mypy to
fail.
Whilst removing a python2 workaround remove reference to __future__.
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/tests/shell/lib/attr.py | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/tools/perf/tests/shell/lib/attr.py b/tools/perf/tests/shell/lib/attr.py
index 3db9a7d78715..bfccc727d9b2 100644
--- a/tools/perf/tests/shell/lib/attr.py
+++ b/tools/perf/tests/shell/lib/attr.py
@@ -1,7 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
-from __future__ import print_function
-
+import configparser
import os
import sys
import glob
@@ -13,11 +12,6 @@ import re
import shutil
import subprocess
-try:
- import configparser
-except ImportError:
- import ConfigParser as configparser
-
def data_equal(a, b):
# Allow multiple values in assignment separated by '|'
a_list = a.split('|')
--
2.49.0.rc0.332.g42c0ae87b1-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v2 6/6] perf python: Fix setup.py mypy errors
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
` (4 preceding siblings ...)
2025-03-11 21:36 ` [PATCH v2 5/6] perf test: Address attr.py mypy error Ian Rogers
@ 2025-03-11 21:36 ` Ian Rogers
2025-03-24 16:15 ` [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
2025-03-25 19:49 ` Namhyung Kim
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-11 21:36 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
getenv may return None, so assert it isn't None for CC and srctree
environmental variables required for the script.
Disable an optional warning related to Popen.
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/util/setup.py | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py
index 649550e9b7aa..dd289d15acfd 100644
--- a/tools/perf/util/setup.py
+++ b/tools/perf/util/setup.py
@@ -3,6 +3,7 @@ from subprocess import Popen, PIPE
from re import sub
cc = getenv("CC")
+assert cc, "Environment variable CC not set"
# Check if CC has options, as is the case in yocto, where it uses CC="cc --sysroot..."
cc_tokens = cc.split()
@@ -12,8 +13,13 @@ if len(cc_tokens) > 1:
else:
cc_options = ""
+# ignore optional stderr could be None as it is set to PIPE to avoid that.
+# mypy: disable-error-code="union-attr"
cc_is_clang = b"clang version" in Popen([cc, "-v"], stderr=PIPE).stderr.readline()
-src_feature_tests = getenv('srctree') + '/tools/build/feature'
+
+srctree = getenv('srctree')
+assert srctree, "Environment variable srctree, for the Linux sources, not set"
+src_feature_tests = f'{srctree}/tools/build/feature'
def clang_has_option(option):
cc_output = Popen([cc, cc_options + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines()
@@ -71,7 +77,7 @@ else:
# The python headers have mixed code with declarations (decls after asserts, for instance)
cflags += [ "-Wno-declaration-after-statement" ]
-src_perf = getenv('srctree') + '/tools/perf'
+src_perf = f'{srctree}/tools/perf'
build_lib = getenv('PYTHON_EXTBUILD_LIB')
build_tmp = getenv('PYTHON_EXTBUILD_TMP')
--
2.49.0.rc0.332.g42c0ae87b1-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH v2 0/6] Mypy and pylint build support
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
` (5 preceding siblings ...)
2025-03-11 21:36 ` [PATCH v2 6/6] perf python: Fix setup.py mypy errors Ian Rogers
@ 2025-03-24 16:15 ` Ian Rogers
2025-03-25 19:49 ` Namhyung Kim
7 siblings, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2025-03-24 16:15 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, Kan Liang, James Clark,
Charlie Jenkins, John Garry, Veronika Molnarova,
Masami Hiramatsu (Google), linux-kernel, linux-perf-users
On Tue, Mar 11, 2025 at 2:36 PM Ian Rogers <irogers@google.com> wrote:
>
> Support building perf checking the python code with mypy and/or
> pylint. Currently there are too many errors to make this a default.
>
> Shellcheck generates no output on success, so linking shellcheck files
> doesn't cause `ld` to fail. Mypy and pylint generate output that will
> break `ld` so change the Makefile.build to ignore test log files.
>
> Address some initial mypy errors.
>
> v2: Rebase. Add James Clark's reviewed-by tags. Add example mypy and
> pylint error output.
Ping.
Thanks,
Ian
> Ian Rogers (6):
> tools/build: Don't pass test log files to linker
> perf build: Rename TEST_LOGS to SHELL_TEST_LOGS
> perf build: Add mypy build tests
> perf build: Add pylint build tests
> perf test: Address attr.py mypy error
> perf python: Fix setup.py mypy errors
>
> tools/build/Makefile.build | 6 +++++-
> tools/perf/Build | 32 +++++++++++++++++++++++++++---
> tools/perf/Makefile.perf | 14 +++++++++++--
> tools/perf/arch/x86/Build | 6 +++---
> tools/perf/arch/x86/tests/Build | 6 +++---
> tools/perf/pmu-events/Build | 25 ++++++++++++++++++++++-
> tools/perf/scripts/Build | 26 ++++++++++++++++++++++++
> tools/perf/tests/Build | 32 +++++++++++++++++++++++++++---
> tools/perf/tests/shell/lib/attr.py | 8 +-------
> tools/perf/trace/beauty/Build | 6 +++---
> tools/perf/util/Build | 31 ++++++++++++++++++++++++++---
> tools/perf/util/setup.py | 10 ++++++++--
> 12 files changed, 171 insertions(+), 31 deletions(-)
>
> --
> 2.49.0.rc0.332.g42c0ae87b1-goog
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v2 0/6] Mypy and pylint build support
2025-03-11 21:36 [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
` (6 preceding siblings ...)
2025-03-24 16:15 ` [PATCH v2 0/6] Mypy and pylint build support Ian Rogers
@ 2025-03-25 19:49 ` Namhyung Kim
7 siblings, 0 replies; 9+ messages in thread
From: Namhyung Kim @ 2025-03-25 19:49 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Mark Rutland, Alexander Shishkin, Jiri Olsa, Adrian Hunter,
Kan Liang, James Clark, Charlie Jenkins, John Garry,
Veronika Molnarova, Masami Hiramatsu (Google), linux-kernel,
linux-perf-users, Ian Rogers
On Tue, 11 Mar 2025 14:36:22 -0700, Ian Rogers wrote:
> Support building perf checking the python code with mypy and/or
> pylint. Currently there are too many errors to make this a default.
>
> Shellcheck generates no output on success, so linking shellcheck files
> doesn't cause `ld` to fail. Mypy and pylint generate output that will
> break `ld` so change the Makefile.build to ignore test log files.
>
> [...]
Applied to perf-tools-next, thanks!
Best regards,
Namhyung
^ permalink raw reply [flat|nested] 9+ messages in thread