Re: [PATCH v1 0/7] Refactor perf python module build

From: James Clark <james.clark@arm.com>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: "Ian Rogers" <irogers@google.com>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Namhyung Kim" <namhyung@kernel.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Alexander Shishkin" <alexander.shishkin@linux.intel.com>,
	"Jiri Olsa" <jolsa@kernel.org>,
	"Adrian Hunter" <adrian.hunter@intel.com>,
	"Kan Liang" <kan.liang@linux.intel.com>,
	"John Garry" <john.g.garry@oracle.com>,
	"Will Deacon" <will@kernel.org>,
	"Mike Leach" <mike.leach@linaro.org>,
	"Leo Yan" <leo.yan@linux.dev>, "Guo Ren" <guoren@kernel.org>,
	"Paul Walmsley" <paul.walmsley@sifive.com>,
	"Palmer Dabbelt" <palmer@dabbelt.com>,
	"Albert Ou" <aou@eecs.berkeley.edu>,
	"Suzuki K Poulose" <suzuki.poulose@arm.com>,
	"Yicong Yang" <yangyicong@hisilicon.com>,
	"Jonathan Cameron" <jonathan.cameron@huawei.com>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Alex Gaynor" <alex.gaynor@gmail.com>,
	"Wedson Almeida Filho" <wedsonaf@gmail.com>,
	"Boqun Feng" <boqun.feng@gmail.com>,
	"Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <benno.lossin@proton.me>,
	"Andreas Hindborg" <a.hindborg@samsung.com>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Nick Terrell" <terrelln@fb.com>,
	"Ravi Bangoria" <ravi.bangoria@amd.com>,
	"Kees Cook" <keescook@chromium.org>,
	"Andrei Vagin" <avagin@google.com>,
	"Athira Jajeev" <atrajeev@linux.vnet.ibm.com>,
	"Oliver Upton" <oliver.upton@linux.dev>,
	"Ze Gao" <zegao2021@gmail.com>,
	linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org,
	linux-riscv@lists.infradead.org, coresight@lists.linaro.org,
	rust-for-linux@vger.kernel.org, bpf@vger.kernel.org
Subject: Re: [PATCH v1 0/7] Refactor perf python module build
Date: Thu, 13 Jun 2024 16:10:39 +0100	[thread overview]
Message-ID: <9814866a-8f9d-4d82-ad2d-4b36203aa196@arm.com> (raw)
In-Reply-To: <Zmr_CfhYsvKePZFt@x1>

On 13/06/2024 15:15, Arnaldo Carvalho de Melo wrote:
> On Thu, Jun 13, 2024 at 10:27:15AM +0100, James Clark wrote:
>> On 12/06/2024 19:31, Ian Rogers wrote:
>>> Refactor the perf python module build to instead of building C files
>>> it links libraries. To support this make static libraries for tests,
>>> ui, util and pmu-events. Doing this allows fewer functions to be
>>> stubbed out, importantly parse_events is no longer stubbed out which
>>> will improve the ability to work with heterogeneous cores.
>>>
>>> Patches 1 to 5 add static libraries for existing parts of the perf
>>> build.
>>>
>>> Patch 6 adds the python build using libraries rather than C source
>>> files.
>>>
>>> Patch 7 cleans up the python dependencies and removes the no longer
>>> needed python-ext-sources.
>>>
>>
>> Reviewed-by: James Clark <james.clark@arm.com>
>>
>> It does require a clean build to avoid some -fPIC errors presumably
>> because not everything that requires it gets rebuilt, for anyone who
>> gets stuck on that.
> 
> We need to find a way to avoid requiring the 'make clean' :-/
> 
> - Arnaldo
>  

Do we need to make it so that if any of the Makefiles are touched it
does a clean? I'm assuming that was the cause of the issue I experienced
here and that the Makefile and/or Build files aren't mentioned as
dependencies of any target.

>>> Ian Rogers (7):
>>>   perf ui: Make ui its own library
>>>   perf pmu-events: Make pmu-events a library
>>>   perf test: Make tests its own library
>>>   perf bench: Make bench its own library
>>>   perf util: Make util its own library
>>>   perf python: Switch module to linking libraries from building source
>>>   perf python: Clean up build dependencies
>>>
>>>  tools/perf/Build                              |  14 +-
>>>  tools/perf/Makefile.config                    |   5 +
>>>  tools/perf/Makefile.perf                      |  66 ++-
>>>  tools/perf/arch/Build                         |   4 +-
>>>  tools/perf/arch/arm/Build                     |   4 +-
>>>  tools/perf/arch/arm/tests/Build               |   8 +-
>>>  tools/perf/arch/arm/util/Build                |  10 +-
>>>  tools/perf/arch/arm64/Build                   |   4 +-
>>>  tools/perf/arch/arm64/tests/Build             |   8 +-
>>>  tools/perf/arch/arm64/util/Build              |  20 +-
>>>  tools/perf/arch/csky/Build                    |   2 +-
>>>  tools/perf/arch/csky/util/Build               |   6 +-
>>>  tools/perf/arch/loongarch/Build               |   2 +-
>>>  tools/perf/arch/loongarch/util/Build          |   8 +-
>>>  tools/perf/arch/mips/Build                    |   2 +-
>>>  tools/perf/arch/mips/util/Build               |   6 +-
>>>  tools/perf/arch/powerpc/Build                 |   4 +-
>>>  tools/perf/arch/powerpc/tests/Build           |   6 +-
>>>  tools/perf/arch/powerpc/util/Build            |  24 +-
>>>  tools/perf/arch/riscv/Build                   |   2 +-
>>>  tools/perf/arch/riscv/util/Build              |   8 +-
>>>  tools/perf/arch/s390/Build                    |   2 +-
>>>  tools/perf/arch/s390/util/Build               |  16 +-
>>>  tools/perf/arch/sh/Build                      |   2 +-
>>>  tools/perf/arch/sh/util/Build                 |   2 +-
>>>  tools/perf/arch/sparc/Build                   |   2 +-
>>>  tools/perf/arch/sparc/util/Build              |   2 +-
>>>  tools/perf/arch/x86/Build                     |   6 +-
>>>  tools/perf/arch/x86/tests/Build               |  20 +-
>>>  tools/perf/arch/x86/util/Build                |  42 +-
>>>  tools/perf/bench/Build                        |  46 +-
>>>  tools/perf/scripts/Build                      |   4 +-
>>>  tools/perf/scripts/perl/Perf-Trace-Util/Build |   2 +-
>>>  .../perf/scripts/python/Perf-Trace-Util/Build |   2 +-
>>>  tools/perf/tests/Build                        | 140 +++----
>>>  tools/perf/tests/workloads/Build              |  12 +-
>>>  tools/perf/ui/Build                           |  18 +-
>>>  tools/perf/ui/browsers/Build                  |  14 +-
>>>  tools/perf/ui/tui/Build                       |   8 +-
>>>  tools/perf/util/Build                         | 394 +++++++++---------
>>>  tools/perf/util/arm-spe-decoder/Build         |   2 +-
>>>  tools/perf/util/cs-etm-decoder/Build          |   2 +-
>>>  tools/perf/util/hisi-ptt-decoder/Build        |   2 +-
>>>  tools/perf/util/intel-pt-decoder/Build        |   2 +-
>>>  tools/perf/util/perf-regs-arch/Build          |  18 +-
>>>  tools/perf/util/python-ext-sources            |  53 ---
>>>  tools/perf/util/python.c                      | 271 +++++-------
>>>  tools/perf/util/scripting-engines/Build       |   4 +-
>>>  tools/perf/util/setup.py                      |  33 +-
>>>  49 files changed, 612 insertions(+), 722 deletions(-)
>>>  delete mode 100644 tools/perf/util/python-ext-sources
>>>