From: Ian Rogers <irogers@google.com>
To: irogers@google.com, acme@kernel.org, james.clark@linaro.org,
namhyung@kernel.org
Cc: 9erthalion6@gmail.com, adrian.hunter@intel.com, alex@ghiti.fr,
alexandre.chartre@oracle.com, andrii@kernel.org,
ankur.a.arora@oracle.com, aou@eecs.berkeley.edu,
bpf@vger.kernel.org, collin.funk1@gmail.com,
costa.shul@redhat.com, daniel@iogearbox.net,
dapeng1.mi@linux.intel.com, dsterba@suse.com, eddyz87@gmail.com,
howardchu95@gmail.com, jolsa@kernel.org, leo.yan@arm.com,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
martin.lau@linux.dev, memxor@gmail.com, mingo@redhat.com,
mmayer@broadcom.com, nathan@kernel.org, palmer@dabbelt.com,
peterz@infradead.org, pjw@kernel.org, qmo@kernel.org,
ricky.ringler@proton.me, song@kernel.org,
swapnil.sapkal@amd.com, terrelln@fb.com, tglozar@redhat.com,
thomas.falcon@intel.com, yonghong.song@linux.dev
Subject: [PATCH v3 13/17] perf pmu-events: Split big_c_string storage into standalone compilation unit
Date: Thu, 14 May 2026 09:34:05 -0700 [thread overview]
Message-ID: <20260514163409.927816-14-irogers@google.com> (raw)
In-Reply-To: <20260514163409.927816-1-irogers@google.com>
Currently, jevents.py emits both the massive 2.8 MB big_c_string literal
and tens of thousands of compact_pmu_event struct arrays into a single
pmu-events.c compilation unit. Compiling this giant file takes ~2.2 seconds
on a single CPU core during Kbuild startup.
Refactor jevents.py to emit big_c_string into a dedicated
pmu-events-string.c compilation unit. This allows Kbuild to compile
pmu-events.o and pmu-events-string.o simultaneously in parallel across
two separate CPU cores, preserving 100% string deduplication and zero
dynamic ELF relocations while cutting C compilation latency in half.
Add pmu-events-string.c to tools/perf/.gitignore to ensure in-tree Kbuild
runs do not leave untracked generated files in the working directory.
To guarantee 100% backward compatibility with GNU Make 4.0+ (avoiding the
Make 4.3+ grouped target &: syntax which causes older Make versions like
4.2.1 to spawn multiple concurrent jevents.py processes during parallel
builds), implement a robust dependency chaining pattern:
$(PMU_EVENTS_C): $(JEVENTS_DEPS)
$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C)
@:
This ensures jevents.py is invoked exactly once. If jevents.py aborts
early, Make's .DELETE_ON_ERROR: purges pmu-events.c, guaranteeing that
subsequent Make invocations correctly re-execute the script and overwrite
pmu-events-string.c, keeping the build fully self-correcting.
Tested-by: James Clark <james.clark@linaro.org>
Assisted-by: Gemini:gemini-3.1-pro-preview
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/.gitignore | 1 +
tools/perf/Makefile.perf | 4 ++--
tools/perf/pmu-events/Build | 16 +++++++++++++++-
tools/perf/pmu-events/jevents.py | 20 ++++++++++++++++----
4 files changed, 34 insertions(+), 7 deletions(-)
diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 0f9451a6e39c..3b968c5158b8 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -38,6 +38,7 @@ arch/*/include/generated/
trace/beauty/generated/
pmu-events/arch/common/common/legacy-cache.json
pmu-events/pmu-events.c
+pmu-events/pmu-events-string.c
pmu-events/jevents
pmu-events/metric_test.log
pmu-events/empty-pmu-events.log
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 01da32cd8988..3be0be5eb43b 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -921,7 +921,7 @@ bpf-skel-clean:
pmu-events-clean:
ifeq ($(OUTPUT),)
$(call QUIET_CLEAN, pmu-events) $(RM) \
- pmu-events/pmu-events.c \
+ pmu-events/pmu-events*.c \
pmu-events/metric_test.log \
pmu-events/test-empty-pmu-events.c \
pmu-events/empty-pmu-events.log
@@ -929,7 +929,7 @@ ifeq ($(OUTPUT),)
-name 'extra-metricgroups.json' -delete
else # When an OUTPUT directory is present, clean up the copied pmu-events/arch directory.
$(call QUIET_CLEAN, pmu-events) $(RM) -r $(OUTPUT)pmu-events/arch \
- $(OUTPUT)pmu-events/pmu-events.c \
+ $(OUTPUT)pmu-events/pmu-events*.c \
$(OUTPUT)pmu-events/metric_test.log \
$(OUTPUT)pmu-events/test-empty-pmu-events.c \
$(OUTPUT)pmu-events/empty-pmu-events.log
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index dc1df2d57ddc..95172a2a851f 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -1,7 +1,12 @@
EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c
# pmu-events.c will be generated by jevents.py or copied from EMPTY_PMU_EVENTS_C
PMU_EVENTS_C = $(OUTPUT)pmu-events/pmu-events.c
+PMU_EVENTS_STRING_C = $(OUTPUT)pmu-events/pmu-events-string.c
+
pmu-events-y += pmu-events.o
+ifneq ($(NO_JEVENTS),1)
+pmu-events-y += pmu-events-string.o
+endif
# pmu-events.c file is generated in the OUTPUT directory so it needs a
# separate rule to depend on it properly
@@ -9,6 +14,10 @@ $(OUTPUT)pmu-events/pmu-events.o: $(PMU_EVENTS_C)
$(call rule_mkdir)
$(call if_changed_dep,cc_o_c)
+$(OUTPUT)pmu-events/pmu-events-string.o: $(PMU_EVENTS_STRING_C)
+ $(call rule_mkdir)
+ $(call if_changed_dep,cc_o_c)
+
# Message for $(call echo-cmd,cp), possibly remove the src file from
# the destination to save space in the build log.
quiet_cmd_cp = COPY $(patsubst %$<,%,$@) <- $<
@@ -118,6 +127,7 @@ CUR_OUT_JSON := $(shell [ -d $(OUT_DIR) ] && find $(OUT_DIR) -type f)
# Things in the OUTPUT directory but shouldn't be there as computed by
# OUT_JSON and GEN_JSON.
+
ORPHAN_FILES := $(filter-out $(OUT_JSON) $(GEN_JSON),$(CUR_OUT_JSON))
# Message for $(call echo-cmd,mkd). There is already a mkdir message
@@ -224,6 +234,10 @@ endif
# and inputs are dependencies.
$(PMU_EVENTS_C): $(JEVENTS_DEPS)
$(call rule_mkdir)
- $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@
+ $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) \
+ $(OUT_DIR) $(PMU_EVENTS_C) $(PMU_EVENTS_STRING_C)
+
+$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C)
+ @:
endif # ifeq ($(NO_JEVENTS),1)
diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
index 3a1bcdcdc685..70a45e62f5d1 100755
--- a/tools/perf/pmu-events/jevents.py
+++ b/tools/perf/pmu-events/jevents.py
@@ -1422,6 +1422,8 @@ such as "arm/cortex-a34".''',
)
ap.add_argument(
'output_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=sys.stdout)
+ ap.add_argument(
+ 'output_string_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=None)
_args = ap.parse_args()
_args.output_file.write(f"""
@@ -1463,10 +1465,20 @@ struct pmu_table_entry {
ftw(arch_path, [], preprocess_one_file)
_bcs.compute()
- _args.output_file.write('static const char *const big_c_string =\n')
- for s in _bcs.big_string:
- _args.output_file.write(s)
- _args.output_file.write(';\n\n')
+ if not _args.output_string_file:
+ _args.output_file.write('static const char *const big_c_string =\n')
+ for s in _bcs.big_string:
+ _args.output_file.write(s)
+ _args.output_file.write(';\n\n')
+ else:
+ _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n')
+ _args.output_string_file.write('/* Autogenerated by jevents.py */\n')
+ _args.output_string_file.write('const char big_c_string[] =\n')
+ for s in _bcs.big_string:
+ _args.output_string_file.write(s)
+ _args.output_string_file.write(';\n')
+ _args.output_string_file.close()
+ _args.output_file.write('extern const char big_c_string[];\n\n')
for arch in archs:
arch_path = f'{_args.starting_dir}/{arch}'
ftw(arch_path, [], process_one_file)
--
2.54.0.563.g4f69b47b94-goog
next prev parent reply other threads:[~2026-05-14 16:34 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-12 5:35 [PATCH v1 00/14] perf build: Reduce build time by one third Ian Rogers
2026-05-12 5:35 ` [PATCH v1 01/14] bpftool build: Restrict feature tests during bootstrap compilation Ian Rogers
2026-05-12 5:35 ` [PATCH v1 02/14] perf trace beauty: Make beauty generated C code standalone .o files Ian Rogers
2026-05-13 5:21 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 03/14] perf build: Decouple pmu-events from prepare umbrella target Ian Rogers
2026-05-12 5:35 ` [PATCH v1 04/14] perf build: Remove empty archheaders target Ian Rogers
2026-05-12 5:35 ` [PATCH v1 05/14] perf build: Move BPF skeleton generation out of Makefile.perf Ian Rogers
2026-05-13 19:52 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 06/14] perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak Ian Rogers
2026-05-13 20:09 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 07/14] perf build: Move static libbpf dependency out of prepare step Ian Rogers
2026-05-13 20:36 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 08/14] perf build: Pre-generate BPF skeletons during umbrella prepare phase Ian Rogers
2026-05-12 5:35 ` [PATCH v1 09/14] perf build: Move libsymbol dependency out of prepare step Ian Rogers
2026-05-13 21:11 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 10/14] perf build: Remove redundant libbpf feature check for static builds Ian Rogers
2026-05-12 5:35 ` [PATCH v1 11/14] tools build: Integrate libdebuginfod into test-all fast path Ian Rogers
2026-05-13 21:40 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 12/14] perf pmu-events: Split big_c_string storage into standalone compilation unit Ian Rogers
2026-05-13 21:56 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 13/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py Ian Rogers
2026-05-13 22:18 ` sashiko-bot
2026-05-12 5:35 ` [PATCH v1 14/14] perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds Ian Rogers
2026-05-12 17:46 ` [PATCH v2 00/18] perf build: Reduce build time by nearly half Ian Rogers
2026-05-12 17:46 ` [PATCH v2 01/18] bpftool build: Restrict feature tests during bootstrap compilation Ian Rogers
2026-05-12 17:46 ` [PATCH v2 02/18] tools build: Integrate libdebuginfod into test-all fast path Ian Rogers
2026-05-13 23:59 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 03/18] tools build: Fix test-clang-bpf-co-re.bin to generate target file Ian Rogers
2026-05-14 0:15 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 04/18] tools scripts: Short-circuit CC_NO_CLANG compiler probe in Makefile.include Ian Rogers
2026-05-14 0:28 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 05/18] perf trace beauty: Make beauty generated C code standalone .o files Ian Rogers
2026-05-14 0:50 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 06/18] perf build: Decouple pmu-events from prepare umbrella target Ian Rogers
2026-05-12 17:46 ` [PATCH v2 07/18] perf build: Remove empty archheaders target Ian Rogers
2026-05-12 17:46 ` [PATCH v2 08/18] perf build: Move BPF skeleton generation out of Makefile.perf Ian Rogers
2026-05-14 1:55 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 09/18] perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak Ian Rogers
2026-05-12 17:46 ` [PATCH v2 10/18] perf build: Move static libbpf dependency out of prepare step Ian Rogers
2026-05-14 3:02 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 11/18] perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase Ian Rogers
2026-05-14 3:39 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 12/18] perf build: Move libsymbol dependency out of prepare step Ian Rogers
2026-05-12 17:46 ` [PATCH v2 13/18] perf build: Remove redundant libbpf feature check for static builds Ian Rogers
2026-05-12 17:46 ` [PATCH v2 14/18] perf pmu-events: Split big_c_string storage into standalone compilation unit Ian Rogers
2026-05-14 4:35 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 15/18] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py Ian Rogers
2026-05-14 5:06 ` sashiko-bot
2026-05-12 17:46 ` [PATCH v2 16/18] perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds Ian Rogers
2026-05-12 17:46 ` [PATCH v2 17/18] perf pmu-events: Convert recursive shell assignments and macros to Make built-ins Ian Rogers
2026-05-12 17:46 ` [PATCH v2 18/18] perf build: Convert llvm-config shell queries to simply expanded variables Ian Rogers
2026-05-14 16:33 ` [PATCH v3 00/17] perf build: Reduce build time by nearly half Ian Rogers
2026-05-14 16:33 ` [PATCH v3 01/17] bpftool build: Restrict feature tests during bootstrap compilation Ian Rogers
2026-05-14 16:33 ` [PATCH v3 02/17] tools build: Integrate libdebuginfod into test-all fast path Ian Rogers
2026-05-14 20:55 ` sashiko-bot
2026-05-14 16:33 ` [PATCH v3 03/17] tools build: Fix test-clang-bpf-co-re.bin to generate target file Ian Rogers
2026-05-14 21:10 ` sashiko-bot
2026-05-14 16:33 ` [PATCH v3 04/17] perf trace beauty: Make beauty generated C code standalone .o files Ian Rogers
2026-05-14 16:33 ` [PATCH v3 05/17] perf build: Decouple pmu-events from prepare umbrella target Ian Rogers
2026-05-14 16:33 ` [PATCH v3 06/17] perf build: Remove empty archheaders target Ian Rogers
2026-05-14 16:33 ` [PATCH v3 07/17] perf build: Move BPF skeleton generation out of Makefile.perf Ian Rogers
2026-05-14 16:34 ` [PATCH v3 08/17] perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak Ian Rogers
2026-05-14 16:34 ` [PATCH v3 09/17] perf build: Move static libbpf dependency out of prepare step Ian Rogers
2026-05-14 23:35 ` sashiko-bot
2026-05-14 16:34 ` [PATCH v3 10/17] perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase Ian Rogers
2026-05-14 23:45 ` sashiko-bot
2026-05-14 16:34 ` [PATCH v3 11/17] perf build: Move libsymbol dependency out of prepare step Ian Rogers
2026-05-14 16:34 ` [PATCH v3 12/17] perf build: Remove redundant libbpf feature check for static builds Ian Rogers
2026-05-14 16:34 ` Ian Rogers [this message]
2026-05-15 0:39 ` [PATCH v3 13/17] perf pmu-events: Split big_c_string storage into standalone compilation unit sashiko-bot
2026-05-14 16:34 ` [PATCH v3 14/17] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py Ian Rogers
2026-05-15 1:00 ` sashiko-bot
2026-05-14 16:34 ` [PATCH v3 15/17] perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds Ian Rogers
2026-05-14 16:34 ` [PATCH v3 16/17] perf pmu-events: Convert recursive shell assignments and macros to Make built-ins Ian Rogers
2026-05-14 16:34 ` [PATCH v3 17/17] perf build: Convert llvm-config shell queries to simply expanded variables Ian Rogers
2026-05-14 22:06 ` [PATCH v3 00/17] perf build: Reduce build time by nearly half Namhyung Kim
2026-05-14 22:23 ` Ian Rogers
2026-05-15 16:20 ` Ian Rogers
2026-05-12 9:36 ` [PATCH v1 00/14] perf build: Reduce build time by one third James Clark
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260514163409.927816-14-irogers@google.com \
--to=irogers@google.com \
--cc=9erthalion6@gmail.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alex@ghiti.fr \
--cc=alexandre.chartre@oracle.com \
--cc=andrii@kernel.org \
--cc=ankur.a.arora@oracle.com \
--cc=aou@eecs.berkeley.edu \
--cc=bpf@vger.kernel.org \
--cc=collin.funk1@gmail.com \
--cc=costa.shul@redhat.com \
--cc=daniel@iogearbox.net \
--cc=dapeng1.mi@linux.intel.com \
--cc=dsterba@suse.com \
--cc=eddyz87@gmail.com \
--cc=howardchu95@gmail.com \
--cc=james.clark@linaro.org \
--cc=jolsa@kernel.org \
--cc=leo.yan@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=mingo@redhat.com \
--cc=mmayer@broadcom.com \
--cc=namhyung@kernel.org \
--cc=nathan@kernel.org \
--cc=palmer@dabbelt.com \
--cc=peterz@infradead.org \
--cc=pjw@kernel.org \
--cc=qmo@kernel.org \
--cc=ricky.ringler@proton.me \
--cc=song@kernel.org \
--cc=swapnil.sapkal@amd.com \
--cc=terrelln@fb.com \
--cc=tglozar@redhat.com \
--cc=thomas.falcon@intel.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.