From: Ian Rogers <irogers@google.com>
To: "Peter Zijlstra" <peterz@infradead.org>,
"Ingo Molnar" <mingo@redhat.com>,
"Arnaldo Carvalho de Melo" <acme@kernel.org>,
"Namhyung Kim" <namhyung@kernel.org>,
"Mark Rutland" <mark.rutland@arm.com>,
"Alexander Shishkin" <alexander.shishkin@linux.intel.com>,
"Jiri Olsa" <jolsa@kernel.org>, "Ian Rogers" <irogers@google.com>,
"Adrian Hunter" <adrian.hunter@intel.com>,
"Kan Liang" <kan.liang@linux.intel.com>,
"Sam James" <sam@gentoo.org>,
"Jesper Juhl" <jesperjuhl76@gmail.com>,
"James Clark" <james.clark@linaro.org>,
"Zhongqiu Han" <quic_zhonhan@quicinc.com>,
"Yicong Yang" <yangyicong@hisilicon.com>,
"Thomas Richter" <tmricht@linux.ibm.com>,
"Michael Petlan" <mpetlan@redhat.com>,
"Veronika Molnarova" <vmolnaro@redhat.com>,
"Anne Macedo" <retpolanne@posteo.net>,
"Dominique Martinet" <asmadeus@codewreck.org>,
"Jean-Philippe Romain" <jean-philippe.romain@foss.st.com>,
"Junhao He" <hejunhao3@huawei.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
"Krzysztof Łopatowski" <krzysztof.m.lopatowski@gmail.com>
Subject: [PATCH v2 0/7] Add io_dir to avoid memory overhead from opendir
Date: Fri, 7 Feb 2025 15:24:41 -0800 [thread overview]
Message-ID: <20250207232452.994822-1-irogers@google.com> (raw)
glibc's opendir allocates a minimum of 32kb, when called recursively
for a directory tree the memory consumption can add up - nearly 300kb
during perf start-up when processing modules. Add a stack allocated
variant of readdir sized a little more than 1kb
v2: Remove the feature test and always use a perf supplied getdents64
to workaround an Alpine Linux issue in v1:
https://lore.kernel.org/lkml/20231207050433.1426834-1-irogers@google.com/
As suggested by Krzysztof Łopatowski
<krzysztof.m.lopatowski@gmail.com> who also pointed to the perf
trace performance improvements in start-up time eliminating stat
calls can achieve:
https://lore.kernel.org/lkml/20250206113314.335376-2-krzysztof.m.lopatowski@gmail.com/
Convert parse-events and hwmon_pmu to use io_dir.
v1: This was previously part of the memory saving change set:
https://lore.kernel.org/lkml/20231127220902.1315692-1-irogers@google.com/
It is separated here and a feature check and syscall workaround
for missing getdents64 added.
Ian Rogers (7):
tools lib api: Add io_dir an allocation free readdir alternative
perf maps: Switch modules tree walk to io_dir__readdir
perf pmu: Switch to io_dir__readdir
perf header: Switch mem topology to io_dir__readdir
perf events: Remove scandir in thread synthesis
perf parse-events: Switch tracepoints to io_dir__readdir
perf hwmon_pmu: Switch event discovery to io_dir__readdir
tools/lib/api/Makefile | 2 +-
tools/lib/api/io_dir.h | 91 ++++++++++++++++++++++++++++++
tools/perf/util/header.c | 31 +++++-----
tools/perf/util/hwmon_pmu.c | 42 ++++++--------
tools/perf/util/machine.c | 19 +++----
tools/perf/util/parse-events.c | 32 ++++++-----
tools/perf/util/pmu.c | 46 +++++++--------
tools/perf/util/pmus.c | 30 ++++------
tools/perf/util/synthetic-events.c | 22 ++++----
9 files changed, 194 insertions(+), 121 deletions(-)
create mode 100644 tools/lib/api/io_dir.h
--
2.48.1.502.g6dc24dfdaf-goog
next reply other threads:[~2025-02-07 23:24 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-07 23:24 Ian Rogers [this message]
2025-02-07 23:24 ` [PATCH v2 1/7] tools lib api: Add io_dir an allocation free readdir alternative Ian Rogers
2025-02-08 17:02 ` [PATCH v2 1/7] tools lib api: Add io_dir() as an allocation free readdir() alternative Markus Elfring
2025-02-19 21:51 ` [PATCH v2 1/7] tools lib api: Add io_dir an allocation free readdir alternative Namhyung Kim
2025-02-19 22:21 ` Ian Rogers
2025-02-21 6:31 ` Namhyung Kim
2025-02-07 23:24 ` [PATCH v2 2/7] perf maps: Switch modules tree walk to io_dir__readdir Ian Rogers
2025-02-07 23:24 ` [PATCH v2 3/7] perf pmu: Switch " Ian Rogers
2025-02-07 23:24 ` [PATCH v2 4/7] perf header: Switch mem topology " Ian Rogers
2025-02-07 23:24 ` [PATCH v2 5/7] perf events: Remove scandir in thread synthesis Ian Rogers
2025-02-07 23:24 ` [PATCH v2 6/7] perf parse-events: Switch tracepoints to io_dir__readdir Ian Rogers
2025-02-07 23:24 ` [PATCH v2 7/7] perf hwmon_pmu: Switch event discovery " Ian Rogers
2025-02-08 10:58 ` [PATCH v2 0/7] Add io_dir to avoid memory overhead from opendir David Laight
2025-02-08 12:15 ` Ian Rogers
2025-02-19 21:54 ` Namhyung Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250207232452.994822-1-irogers@google.com \
--to=irogers@google.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=asmadeus@codewreck.org \
--cc=hejunhao3@huawei.com \
--cc=james.clark@linaro.org \
--cc=jean-philippe.romain@foss.st.com \
--cc=jesperjuhl76@gmail.com \
--cc=jolsa@kernel.org \
--cc=kan.liang@linux.intel.com \
--cc=krzysztof.m.lopatowski@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=mpetlan@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=quic_zhonhan@quicinc.com \
--cc=retpolanne@posteo.net \
--cc=sam@gentoo.org \
--cc=tmricht@linux.ibm.com \
--cc=vmolnaro@redhat.com \
--cc=yangyicong@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).