All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ian Rogers <irogers@google.com>
To: "Peter Zijlstra" <peterz@infradead.org>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Arnaldo Carvalho de Melo" <acme@kernel.org>,
	"Namhyung Kim" <namhyung@kernel.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Alexander Shishkin" <alexander.shishkin@linux.intel.com>,
	"Jiri Olsa" <jolsa@kernel.org>, "Ian Rogers" <irogers@google.com>,
	"Adrian Hunter" <adrian.hunter@intel.com>,
	"Kan Liang" <kan.liang@linux.intel.com>,
	"Sam James" <sam@gentoo.org>,
	"Jesper Juhl" <jesperjuhl76@gmail.com>,
	"James Clark" <james.clark@linaro.org>,
	"Zhongqiu Han" <quic_zhonhan@quicinc.com>,
	"Yicong Yang" <yangyicong@hisilicon.com>,
	"Thomas Richter" <tmricht@linux.ibm.com>,
	"Michael Petlan" <mpetlan@redhat.com>,
	"Veronika Molnarova" <vmolnaro@redhat.com>,
	"Anne Macedo" <retpolanne@posteo.net>,
	"Dominique Martinet" <asmadeus@codewreck.org>,
	"Jean-Philippe Romain" <jean-philippe.romain@foss.st.com>,
	"Junhao He" <hejunhao3@huawei.com>,
	linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
	"Krzysztof Łopatowski" <krzysztof.m.lopatowski@gmail.com>
Subject: [PATCH v2 0/7] Add io_dir to avoid memory overhead from opendir
Date: Fri,  7 Feb 2025 15:24:41 -0800	[thread overview]
Message-ID: <20250207232452.994822-1-irogers@google.com> (raw)

glibc's opendir allocates a minimum of 32kb, when called recursively
for a directory tree the memory consumption can add up - nearly 300kb
during perf start-up when processing modules. Add a stack allocated
variant of readdir sized a little more than 1kb

v2: Remove the feature test and always use a perf supplied getdents64
    to workaround an Alpine Linux issue in v1:
    https://lore.kernel.org/lkml/20231207050433.1426834-1-irogers@google.com/
    As suggested by Krzysztof Łopatowski
    <krzysztof.m.lopatowski@gmail.com> who also pointed to the perf
    trace performance improvements in start-up time eliminating stat
    calls can achieve:
    https://lore.kernel.org/lkml/20250206113314.335376-2-krzysztof.m.lopatowski@gmail.com/
    Convert parse-events and hwmon_pmu to use io_dir.
v1: This was previously part of the memory saving change set:
    https://lore.kernel.org/lkml/20231127220902.1315692-1-irogers@google.com/
    It is separated here and a feature check and syscall workaround
    for missing getdents64 added.

Ian Rogers (7):
  tools lib api: Add io_dir an allocation free readdir alternative
  perf maps: Switch modules tree walk to io_dir__readdir
  perf pmu: Switch to io_dir__readdir
  perf header: Switch mem topology to io_dir__readdir
  perf events: Remove scandir in thread synthesis
  perf parse-events: Switch tracepoints to io_dir__readdir
  perf hwmon_pmu: Switch event discovery to io_dir__readdir

 tools/lib/api/Makefile             |  2 +-
 tools/lib/api/io_dir.h             | 91 ++++++++++++++++++++++++++++++
 tools/perf/util/header.c           | 31 +++++-----
 tools/perf/util/hwmon_pmu.c        | 42 ++++++--------
 tools/perf/util/machine.c          | 19 +++----
 tools/perf/util/parse-events.c     | 32 ++++++-----
 tools/perf/util/pmu.c              | 46 +++++++--------
 tools/perf/util/pmus.c             | 30 ++++------
 tools/perf/util/synthetic-events.c | 22 ++++----
 9 files changed, 194 insertions(+), 121 deletions(-)
 create mode 100644 tools/lib/api/io_dir.h

-- 
2.48.1.502.g6dc24dfdaf-goog


             reply	other threads:[~2025-02-07 23:24 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-07 23:24 Ian Rogers [this message]
2025-02-07 23:24 ` [PATCH v2 1/7] tools lib api: Add io_dir an allocation free readdir alternative Ian Rogers
2025-02-08 17:02   ` [PATCH v2 1/7] tools lib api: Add io_dir() as an allocation free readdir() alternative Markus Elfring
2025-02-19 21:51   ` [PATCH v2 1/7] tools lib api: Add io_dir an allocation free readdir alternative Namhyung Kim
2025-02-19 22:21     ` Ian Rogers
2025-02-21  6:31       ` Namhyung Kim
2025-02-07 23:24 ` [PATCH v2 2/7] perf maps: Switch modules tree walk to io_dir__readdir Ian Rogers
2025-02-07 23:24 ` [PATCH v2 3/7] perf pmu: Switch " Ian Rogers
2025-02-07 23:24 ` [PATCH v2 4/7] perf header: Switch mem topology " Ian Rogers
2025-02-07 23:24 ` [PATCH v2 5/7] perf events: Remove scandir in thread synthesis Ian Rogers
2025-02-07 23:24 ` [PATCH v2 6/7] perf parse-events: Switch tracepoints to io_dir__readdir Ian Rogers
2025-02-07 23:24 ` [PATCH v2 7/7] perf hwmon_pmu: Switch event discovery " Ian Rogers
2025-02-08 10:58 ` [PATCH v2 0/7] Add io_dir to avoid memory overhead from opendir David Laight
2025-02-08 12:15   ` Ian Rogers
2025-02-19 21:54 ` Namhyung Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250207232452.994822-1-irogers@google.com \
    --to=irogers@google.com \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=asmadeus@codewreck.org \
    --cc=hejunhao3@huawei.com \
    --cc=james.clark@linaro.org \
    --cc=jean-philippe.romain@foss.st.com \
    --cc=jesperjuhl76@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kan.liang@linux.intel.com \
    --cc=krzysztof.m.lopatowski@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=mpetlan@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=quic_zhonhan@quicinc.com \
    --cc=retpolanne@posteo.net \
    --cc=sam@gentoo.org \
    --cc=tmricht@linux.ibm.com \
    --cc=vmolnaro@redhat.com \
    --cc=yangyicong@hisilicon.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.