From: Ian Rogers <irogers@google.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Namhyung Kim <namhyung@kernel.org>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Nick Terrell <terrelln@fb.com>,
Kan Liang <kan.liang@linux.intel.com>,
Song Liu <song@kernel.org>, Sandipan Das <sandipan.das@amd.com>,
Anshuman Khandual <anshuman.khandual@arm.com>,
James Clark <james.clark@arm.com>,
Liam Howlett <liam.howlett@oracle.com>,
Miguel Ojeda <ojeda@kernel.org>, Leo Yan <leo.yan@linaro.org>,
German Gomez <german.gomez@arm.com>,
Ravi Bangoria <ravi.bangoria@amd.com>,
Artem Savkov <asavkov@redhat.com>,
Athira Rajeev <atrajeev@linux.vnet.ibm.com>,
Andi Kleen <ak@linux.intel.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
bpf@vger.kernel.org
Subject: [PATCH v2 00/13] Improvements to memory use
Date: Wed, 11 Oct 2023 23:23:46 -0700 [thread overview]
Message-ID: <20231012062359.1616786-1-irogers@google.com> (raw)
Fix memory leaks detected by address/leak sanitizer affecting LBR
call-graphs, perf mem and BPF offcpu.
Make branch_type_stat in callchain_list optional as it is large and
not always necessary - in particular it isn't used by perf top.
Make the allocations of zstd streams, kernel symbols and event copies
lazier in order to save memory in cases like perf record.
Handle the thread exit event and have it remove the thread from the
threads set in machine. Don't do this for perf report as it causes a
regression for task lists, which assume threads are never removed from
the machine's set, and offcpu events, that may sythensize samples for
threads that have exited.
The overall effect is to reduce memory consumption significantly for
perf top - with call graphs enabled running longer before 1GB of
memory is consumed. For a perf record of 'true', the memory
consumption goes from 39912kb max resident to 20820kb max resident -
nearly halved.
v2: Add additional memory fixes on top of initial LBR and rc check
fixes.
Ian Rogers (13):
perf machine: Avoid out of bounds LBR memory read
libperf rc_check: Make implicit enabling work for GCC
perf hist: Add missing puts to hist__account_cycles
perf threads: Remove unused dead thread list
perf offcpu: Add missed btf_free
perf callchain: Make display use of branch_type_stat const
perf callchain: Make brtype_stat in callchain_list optional
perf callchain: Minor layout changes to callchain_list
perf mem_info: Add and use map_symbol__exit and addr_map_symbol__exit
perf record: Lazy load kernel symbols
libperf: Lazily allocate mmap event copy
perf mmap: Lazily initialize zstd streams
perf machine thread: Remove exited threads by default
tools/lib/perf/include/internal/mmap.h | 2 +-
tools/lib/perf/include/internal/rc_check.h | 6 ++-
tools/lib/perf/mmap.c | 9 ++++
tools/perf/builtin-inject.c | 4 ++
tools/perf/builtin-record.c | 2 +
tools/perf/builtin-report.c | 7 +++
tools/perf/util/Build | 1 +
tools/perf/util/bpf_off_cpu.c | 10 ++--
tools/perf/util/branch.c | 4 +-
tools/perf/util/branch.h | 4 +-
tools/perf/util/callchain.c | 62 ++++++++++++++--------
tools/perf/util/callchain.h | 18 +++----
tools/perf/util/compress.h | 1 +
tools/perf/util/event.c | 4 +-
tools/perf/util/hist.c | 16 +++---
tools/perf/util/machine.c | 39 +++++++-------
tools/perf/util/machine.h | 1 -
tools/perf/util/map_symbol.c | 15 ++++++
tools/perf/util/map_symbol.h | 4 ++
tools/perf/util/mmap.c | 5 +-
tools/perf/util/mmap.h | 1 -
tools/perf/util/symbol.c | 5 +-
tools/perf/util/symbol_conf.h | 4 +-
tools/perf/util/thread.h | 14 +++++
tools/perf/util/zstd.c | 61 +++++++++++----------
25 files changed, 196 insertions(+), 103 deletions(-)
create mode 100644 tools/perf/util/map_symbol.c
--
2.42.0.609.gbb76f46606-goog
next reply other threads:[~2023-10-12 6:24 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-12 6:23 Ian Rogers [this message]
2023-10-12 6:23 ` [PATCH v2 01/13] perf machine: Avoid out of bounds LBR memory read Ian Rogers
2023-10-12 6:23 ` [PATCH v2 02/13] libperf rc_check: Make implicit enabling work for GCC Ian Rogers
2023-10-12 6:23 ` [PATCH v2 03/13] perf hist: Add missing puts to hist__account_cycles Ian Rogers
2023-10-18 23:16 ` Namhyung Kim
2023-10-24 16:12 ` Ian Rogers
2023-10-12 6:23 ` [PATCH v2 04/13] perf threads: Remove unused dead thread list Ian Rogers
2023-10-12 6:23 ` [PATCH v2 05/13] perf offcpu: Add missed btf_free Ian Rogers
2023-10-12 6:23 ` [PATCH v2 06/13] perf callchain: Make display use of branch_type_stat const Ian Rogers
2023-10-12 6:23 ` [PATCH v2 07/13] perf callchain: Make brtype_stat in callchain_list optional Ian Rogers
2023-10-12 6:23 ` [PATCH v2 08/13] perf callchain: Minor layout changes to callchain_list Ian Rogers
2023-10-12 6:23 ` [PATCH v2 09/13] perf mem_info: Add and use map_symbol__exit and addr_map_symbol__exit Ian Rogers
2023-10-18 23:20 ` Namhyung Kim
2023-10-24 20:52 ` Ian Rogers
2023-10-12 6:23 ` [PATCH v2 10/13] perf record: Lazy load kernel symbols Ian Rogers
2023-10-19 11:02 ` Adrian Hunter
2023-10-24 18:42 ` Ian Rogers
2023-10-12 6:23 ` [PATCH v2 11/13] libperf: Lazily allocate mmap event copy Ian Rogers
2023-10-12 6:23 ` [PATCH v2 12/13] perf mmap: Lazily initialize zstd streams Ian Rogers
2023-10-18 23:21 ` Namhyung Kim
2023-10-24 19:21 ` Ian Rogers
2023-10-12 6:23 ` [PATCH v2 13/13] perf machine thread: Remove exited threads by default Ian Rogers
2023-10-18 23:30 ` Namhyung Kim
2023-10-19 0:39 ` Ian Rogers
2023-10-20 21:08 ` Namhyung Kim
2023-10-23 14:24 ` Adrian Hunter
2023-10-23 18:49 ` Ian Rogers
2023-10-24 5:19 ` Adrian Hunter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231012062359.1616786-1-irogers@google.com \
--to=irogers@google.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=anshuman.khandual@arm.com \
--cc=asavkov@redhat.com \
--cc=atrajeev@linux.vnet.ibm.com \
--cc=bpf@vger.kernel.org \
--cc=german.gomez@arm.com \
--cc=james.clark@arm.com \
--cc=jolsa@kernel.org \
--cc=kan.liang@linux.intel.com \
--cc=leo.yan@linaro.org \
--cc=liam.howlett@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=ojeda@kernel.org \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=sandipan.das@amd.com \
--cc=song@kernel.org \
--cc=terrelln@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox