[PATCH 0/4] perf script: Bounded and lazy symbol loading

All of lore.kernel.org
 help / color / mirror / Atom feed

* [PATCH 0/4] perf script: Bounded and lazy symbol loading
@ 2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

perf script loads the entire ELF symbol table of every DSO that appears
in a sample, allocating each symbol into an rb-tree held until process
exit.  Therefore, a large enough profile turns symbol loading into an OOM kill.
This is not scalable for profiling a large cgroup with a lot of large
binaries on a production system with limited free memory.

This series adds two independent, opt-in mechanisms and a leading
regression fix they build on:

  [1/4] Fix a broken "#ifdef ELF_C_READ_MMAP" guard so perf actually
        mmaps ELF files instead of malloc'ing section data. This is a
        standalone regression fix introduced by 22dd1ac91a77.

  [2/4] --max-symbol-bytes <size>: a byte budget on struct symbol
        allocations (and the lazy index) enforced at the ELF symbol
        loader, degrading to [unknown] with a warning past the cap.
        An unbounded profile doesn't just risk OOM-killing itself.  It
        also forces memory pressure on the whole host, pushing the kernel
        to reclaim from co-located latency-sensitive processes.  Capping
        it lets the user bound that footprint up front and choose the
        trade-off explicitly.

  [3/4] --lazy-load-symbols: build a compact per-DSO sorted index and
        resolve only the sampled addresses, reading names via pread()
        from the file's string table at lookup time. On the production fixture,
        peak RssAnon drops from 265 MiB to 39 MiB (6.8x) and wall time
        from 3.1 s to 1.85 s (1.7x). Memory optimizations usually cost
        time; this one does not because lazy loading skips a lot of calloc and
        demangle calls. The output is byte-identical to the default loader.

  [4/4] Documentation and a shell test.

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
Alireza Haghdoost (4):
      perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
      perf script: Add --max-symbol-bytes to bound ELF symbol memory
      perf script: Add --lazy-load-symbols for lazy symbol loading
      perf script: Document and test --lazy-load-symbols and --max-symbol-bytes

 tools/perf/Documentation/perf-script.txt           |  24 +
 tools/perf/builtin-script.c                        |  33 ++
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 120 +++++
 tools/perf/util/dso.c                              |  10 +
 tools/perf/util/dso.h                              |  29 ++
 tools/perf/util/map.c                              |   9 +-
 tools/perf/util/symbol-elf.c                       | 561 +++++++++++++++++++++
 tools/perf/util/symbol-minimal.c                   |  11 +
 tools/perf/util/symbol.c                           |  87 +++-
 tools/perf/util/symbol.h                           |  18 +-
 tools/perf/util/symbol_conf.h                      |   2 +
 11 files changed, 875 insertions(+), 29 deletions(-)
---
base-commit: aa18964dd64511305de0711fed912054da6f5d18
change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9

Best regards,
--  
Alireza Haghdoost <haghdoost@uber.com>


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH 0/4] perf script: Bounded and lazy symbol loading
@ 2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

perf script loads the entire ELF symbol table of every DSO that appears
in a sample, allocating each symbol into an rb-tree held until process
exit.  Therefore, a large enough profile turns symbol loading into an OOM kill.
This is not scalable for profiling a large cgroup with a lot of large
binaries on a production system with limited free memory.

This series adds two independent, opt-in mechanisms and a leading
regression fix they build on:

  [1/4] Fix a broken "#ifdef ELF_C_READ_MMAP" guard so perf actually
        mmaps ELF files instead of malloc'ing section data. This is a
        standalone regression fix introduced by 22dd1ac91a77.

  [2/4] --max-symbol-bytes <size>: a byte budget on struct symbol
        allocations (and the lazy index) enforced at the ELF symbol
        loader, degrading to [unknown] with a warning past the cap.
        An unbounded profile doesn't just risk OOM-killing itself.  It
        also forces memory pressure on the whole host, pushing the kernel
        to reclaim from co-located latency-sensitive processes.  Capping
        it lets the user bound that footprint up front and choose the
        trade-off explicitly.

  [3/4] --lazy-load-symbols: build a compact per-DSO sorted index and
        resolve only the sampled addresses, reading names via pread()
        from the file's string table at lookup time. On the production fixture,
        peak RssAnon drops from 265 MiB to 39 MiB (6.8x) and wall time
        from 3.1 s to 1.85 s (1.7x). Memory optimizations usually cost
        time; this one does not because lazy loading skips a lot of calloc and
        demangle calls. The output is byte-identical to the default loader.

  [4/4] Documentation and a shell test.

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
Alireza Haghdoost (4):
      perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
      perf script: Add --max-symbol-bytes to bound ELF symbol memory
      perf script: Add --lazy-load-symbols for lazy symbol loading
      perf script: Document and test --lazy-load-symbols and --max-symbol-bytes

 tools/perf/Documentation/perf-script.txt           |  24 +
 tools/perf/builtin-script.c                        |  33 ++
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 120 +++++
 tools/perf/util/dso.c                              |  10 +
 tools/perf/util/dso.h                              |  29 ++
 tools/perf/util/map.c                              |   9 +-
 tools/perf/util/symbol-elf.c                       | 561 +++++++++++++++++++++
 tools/perf/util/symbol-minimal.c                   |  11 +
 tools/perf/util/symbol.c                           |  87 +++-
 tools/perf/util/symbol.h                           |  18 +-
 tools/perf/util/symbol_conf.h                      |   2 +
 11 files changed, 875 insertions(+), 29 deletions(-)
---
base-commit: aa18964dd64511305de0711fed912054da6f5d18
change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9

Best regards,
--  
Alireza Haghdoost <haghdoost@uber.com>



^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
  2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  -1 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

This patch makes perf mmap ELF files.  This was the intended behavior,
regressed by commit 22dd1ac91a77 ("tools: Remove feature-libelf-mmap
feature detection").  ELF_C_READ_MMAP is an Elf_Cmd enumerator, not a
preprocessor macro, so the #ifdef is false on every libelf that provides
it.  objtool uses ELF_C_READ_MMAP without an #ifdef guard and has been
fine for five years.

elf_getdata() still allocates when translation is needed (32-on-64 or
cross-endian), and libelf falls back to internal reads on non-seekable
fds.

Fixes: 22dd1ac91a77 ("tools: Remove feature-libelf-mmap feature detection")
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/util/symbol.h | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index d0bac824c79c..e5cef16b240d 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -57,15 +57,8 @@ static inline bool is_livepatch_symbol(const char *str)
 	return strstarts(str, KLP_SYM_PREFIX);
 }
 
-/*
- * libelf 0.8.x and earlier do not support ELF_C_READ_MMAP;
- * for newer versions we can use mmap to reduce memory usage:
- */
-#ifdef ELF_C_READ_MMAP
-# define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
-#else
-# define PERF_ELF_C_READ_MMAP ELF_C_READ
-#endif
+/* libelf falls back to internal reads when mmap fails (e.g. non-seekable fd). */
+#define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
 
 #ifdef HAVE_LIBELF_SUPPORT
 Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,

-- 
Git-155)


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

From: Alireza Haghdoost <haghdoost@uber.com>

This patch makes perf mmap ELF files.  This was the intended behavior,
regressed by commit 22dd1ac91a77 ("tools: Remove feature-libelf-mmap
feature detection").  ELF_C_READ_MMAP is an Elf_Cmd enumerator, not a
preprocessor macro, so the #ifdef is false on every libelf that provides
it.  objtool uses ELF_C_READ_MMAP without an #ifdef guard and has been
fine for five years.

elf_getdata() still allocates when translation is needed (32-on-64 or
cross-endian), and libelf falls back to internal reads on non-seekable
fds.

Fixes: 22dd1ac91a77 ("tools: Remove feature-libelf-mmap feature detection")
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/util/symbol.h | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index d0bac824c79c..e5cef16b240d 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -57,15 +57,8 @@ static inline bool is_livepatch_symbol(const char *str)
 	return strstarts(str, KLP_SYM_PREFIX);
 }
 
-/*
- * libelf 0.8.x and earlier do not support ELF_C_READ_MMAP;
- * for newer versions we can use mmap to reduce memory usage:
- */
-#ifdef ELF_C_READ_MMAP
-# define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
-#else
-# define PERF_ELF_C_READ_MMAP ELF_C_READ
-#endif
+/* libelf falls back to internal reads when mmap fails (e.g. non-seekable fd). */
+#define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
 
 #ifdef HAVE_LIBELF_SUPPORT
 Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory
  2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  -1 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

perf script eagerly materializes every ELF symbol into an rb-tree kept
until process exit. Large profiles can therefore consume substantial
anonymous memory, causing perf script to be OOM-killed or forcing the
kernel to reclaim memory from co-located workloads.

This patch adds --max-symbol-bytes to bound struct symbol allocations.
Once the budget is reached, the ELF loader stops loading symbols, warns
once, and lets unresolved addresses appear as [unknown].
This allows users to bound the memory footprint upfront and explicitly
choose between complete symbolization and avoiding unbounded host memory
pressure. perf record already provides a similar --max-size option to
bound disk usage.

The counter includes every symbol__new() allocation, but this patch
enforces the limit only in the ELF loader, which is the source of the
unbounded memory growth addressed here. In this path, reaching the limit
can safely produce [unknown] symbols. Other loaders currently treat a
failed symbol allocation as an error. Capping those paths would therefore
require separate changes whose complexity may outweigh the potential
memory savings.

Sizes require a B/K/M/G suffix. Zero, the default, means unlimited.

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/builtin-script.c   | 31 +++++++++++++++++++++++++++++++
 tools/perf/util/symbol-elf.c  |  7 +++++++
 tools/perf/util/symbol.c      | 29 +++++++++++++++++++++++++++--
 tools/perf/util/symbol.h      |  3 +++
 tools/perf/util/symbol_conf.h |  1 +
 5 files changed, 69 insertions(+), 2 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index ad8ca08ceb5f..50fa6ca6455a 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -68,6 +68,7 @@
 #include "util/thread.h"
 #include "util/thread_map.h"
 #include "util/time-utils.h"
+#include "util/units.h"
 #include "util/tool.h"
 #include "util/trace-event.h"
 #include "util/unwind.h"
@@ -4035,6 +4036,33 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
 	return 0;
 }
 
+static int parse_max_symbol_bytes(const struct option *opt,
+				  const char *str, int unset)
+{
+	unsigned long *max_bytes = (unsigned long *)opt->value;
+	static struct parse_tag size_tags[] = {
+		{ .tag  = 'B', .mult = 1       },
+		{ .tag  = 'K', .mult = 1 << 10 },
+		{ .tag  = 'M', .mult = 1 << 20 },
+		{ .tag  = 'G', .mult = 1 << 30 },
+		{ .tag  = 0 },
+	};
+	unsigned long bytes;
+
+	if (unset) {
+		*max_bytes = 0;
+		return 0;
+	}
+
+	bytes = parse_tag_value(str, size_tags);
+	if (bytes != (unsigned long)-1) {
+		*max_bytes = bytes;
+		return 0;
+	}
+
+	return -1;
+}
+
 int cmd_script(int argc, const char **argv)
 {
 	bool show_full_info = false;
@@ -4135,6 +4163,9 @@ int cmd_script(int argc, const char **argv)
 		     "Set the maximum stack depth when parsing the callchain, "
 		     "anything beyond the specified depth will be ignored. "
 		     "Default: kernel.perf_event_max_stack or " __stringify(PERF_MAX_STACK_DEPTH)),
+	OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
+		     "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
+		     parse_max_symbol_bytes),
 	OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
 	OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
 	OPT_BOOLEAN('I', "show-info", &show_full_info,
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index e955c3feddcd..914e42d21f70 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -1634,6 +1634,13 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 		int is_label = elf_sym__is_label(&sym);
 		const char *section_name;
 		bool used_opd = false;
+		if (symbol_conf.max_symbol_bytes &&
+		    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
+			pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
+					"remaining symbols will be [unknown]\n",
+					symbol_conf.max_symbol_bytes);
+			break;
+		}
 
 		if (!is_label && !elf_sym__filter(&sym))
 			continue;
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 3587ad243159..62a4f91c2f5d 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -310,14 +310,35 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms)
 		curr->end = roundup(curr->start, 4096) + 4096;
 }
 
+static size_t symbol_bytes_used;
+
+size_t symbol__bytes_used(void)
+{
+	return symbol_bytes_used;
+}
+
+void symbol__account_bytes(size_t bytes)
+{
+	symbol_bytes_used += bytes;
+}
+
+void symbol__unaccount_bytes(size_t bytes)
+{
+	symbol_bytes_used -= bytes;
+}
+
 struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
 {
 	size_t namelen = strlen(name) + 1;
-	struct symbol *sym = calloc(1, (symbol_conf.priv_size +
-					sizeof(*sym) + namelen));
+	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) + namelen;
+	struct symbol *sym;
+
+	sym = calloc(1, alloc_size);
 	if (sym == NULL)
 		return NULL;
 
+	symbol__account_bytes(alloc_size);
+
 	if (symbol_conf.priv_size) {
 		if (symbol_conf.init_annotation) {
 			struct annotation *notes = (void *)sym;
@@ -341,6 +362,9 @@ struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *
 
 void symbol__delete(struct symbol *sym)
 {
+	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) +
+			    sym->namelen + 1;
+
 	if (symbol_conf.priv_size) {
 		if (symbol_conf.init_annotation) {
 			struct annotation *notes = symbol__annotation(sym);
@@ -348,6 +372,7 @@ void symbol__delete(struct symbol *sym)
 			annotation__exit(notes);
 		}
 	}
+	symbol__unaccount_bytes(alloc_size);
 	free(((void *)sym) - symbol_conf.priv_size);
 }
 
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index e5cef16b240d..0d5d3792aac1 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -228,6 +228,9 @@ void symbol__elf_init(void);
 int symbol__annotation_init(void);
 
 struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name);
+size_t symbol__bytes_used(void);
+void symbol__account_bytes(size_t bytes);
+void symbol__unaccount_bytes(size_t bytes);
 size_t __symbol__fprintf_symname_offs(const struct symbol *sym,
 				      const struct addr_location *al,
 				      bool unknown_as_addr,
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index 71f60081a85b..6a16c5badd5e 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -120,6 +120,7 @@ struct symbol_conf {
 	int		pad_output_len_dso;
 	int		group_sort_idx;
 	int		addr_range;
+	unsigned long	max_symbol_bytes;
 	DECLARE_BITMAP(parallelism_filter, MAX_NR_CPUS + 1);
 };
 

-- 
Git-155)


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

From: Alireza Haghdoost <haghdoost@uber.com>

perf script eagerly materializes every ELF symbol into an rb-tree kept
until process exit. Large profiles can therefore consume substantial
anonymous memory, causing perf script to be OOM-killed or forcing the
kernel to reclaim memory from co-located workloads.

This patch adds --max-symbol-bytes to bound struct symbol allocations.
Once the budget is reached, the ELF loader stops loading symbols, warns
once, and lets unresolved addresses appear as [unknown].
This allows users to bound the memory footprint upfront and explicitly
choose between complete symbolization and avoiding unbounded host memory
pressure. perf record already provides a similar --max-size option to
bound disk usage.

The counter includes every symbol__new() allocation, but this patch
enforces the limit only in the ELF loader, which is the source of the
unbounded memory growth addressed here. In this path, reaching the limit
can safely produce [unknown] symbols. Other loaders currently treat a
failed symbol allocation as an error. Capping those paths would therefore
require separate changes whose complexity may outweigh the potential
memory savings.

Sizes require a B/K/M/G suffix. Zero, the default, means unlimited.

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/builtin-script.c   | 31 +++++++++++++++++++++++++++++++
 tools/perf/util/symbol-elf.c  |  7 +++++++
 tools/perf/util/symbol.c      | 29 +++++++++++++++++++++++++++--
 tools/perf/util/symbol.h      |  3 +++
 tools/perf/util/symbol_conf.h |  1 +
 5 files changed, 69 insertions(+), 2 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index ad8ca08ceb5f..50fa6ca6455a 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -68,6 +68,7 @@
 #include "util/thread.h"
 #include "util/thread_map.h"
 #include "util/time-utils.h"
+#include "util/units.h"
 #include "util/tool.h"
 #include "util/trace-event.h"
 #include "util/unwind.h"
@@ -4035,6 +4036,33 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
 	return 0;
 }
 
+static int parse_max_symbol_bytes(const struct option *opt,
+				  const char *str, int unset)
+{
+	unsigned long *max_bytes = (unsigned long *)opt->value;
+	static struct parse_tag size_tags[] = {
+		{ .tag  = 'B', .mult = 1       },
+		{ .tag  = 'K', .mult = 1 << 10 },
+		{ .tag  = 'M', .mult = 1 << 20 },
+		{ .tag  = 'G', .mult = 1 << 30 },
+		{ .tag  = 0 },
+	};
+	unsigned long bytes;
+
+	if (unset) {
+		*max_bytes = 0;
+		return 0;
+	}
+
+	bytes = parse_tag_value(str, size_tags);
+	if (bytes != (unsigned long)-1) {
+		*max_bytes = bytes;
+		return 0;
+	}
+
+	return -1;
+}
+
 int cmd_script(int argc, const char **argv)
 {
 	bool show_full_info = false;
@@ -4135,6 +4163,9 @@ int cmd_script(int argc, const char **argv)
 		     "Set the maximum stack depth when parsing the callchain, "
 		     "anything beyond the specified depth will be ignored. "
 		     "Default: kernel.perf_event_max_stack or " __stringify(PERF_MAX_STACK_DEPTH)),
+	OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
+		     "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
+		     parse_max_symbol_bytes),
 	OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
 	OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
 	OPT_BOOLEAN('I', "show-info", &show_full_info,
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index e955c3feddcd..914e42d21f70 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -1634,6 +1634,13 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 		int is_label = elf_sym__is_label(&sym);
 		const char *section_name;
 		bool used_opd = false;
+		if (symbol_conf.max_symbol_bytes &&
+		    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
+			pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
+					"remaining symbols will be [unknown]\n",
+					symbol_conf.max_symbol_bytes);
+			break;
+		}
 
 		if (!is_label && !elf_sym__filter(&sym))
 			continue;
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 3587ad243159..62a4f91c2f5d 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -310,14 +310,35 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms)
 		curr->end = roundup(curr->start, 4096) + 4096;
 }
 
+static size_t symbol_bytes_used;
+
+size_t symbol__bytes_used(void)
+{
+	return symbol_bytes_used;
+}
+
+void symbol__account_bytes(size_t bytes)
+{
+	symbol_bytes_used += bytes;
+}
+
+void symbol__unaccount_bytes(size_t bytes)
+{
+	symbol_bytes_used -= bytes;
+}
+
 struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
 {
 	size_t namelen = strlen(name) + 1;
-	struct symbol *sym = calloc(1, (symbol_conf.priv_size +
-					sizeof(*sym) + namelen));
+	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) + namelen;
+	struct symbol *sym;
+
+	sym = calloc(1, alloc_size);
 	if (sym == NULL)
 		return NULL;
 
+	symbol__account_bytes(alloc_size);
+
 	if (symbol_conf.priv_size) {
 		if (symbol_conf.init_annotation) {
 			struct annotation *notes = (void *)sym;
@@ -341,6 +362,9 @@ struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *
 
 void symbol__delete(struct symbol *sym)
 {
+	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) +
+			    sym->namelen + 1;
+
 	if (symbol_conf.priv_size) {
 		if (symbol_conf.init_annotation) {
 			struct annotation *notes = symbol__annotation(sym);
@@ -348,6 +372,7 @@ void symbol__delete(struct symbol *sym)
 			annotation__exit(notes);
 		}
 	}
+	symbol__unaccount_bytes(alloc_size);
 	free(((void *)sym) - symbol_conf.priv_size);
 }
 
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index e5cef16b240d..0d5d3792aac1 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -228,6 +228,9 @@ void symbol__elf_init(void);
 int symbol__annotation_init(void);
 
 struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name);
+size_t symbol__bytes_used(void);
+void symbol__account_bytes(size_t bytes);
+void symbol__unaccount_bytes(size_t bytes);
 size_t __symbol__fprintf_symname_offs(const struct symbol *sym,
 				      const struct addr_location *al,
 				      bool unknown_as_addr,
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index 71f60081a85b..6a16c5badd5e 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -120,6 +120,7 @@ struct symbol_conf {
 	int		pad_output_len_dso;
 	int		group_sort_idx;
 	int		addr_range;
+	unsigned long	max_symbol_bytes;
 	DECLARE_BITMAP(parallelism_filter, MAX_NR_CPUS + 1);
 };
 

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading
  2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  -1 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

perf script eagerly materializes eligible symbols from every DSO
encountered in samples. On a production fixture, it loaded about 765k
symbols to resolve about 45k distinct (DSO, symbol) frames, exceeding the
memory available in a memory-constrained cgroup.

This patch adds --lazy-load-symbols for userspace ELF DSOs. It builds a
compact sorted index, resolves sampled addresses by binary search, reads
symbol names with pread(), and caches resolved symbols in the existing
rb-tree. Lazy lookup retains one CLOEXEC file descriptor per indexed DSO.
If the descriptor cannot be retained, perf discards the index and eagerly
loads that DSO instead.

On the same fixture, peak RssAnon drops from 265 MiB to 39 MiB and wall
time from 3.1 seconds to 1.85 seconds. Memory optimizations usually cost
time; this one does not because lazy loading skips many unnecessary
calloc() calls and demangling operations. Output was byte-identical on
the tested x86-64 workloads and an aarch64 capture.

Lazy loading is most effective when samples reference only a small
fraction of the available symbols, such as profiles spanning many large
DSOs. It still builds an index proportional to the total symbol count.
Eager loading remains available for dense symbol coverage or cases
requiring its broader ELF and architecture support.

This does not claim full parity with the eager loader. Lazy loading
supports the common userspace ELF symtab/dynsym case, with these known
differences:

  - .gnu_debugdata merging and PPC64 .opd are not handled.
  - SHT_NOBITS re-reading, IFUNC PLT naming, and exact-tie alias ordering
    are unreachable or output-equivalent on x86-64.

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/builtin-script.c      |   4 +-
 tools/perf/util/dso.c            |  10 +
 tools/perf/util/dso.h            |  29 ++
 tools/perf/util/map.c            |   9 +-
 tools/perf/util/symbol-elf.c     | 560 ++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/symbol-minimal.c |  11 +
 tools/perf/util/symbol.c         |  58 ++--
 tools/perf/util/symbol.h         |   4 +
 tools/perf/util/symbol_conf.h    |   1 +
 9 files changed, 664 insertions(+), 22 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 50fa6ca6455a..81e56378d3d2 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -4037,7 +4037,7 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
 }
 
 static int parse_max_symbol_bytes(const struct option *opt,
-				  const char *str, int unset)
+				const char *str, int unset)
 {
 	unsigned long *max_bytes = (unsigned long *)opt->value;
 	static struct parse_tag size_tags[] = {
@@ -4166,6 +4166,8 @@ int cmd_script(int argc, const char **argv)
 	OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
 		     "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
 		     parse_max_symbol_bytes),
+	OPT_BOOLEAN(0, "lazy-load-symbols", &symbol_conf.lazy_load_symbols,
+		    "Resolve symbols lazily instead of loading full symtabs"),
 	OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
 	OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
 	OPT_BOOLEAN('I', "show-info", &show_full_info,
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 42bfe30a3b51..196cd27e79e2 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -1755,6 +1755,16 @@ void dso__delete(struct dso *dso)
 
 	dso__data_close(dso);
 	auxtrace_cache__free(RC_CHK_ACCESS(dso)->auxtrace_cache);
+	if (RC_CHK_ACCESS(dso)->ondemand) {
+		struct dso_ondemand *od = RC_CHK_ACCESS(dso)->ondemand;
+
+		free(od->sorted);
+		symbol__unaccount_bytes(od->nr_alloc * sizeof(struct sym_idx));
+		if (od->fd >= 0)
+			close(od->fd);
+		free(od);
+		RC_CHK_ACCESS(dso)->ondemand = NULL;
+	}
 	dso_cache__free(dso);
 	dso__free_a2l(dso);
 	dso__free_libdw(dso);
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index 55c4aaa53c38..50c4d64e5081 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -282,6 +282,23 @@ struct dso_bpf_prog {
 	struct perf_env	*env;
 };
 
+struct sym_idx {
+	u64	start;	/* adjusted st_value (same space as sym->start) */
+	u64	end;	/* start + st_size, or next start if st_size==0 */
+	u32	name_off; /* symbol's st_name: offset into the strtab */
+	u8	binding;
+	u8	type;
+};
+
+struct dso_ondemand {
+	int		 fd;		/* kept open for pread */
+	u64		 strtab_offset;	/* file offset of strtab section */
+	u64		 strtab_size;
+	struct sym_idx	*sorted;
+	u32		 nr_sorted;	/* deduped count */
+	u32		 nr_alloc;	/* allocated count, for accounting */
+};
+
 struct auxtrace_cache;
 
 DECLARE_RC_STRUCT(dso) {
@@ -308,6 +325,7 @@ DECLARE_RC_STRUCT(dso) {
 	char		 *symsrc_filename;
 	struct nsinfo	*nsinfo;
 	struct auxtrace_cache *auxtrace_cache;
+	struct dso_ondemand *ondemand;
 	union { /* Tool specific area */
 		void	 *priv;
 		u64	 db_id;
@@ -448,6 +466,16 @@ static inline void dso__set_auxtrace_cache(struct dso *dso, struct auxtrace_cach
 	RC_CHK_ACCESS(dso)->auxtrace_cache = cache;
 }
 
+static inline struct dso_ondemand *dso__ondemand(struct dso *dso)
+{
+	return RC_CHK_ACCESS(dso)->ondemand;
+}
+
+static inline void dso__set_ondemand(struct dso *dso, struct dso_ondemand *od)
+{
+	RC_CHK_ACCESS(dso)->ondemand = od;
+}
+
 static inline struct dso_bpf_prog *dso__bpf_prog(struct dso *dso)
 {
 	return &RC_CHK_ACCESS(dso)->bpf_prog;
@@ -823,6 +851,7 @@ int dso__read_binary_type_filename(const struct dso *dso, enum dso_binary_type t
 				   const char *root_dir, char *filename, size_t size);
 bool is_kernel_module(const char *pathname, int cpumode);
 bool dso__needs_decompress(struct dso *dso);
+struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr);
 int dso__decompress_kmodule_fd(struct dso *dso, const char *name);
 int dso__decompress_kmodule_path(struct dso *dso, const char *name,
 				 char *pathname, size_t len);
diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 41cdddc987ee..68ffa01c9f9e 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -382,10 +382,17 @@ int map__load(struct map *map)
 
 struct symbol *map__find_symbol(struct map *map, u64 addr)
 {
+	struct dso *dso;
+	struct symbol *sym;
+
 	if (map__load(map) < 0)
 		return NULL;
 
-	return dso__find_symbol(map__dso(map), addr);
+	dso = map__dso(map);
+	sym = dso__find_symbol(dso, addr);
+	if (!sym && dso__ondemand(dso))
+		sym = dso__find_symbol_ondemand(dso, addr);
+	return sym;
 }
 
 struct symbol *map__find_symbol_by_name_idx(struct map *map, const char *name, size_t *idx)
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 914e42d21f70..e4d77e46e883 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -2,6 +2,7 @@
 #include <fcntl.h>
 #include <stdio.h>
 #include <errno.h>
+#include <stdint.h>
 #include <stdlib.h>
 #include <string.h>
 #include <unistd.h>
@@ -600,6 +601,32 @@ static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
  * And always look at the original dso, not at debuginfo packages, that
  * have the PLT data stripped out (shdr_rel_plt.sh_type == SHT_NOBITS).
  */
+static void dso__clip_ondemand_symbols_at(struct dso *dso, u64 addr)
+{
+	struct dso_ondemand *od = dso__ondemand(dso);
+	struct sym_idx *sym;
+	u32 lo = 0, hi, mid;
+
+	if (!od)
+		return;
+
+	hi = od->nr_sorted;
+	while (lo < hi) {
+		mid = (lo + hi) / 2;
+		if (od->sorted[mid].start < addr)
+			lo = mid + 1;
+		else
+			hi = mid;
+	}
+
+	if (!lo)
+		return;
+
+	sym = &od->sorted[lo - 1];
+	if (sym->end > addr)
+		sym->end = addr;
+}
+
 int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
 {
 	uint32_t idx;
@@ -623,6 +650,8 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
 	if (!elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL))
 		return 0;
 
+	dso__clip_ondemand_symbols_at(dso, shdr_plt.sh_offset);
+
 	/*
 	 * A symbol from a previous section (e.g. .init) can have been expanded
 	 * by symbols__fixup_end() to overlap .plt. Truncate it before adding
@@ -1515,6 +1544,504 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
 	return 0;
 }
 
+/*
+ * On-demand symbol loading: build a sorted in-memory index of
+ * (adjusted_start, end, name strtab offset, binding, type) from the ELF
+ * symtab, then resolve individual addresses via binary search.  Symbol
+ * names are read from the file's string table with pread() at lookup
+ * time, so the steady-state footprint is only the index itself, not
+ * the full set of struct symbol allocations.
+ */
+
+static void symbol_budget_warning(void)
+{
+	pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
+			"remaining symbols will be [unknown]\n",
+			symbol_conf.max_symbol_bytes);
+}
+
+static int cmp_sym_idx(const void *a, const void *b)
+{
+	const struct sym_idx *sa = a, *sb = b;
+
+	if (sa->start != sb->start)
+		return sa->start < sb->start ? -1 : 1;
+	/*
+	 * qsort is not stable, so make the order total: tiebreak on
+	 * name_off (the symbol's st_name, a strtab offset) so that an
+	 * exact start tie keeps a deterministic first entry regardless of
+	 * how qsort permutes equal-start aliases.
+	 *
+	 * Note: st_name is a strtab offset, not a symtab index, and the two
+	 * are not monotonic -- so on an exact heuristic tie this is
+	 * deterministic but NOT necessarily eager's symtab order (eager
+	 * falls to arch__choose_best_symbol -> SYMBOL_A, i.e. insertion
+	 * order).  See "Known divergences" in the commit message.
+	 */
+	if (sa->name_off != sb->name_off)
+		return sa->name_off < sb->name_off ? -1 : 1;
+	return 0;
+}
+
+/*
+ * Return true if a symbol should be included in the on-demand index.
+ * This mirrors the filter used by the eager dso__load_sym_internal()
+ * loop: functions, objects and IFUNCs, plus STT_NOTYPE "labels" in
+ * text/data sections, all restricted to SHF_ALLOC sections.
+ */
+static bool ondemand_sym_ok(Elf *elf, Elf_Data *secstrs,
+			    const GElf_Sym *sym, u32 sh_link,
+			    uint16_t e_machine)
+{
+	Elf_Scn *sym_sec;
+	GElf_Shdr sym_shdr;
+	int is_label = elf_sym__is_label(sym);
+	const char *name;
+
+	if (!is_label && !elf_sym__filter((GElf_Sym *)sym))
+		return false;
+
+	if (sym->st_shndx == SHN_ABS)
+		return false;
+
+	sym_sec = elf_getscn(elf, sym->st_shndx);
+	if (!sym_sec)
+		return false;
+	if (!gelf_getshdr(sym_sec, &sym_shdr))
+		return false;
+	if (!(sym_shdr.sh_flags & SHF_ALLOC))
+		return false;
+
+	if (is_label && (!secstrs || !elf_sec__filter(&sym_shdr, secstrs)))
+		return false;
+
+	name = elf_strptr(elf, sh_link, sym->st_name);
+	if (!name)
+		return false;
+
+	/*
+	 * Reject ARM/AArch64/RISC-V "mapping symbols" ($a/$d/$t/$x), as
+	 * the eager loop does.  They are zero-size STT_NOTYPE labels in
+	 * allocated sections that would otherwise be indexed and fill
+	 * forward over real functions, misattributing everything after
+	 * them.
+	 */
+	if (e_machine == EM_ARM || e_machine == EM_AARCH64) {
+		if (name[0] == '$' && strchr("adtx", name[1]) &&
+		    (name[2] == '\0' || name[2] == '.'))
+			return false;
+	}
+	if (e_machine == EM_RISCV) {
+		if (name[0] == '$' && strchr("dx", name[1]))
+			return false;
+	}
+
+	return true;
+}
+
+static int dso__build_ondemand_index(struct dso *dso, struct symsrc *syms_ss,
+				     struct symsrc *runtime_ss,
+				     int dynsym)
+{
+	struct dso_ondemand *od;
+	Elf *elf = syms_ss->elf;
+	GElf_Ehdr ehdr = syms_ss->ehdr;
+	GElf_Shdr shdr;
+	GElf_Shdr strshdr;
+	Elf_Scn *strscn, *sec_strndx;
+	Elf_Data *syms;
+	GElf_Sym sym;
+	Elf_Data *secstrs = NULL;
+	size_t i;
+	u32 count = 0, j;
+	u64 nr_entries, strtab_offset;
+
+	if (dynsym)
+		shdr = syms_ss->dynshdr;
+	else
+		shdr = syms_ss->symshdr;
+
+	syms = elf_getdata(dynsym ? syms_ss->dynsym : syms_ss->symtab, NULL);
+	if (!syms)
+		return -1;
+
+	if (!shdr.sh_entsize)
+		return 0;
+
+	nr_entries = shdr.sh_size / shdr.sh_entsize;
+	if (nr_entries > UINT32_MAX)
+		return -EOVERFLOW;
+
+	/*
+	 * File offset of the string table linked from the symbol table.
+	 * Symbol names are pread() from the file at lookup time, so we
+	 * only need the offset and size here, not the strings themselves.
+	 */
+	strscn = elf_getscn(elf, shdr.sh_link);
+	if (!strscn || !gelf_getshdr(strscn, &strshdr))
+		return -1;
+	strtab_offset = strshdr.sh_offset;
+
+	/*
+	 * Section name string table, used to match the eager path's
+	 * elf_sec__filter() (text/data section check for STT_NOTYPE labels).
+	 */
+	sec_strndx = elf_getscn(elf, ehdr.e_shstrndx);
+	if (sec_strndx)
+		secstrs = elf_getdata(sec_strndx, NULL);
+
+	/* Count symbols that pass the filter (same filter as fill below) */
+	for (i = 0; i < nr_entries; i++) {
+		if (!gelf_getsym(syms, i, &sym))
+			continue;
+		if (ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
+				    ehdr.e_machine))
+			count++;
+	}
+
+	if (!count)
+		return 0;
+	if (count > SIZE_MAX / sizeof(*od->sorted))
+		return -EOVERFLOW;
+
+	/*
+	 * Account the index against the symbol memory budget: at 24
+	 * bytes/symbol it is the dominant on-demand cost and must count
+	 * toward --max-symbol-bytes just like struct symbol allocations do.
+	 */
+	if (symbol_conf.max_symbol_bytes &&
+	    symbol__bytes_used() + count * sizeof(struct sym_idx) >
+	    symbol_conf.max_symbol_bytes) {
+		symbol_budget_warning();
+		return 0; /* fall back to the eager loader's per-symbol budget */
+	}
+
+	od = zalloc(sizeof(*od));
+	if (!od)
+		return -1;
+
+	od->sorted = malloc(count * sizeof(*od->sorted));
+	if (!od->sorted) {
+		free(od);
+		return -1;
+	}
+	od->nr_alloc = count;	/* allocated; the deduped count may shrink */
+
+	/* Fill the index with adjusted addresses */
+	j = 0;
+	for (i = 0; i < nr_entries; i++) {
+		u64 adjusted;
+		GElf_Phdr phdr;
+
+		if (!gelf_getsym(syms, i, &sym))
+			continue;
+		if (!ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
+				     ehdr.e_machine))
+			continue;
+
+		adjusted = sym.st_value;
+
+		/* ARM Thumb bit fix (same as eager path, FUNC only) */
+		if ((ehdr.e_machine == EM_ARM) &&
+		    (GELF_ST_TYPE(sym.st_info) == STT_FUNC) &&
+		    (adjusted & 1))
+			--adjusted;
+
+		/*
+		 * Program header adjustment, identical to the eager loop:
+		 * read the PT_LOAD containing the symbol from the runtime
+		 * ELF (the debug-info file may have zeroed p_offset), and
+		 * fall back to the section-header bias when no program
+		 * header matches -- exactly what the eager path does when
+		 * elf_read_program_header fails.
+		 */
+		if (elf_read_program_header(runtime_ss->elf, adjusted,
+					    &phdr) == 0) {
+			adjusted -= phdr.p_vaddr - phdr.p_offset;
+		} else {
+			Elf_Scn *sym_sec = elf_getscn(elf, sym.st_shndx);
+			GElf_Shdr sym_shdr;
+
+			if (sym_sec && gelf_getshdr(sym_sec, &sym_shdr))
+				adjusted -= sym_shdr.sh_addr - sym_shdr.sh_offset;
+		}
+
+		od->sorted[j].start = adjusted;
+		od->sorted[j].end = sym.st_size; /* st_size for now, converted later */
+		od->sorted[j].name_off = sym.st_name; /* strtab-relative */
+		od->sorted[j].binding = GELF_ST_BIND(sym.st_info);
+		od->sorted[j].type = GELF_ST_TYPE(sym.st_info);
+		j++;
+	}
+
+	/* Sort by adjusted start address */
+	qsort(od->sorted, count, sizeof(*od->sorted), cmp_sym_idx);
+
+	/* Alias dedup: keep only the best symbol for each start address */
+	if (!symbol_conf.allow_aliases) {
+		u32 out = 0;
+
+		for (i = 0; i < count; i++) {
+			u32 best = i;
+			const char *na = NULL, *nb;
+			char *da = NULL, *db;
+
+			/* name_off is the strtab index (st_name) */
+			na = elf_strptr(elf, shdr.sh_link,
+					od->sorted[best].name_off);
+			if (na) {
+				da = dso__demangle_sym(dso, 0, na);
+				if (da)
+					na = da;
+			}
+
+			/* Find the best among all entries with this start */
+			for (j = i + 1; j < count &&
+			     od->sorted[j].start == od->sorted[i].start; j++) {
+				nb = elf_strptr(elf, shdr.sh_link,
+						od->sorted[j].name_off);
+				if (!na || !nb)
+					continue;
+
+				/* Demangle for comparison, matching eager path */
+				db = dso__demangle_sym(dso, 0, nb);
+				if (db)
+					nb = db;
+
+				/* od->sorted[].end holds st_size at this point */
+				if (choose_best_symbol_raw(
+					    od->sorted[best].end,
+					    od->sorted[best].type,
+					    od->sorted[best].binding, na,
+					    od->sorted[j].end,
+					    od->sorted[j].type,
+					    od->sorted[j].binding, nb) == SYMBOL_B) {
+					best = j;
+					free(da);
+					da = db;
+					na = nb;
+				} else {
+					free(db);
+				}
+			}
+
+			free(da);
+			od->sorted[out++] = od->sorted[best];
+			i = j - 1; /* skip past all aliases of this start */
+		}
+
+		if (out < count) {
+			struct sym_idx *shrunk;
+
+			shrunk = realloc(od->sorted, out * sizeof(*od->sorted));
+			if (shrunk) {
+				od->sorted = shrunk;
+				od->nr_alloc = out;
+			}
+		}
+		count = out;
+	}
+
+	/* Convert st_size to end addresses */
+	for (i = 0; i < count; i++) {
+		u64 size = od->sorted[i].end; /* was st_size */
+
+		if (size > 0)
+			od->sorted[i].end = od->sorted[i].start + size;
+		else if (i + 1 < count)
+			od->sorted[i].end = od->sorted[i + 1].start;
+		else
+			/* Match symbols__fixup_end's last-symbol formula. */
+			od->sorted[i].end = roundup(od->sorted[i].start, 4096) + 4096;
+	}
+
+	/*
+	 * Keep a private fd open for pread() of symbol names.  Dup with
+	 * O_CLOEXEC so children don't inherit it, and so that
+	 * symsrc__destroy() can close the original regardless of whether
+	 * it is a real file or a temporary debugdata extraction.
+	 *
+	 * If the dup fails (e.g. fd exhaustion), decline by returning 0
+	 * without setting the index: the caller falls back to the eager
+	 * loader so the DSO still gets symbols rather than going symbol-less.
+	 */
+	od->fd = fcntl(syms_ss->fd, F_DUPFD_CLOEXEC, 0);
+	if (od->fd < 0) {
+		free(od->sorted);
+		free(od);
+		return 0;
+	}
+	od->strtab_offset = strtab_offset;
+	od->strtab_size = strshdr.sh_size;
+	od->nr_sorted = count;
+
+	symbol__account_bytes(od->nr_alloc * sizeof(*od->sorted));
+
+	dso__set_ondemand(dso, od);
+
+	pr_debug("%s: on-demand index: %u symbols\n",
+		 dso__long_name(dso), count);
+
+	return 1;
+}
+
+/*
+ * Read a NUL-terminated symbol name from the file's string table at
+ * file offset @off.  The fast path uses a stack buffer; if the NUL is
+ * not found within it (names can exceed 1 KiB for template-heavy C++
+ * mangled names), grow a heap buffer geometrically from 4 KiB, doubling
+ * until the terminator is found or the strtab is exhausted.  This keeps
+ * a single long name cheap even when it sits near the start of a large
+ * (tens of MB) strtab, while bounding a corrupt/missing terminator by
+ * the remaining strtab size.
+ *
+ * Returns a pointer to the name (either @buf or a heap allocation) and
+ * sets *@to_free to the buffer that must be free()d (NULL for @buf).
+ * Returns NULL on read error or if no NUL terminator exists within the
+ * strtab bounds.
+ */
+static const char *ondemand_read_name(int fd, u64 strtab_offset,
+				      u64 strtab_size, u64 name_off,
+				      char *buf, size_t buflen,
+				      char **to_free)
+{
+	ssize_t n;
+	u64 remain;
+	u64 file_off;
+	size_t cap;
+
+	*to_free = NULL;
+
+	/* name_off is strtab-relative (the symbol's st_name). */
+	if (name_off >= strtab_size)
+		return NULL;
+	file_off = strtab_offset + name_off;
+	remain = strtab_size - name_off;
+
+	/*
+	 * Fast path: stack buffer, expect the name to fit.  Cap at the
+	 * remaining strtab so a missing terminator can't read past the
+	 * section into adjacent file data.
+	 */
+	n = pread(fd, buf, min((u64)(buflen - 1), remain), file_off);
+	if (n <= 0)
+		return NULL;
+	buf[n] = '\0';
+	if (memchr(buf, '\0', n))
+		return buf;
+
+	/*
+	 * Slow path: the name is longer than buflen.  Grow a heap buffer
+	 * geometrically, doubling until the terminator appears, so a long
+	 * name costs O(name length), not O(remaining strtab size).
+	 */
+	cap = 4096;
+	for (;;) {
+		char *tmp;
+		size_t want = cap;
+
+		if (want > remain)
+			want = remain;
+		if (want == 0)
+			break;
+
+		tmp = *to_free ? realloc(*to_free, want + 1) : malloc(want + 1);
+		if (!tmp) {
+			free(*to_free);
+			*to_free = NULL;
+			return NULL;
+		}
+		*to_free = tmp;
+
+		n = pread(fd, *to_free, want, file_off);
+		if (n <= 0) {
+			free(*to_free);
+			*to_free = NULL;
+			return NULL;
+		}
+		(*to_free)[n] = '\0';
+
+		if (memchr(*to_free, '\0', n))
+			return *to_free;
+
+		/*
+		 * Read the whole remaining strtab (or hit EOF) with no
+		 * terminator: corrupt file, bail instead of re-reading.
+		 */
+		if (want >= remain || (u64)n >= remain)
+			break;
+
+		/* Avoid size_t overflow on absurdly large strtabs. */
+		if (cap > SIZE_MAX / 2)
+			break;
+		cap *= 2;
+	}
+
+	free(*to_free);
+	*to_free = NULL;
+	return NULL;
+}
+
+struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr)
+{
+	struct dso_ondemand *od = dso__ondemand(dso);
+	u32 lo, hi, mid;
+	const char *name;
+	char namebuf[1024];
+	char *name_heap = NULL;
+	char *demangled;
+	struct symbol *s = NULL;
+
+	if (!od || !od->sorted || od->fd < 0)
+		return NULL;
+
+	lo = 0;
+	hi = od->nr_sorted;
+	while (lo < hi) {
+		mid = (lo + hi) / 2;
+		if (addr < od->sorted[mid].start)
+			hi = mid;
+		else if (addr >= od->sorted[mid].end)
+			lo = mid + 1;
+		else
+			goto found;
+	}
+
+	/* Not found */
+	return NULL;
+
+found:
+	/*
+	 * Check the budget before doing any name I/O or demangling, so an
+	 * over-budget DSO stops paying pread+demangle on every later miss.
+	 */
+	if (symbol_conf.max_symbol_bytes &&
+	    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
+		symbol_budget_warning();
+		return NULL;
+	}
+
+	name = ondemand_read_name(od->fd, od->strtab_offset, od->strtab_size,
+				  od->sorted[mid].name_off,
+				  namebuf, sizeof(namebuf), &name_heap);
+	if (!name)
+		return NULL;
+
+	demangled = dso__demangle_sym(dso, 0, name);
+	if (demangled)
+		name = demangled;
+
+	s = symbol__new(od->sorted[mid].start,
+			od->sorted[mid].end - od->sorted[mid].start,
+			od->sorted[mid].binding,
+			od->sorted[mid].type, name);
+	free(demangled);
+	free(name_heap);
+	if (s)
+		__symbols__insert(dso__symbols(dso), s);
+	return s;
+}
+
 static int
 dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 		       struct symsrc *runtime_ss, int kmodule, int dynsym)
@@ -1626,6 +2153,35 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 	if (kmodule && adjust_kernel_syms)
 		max_text_sh_offset = max_text_section(runtime_ss->elf, &runtime_ss->ehdr);
 
+	/*
+	 * On-demand mode: build a sorted index and resolve symbols
+	 * per-address instead of loading the entire symtab.
+	 * Only for userspace ELF DSOs.
+	 */
+	if (symbol_conf.lazy_load_symbols && !dso__kernel(dso) && !kmodule) {
+		int oret = 0;
+
+		/* Use symtab if available, fall back to dynsym (e.g. vdso) */
+		if (!dynsym && syms_ss->symtab)
+			oret = dso__build_ondemand_index(dso, syms_ss,
+							 runtime_ss, 0);
+		else if (dynsym && !dso__ondemand(dso) && syms_ss->dynsym)
+			oret = dso__build_ondemand_index(dso, syms_ss,
+							 runtime_ss, 1);
+
+		/*
+		 * On hard error, propagate it.  If an index was built, the
+		 * DSO resolves on demand; skip the eager loop below.  If the
+		 * build declined (oret == 0, no index -- e.g. no usable
+		 * symbols, or fd duplication failed), continue with the eager
+		 * loop so the DSO still gets symbols.
+		 */
+		if (oret < 0)
+			return oret;
+		if (dso__ondemand(dso))
+			return 1;
+	}
+
 	curr_dso = dso__get(dso);
 	elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
 		struct symbol *f;
@@ -1636,9 +2192,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 		bool used_opd = false;
 		if (symbol_conf.max_symbol_bytes &&
 		    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
-			pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
-					"remaining symbols will be [unknown]\n",
-					symbol_conf.max_symbol_bytes);
+			symbol_budget_warning();
 			break;
 		}
 
diff --git a/tools/perf/util/symbol-minimal.c b/tools/perf/util/symbol-minimal.c
index 0a71d1463952..1a7088389d62 100644
--- a/tools/perf/util/symbol-minimal.c
+++ b/tools/perf/util/symbol-minimal.c
@@ -373,6 +373,17 @@ void symbol__elf_init(void)
 {
 }
 
+/*
+ * Without libelf there is no on-demand index to resolve against; the
+ * DSO can never be in on-demand mode, so this is only reached when a
+ * caller ignores the NULL from dso__ondemand().
+ */
+struct symbol *dso__find_symbol_ondemand(struct dso *dso __maybe_unused,
+					 u64 addr __maybe_unused)
+{
+	return NULL;
+}
+
 bool filename__has_section(const char *filename __maybe_unused, const char *sec __maybe_unused)
 {
 	return false;
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 62a4f91c2f5d..7e90f3c11832 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -155,60 +155,76 @@ int __weak arch__choose_best_symbol(struct symbol *syma,
 	return SYMBOL_A;
 }
 
-static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
+int choose_best_symbol_raw(u64 a_size, u8 a_type, u8 a_binding,
+			   const char *a_name,
+			   u64 b_size, u8 b_type, u8 b_binding,
+			   const char *b_name)
 {
 	s64 a;
 	s64 b;
 	size_t na, nb;
 
 	/* Prefer a symbol with non zero length */
-	a = syma->end - syma->start;
-	b = symb->end - symb->start;
-	if ((b == 0) && (a > 0))
+	if ((b_size == 0) && (a_size > 0))
 		return SYMBOL_A;
-	else if ((a == 0) && (b > 0))
+	else if ((a_size == 0) && (b_size > 0))
 		return SYMBOL_B;
 
-	if (symbol__type(syma) != symbol__type(symb)) {
-		if (symbol__type(syma) == STT_NOTYPE)
+	if (a_type != b_type) {
+		if (a_type == STT_NOTYPE)
 			return SYMBOL_B;
-		if (symbol__type(symb) == STT_NOTYPE)
+		if (b_type == STT_NOTYPE)
 			return SYMBOL_A;
 	}
 
 	/* Prefer a non weak symbol over a weak one */
-	a = symbol__binding(syma) == STB_WEAK;
-	b = symbol__binding(symb) == STB_WEAK;
+	a = a_binding == STB_WEAK;
+	b = b_binding == STB_WEAK;
 	if (b && !a)
 		return SYMBOL_A;
 	if (a && !b)
 		return SYMBOL_B;
 
 	/* Prefer a global symbol over a non global one */
-	a = symbol__binding(syma) == STB_GLOBAL;
-	b = symbol__binding(symb) == STB_GLOBAL;
+	a = a_binding == STB_GLOBAL;
+	b = b_binding == STB_GLOBAL;
 	if (a && !b)
 		return SYMBOL_A;
 	if (b && !a)
 		return SYMBOL_B;
 
 	/* Prefer a symbol with less underscores */
-	a = prefix_underscores_count(syma->name);
-	b = prefix_underscores_count(symb->name);
+	a = prefix_underscores_count(a_name);
+	b = prefix_underscores_count(b_name);
 	if (b > a)
 		return SYMBOL_A;
 	else if (a > b)
 		return SYMBOL_B;
 
 	/* Choose the symbol with the longest name */
-	na = strlen(syma->name);
-	nb = strlen(symb->name);
+	na = strlen(a_name);
+	nb = strlen(b_name);
 	if (na > nb)
 		return SYMBOL_A;
 	else if (na < nb)
 		return SYMBOL_B;
 
-	return arch__choose_best_symbol(syma, symb);
+	/* All heuristics tied: let the caller decide (arch hook, etc.) */
+	return -1;
+}
+
+static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
+{
+	int ret;
+
+	ret = choose_best_symbol_raw(syma->end - syma->start,
+				     symbol__type(syma), symbol__binding(syma), syma->name,
+				     symb->end - symb->start,
+				     symbol__type(symb), symbol__binding(symb), symb->name);
+	if (ret < 0)
+		return arch__choose_best_symbol(syma, symb);
+
+	return ret;
 }
 
 void symbols__fixup_duplicate(struct rb_root_cached *symbols)
@@ -1940,11 +1956,19 @@ int dso__load(struct dso *dso, struct map *map)
 		}
 
 #ifdef HAVE_LIBBFD_SUPPORT
+#ifdef HAVE_LIBELF_SUPPORT
+		if (is_reg && !symbol_conf.lazy_load_symbols)
+#else
 		if (is_reg)
+#endif
 			bfdrc = dso__load_bfd_symbols(dso, name);
 #endif
 		if (is_reg && bfdrc < 0)
 			sirc = symsrc__init(ss, dso, name, symtab_type);
+#if defined(HAVE_LIBBFD_SUPPORT) && defined(HAVE_LIBELF_SUPPORT)
+		if (is_reg && symbol_conf.lazy_load_symbols && sirc < 0)
+			bfdrc = dso__load_bfd_symbols(dso, name);
+#endif
 
 		if (nsexit)
 			nsinfo__mountns_enter(dso__nsinfo(dso), &nsc);
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 0d5d3792aac1..db1b662988e3 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -231,6 +231,10 @@ struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *
 size_t symbol__bytes_used(void);
 void symbol__account_bytes(size_t bytes);
 void symbol__unaccount_bytes(size_t bytes);
+int choose_best_symbol_raw(u64 a_size, u8 a_type, u8 a_binding,
+			   const char *a_name,
+			   u64 b_size, u8 b_type, u8 b_binding,
+			   const char *b_name);
 size_t __symbol__fprintf_symname_offs(const struct symbol *sym,
 				      const struct addr_location *al,
 				      bool unknown_as_addr,
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index 6a16c5badd5e..0f8d044eba20 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -74,6 +74,7 @@ struct symbol_conf {
 			no_buildid_mmap2,
 			guest_code,
 			lazy_load_kernel_maps,
+			lazy_load_symbols,
 			keep_exited_threads,
 			annotate_data_member,
 			annotate_data_sample,

-- 
Git-155)


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

From: Alireza Haghdoost <haghdoost@uber.com>

perf script eagerly materializes eligible symbols from every DSO
encountered in samples. On a production fixture, it loaded about 765k
symbols to resolve about 45k distinct (DSO, symbol) frames, exceeding the
memory available in a memory-constrained cgroup.

This patch adds --lazy-load-symbols for userspace ELF DSOs. It builds a
compact sorted index, resolves sampled addresses by binary search, reads
symbol names with pread(), and caches resolved symbols in the existing
rb-tree. Lazy lookup retains one CLOEXEC file descriptor per indexed DSO.
If the descriptor cannot be retained, perf discards the index and eagerly
loads that DSO instead.

On the same fixture, peak RssAnon drops from 265 MiB to 39 MiB and wall
time from 3.1 seconds to 1.85 seconds. Memory optimizations usually cost
time; this one does not because lazy loading skips many unnecessary
calloc() calls and demangling operations. Output was byte-identical on
the tested x86-64 workloads and an aarch64 capture.

Lazy loading is most effective when samples reference only a small
fraction of the available symbols, such as profiles spanning many large
DSOs. It still builds an index proportional to the total symbol count.
Eager loading remains available for dense symbol coverage or cases
requiring its broader ELF and architecture support.

This does not claim full parity with the eager loader. Lazy loading
supports the common userspace ELF symtab/dynsym case, with these known
differences:

  - .gnu_debugdata merging and PPC64 .opd are not handled.
  - SHT_NOBITS re-reading, IFUNC PLT naming, and exact-tie alias ordering
    are unreachable or output-equivalent on x86-64.

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/builtin-script.c      |   4 +-
 tools/perf/util/dso.c            |  10 +
 tools/perf/util/dso.h            |  29 ++
 tools/perf/util/map.c            |   9 +-
 tools/perf/util/symbol-elf.c     | 560 ++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/symbol-minimal.c |  11 +
 tools/perf/util/symbol.c         |  58 ++--
 tools/perf/util/symbol.h         |   4 +
 tools/perf/util/symbol_conf.h    |   1 +
 9 files changed, 664 insertions(+), 22 deletions(-)

diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 50fa6ca6455a..81e56378d3d2 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -4037,7 +4037,7 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
 }
 
 static int parse_max_symbol_bytes(const struct option *opt,
-				  const char *str, int unset)
+				const char *str, int unset)
 {
 	unsigned long *max_bytes = (unsigned long *)opt->value;
 	static struct parse_tag size_tags[] = {
@@ -4166,6 +4166,8 @@ int cmd_script(int argc, const char **argv)
 	OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
 		     "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
 		     parse_max_symbol_bytes),
+	OPT_BOOLEAN(0, "lazy-load-symbols", &symbol_conf.lazy_load_symbols,
+		    "Resolve symbols lazily instead of loading full symtabs"),
 	OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
 	OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
 	OPT_BOOLEAN('I', "show-info", &show_full_info,
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 42bfe30a3b51..196cd27e79e2 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -1755,6 +1755,16 @@ void dso__delete(struct dso *dso)
 
 	dso__data_close(dso);
 	auxtrace_cache__free(RC_CHK_ACCESS(dso)->auxtrace_cache);
+	if (RC_CHK_ACCESS(dso)->ondemand) {
+		struct dso_ondemand *od = RC_CHK_ACCESS(dso)->ondemand;
+
+		free(od->sorted);
+		symbol__unaccount_bytes(od->nr_alloc * sizeof(struct sym_idx));
+		if (od->fd >= 0)
+			close(od->fd);
+		free(od);
+		RC_CHK_ACCESS(dso)->ondemand = NULL;
+	}
 	dso_cache__free(dso);
 	dso__free_a2l(dso);
 	dso__free_libdw(dso);
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index 55c4aaa53c38..50c4d64e5081 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -282,6 +282,23 @@ struct dso_bpf_prog {
 	struct perf_env	*env;
 };
 
+struct sym_idx {
+	u64	start;	/* adjusted st_value (same space as sym->start) */
+	u64	end;	/* start + st_size, or next start if st_size==0 */
+	u32	name_off; /* symbol's st_name: offset into the strtab */
+	u8	binding;
+	u8	type;
+};
+
+struct dso_ondemand {
+	int		 fd;		/* kept open for pread */
+	u64		 strtab_offset;	/* file offset of strtab section */
+	u64		 strtab_size;
+	struct sym_idx	*sorted;
+	u32		 nr_sorted;	/* deduped count */
+	u32		 nr_alloc;	/* allocated count, for accounting */
+};
+
 struct auxtrace_cache;
 
 DECLARE_RC_STRUCT(dso) {
@@ -308,6 +325,7 @@ DECLARE_RC_STRUCT(dso) {
 	char		 *symsrc_filename;
 	struct nsinfo	*nsinfo;
 	struct auxtrace_cache *auxtrace_cache;
+	struct dso_ondemand *ondemand;
 	union { /* Tool specific area */
 		void	 *priv;
 		u64	 db_id;
@@ -448,6 +466,16 @@ static inline void dso__set_auxtrace_cache(struct dso *dso, struct auxtrace_cach
 	RC_CHK_ACCESS(dso)->auxtrace_cache = cache;
 }
 
+static inline struct dso_ondemand *dso__ondemand(struct dso *dso)
+{
+	return RC_CHK_ACCESS(dso)->ondemand;
+}
+
+static inline void dso__set_ondemand(struct dso *dso, struct dso_ondemand *od)
+{
+	RC_CHK_ACCESS(dso)->ondemand = od;
+}
+
 static inline struct dso_bpf_prog *dso__bpf_prog(struct dso *dso)
 {
 	return &RC_CHK_ACCESS(dso)->bpf_prog;
@@ -823,6 +851,7 @@ int dso__read_binary_type_filename(const struct dso *dso, enum dso_binary_type t
 				   const char *root_dir, char *filename, size_t size);
 bool is_kernel_module(const char *pathname, int cpumode);
 bool dso__needs_decompress(struct dso *dso);
+struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr);
 int dso__decompress_kmodule_fd(struct dso *dso, const char *name);
 int dso__decompress_kmodule_path(struct dso *dso, const char *name,
 				 char *pathname, size_t len);
diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 41cdddc987ee..68ffa01c9f9e 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -382,10 +382,17 @@ int map__load(struct map *map)
 
 struct symbol *map__find_symbol(struct map *map, u64 addr)
 {
+	struct dso *dso;
+	struct symbol *sym;
+
 	if (map__load(map) < 0)
 		return NULL;
 
-	return dso__find_symbol(map__dso(map), addr);
+	dso = map__dso(map);
+	sym = dso__find_symbol(dso, addr);
+	if (!sym && dso__ondemand(dso))
+		sym = dso__find_symbol_ondemand(dso, addr);
+	return sym;
 }
 
 struct symbol *map__find_symbol_by_name_idx(struct map *map, const char *name, size_t *idx)
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 914e42d21f70..e4d77e46e883 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -2,6 +2,7 @@
 #include <fcntl.h>
 #include <stdio.h>
 #include <errno.h>
+#include <stdint.h>
 #include <stdlib.h>
 #include <string.h>
 #include <unistd.h>
@@ -600,6 +601,32 @@ static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
  * And always look at the original dso, not at debuginfo packages, that
  * have the PLT data stripped out (shdr_rel_plt.sh_type == SHT_NOBITS).
  */
+static void dso__clip_ondemand_symbols_at(struct dso *dso, u64 addr)
+{
+	struct dso_ondemand *od = dso__ondemand(dso);
+	struct sym_idx *sym;
+	u32 lo = 0, hi, mid;
+
+	if (!od)
+		return;
+
+	hi = od->nr_sorted;
+	while (lo < hi) {
+		mid = (lo + hi) / 2;
+		if (od->sorted[mid].start < addr)
+			lo = mid + 1;
+		else
+			hi = mid;
+	}
+
+	if (!lo)
+		return;
+
+	sym = &od->sorted[lo - 1];
+	if (sym->end > addr)
+		sym->end = addr;
+}
+
 int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
 {
 	uint32_t idx;
@@ -623,6 +650,8 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
 	if (!elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL))
 		return 0;
 
+	dso__clip_ondemand_symbols_at(dso, shdr_plt.sh_offset);
+
 	/*
 	 * A symbol from a previous section (e.g. .init) can have been expanded
 	 * by symbols__fixup_end() to overlap .plt. Truncate it before adding
@@ -1515,6 +1544,504 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
 	return 0;
 }
 
+/*
+ * On-demand symbol loading: build a sorted in-memory index of
+ * (adjusted_start, end, name strtab offset, binding, type) from the ELF
+ * symtab, then resolve individual addresses via binary search.  Symbol
+ * names are read from the file's string table with pread() at lookup
+ * time, so the steady-state footprint is only the index itself, not
+ * the full set of struct symbol allocations.
+ */
+
+static void symbol_budget_warning(void)
+{
+	pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
+			"remaining symbols will be [unknown]\n",
+			symbol_conf.max_symbol_bytes);
+}
+
+static int cmp_sym_idx(const void *a, const void *b)
+{
+	const struct sym_idx *sa = a, *sb = b;
+
+	if (sa->start != sb->start)
+		return sa->start < sb->start ? -1 : 1;
+	/*
+	 * qsort is not stable, so make the order total: tiebreak on
+	 * name_off (the symbol's st_name, a strtab offset) so that an
+	 * exact start tie keeps a deterministic first entry regardless of
+	 * how qsort permutes equal-start aliases.
+	 *
+	 * Note: st_name is a strtab offset, not a symtab index, and the two
+	 * are not monotonic -- so on an exact heuristic tie this is
+	 * deterministic but NOT necessarily eager's symtab order (eager
+	 * falls to arch__choose_best_symbol -> SYMBOL_A, i.e. insertion
+	 * order).  See "Known divergences" in the commit message.
+	 */
+	if (sa->name_off != sb->name_off)
+		return sa->name_off < sb->name_off ? -1 : 1;
+	return 0;
+}
+
+/*
+ * Return true if a symbol should be included in the on-demand index.
+ * This mirrors the filter used by the eager dso__load_sym_internal()
+ * loop: functions, objects and IFUNCs, plus STT_NOTYPE "labels" in
+ * text/data sections, all restricted to SHF_ALLOC sections.
+ */
+static bool ondemand_sym_ok(Elf *elf, Elf_Data *secstrs,
+			    const GElf_Sym *sym, u32 sh_link,
+			    uint16_t e_machine)
+{
+	Elf_Scn *sym_sec;
+	GElf_Shdr sym_shdr;
+	int is_label = elf_sym__is_label(sym);
+	const char *name;
+
+	if (!is_label && !elf_sym__filter((GElf_Sym *)sym))
+		return false;
+
+	if (sym->st_shndx == SHN_ABS)
+		return false;
+
+	sym_sec = elf_getscn(elf, sym->st_shndx);
+	if (!sym_sec)
+		return false;
+	if (!gelf_getshdr(sym_sec, &sym_shdr))
+		return false;
+	if (!(sym_shdr.sh_flags & SHF_ALLOC))
+		return false;
+
+	if (is_label && (!secstrs || !elf_sec__filter(&sym_shdr, secstrs)))
+		return false;
+
+	name = elf_strptr(elf, sh_link, sym->st_name);
+	if (!name)
+		return false;
+
+	/*
+	 * Reject ARM/AArch64/RISC-V "mapping symbols" ($a/$d/$t/$x), as
+	 * the eager loop does.  They are zero-size STT_NOTYPE labels in
+	 * allocated sections that would otherwise be indexed and fill
+	 * forward over real functions, misattributing everything after
+	 * them.
+	 */
+	if (e_machine == EM_ARM || e_machine == EM_AARCH64) {
+		if (name[0] == '$' && strchr("adtx", name[1]) &&
+		    (name[2] == '\0' || name[2] == '.'))
+			return false;
+	}
+	if (e_machine == EM_RISCV) {
+		if (name[0] == '$' && strchr("dx", name[1]))
+			return false;
+	}
+
+	return true;
+}
+
+static int dso__build_ondemand_index(struct dso *dso, struct symsrc *syms_ss,
+				     struct symsrc *runtime_ss,
+				     int dynsym)
+{
+	struct dso_ondemand *od;
+	Elf *elf = syms_ss->elf;
+	GElf_Ehdr ehdr = syms_ss->ehdr;
+	GElf_Shdr shdr;
+	GElf_Shdr strshdr;
+	Elf_Scn *strscn, *sec_strndx;
+	Elf_Data *syms;
+	GElf_Sym sym;
+	Elf_Data *secstrs = NULL;
+	size_t i;
+	u32 count = 0, j;
+	u64 nr_entries, strtab_offset;
+
+	if (dynsym)
+		shdr = syms_ss->dynshdr;
+	else
+		shdr = syms_ss->symshdr;
+
+	syms = elf_getdata(dynsym ? syms_ss->dynsym : syms_ss->symtab, NULL);
+	if (!syms)
+		return -1;
+
+	if (!shdr.sh_entsize)
+		return 0;
+
+	nr_entries = shdr.sh_size / shdr.sh_entsize;
+	if (nr_entries > UINT32_MAX)
+		return -EOVERFLOW;
+
+	/*
+	 * File offset of the string table linked from the symbol table.
+	 * Symbol names are pread() from the file at lookup time, so we
+	 * only need the offset and size here, not the strings themselves.
+	 */
+	strscn = elf_getscn(elf, shdr.sh_link);
+	if (!strscn || !gelf_getshdr(strscn, &strshdr))
+		return -1;
+	strtab_offset = strshdr.sh_offset;
+
+	/*
+	 * Section name string table, used to match the eager path's
+	 * elf_sec__filter() (text/data section check for STT_NOTYPE labels).
+	 */
+	sec_strndx = elf_getscn(elf, ehdr.e_shstrndx);
+	if (sec_strndx)
+		secstrs = elf_getdata(sec_strndx, NULL);
+
+	/* Count symbols that pass the filter (same filter as fill below) */
+	for (i = 0; i < nr_entries; i++) {
+		if (!gelf_getsym(syms, i, &sym))
+			continue;
+		if (ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
+				    ehdr.e_machine))
+			count++;
+	}
+
+	if (!count)
+		return 0;
+	if (count > SIZE_MAX / sizeof(*od->sorted))
+		return -EOVERFLOW;
+
+	/*
+	 * Account the index against the symbol memory budget: at 24
+	 * bytes/symbol it is the dominant on-demand cost and must count
+	 * toward --max-symbol-bytes just like struct symbol allocations do.
+	 */
+	if (symbol_conf.max_symbol_bytes &&
+	    symbol__bytes_used() + count * sizeof(struct sym_idx) >
+	    symbol_conf.max_symbol_bytes) {
+		symbol_budget_warning();
+		return 0; /* fall back to the eager loader's per-symbol budget */
+	}
+
+	od = zalloc(sizeof(*od));
+	if (!od)
+		return -1;
+
+	od->sorted = malloc(count * sizeof(*od->sorted));
+	if (!od->sorted) {
+		free(od);
+		return -1;
+	}
+	od->nr_alloc = count;	/* allocated; the deduped count may shrink */
+
+	/* Fill the index with adjusted addresses */
+	j = 0;
+	for (i = 0; i < nr_entries; i++) {
+		u64 adjusted;
+		GElf_Phdr phdr;
+
+		if (!gelf_getsym(syms, i, &sym))
+			continue;
+		if (!ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
+				     ehdr.e_machine))
+			continue;
+
+		adjusted = sym.st_value;
+
+		/* ARM Thumb bit fix (same as eager path, FUNC only) */
+		if ((ehdr.e_machine == EM_ARM) &&
+		    (GELF_ST_TYPE(sym.st_info) == STT_FUNC) &&
+		    (adjusted & 1))
+			--adjusted;
+
+		/*
+		 * Program header adjustment, identical to the eager loop:
+		 * read the PT_LOAD containing the symbol from the runtime
+		 * ELF (the debug-info file may have zeroed p_offset), and
+		 * fall back to the section-header bias when no program
+		 * header matches -- exactly what the eager path does when
+		 * elf_read_program_header fails.
+		 */
+		if (elf_read_program_header(runtime_ss->elf, adjusted,
+					    &phdr) == 0) {
+			adjusted -= phdr.p_vaddr - phdr.p_offset;
+		} else {
+			Elf_Scn *sym_sec = elf_getscn(elf, sym.st_shndx);
+			GElf_Shdr sym_shdr;
+
+			if (sym_sec && gelf_getshdr(sym_sec, &sym_shdr))
+				adjusted -= sym_shdr.sh_addr - sym_shdr.sh_offset;
+		}
+
+		od->sorted[j].start = adjusted;
+		od->sorted[j].end = sym.st_size; /* st_size for now, converted later */
+		od->sorted[j].name_off = sym.st_name; /* strtab-relative */
+		od->sorted[j].binding = GELF_ST_BIND(sym.st_info);
+		od->sorted[j].type = GELF_ST_TYPE(sym.st_info);
+		j++;
+	}
+
+	/* Sort by adjusted start address */
+	qsort(od->sorted, count, sizeof(*od->sorted), cmp_sym_idx);
+
+	/* Alias dedup: keep only the best symbol for each start address */
+	if (!symbol_conf.allow_aliases) {
+		u32 out = 0;
+
+		for (i = 0; i < count; i++) {
+			u32 best = i;
+			const char *na = NULL, *nb;
+			char *da = NULL, *db;
+
+			/* name_off is the strtab index (st_name) */
+			na = elf_strptr(elf, shdr.sh_link,
+					od->sorted[best].name_off);
+			if (na) {
+				da = dso__demangle_sym(dso, 0, na);
+				if (da)
+					na = da;
+			}
+
+			/* Find the best among all entries with this start */
+			for (j = i + 1; j < count &&
+			     od->sorted[j].start == od->sorted[i].start; j++) {
+				nb = elf_strptr(elf, shdr.sh_link,
+						od->sorted[j].name_off);
+				if (!na || !nb)
+					continue;
+
+				/* Demangle for comparison, matching eager path */
+				db = dso__demangle_sym(dso, 0, nb);
+				if (db)
+					nb = db;
+
+				/* od->sorted[].end holds st_size at this point */
+				if (choose_best_symbol_raw(
+					    od->sorted[best].end,
+					    od->sorted[best].type,
+					    od->sorted[best].binding, na,
+					    od->sorted[j].end,
+					    od->sorted[j].type,
+					    od->sorted[j].binding, nb) == SYMBOL_B) {
+					best = j;
+					free(da);
+					da = db;
+					na = nb;
+				} else {
+					free(db);
+				}
+			}
+
+			free(da);
+			od->sorted[out++] = od->sorted[best];
+			i = j - 1; /* skip past all aliases of this start */
+		}
+
+		if (out < count) {
+			struct sym_idx *shrunk;
+
+			shrunk = realloc(od->sorted, out * sizeof(*od->sorted));
+			if (shrunk) {
+				od->sorted = shrunk;
+				od->nr_alloc = out;
+			}
+		}
+		count = out;
+	}
+
+	/* Convert st_size to end addresses */
+	for (i = 0; i < count; i++) {
+		u64 size = od->sorted[i].end; /* was st_size */
+
+		if (size > 0)
+			od->sorted[i].end = od->sorted[i].start + size;
+		else if (i + 1 < count)
+			od->sorted[i].end = od->sorted[i + 1].start;
+		else
+			/* Match symbols__fixup_end's last-symbol formula. */
+			od->sorted[i].end = roundup(od->sorted[i].start, 4096) + 4096;
+	}
+
+	/*
+	 * Keep a private fd open for pread() of symbol names.  Dup with
+	 * O_CLOEXEC so children don't inherit it, and so that
+	 * symsrc__destroy() can close the original regardless of whether
+	 * it is a real file or a temporary debugdata extraction.
+	 *
+	 * If the dup fails (e.g. fd exhaustion), decline by returning 0
+	 * without setting the index: the caller falls back to the eager
+	 * loader so the DSO still gets symbols rather than going symbol-less.
+	 */
+	od->fd = fcntl(syms_ss->fd, F_DUPFD_CLOEXEC, 0);
+	if (od->fd < 0) {
+		free(od->sorted);
+		free(od);
+		return 0;
+	}
+	od->strtab_offset = strtab_offset;
+	od->strtab_size = strshdr.sh_size;
+	od->nr_sorted = count;
+
+	symbol__account_bytes(od->nr_alloc * sizeof(*od->sorted));
+
+	dso__set_ondemand(dso, od);
+
+	pr_debug("%s: on-demand index: %u symbols\n",
+		 dso__long_name(dso), count);
+
+	return 1;
+}
+
+/*
+ * Read a NUL-terminated symbol name from the file's string table at
+ * file offset @off.  The fast path uses a stack buffer; if the NUL is
+ * not found within it (names can exceed 1 KiB for template-heavy C++
+ * mangled names), grow a heap buffer geometrically from 4 KiB, doubling
+ * until the terminator is found or the strtab is exhausted.  This keeps
+ * a single long name cheap even when it sits near the start of a large
+ * (tens of MB) strtab, while bounding a corrupt/missing terminator by
+ * the remaining strtab size.
+ *
+ * Returns a pointer to the name (either @buf or a heap allocation) and
+ * sets *@to_free to the buffer that must be free()d (NULL for @buf).
+ * Returns NULL on read error or if no NUL terminator exists within the
+ * strtab bounds.
+ */
+static const char *ondemand_read_name(int fd, u64 strtab_offset,
+				      u64 strtab_size, u64 name_off,
+				      char *buf, size_t buflen,
+				      char **to_free)
+{
+	ssize_t n;
+	u64 remain;
+	u64 file_off;
+	size_t cap;
+
+	*to_free = NULL;
+
+	/* name_off is strtab-relative (the symbol's st_name). */
+	if (name_off >= strtab_size)
+		return NULL;
+	file_off = strtab_offset + name_off;
+	remain = strtab_size - name_off;
+
+	/*
+	 * Fast path: stack buffer, expect the name to fit.  Cap at the
+	 * remaining strtab so a missing terminator can't read past the
+	 * section into adjacent file data.
+	 */
+	n = pread(fd, buf, min((u64)(buflen - 1), remain), file_off);
+	if (n <= 0)
+		return NULL;
+	buf[n] = '\0';
+	if (memchr(buf, '\0', n))
+		return buf;
+
+	/*
+	 * Slow path: the name is longer than buflen.  Grow a heap buffer
+	 * geometrically, doubling until the terminator appears, so a long
+	 * name costs O(name length), not O(remaining strtab size).
+	 */
+	cap = 4096;
+	for (;;) {
+		char *tmp;
+		size_t want = cap;
+
+		if (want > remain)
+			want = remain;
+		if (want == 0)
+			break;
+
+		tmp = *to_free ? realloc(*to_free, want + 1) : malloc(want + 1);
+		if (!tmp) {
+			free(*to_free);
+			*to_free = NULL;
+			return NULL;
+		}
+		*to_free = tmp;
+
+		n = pread(fd, *to_free, want, file_off);
+		if (n <= 0) {
+			free(*to_free);
+			*to_free = NULL;
+			return NULL;
+		}
+		(*to_free)[n] = '\0';
+
+		if (memchr(*to_free, '\0', n))
+			return *to_free;
+
+		/*
+		 * Read the whole remaining strtab (or hit EOF) with no
+		 * terminator: corrupt file, bail instead of re-reading.
+		 */
+		if (want >= remain || (u64)n >= remain)
+			break;
+
+		/* Avoid size_t overflow on absurdly large strtabs. */
+		if (cap > SIZE_MAX / 2)
+			break;
+		cap *= 2;
+	}
+
+	free(*to_free);
+	*to_free = NULL;
+	return NULL;
+}
+
+struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr)
+{
+	struct dso_ondemand *od = dso__ondemand(dso);
+	u32 lo, hi, mid;
+	const char *name;
+	char namebuf[1024];
+	char *name_heap = NULL;
+	char *demangled;
+	struct symbol *s = NULL;
+
+	if (!od || !od->sorted || od->fd < 0)
+		return NULL;
+
+	lo = 0;
+	hi = od->nr_sorted;
+	while (lo < hi) {
+		mid = (lo + hi) / 2;
+		if (addr < od->sorted[mid].start)
+			hi = mid;
+		else if (addr >= od->sorted[mid].end)
+			lo = mid + 1;
+		else
+			goto found;
+	}
+
+	/* Not found */
+	return NULL;
+
+found:
+	/*
+	 * Check the budget before doing any name I/O or demangling, so an
+	 * over-budget DSO stops paying pread+demangle on every later miss.
+	 */
+	if (symbol_conf.max_symbol_bytes &&
+	    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
+		symbol_budget_warning();
+		return NULL;
+	}
+
+	name = ondemand_read_name(od->fd, od->strtab_offset, od->strtab_size,
+				  od->sorted[mid].name_off,
+				  namebuf, sizeof(namebuf), &name_heap);
+	if (!name)
+		return NULL;
+
+	demangled = dso__demangle_sym(dso, 0, name);
+	if (demangled)
+		name = demangled;
+
+	s = symbol__new(od->sorted[mid].start,
+			od->sorted[mid].end - od->sorted[mid].start,
+			od->sorted[mid].binding,
+			od->sorted[mid].type, name);
+	free(demangled);
+	free(name_heap);
+	if (s)
+		__symbols__insert(dso__symbols(dso), s);
+	return s;
+}
+
 static int
 dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 		       struct symsrc *runtime_ss, int kmodule, int dynsym)
@@ -1626,6 +2153,35 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 	if (kmodule && adjust_kernel_syms)
 		max_text_sh_offset = max_text_section(runtime_ss->elf, &runtime_ss->ehdr);
 
+	/*
+	 * On-demand mode: build a sorted index and resolve symbols
+	 * per-address instead of loading the entire symtab.
+	 * Only for userspace ELF DSOs.
+	 */
+	if (symbol_conf.lazy_load_symbols && !dso__kernel(dso) && !kmodule) {
+		int oret = 0;
+
+		/* Use symtab if available, fall back to dynsym (e.g. vdso) */
+		if (!dynsym && syms_ss->symtab)
+			oret = dso__build_ondemand_index(dso, syms_ss,
+							 runtime_ss, 0);
+		else if (dynsym && !dso__ondemand(dso) && syms_ss->dynsym)
+			oret = dso__build_ondemand_index(dso, syms_ss,
+							 runtime_ss, 1);
+
+		/*
+		 * On hard error, propagate it.  If an index was built, the
+		 * DSO resolves on demand; skip the eager loop below.  If the
+		 * build declined (oret == 0, no index -- e.g. no usable
+		 * symbols, or fd duplication failed), continue with the eager
+		 * loop so the DSO still gets symbols.
+		 */
+		if (oret < 0)
+			return oret;
+		if (dso__ondemand(dso))
+			return 1;
+	}
+
 	curr_dso = dso__get(dso);
 	elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
 		struct symbol *f;
@@ -1636,9 +2192,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
 		bool used_opd = false;
 		if (symbol_conf.max_symbol_bytes &&
 		    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
-			pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
-					"remaining symbols will be [unknown]\n",
-					symbol_conf.max_symbol_bytes);
+			symbol_budget_warning();
 			break;
 		}
 
diff --git a/tools/perf/util/symbol-minimal.c b/tools/perf/util/symbol-minimal.c
index 0a71d1463952..1a7088389d62 100644
--- a/tools/perf/util/symbol-minimal.c
+++ b/tools/perf/util/symbol-minimal.c
@@ -373,6 +373,17 @@ void symbol__elf_init(void)
 {
 }
 
+/*
+ * Without libelf there is no on-demand index to resolve against; the
+ * DSO can never be in on-demand mode, so this is only reached when a
+ * caller ignores the NULL from dso__ondemand().
+ */
+struct symbol *dso__find_symbol_ondemand(struct dso *dso __maybe_unused,
+					 u64 addr __maybe_unused)
+{
+	return NULL;
+}
+
 bool filename__has_section(const char *filename __maybe_unused, const char *sec __maybe_unused)
 {
 	return false;
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 62a4f91c2f5d..7e90f3c11832 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -155,60 +155,76 @@ int __weak arch__choose_best_symbol(struct symbol *syma,
 	return SYMBOL_A;
 }
 
-static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
+int choose_best_symbol_raw(u64 a_size, u8 a_type, u8 a_binding,
+			   const char *a_name,
+			   u64 b_size, u8 b_type, u8 b_binding,
+			   const char *b_name)
 {
 	s64 a;
 	s64 b;
 	size_t na, nb;
 
 	/* Prefer a symbol with non zero length */
-	a = syma->end - syma->start;
-	b = symb->end - symb->start;
-	if ((b == 0) && (a > 0))
+	if ((b_size == 0) && (a_size > 0))
 		return SYMBOL_A;
-	else if ((a == 0) && (b > 0))
+	else if ((a_size == 0) && (b_size > 0))
 		return SYMBOL_B;
 
-	if (symbol__type(syma) != symbol__type(symb)) {
-		if (symbol__type(syma) == STT_NOTYPE)
+	if (a_type != b_type) {
+		if (a_type == STT_NOTYPE)
 			return SYMBOL_B;
-		if (symbol__type(symb) == STT_NOTYPE)
+		if (b_type == STT_NOTYPE)
 			return SYMBOL_A;
 	}
 
 	/* Prefer a non weak symbol over a weak one */
-	a = symbol__binding(syma) == STB_WEAK;
-	b = symbol__binding(symb) == STB_WEAK;
+	a = a_binding == STB_WEAK;
+	b = b_binding == STB_WEAK;
 	if (b && !a)
 		return SYMBOL_A;
 	if (a && !b)
 		return SYMBOL_B;
 
 	/* Prefer a global symbol over a non global one */
-	a = symbol__binding(syma) == STB_GLOBAL;
-	b = symbol__binding(symb) == STB_GLOBAL;
+	a = a_binding == STB_GLOBAL;
+	b = b_binding == STB_GLOBAL;
 	if (a && !b)
 		return SYMBOL_A;
 	if (b && !a)
 		return SYMBOL_B;
 
 	/* Prefer a symbol with less underscores */
-	a = prefix_underscores_count(syma->name);
-	b = prefix_underscores_count(symb->name);
+	a = prefix_underscores_count(a_name);
+	b = prefix_underscores_count(b_name);
 	if (b > a)
 		return SYMBOL_A;
 	else if (a > b)
 		return SYMBOL_B;
 
 	/* Choose the symbol with the longest name */
-	na = strlen(syma->name);
-	nb = strlen(symb->name);
+	na = strlen(a_name);
+	nb = strlen(b_name);
 	if (na > nb)
 		return SYMBOL_A;
 	else if (na < nb)
 		return SYMBOL_B;
 
-	return arch__choose_best_symbol(syma, symb);
+	/* All heuristics tied: let the caller decide (arch hook, etc.) */
+	return -1;
+}
+
+static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
+{
+	int ret;
+
+	ret = choose_best_symbol_raw(syma->end - syma->start,
+				     symbol__type(syma), symbol__binding(syma), syma->name,
+				     symb->end - symb->start,
+				     symbol__type(symb), symbol__binding(symb), symb->name);
+	if (ret < 0)
+		return arch__choose_best_symbol(syma, symb);
+
+	return ret;
 }
 
 void symbols__fixup_duplicate(struct rb_root_cached *symbols)
@@ -1940,11 +1956,19 @@ int dso__load(struct dso *dso, struct map *map)
 		}
 
 #ifdef HAVE_LIBBFD_SUPPORT
+#ifdef HAVE_LIBELF_SUPPORT
+		if (is_reg && !symbol_conf.lazy_load_symbols)
+#else
 		if (is_reg)
+#endif
 			bfdrc = dso__load_bfd_symbols(dso, name);
 #endif
 		if (is_reg && bfdrc < 0)
 			sirc = symsrc__init(ss, dso, name, symtab_type);
+#if defined(HAVE_LIBBFD_SUPPORT) && defined(HAVE_LIBELF_SUPPORT)
+		if (is_reg && symbol_conf.lazy_load_symbols && sirc < 0)
+			bfdrc = dso__load_bfd_symbols(dso, name);
+#endif
 
 		if (nsexit)
 			nsinfo__mountns_enter(dso__nsinfo(dso), &nsc);
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 0d5d3792aac1..db1b662988e3 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -231,6 +231,10 @@ struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *
 size_t symbol__bytes_used(void);
 void symbol__account_bytes(size_t bytes);
 void symbol__unaccount_bytes(size_t bytes);
+int choose_best_symbol_raw(u64 a_size, u8 a_type, u8 a_binding,
+			   const char *a_name,
+			   u64 b_size, u8 b_type, u8 b_binding,
+			   const char *b_name);
 size_t __symbol__fprintf_symname_offs(const struct symbol *sym,
 				      const struct addr_location *al,
 				      bool unknown_as_addr,
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index 6a16c5badd5e..0f8d044eba20 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -74,6 +74,7 @@ struct symbol_conf {
 			no_buildid_mmap2,
 			guest_code,
 			lazy_load_kernel_maps,
+			lazy_load_symbols,
 			keep_exited_threads,
 			annotate_data_member,
 			annotate_data_sample,

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 4/4] perf script: Document and test --lazy-load-symbols and --max-symbol-bytes
  2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  -1 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

Document both new options in perf-script.txt: their interaction, the
memory tradeoff, that lazy loading applies only to userspace ELF DSOs
(kernel DSOs and modules always load eagerly), and that output may
differ from the default loader for some targets.

Add a shell test that records a small profile with callchains and
asserts:

- --lazy-load-symbols produces output byte-identical to the eager
  loader, and that at least one symbol actually resolved (so the
  comparison can't pass vacuously on all-[unknown] output);
- --max-symbol-bytes=1K, with and without lazy loading, forces
  [unknown] resolution with a single warning and exit code 0 (no crash).

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/Documentation/perf-script.txt           |  24 +++++
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 120 +++++++++++++++++++++
 2 files changed, 144 insertions(+)

diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index 200ea25891d8..b5a90ff22342 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -412,6 +412,30 @@ include::itrace.txt[]
 
         Default: 127
 
+--lazy-load-symbols::
+	Resolve symbols lazily instead of eagerly loading the full
+	symbol table of every DSO that appears in a sample. A compact
+	sorted index is built per DSO and only the addresses that appear
+	in samples are materialized into symbols, with names read from the
+	file's string table at lookup time. This sharply reduces memory
+	(and usually time) for profiles of large binaries where only a
+	small fraction of the symbol table is referenced. This applies only
+	to userspace ELF DSOs; kernel DSOs and modules always load eagerly.
+	Output may differ from the default loader for some targets
+	(e.g. PPC64 .opd, .gnu_debugdata, or split debuginfo). Default: off.
+
+--max-symbol-bytes::
+	Limit the bytes held in struct symbol allocations (and, with
+	--lazy-load-symbols, the lazy index) for DSOs on the ELF symbol
+	loader path -- userspace DSOs plus vmlinux-as-ELF and kernel
+	modules. This is not a cap on all symbol memory or RSS: symbols
+	from kallsyms, JIT maps, PLT synthesis, and libbfd are counted
+	but not capped. Accepts a size with a B/K/M/G suffix (e.g. 128M).
+	When the budget is exceeded, further symbols resolve to [unknown]
+	and a warning is printed. This is a safety net independent of
+	--lazy-load-symbols and can be used with or without it. Default: 0
+	(unlimited).
+
 --ns::
 	Use 9 decimal places when displaying time (i.e. show the nanoseconds)
 
diff --git a/tools/perf/tests/shell/script_lazy_load_symbols.sh b/tools/perf/tests/shell/script_lazy_load_symbols.sh
new file mode 100755
index 000000000000..799c61e2f3e8
--- /dev/null
+++ b/tools/perf/tests/shell/script_lazy_load_symbols.sh
@@ -0,0 +1,120 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# perf script lazy symbol loading tests
+#
+# Verifies that --lazy-load-symbols produces output identical to the default
+# eager symbol loader, and that --max-symbol-bytes caps symbol allocations
+# (emitting [unknown] plus a warning) without crashing.
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/perf_has_symbol.sh
+. "${shelldir}"/lib/perf_has_symbol.sh
+
+testsym="test_loop"
+
+skip_test_missing_symbol ${testsym}
+
+err=0
+temp_dir=$(mktemp -d /tmp/__perf_test.lazy_load.XXXXX)
+perfdata="${temp_dir}/perf.data"
+eager_out="${temp_dir}/eager.out"
+lazy_out="${temp_dir}/lazy.out"
+
+cleanup() {
+	rm -rf "${temp_dir}"
+	trap - EXIT TERM INT
+}
+
+trap_cleanup() {
+	echo "Unexpected signal in ${FUNCNAME[1]}"
+	cleanup
+	exit 1
+}
+trap trap_cleanup EXIT TERM INT
+
+test_lazy_load_identical() {
+	echo "Lazy-load output matches eager loader"
+
+	# Record a small profile with callchains so symbol resolution runs.
+	if ! perf record -o "${perfdata}" -g -- perf test -w thloop 2> /dev/null
+	then
+		echo "Lazy-load identical [Skipped record not supported]"
+		return
+	fi
+
+	if ! perf script -i "${perfdata}" 2> /dev/null > "${eager_out}" || \
+	   ! perf script --lazy-load-symbols -i "${perfdata}" 2> /dev/null > "${lazy_out}"
+	then
+		echo "Lazy-load identical [Failed perf script error]"
+		err=1
+		return
+	fi
+
+	# The comparison is only meaningful if something actually resolved;
+	# two all-[unknown] outputs would also match.
+	if ! grep -q "${testsym}" "${eager_out}"
+	then
+		echo "Lazy-load identical [Skipped no ${testsym} resolved]"
+		return
+	fi
+
+	if ! cmp -s "${eager_out}" "${lazy_out}"
+	then
+		echo "Lazy-load identical [Failed output differs]"
+		err=1
+		return
+	fi
+	echo "Lazy-load identical [Success]"
+}
+
+test_max_symbol_bytes() {
+	echo "--max-symbol-bytes budget enforcement"
+
+	# Depends on ${perfdata} from test_lazy_load_identical.
+	if [ ! -s "${perfdata}" ]
+	then
+		echo "--max-symbol-bytes budget [Skipped record not supported]"
+		return
+	fi
+
+	# A tiny budget forces most symbols to be dropped as [unknown],
+	# with a single warning, and must not crash.
+	if ! perf script --max-symbol-bytes=1K -i "${perfdata}" > /dev/null \
+		2> "${temp_dir}/budget.err"
+	then
+		echo "--max-symbol-bytes budget [Failed nonzero exit]"
+		err=1
+		return
+	fi
+	if ! grep -q "symbol memory budget exceeded" "${temp_dir}/budget.err"
+	then
+		echo "--max-symbol-bytes budget [Failed missing warning]"
+		err=1
+		return
+	fi
+
+	if ! perf script --lazy-load-symbols --max-symbol-bytes=1K \
+		-i "${perfdata}" > /dev/null 2> "${temp_dir}/lazy-budget.err"
+	then
+		echo "--max-symbol-bytes lazy budget [Failed nonzero exit]"
+		err=1
+		return
+	fi
+	warnings=$(grep -c "symbol memory budget exceeded" \
+		"${temp_dir}/lazy-budget.err" || true)
+	if [ "${warnings}" -ne 1 ]
+	then
+		echo "--max-symbol-bytes lazy budget [Failed warning count: ${warnings}]"
+		err=1
+		return
+	fi
+	echo "--max-symbol-bytes budget [Success]"
+}
+
+test_lazy_load_identical
+test_max_symbol_bytes
+
+cleanup
+exit $err

-- 
Git-155)


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 4/4] perf script: Document and test --lazy-load-symbols and --max-symbol-bytes
@ 2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-15 18:42 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Andrii Nakryiko,
	Alexei Starovoitov
  Cc: linux-perf-users, linux-kernel, Alireza Haghdoost

From: Alireza Haghdoost <haghdoost@uber.com>

Document both new options in perf-script.txt: their interaction, the
memory tradeoff, that lazy loading applies only to userspace ELF DSOs
(kernel DSOs and modules always load eagerly), and that output may
differ from the default loader for some targets.

Add a shell test that records a small profile with callchains and
asserts:

- --lazy-load-symbols produces output byte-identical to the eager
  loader, and that at least one symbol actually resolved (so the
  comparison can't pass vacuously on all-[unknown] output);
- --max-symbol-bytes=1K, with and without lazy loading, forces
  [unknown] resolution with a single warning and exit code 0 (no crash).

Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Kimi:K3
---
 tools/perf/Documentation/perf-script.txt           |  24 +++++
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 120 +++++++++++++++++++++
 2 files changed, 144 insertions(+)

diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index 200ea25891d8..b5a90ff22342 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -412,6 +412,30 @@ include::itrace.txt[]
 
         Default: 127
 
+--lazy-load-symbols::
+	Resolve symbols lazily instead of eagerly loading the full
+	symbol table of every DSO that appears in a sample. A compact
+	sorted index is built per DSO and only the addresses that appear
+	in samples are materialized into symbols, with names read from the
+	file's string table at lookup time. This sharply reduces memory
+	(and usually time) for profiles of large binaries where only a
+	small fraction of the symbol table is referenced. This applies only
+	to userspace ELF DSOs; kernel DSOs and modules always load eagerly.
+	Output may differ from the default loader for some targets
+	(e.g. PPC64 .opd, .gnu_debugdata, or split debuginfo). Default: off.
+
+--max-symbol-bytes::
+	Limit the bytes held in struct symbol allocations (and, with
+	--lazy-load-symbols, the lazy index) for DSOs on the ELF symbol
+	loader path -- userspace DSOs plus vmlinux-as-ELF and kernel
+	modules. This is not a cap on all symbol memory or RSS: symbols
+	from kallsyms, JIT maps, PLT synthesis, and libbfd are counted
+	but not capped. Accepts a size with a B/K/M/G suffix (e.g. 128M).
+	When the budget is exceeded, further symbols resolve to [unknown]
+	and a warning is printed. This is a safety net independent of
+	--lazy-load-symbols and can be used with or without it. Default: 0
+	(unlimited).
+
 --ns::
 	Use 9 decimal places when displaying time (i.e. show the nanoseconds)
 
diff --git a/tools/perf/tests/shell/script_lazy_load_symbols.sh b/tools/perf/tests/shell/script_lazy_load_symbols.sh
new file mode 100755
index 000000000000..799c61e2f3e8
--- /dev/null
+++ b/tools/perf/tests/shell/script_lazy_load_symbols.sh
@@ -0,0 +1,120 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# perf script lazy symbol loading tests
+#
+# Verifies that --lazy-load-symbols produces output identical to the default
+# eager symbol loader, and that --max-symbol-bytes caps symbol allocations
+# (emitting [unknown] plus a warning) without crashing.
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/perf_has_symbol.sh
+. "${shelldir}"/lib/perf_has_symbol.sh
+
+testsym="test_loop"
+
+skip_test_missing_symbol ${testsym}
+
+err=0
+temp_dir=$(mktemp -d /tmp/__perf_test.lazy_load.XXXXX)
+perfdata="${temp_dir}/perf.data"
+eager_out="${temp_dir}/eager.out"
+lazy_out="${temp_dir}/lazy.out"
+
+cleanup() {
+	rm -rf "${temp_dir}"
+	trap - EXIT TERM INT
+}
+
+trap_cleanup() {
+	echo "Unexpected signal in ${FUNCNAME[1]}"
+	cleanup
+	exit 1
+}
+trap trap_cleanup EXIT TERM INT
+
+test_lazy_load_identical() {
+	echo "Lazy-load output matches eager loader"
+
+	# Record a small profile with callchains so symbol resolution runs.
+	if ! perf record -o "${perfdata}" -g -- perf test -w thloop 2> /dev/null
+	then
+		echo "Lazy-load identical [Skipped record not supported]"
+		return
+	fi
+
+	if ! perf script -i "${perfdata}" 2> /dev/null > "${eager_out}" || \
+	   ! perf script --lazy-load-symbols -i "${perfdata}" 2> /dev/null > "${lazy_out}"
+	then
+		echo "Lazy-load identical [Failed perf script error]"
+		err=1
+		return
+	fi
+
+	# The comparison is only meaningful if something actually resolved;
+	# two all-[unknown] outputs would also match.
+	if ! grep -q "${testsym}" "${eager_out}"
+	then
+		echo "Lazy-load identical [Skipped no ${testsym} resolved]"
+		return
+	fi
+
+	if ! cmp -s "${eager_out}" "${lazy_out}"
+	then
+		echo "Lazy-load identical [Failed output differs]"
+		err=1
+		return
+	fi
+	echo "Lazy-load identical [Success]"
+}
+
+test_max_symbol_bytes() {
+	echo "--max-symbol-bytes budget enforcement"
+
+	# Depends on ${perfdata} from test_lazy_load_identical.
+	if [ ! -s "${perfdata}" ]
+	then
+		echo "--max-symbol-bytes budget [Skipped record not supported]"
+		return
+	fi
+
+	# A tiny budget forces most symbols to be dropped as [unknown],
+	# with a single warning, and must not crash.
+	if ! perf script --max-symbol-bytes=1K -i "${perfdata}" > /dev/null \
+		2> "${temp_dir}/budget.err"
+	then
+		echo "--max-symbol-bytes budget [Failed nonzero exit]"
+		err=1
+		return
+	fi
+	if ! grep -q "symbol memory budget exceeded" "${temp_dir}/budget.err"
+	then
+		echo "--max-symbol-bytes budget [Failed missing warning]"
+		err=1
+		return
+	fi
+
+	if ! perf script --lazy-load-symbols --max-symbol-bytes=1K \
+		-i "${perfdata}" > /dev/null 2> "${temp_dir}/lazy-budget.err"
+	then
+		echo "--max-symbol-bytes lazy budget [Failed nonzero exit]"
+		err=1
+		return
+	fi
+	warnings=$(grep -c "symbol memory budget exceeded" \
+		"${temp_dir}/lazy-budget.err" || true)
+	if [ "${warnings}" -ne 1 ]
+	then
+		echo "--max-symbol-bytes lazy budget [Failed warning count: ${warnings}]"
+		err=1
+		return
+	fi
+	echo "--max-symbol-bytes budget [Success]"
+}
+
+test_lazy_load_identical
+test_max_symbol_bytes
+
+cleanup
+exit $err

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 23+ messages in thread

* Re: [PATCH 4/4] perf script: Document and test --lazy-load-symbols and --max-symbol-bytes
  2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  (?)
@ 2026-09-15 18:50   ` sashiko-bot
  2026-09-16 16:21     ` Alireza Haghdoost
  -1 siblings, 1 reply; 23+ messages in thread
From: sashiko-bot @ 2026-09-15 18:50 UTC (permalink / raw)
  To: Alireza Haghdoost; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] perf test: missing err=2 assignments cause skips to be reported as success
- [Medium] perf test: strict byte comparison fails on platforms with differing debug output
--

commit 152590a92c1dd38a6f7f70f13ced82231e7560bb
Author: Alireza Haghdoost <haghdoost@uber.com>

perf script: Document and test --lazy-load-symbols and --max-symbol-bytes

Document both new options in perf-script.txt detailing their interaction,
the memory tradeoffs, and output behavior for certain targets. Add a shell
test that records a small profile and asserts that lazy loading produces
identical output to the eager loader, and that the max symbol budget is
enforced.

> diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
> index 200ea25891d88..b5a90ff22342f 100644
> --- a/tools/perf/Documentation/perf-script.txt
> +++ b/tools/perf/Documentation/perf-script.txt
> @@ -412,6 +412,30 @@ include::itrace.txt[]
>  
>          Default: 127
>  
> +--lazy-load-symbols::
> +	Resolve symbols lazily instead of eagerly loading the full
> +	symbol table of every DSO that appears in a sample. A compact
> +	sorted index is built per DSO and only the addresses that appear
> +	in samples are materialized into symbols, with names read from the
> +	file's string table at lookup time. This sharply reduces memory
> +	(and usually time) for profiles of large binaries where only a
> +	small fraction of the symbol table is referenced. This applies only
> +	to userspace ELF DSOs; kernel DSOs and modules always load eagerly.
> +	Output may differ from the default loader for some targets
> +	(e.g. PPC64 .opd, .gnu_debugdata, or split debuginfo). Default: off.

[ ... ]

> diff --git a/tools/perf/tests/shell/script_lazy_load_symbols.sh b/tools/perf/tests/shell/script_lazy_load_symbols.sh
> new file mode 100755
> index 0000000000000..799c61e2f3e87
> --- /dev/null
> +++ b/tools/perf/tests/shell/script_lazy_load_symbols.sh
> @@ -0,0 +1,120 @@
[ ... ]
> +test_lazy_load_identical() {
> +	echo "Lazy-load output matches eager loader"
> +
> +	# Record a small profile with callchains so symbol resolution runs.
> +	if ! perf record -o "${perfdata}" -g -- perf test -w thloop 2> /dev/null
> +	then
> +		echo "Lazy-load identical [Skipped record not supported]"
> +		return

[Severity: Medium]
Will this script falsely report success when it skips tests? It looks like
the function returns here without setting err=2, which would cause the test
script to exit with 0 (pass) instead of 2 (skip) when perf record fails.

> +	fi
> +
> +	if ! perf script -i "${perfdata}" 2> /dev/null > "${eager_out}" || \
> +	   ! perf script --lazy-load-symbols -i "${perfdata}" 2> /dev/null > "${lazy_out}"
> +	then
> +		echo "Lazy-load identical [Failed perf script error]"
> +		err=1
> +		return
> +	fi
> +
> +	# The comparison is only meaningful if something actually resolved;
> +	# two all-[unknown] outputs would also match.
> +	if ! grep -q "${testsym}" "${eager_out}"
> +	then
> +		echo "Lazy-load identical [Skipped no ${testsym} resolved]"
> +		return
> +	fi
> +
> +	if ! cmp -s "${eager_out}" "${lazy_out}"
> +	then
> +		echo "Lazy-load identical [Failed output differs]"
> +		err=1
> +		return

[Severity: Medium]
Since the documentation mentions that output may differ from the default loader
for some targets (e.g., PPC64, .gnu_debugdata, or split debuginfo), will this
strict byte-identical comparison fail spuriously on those platforms?

> +	fi
> +	echo "Lazy-load identical [Success]"
> +}
> +
> +test_max_symbol_bytes() {
> +	echo "--max-symbol-bytes budget enforcement"
> +
> +	# Depends on ${perfdata} from test_lazy_load_identical.
> +	if [ ! -s "${perfdata}" ]
> +	then
> +		echo "--max-symbol-bytes budget [Skipped record not supported]"
> +		return

[Severity: Medium]
Similar to the skip path in test_lazy_load_identical(), should err=2 be set
here so the script correctly returns the skip exit code rather than success?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com?part=4

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
  2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  (?)
@ 2026-09-15 18:53   ` sashiko-bot
  2026-09-15 19:04     ` Alireza Haghdoost
  -1 siblings, 1 reply; 23+ messages in thread
From: sashiko-bot @ 2026-09-15 18:53 UTC (permalink / raw)
  To: Alireza Haghdoost; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] perf: build failure on systems lacking ELF_C_READ_MMAP enum
--

commit d38dd3de1c465f53af2cb4e7df04744c926b7ed6
Author: Alireza Haghdoost <haghdoost@uber.com>

perf symbols: Fix broken ELF_C_READ_MMAP fallback guard

This patch attempts to make perf mmap ELF files by unconditionally defining
PERF_ELF_C_READ_MMAP to ELF_C_READ_MMAP. It removes the previous guard
because ELF_C_READ_MMAP is an enum rather than a preprocessor macro.

> diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
> --- a/tools/perf/util/symbol.h
> +++ b/tools/perf/util/symbol.h
> @@ -57,15 +57,8 @@ static inline bool is_livepatch_symbol(const char *str)
>  	return strstarts(str, KLP_SYM_PREFIX);
>  }
>  
> -/*
> - * libelf 0.8.x and earlier do not support ELF_C_READ_MMAP;
> - * for newer versions we can use mmap to reduce memory usage:
> - */
> -#ifdef ELF_C_READ_MMAP
> -# define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
> -#else
> -# define PERF_ELF_C_READ_MMAP ELF_C_READ
> -#endif
> +/* libelf falls back to internal reads when mmap fails (e.g. non-seekable fd). */
> +#define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP

[Severity: Low]
Will this cause compilation errors on systems where the libelf.h header lacks
the ELF_C_READ_MMAP enum?

By unconditionally assigning PERF_ELF_C_READ_MMAP to ELF_C_READ_MMAP in
tools/perf/util/symbol.h, this bypasses build feature detection. On systems
with libelf implementations that do not provide ELF_C_READ_MMAP, such as
elftoolchain on FreeBSD and macOS, this triggers an undeclared identifier
compilation error.

The commit message notes that objtool uses an #ifndef guard which works fine,
but since ELF_C_READ_MMAP is an enum in libelf, that #ifndef in objtool
silently evaluates to true and incorrectly downgrades all mmap reads to
standard reads instead of actually using mmap.

Could we restore a proper feature detection mechanism here rather than
relying on an unconditional definition?

>  
>  #ifdef HAVE_LIBELF_SUPPORT
>  Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com?part=1

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading
  2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  (?)
@ 2026-09-15 18:54   ` sashiko-bot
  2026-09-16 16:18     ` Alireza Haghdoost
  -1 siblings, 1 reply; 23+ messages in thread
From: sashiko-bot @ 2026-09-15 18:54 UTC (permalink / raw)
  To: Alireza Haghdoost; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf: memory exhaustion in ondemand_read_name() via truncated ELF
--

commit 9189c705674e1061e4de963daab5ec07bd55d920
Author: Alireza Haghdoost <haghdoost@uber.com>

perf script: Add --lazy-load-symbols for lazy symbol loading

This commit introduces lazy symbol loading for userspace ELF DSOs in perf
script. It builds a sorted in-memory index of symbols and resolves them via
binary search, reading symbol names on-demand from the string table using
pread() to reduce memory footprint.

diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 914e42d21f70f..e4d77e46e8830 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c

[ ... ]

> +static const char *ondemand_read_name(int fd, u64 strtab_offset,
> +				      u64 strtab_size, u64 name_off,
> +				      char *buf, size_t buflen,
> +				      char **to_free)
> +{

[ ... ]

> +	/*
> +	 * Slow path: the name is longer than buflen.  Grow a heap buffer
> +	 * geometrically, doubling until the terminator appears, so a long
> +	 * name costs O(name length), not O(remaining strtab size).
> +	 */
> +	cap = 4096;
> +	for (;;) {
> +		char *tmp;
> +		size_t want = cap;
> +
> +		if (want > remain)
> +			want = remain;
> +		if (want == 0)
> +			break;
> +
> +		tmp = *to_free ? realloc(*to_free, want + 1) : malloc(want + 1);
> +		if (!tmp) {
> +			free(*to_free);
> +			*to_free = NULL;
> +			return NULL;
> +		}
> +		*to_free = tmp;
> +
> +		n = pread(fd, *to_free, want, file_off);
> +		if (n <= 0) {
> +			free(*to_free);
> +			*to_free = NULL;
> +			return NULL;
> +		}
> +		(*to_free)[n] = '\0';
> +
> +		if (memchr(*to_free, '\0', n))
> +			return *to_free;
> +
> +		/*
> +		 * Read the whole remaining strtab (or hit EOF) with no
> +		 * terminator: corrupt file, bail instead of re-reading.
> +		 */
> +		if (want >= remain || (u64)n >= remain)
> +			break;

[Severity: Medium]
Does this loop properly handle a short read (EOF) on a truncated ELF file?

If pread() hits EOF before filling the requested want bytes, n will be smaller
than want but greater than 0. The loop condition currently checks if
want >= remain or (u64)n >= remain, but it appears to miss an explicit
short-read/EOF check like (size_t)n < want.

Because of this, if remain is very large (based on an unvalidated sh_size
in the ELF header), the loop could continuously geometrically reallocate up
to remain bytes while reading the exact same short bytes from the constant
file_off. Could this cause memory exhaustion / an OOM allocation failure?

> +
> +		/* Avoid size_t overflow on absurdly large strtabs. */
> +		if (cap > SIZE_MAX / 2)
> +			break;
> +		cap *= 2;
> +	}
> +
> +	free(*to_free);
> +	*to_free = NULL;
> +	return NULL;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com?part=3

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory
  2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  (?)
@ 2026-09-15 19:01   ` sashiko-bot
  2026-09-16 16:15     ` Alireza Haghdoost
  -1 siblings, 1 reply; 23+ messages in thread
From: sashiko-bot @ 2026-09-15 19:01 UTC (permalink / raw)
  To: Alireza Haghdoost; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] perf: data race and underflow on global symbol_bytes_used counter
- [High] perf: early abort of ELF symbol loading stretches symbols over address gaps
- [Medium] perf: u16 truncation in sym->namelen causes permanent memory budget leak
--

commit c9eea40a0a34f49638af63c080650086fd45e844
Author: Alireza Haghdoost <haghdoost@uber.com>

perf script: Add --max-symbol-bytes to bound ELF symbol memory

This patch adds a --max-symbol-bytes option to bound the memory allocated
for struct symbol when loading ELF symbols. Once the budget is reached,
further loading is aborted to prevent OOM kills.

> diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
> index e955c3feddcd6..914e42d21f70f 100644
> --- a/tools/perf/util/symbol-elf.c
> +++ b/tools/perf/util/symbol-elf.c
> @@ -1634,6 +1634,13 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
>  		int is_label = elf_sym__is_label(&sym);
>  		const char *section_name;
>  		bool used_opd = false;
> +		if (symbol_conf.max_symbol_bytes &&
> +		    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
> +			pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
> +					"remaining symbols will be [unknown]\n",
> +					symbol_conf.max_symbol_bytes);
> +			break;

[Severity: High]
Since ELF symbol tables are generally not sorted by address (they are grouped
by binding), breaking out of the loop early creates random gaps in the
address space. Following this, symbols__fixup_end() iterates the partially
loaded rb-tree and stretches any zero-length symbols to the start address of
the very next loaded symbol (prev->end = curr->start).

Will this cause execution samples originating in the missing un-symbolized
gaps to be incorrectly attributed to completely unrelated, stretched
zero-length symbols instead of falling back to [unknown]?

[ ... ]
> diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
> index 3587ad2431590..62a4f91c2f5db 100644
> --- a/tools/perf/util/symbol.c
> +++ b/tools/perf/util/symbol.c
[ ... ]
> +static size_t symbol_bytes_used;
> +
> +size_t symbol__bytes_used(void)
> +{
> +	return symbol_bytes_used;
> +}
> +
> +void symbol__account_bytes(size_t bytes)
> +{
> +	symbol_bytes_used += bytes;
> +}
> +
> +void symbol__unaccount_bytes(size_t bytes)
> +{
> +	symbol_bytes_used -= bytes;
> +}

[Severity: High]
The global symbol_bytes_used is updated without atomic operations. In
multi-threaded tools like perf top where different DSOs are loaded
concurrently by different threads, will concurrent updates cause lost
increments?

If subsequent decrements succeed, this could underflow symbol_bytes_used to
SIZE_MAX, instantly exhausting the memory budget and breaking all further
symbol resolution.

> +
>  struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
>  {
>  	size_t namelen = strlen(name) + 1;
> -	struct symbol *sym = calloc(1, (symbol_conf.priv_size +
> -					sizeof(*sym) + namelen));
> +	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) + namelen;
> +	struct symbol *sym;
> +
> +	sym = calloc(1, alloc_size);
>  	if (sym == NULL)
>  		return NULL;
>  
> +	symbol__account_bytes(alloc_size);
> +
[ ... ]
>  void symbol__delete(struct symbol *sym)
>  {
> +	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) +
> +			    sym->namelen + 1;

[Severity: Medium]
If a symbol name exceeds 65,535 bytes (which is possible with deeply mangled
C++ or Rust templates), the u16 sym->namelen field silently wraps when it is
assigned during allocation.

When symbol__delete() recalculates the budget to subtract using this truncated
u16 value, will it result in a much smaller subtraction than the original
addition, acting as a permanent memory budget leak that artificially exhausts
the limit?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com?part=2

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
  2026-09-15 18:53   ` sashiko-bot
@ 2026-09-15 19:04     ` Alireza Haghdoost
  2026-09-17  0:30       ` Namhyung Kim
  0 siblings, 1 reply; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-15 19:04 UTC (permalink / raw)
  To: sashiko-reviews; +Cc: linux-perf-users

> Could we restore a proper feature detection mechanism here rather than
> relying on an unconditional definition?

Thanks, this is a valid concern. I’ll restore the compile-time
feature-libelf-mmap probe removed by 22dd1ac91a77 and use it to select
between ELF_C_READ_MMAP and ELF_C_READ in v2.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory
  2026-09-15 19:01   ` sashiko-bot
@ 2026-09-16 16:15     ` Alireza Haghdoost
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-16 16:15 UTC (permalink / raw)
  To: sashiko-reviews; +Cc: linux-perf-users

Thanks, all three findings are valid.

I’ll make the global accounting counter atomic, ensure that a
budget-truncated symbol tree does not fill ranges containing omitted
symbols, and use the full stored name length instead of the truncated
u16 namelen when unaccounting allocations.

I’ll include these fixes in v2.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading
  2026-09-15 18:54   ` sashiko-bot
@ 2026-09-16 16:18     ` Alireza Haghdoost
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-16 16:18 UTC (permalink / raw)
  To: sashiko-reviews; +Cc: linux-perf-users

> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] perf: memory exhaustion in ondemand_read_name() via truncated ELF
> --
Thanks, this is valid. The constant file offset is intentional because
each iteration rereads a larger prefix, but a positive short read must
be treated as EOF. I’ll detect n < want after checking the returned
bytes for a terminator, abort the lookup as a truncated string table,
and add this fix to v2.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 4/4] perf script: Document and test --lazy-load-symbols and --max-symbol-bytes
  2026-09-15 18:50   ` sashiko-bot
@ 2026-09-16 16:21     ` Alireza Haghdoost
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-16 16:21 UTC (permalink / raw)
  To: sashiko-reviews; +Cc: linux-perf-users

> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [Medium] perf test: missing err=2 assignments cause skips to be reported as success
> - [Medium] perf test: strict byte comparison fails on platforms with differing debug output
> --

Thanks, both findings are valid. I’ll make the skip paths return
status 2 without overriding a prior failure, and narrow the
eager-versus-lazy comparison to the controlled test symbol rather than
comparing all output from every DSO. I’ll include these fixes in v2.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
  2026-09-15 19:04     ` Alireza Haghdoost
@ 2026-09-17  0:30       ` Namhyung Kim
  0 siblings, 0 replies; 23+ messages in thread
From: Namhyung Kim @ 2026-09-17  0:30 UTC (permalink / raw)
  To: Alireza Haghdoost; +Cc: sashiko-reviews, linux-perf-users

Hello,

On Tue, Sep 15, 2026 at 12:04:27PM -0700, Alireza Haghdoost wrote:
> > Could we restore a proper feature detection mechanism here rather than
> > relying on an unconditional definition?
> 
> Thanks, this is a valid concern. I’ll restore the compile-time
> feature-libelf-mmap probe removed by 22dd1ac91a77 and use it to select
> between ELF_C_READ_MMAP and ELF_C_READ in v2.

Please don't.

I think we already require much recent versions of elfutils like 0.157
for libdw which is released more than 10 years ago.  So I think it's ok
just to assume _MMAP is available.

Thanks,
Namhyung


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory
  2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  (?)
  (?)
@ 2026-09-17  6:53   ` Namhyung Kim
  2026-09-19 18:37     ` Alireza Haghdoost
  -1 siblings, 1 reply; 23+ messages in thread
From: Namhyung Kim @ 2026-09-17  6:53 UTC (permalink / raw)
  To: haghdoost
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Ian Rogers,
	Adrian Hunter, James Clark, Andrii Nakryiko, Alexei Starovoitov,
	linux-perf-users, linux-kernel

On Tue, Sep 15, 2026 at 11:42:44AM -0700, Alireza Haghdoost via B4 Relay wrote:
> From: Alireza Haghdoost <haghdoost@uber.com>
> 
> perf script eagerly materializes every ELF symbol into an rb-tree kept
> until process exit. Large profiles can therefore consume substantial
> anonymous memory, causing perf script to be OOM-killed or forcing the
> kernel to reclaim memory from co-located workloads.

It's not only for perf script.  Other commands like perf report and perf
annotate would have the same issue.

> 
> This patch adds --max-symbol-bytes to bound struct symbol allocations.
> Once the budget is reached, the ELF loader stops loading symbols, warns
> once, and lets unresolved addresses appear as [unknown].
> This allows users to bound the memory footprint upfront and explicitly
> choose between complete symbolization and avoiding unbounded host memory
> pressure. perf record already provides a similar --max-size option to
> bound disk usage.

Please add documentation when you add a new command line option.

> 
> The counter includes every symbol__new() allocation, but this patch
> enforces the limit only in the ELF loader, which is the source of the
> unbounded memory growth addressed here. In this path, reaching the limit
> can safely produce [unknown] symbols. Other loaders currently treat a
> failed symbol allocation as an error. Capping those paths would therefore
> require separate changes whose complexity may outweigh the potential
> memory savings.

What's the other loaders you meant?

I don't see it produces [unknown] symbols.  Do you mean it just stops
loading symbols when it hits the limit?

> 
> Sizes require a B/K/M/G suffix. Zero, the default, means unlimited.
> 
> Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
> Assisted-by: Kimi:K3
> ---
>  tools/perf/builtin-script.c   | 31 +++++++++++++++++++++++++++++++
>  tools/perf/util/symbol-elf.c  |  7 +++++++
>  tools/perf/util/symbol.c      | 29 +++++++++++++++++++++++++++--
>  tools/perf/util/symbol.h      |  3 +++
>  tools/perf/util/symbol_conf.h |  1 +
>  5 files changed, 69 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index ad8ca08ceb5f..50fa6ca6455a 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -68,6 +68,7 @@
>  #include "util/thread.h"
>  #include "util/thread_map.h"
>  #include "util/time-utils.h"
> +#include "util/units.h"
>  #include "util/tool.h"
>  #include "util/trace-event.h"
>  #include "util/unwind.h"
> @@ -4035,6 +4036,33 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
>  	return 0;
>  }
>  
> +static int parse_max_symbol_bytes(const struct option *opt,
> +				  const char *str, int unset)
> +{
> +	unsigned long *max_bytes = (unsigned long *)opt->value;
> +	static struct parse_tag size_tags[] = {
> +		{ .tag  = 'B', .mult = 1       },
> +		{ .tag  = 'K', .mult = 1 << 10 },
> +		{ .tag  = 'M', .mult = 1 << 20 },
> +		{ .tag  = 'G', .mult = 1 << 30 },
> +		{ .tag  = 0 },
> +	};
> +	unsigned long bytes;
> +
> +	if (unset) {
> +		*max_bytes = 0;
> +		return 0;
> +	}
> +
> +	bytes = parse_tag_value(str, size_tags);
> +	if (bytes != (unsigned long)-1) {
> +		*max_bytes = bytes;
> +		return 0;
> +	}
> +
> +	return -1;
> +}
> +
>  int cmd_script(int argc, const char **argv)
>  {
>  	bool show_full_info = false;
> @@ -4135,6 +4163,9 @@ int cmd_script(int argc, const char **argv)
>  		     "Set the maximum stack depth when parsing the callchain, "
>  		     "anything beyond the specified depth will be ignored. "
>  		     "Default: kernel.perf_event_max_stack or " __stringify(PERF_MAX_STACK_DEPTH)),
> +	OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
> +		     "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
> +		     parse_max_symbol_bytes),
>  	OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
>  	OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
>  	OPT_BOOLEAN('I', "show-info", &show_full_info,
> diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
> index e955c3feddcd..914e42d21f70 100644
> --- a/tools/perf/util/symbol-elf.c
> +++ b/tools/perf/util/symbol-elf.c
> @@ -1634,6 +1634,13 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
>  		int is_label = elf_sym__is_label(&sym);
>  		const char *section_name;
>  		bool used_opd = false;

Please keep a blank line after declaration.


> +		if (symbol_conf.max_symbol_bytes &&
> +		    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
> +			pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
> +					"remaining symbols will be [unknown]\n",
> +					symbol_conf.max_symbol_bytes);
> +			break;
> +		}
>  
>  		if (!is_label && !elf_sym__filter(&sym))
>  			continue;
> diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
> index 3587ad243159..62a4f91c2f5d 100644
> --- a/tools/perf/util/symbol.c
> +++ b/tools/perf/util/symbol.c
> @@ -310,14 +310,35 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms)
>  		curr->end = roundup(curr->start, 4096) + 4096;
>  }
>  
> +static size_t symbol_bytes_used;
> +
> +size_t symbol__bytes_used(void)
> +{
> +	return symbol_bytes_used;
> +}
> +
> +void symbol__account_bytes(size_t bytes)
> +{
> +	symbol_bytes_used += bytes;
> +}
> +
> +void symbol__unaccount_bytes(size_t bytes)
> +{
> +	symbol_bytes_used -= bytes;
> +}
> +
>  struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
>  {
>  	size_t namelen = strlen(name) + 1;
> -	struct symbol *sym = calloc(1, (symbol_conf.priv_size +
> -					sizeof(*sym) + namelen));
> +	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) + namelen;

The convention is 'sizeof(*sym)' rather than 'sizeof(struct symbol)'.
So that it can easily handle type changes in the future.


> +	struct symbol *sym;
> +
> +	sym = calloc(1, alloc_size);
>  	if (sym == NULL)
>  		return NULL;
>  
> +	symbol__account_bytes(alloc_size);
> +
>  	if (symbol_conf.priv_size) {
>  		if (symbol_conf.init_annotation) {
>  			struct annotation *notes = (void *)sym;
> @@ -341,6 +362,9 @@ struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *
>  
>  void symbol__delete(struct symbol *sym)
>  {
> +	size_t alloc_size = symbol_conf.priv_size + sizeof(struct symbol) +
> +			    sym->namelen + 1;

Ditto.

Thanks,
Namhyung

> +
>  	if (symbol_conf.priv_size) {
>  		if (symbol_conf.init_annotation) {
>  			struct annotation *notes = symbol__annotation(sym);
> @@ -348,6 +372,7 @@ void symbol__delete(struct symbol *sym)
>  			annotation__exit(notes);
>  		}
>  	}
> +	symbol__unaccount_bytes(alloc_size);
>  	free(((void *)sym) - symbol_conf.priv_size);
>  }
>  
> diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
> index e5cef16b240d..0d5d3792aac1 100644
> --- a/tools/perf/util/symbol.h
> +++ b/tools/perf/util/symbol.h
> @@ -228,6 +228,9 @@ void symbol__elf_init(void);
>  int symbol__annotation_init(void);
>  
>  struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name);
> +size_t symbol__bytes_used(void);
> +void symbol__account_bytes(size_t bytes);
> +void symbol__unaccount_bytes(size_t bytes);
>  size_t __symbol__fprintf_symname_offs(const struct symbol *sym,
>  				      const struct addr_location *al,
>  				      bool unknown_as_addr,
> diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
> index 71f60081a85b..6a16c5badd5e 100644
> --- a/tools/perf/util/symbol_conf.h
> +++ b/tools/perf/util/symbol_conf.h
> @@ -120,6 +120,7 @@ struct symbol_conf {
>  	int		pad_output_len_dso;
>  	int		group_sort_idx;
>  	int		addr_range;
> +	unsigned long	max_symbol_bytes;
>  	DECLARE_BITMAP(parallelism_filter, MAX_NR_CPUS + 1);
>  };
>  
> 
> -- 
> Git-155)
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading
  2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
  (?)
  (?)
@ 2026-09-17  7:24   ` Namhyung Kim
  2026-09-19 23:22     ` Alireza Haghdoost
  -1 siblings, 1 reply; 23+ messages in thread
From: Namhyung Kim @ 2026-09-17  7:24 UTC (permalink / raw)
  To: haghdoost
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Ian Rogers,
	Adrian Hunter, James Clark, Andrii Nakryiko, Alexei Starovoitov,
	linux-perf-users, linux-kernel

On Tue, Sep 15, 2026 at 11:42:45AM -0700, Alireza Haghdoost via B4 Relay wrote:
> From: Alireza Haghdoost <haghdoost@uber.com>
> 
> perf script eagerly materializes eligible symbols from every DSO
> encountered in samples. On a production fixture, it loaded about 765k
> symbols to resolve about 45k distinct (DSO, symbol) frames, exceeding the
> memory available in a memory-constrained cgroup.
> 
> This patch adds --lazy-load-symbols for userspace ELF DSOs. It builds a
> compact sorted index, resolves sampled addresses by binary search, reads
> symbol names with pread(), and caches resolved symbols in the existing
> rb-tree. Lazy lookup retains one CLOEXEC file descriptor per indexed DSO.
> If the descriptor cannot be retained, perf discards the index and eagerly
> loads that DSO instead.
> 
> On the same fixture, peak RssAnon drops from 265 MiB to 39 MiB and wall
> time from 3.1 seconds to 1.85 seconds. Memory optimizations usually cost
> time; this one does not because lazy loading skips many unnecessary
> calloc() calls and demangling operations. Output was byte-identical on
> the tested x86-64 workloads and an aarch64 capture.

Interesting!

> 
> Lazy loading is most effective when samples reference only a small
> fraction of the available symbols, such as profiles spanning many large
> DSOs. It still builds an index proportional to the total symbol count.
> Eager loading remains available for dense symbol coverage or cases
> requiring its broader ELF and architecture support.
> 
> This does not claim full parity with the eager loader. Lazy loading
> supports the common userspace ELF symtab/dynsym case, with these known
> differences:
> 
>   - .gnu_debugdata merging and PPC64 .opd are not handled.
>   - SHT_NOBITS re-reading, IFUNC PLT naming, and exact-tie alias ordering
>     are unreachable or output-equivalent on x86-64.

I'm curious how it could handle map__find_symbol_by_name().

> 
> Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
> Assisted-by: Kimi:K3
> ---
>  tools/perf/builtin-script.c      |   4 +-
>  tools/perf/util/dso.c            |  10 +
>  tools/perf/util/dso.h            |  29 ++
>  tools/perf/util/map.c            |   9 +-
>  tools/perf/util/symbol-elf.c     | 560 ++++++++++++++++++++++++++++++++++++++-
>  tools/perf/util/symbol-minimal.c |  11 +
>  tools/perf/util/symbol.c         |  58 ++--
>  tools/perf/util/symbol.h         |   4 +
>  tools/perf/util/symbol_conf.h    |   1 +
>  9 files changed, 664 insertions(+), 22 deletions(-)
> 
> diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
> index 50fa6ca6455a..81e56378d3d2 100644
> --- a/tools/perf/builtin-script.c
> +++ b/tools/perf/builtin-script.c
> @@ -4037,7 +4037,7 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
>  }
>  
>  static int parse_max_symbol_bytes(const struct option *opt,
> -				  const char *str, int unset)
> +				const char *str, int unset)
>  {
>  	unsigned long *max_bytes = (unsigned long *)opt->value;
>  	static struct parse_tag size_tags[] = {
> @@ -4166,6 +4166,8 @@ int cmd_script(int argc, const char **argv)
>  	OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
>  		     "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
>  		     parse_max_symbol_bytes),
> +	OPT_BOOLEAN(0, "lazy-load-symbols", &symbol_conf.lazy_load_symbols,
> +		    "Resolve symbols lazily instead of loading full symtabs"),

Please add documentation as well.


>  	OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
>  	OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
>  	OPT_BOOLEAN('I', "show-info", &show_full_info,
[SNIP]
> diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
> index 914e42d21f70..e4d77e46e883 100644
> --- a/tools/perf/util/symbol-elf.c
> +++ b/tools/perf/util/symbol-elf.c
> @@ -2,6 +2,7 @@
>  #include <fcntl.h>
>  #include <stdio.h>
>  #include <errno.h>
> +#include <stdint.h>
>  #include <stdlib.h>
>  #include <string.h>
>  #include <unistd.h>
> @@ -600,6 +601,32 @@ static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
>   * And always look at the original dso, not at debuginfo packages, that
>   * have the PLT data stripped out (shdr_rel_plt.sh_type == SHT_NOBITS).
>   */

It seems the above comment belongs to the original function.


> +static void dso__clip_ondemand_symbols_at(struct dso *dso, u64 addr)
> +{
> +	struct dso_ondemand *od = dso__ondemand(dso);
> +	struct sym_idx *sym;
> +	u32 lo = 0, hi, mid;
> +
> +	if (!od)
> +		return;
> +
> +	hi = od->nr_sorted;
> +	while (lo < hi) {
> +		mid = (lo + hi) / 2;
> +		if (od->sorted[mid].start < addr)
> +			lo = mid + 1;
> +		else
> +			hi = mid;
> +	}

Why not use bsearch()?

> +
> +	if (!lo)
> +		return;
> +
> +	sym = &od->sorted[lo - 1];
> +	if (sym->end > addr)
> +		sym->end = addr;
> +}
> +
>  int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
>  {
>  	uint32_t idx;
> @@ -623,6 +650,8 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
>  	if (!elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL))
>  		return 0;
>  
> +	dso__clip_ondemand_symbols_at(dso, shdr_plt.sh_offset);

Why is this needed?

> +
>  	/*
>  	 * A symbol from a previous section (e.g. .init) can have been expanded
>  	 * by symbols__fixup_end() to overlap .plt. Truncate it before adding
[SNIP]
> +static int dso__build_ondemand_index(struct dso *dso, struct symsrc *syms_ss,
> +				     struct symsrc *runtime_ss,
> +				     int dynsym)
> +{
> +	struct dso_ondemand *od;
> +	Elf *elf = syms_ss->elf;
> +	GElf_Ehdr ehdr = syms_ss->ehdr;
> +	GElf_Shdr shdr;
> +	GElf_Shdr strshdr;
> +	Elf_Scn *strscn, *sec_strndx;
> +	Elf_Data *syms;
> +	GElf_Sym sym;
> +	Elf_Data *secstrs = NULL;
> +	size_t i;
> +	u32 count = 0, j;
> +	u64 nr_entries, strtab_offset;
> +
> +	if (dynsym)
> +		shdr = syms_ss->dynshdr;
> +	else
> +		shdr = syms_ss->symshdr;
> +
> +	syms = elf_getdata(dynsym ? syms_ss->dynsym : syms_ss->symtab, NULL);
> +	if (!syms)
> +		return -1;
> +
> +	if (!shdr.sh_entsize)
> +		return 0;
> +
> +	nr_entries = shdr.sh_size / shdr.sh_entsize;
> +	if (nr_entries > UINT32_MAX)
> +		return -EOVERFLOW;
> +
> +	/*
> +	 * File offset of the string table linked from the symbol table.
> +	 * Symbol names are pread() from the file at lookup time, so we
> +	 * only need the offset and size here, not the strings themselves.
> +	 */
> +	strscn = elf_getscn(elf, shdr.sh_link);
> +	if (!strscn || !gelf_getshdr(strscn, &strshdr))
> +		return -1;
> +	strtab_offset = strshdr.sh_offset;
> +
> +	/*
> +	 * Section name string table, used to match the eager path's
> +	 * elf_sec__filter() (text/data section check for STT_NOTYPE labels).
> +	 */
> +	sec_strndx = elf_getscn(elf, ehdr.e_shstrndx);
> +	if (sec_strndx)
> +		secstrs = elf_getdata(sec_strndx, NULL);
> +
> +	/* Count symbols that pass the filter (same filter as fill below) */
> +	for (i = 0; i < nr_entries; i++) {
> +		if (!gelf_getsym(syms, i, &sym))
> +			continue;
> +		if (ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
> +				    ehdr.e_machine))
> +			count++;
> +	}
> +
> +	if (!count)
> +		return 0;
> +	if (count > SIZE_MAX / sizeof(*od->sorted))
> +		return -EOVERFLOW;
> +
> +	/*
> +	 * Account the index against the symbol memory budget: at 24
> +	 * bytes/symbol it is the dominant on-demand cost and must count
> +	 * toward --max-symbol-bytes just like struct symbol allocations do.
> +	 */
> +	if (symbol_conf.max_symbol_bytes &&
> +	    symbol__bytes_used() + count * sizeof(struct sym_idx) >
> +	    symbol_conf.max_symbol_bytes) {
> +		symbol_budget_warning();
> +		return 0; /* fall back to the eager loader's per-symbol budget */
> +	}
> +
> +	od = zalloc(sizeof(*od));
> +	if (!od)
> +		return -1;
> +
> +	od->sorted = malloc(count * sizeof(*od->sorted));
> +	if (!od->sorted) {
> +		free(od);
> +		return -1;
> +	}
> +	od->nr_alloc = count;	/* allocated; the deduped count may shrink */
> +
> +	/* Fill the index with adjusted addresses */
> +	j = 0;
> +	for (i = 0; i < nr_entries; i++) {
> +		u64 adjusted;
> +		GElf_Phdr phdr;
> +
> +		if (!gelf_getsym(syms, i, &sym))
> +			continue;
> +		if (!ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
> +				     ehdr.e_machine))
> +			continue;
> +
> +		adjusted = sym.st_value;
> +
> +		/* ARM Thumb bit fix (same as eager path, FUNC only) */
> +		if ((ehdr.e_machine == EM_ARM) &&
> +		    (GELF_ST_TYPE(sym.st_info) == STT_FUNC) &&
> +		    (adjusted & 1))
> +			--adjusted;
> +
> +		/*
> +		 * Program header adjustment, identical to the eager loop:
> +		 * read the PT_LOAD containing the symbol from the runtime
> +		 * ELF (the debug-info file may have zeroed p_offset), and
> +		 * fall back to the section-header bias when no program
> +		 * header matches -- exactly what the eager path does when
> +		 * elf_read_program_header fails.
> +		 */
> +		if (elf_read_program_header(runtime_ss->elf, adjusted,
> +					    &phdr) == 0) {
> +			adjusted -= phdr.p_vaddr - phdr.p_offset;
> +		} else {
> +			Elf_Scn *sym_sec = elf_getscn(elf, sym.st_shndx);
> +			GElf_Shdr sym_shdr;
> +
> +			if (sym_sec && gelf_getshdr(sym_sec, &sym_shdr))
> +				adjusted -= sym_shdr.sh_addr - sym_shdr.sh_offset;
> +		}
> +
> +		od->sorted[j].start = adjusted;
> +		od->sorted[j].end = sym.st_size; /* st_size for now, converted later */
> +		od->sorted[j].name_off = sym.st_name; /* strtab-relative */
> +		od->sorted[j].binding = GELF_ST_BIND(sym.st_info);
> +		od->sorted[j].type = GELF_ST_TYPE(sym.st_info);
> +		j++;
> +	}
> +
> +	/* Sort by adjusted start address */
> +	qsort(od->sorted, count, sizeof(*od->sorted), cmp_sym_idx);
> +
> +	/* Alias dedup: keep only the best symbol for each start address */
> +	if (!symbol_conf.allow_aliases) {

Probably better to factor out the dedup logic into a function.


> +		u32 out = 0;
> +
> +		for (i = 0; i < count; i++) {
> +			u32 best = i;
> +			const char *na = NULL, *nb;
> +			char *da = NULL, *db;
> +
> +			/* name_off is the strtab index (st_name) */
> +			na = elf_strptr(elf, shdr.sh_link,
> +					od->sorted[best].name_off);
> +			if (na) {
> +				da = dso__demangle_sym(dso, 0, na);
> +				if (da)
> +					na = da;
> +			}
> +
> +			/* Find the best among all entries with this start */
> +			for (j = i + 1; j < count &&
> +			     od->sorted[j].start == od->sorted[i].start; j++) {
> +				nb = elf_strptr(elf, shdr.sh_link,
> +						od->sorted[j].name_off);
> +				if (!na || !nb)
> +					continue;
> +
> +				/* Demangle for comparison, matching eager path */
> +				db = dso__demangle_sym(dso, 0, nb);
> +				if (db)
> +					nb = db;
> +
> +				/* od->sorted[].end holds st_size at this point */
> +				if (choose_best_symbol_raw(
> +					    od->sorted[best].end,
> +					    od->sorted[best].type,
> +					    od->sorted[best].binding, na,
> +					    od->sorted[j].end,
> +					    od->sorted[j].type,
> +					    od->sorted[j].binding, nb) == SYMBOL_B) {
> +					best = j;
> +					free(da);
> +					da = db;
> +					na = nb;
> +				} else {
> +					free(db);
> +				}
> +			}
> +
> +			free(da);
> +			od->sorted[out++] = od->sorted[best];
> +			i = j - 1; /* skip past all aliases of this start */
> +		}
> +
> +		if (out < count) {
> +			struct sym_idx *shrunk;
> +
> +			shrunk = realloc(od->sorted, out * sizeof(*od->sorted));
> +			if (shrunk) {
> +				od->sorted = shrunk;
> +				od->nr_alloc = out;
> +			}
> +		}
> +		count = out;
> +	}
> +
> +	/* Convert st_size to end addresses */
> +	for (i = 0; i < count; i++) {
> +		u64 size = od->sorted[i].end; /* was st_size */
> +
> +		if (size > 0)
> +			od->sorted[i].end = od->sorted[i].start + size;
> +		else if (i + 1 < count)
> +			od->sorted[i].end = od->sorted[i + 1].start;
> +		else
> +			/* Match symbols__fixup_end's last-symbol formula. */
> +			od->sorted[i].end = roundup(od->sorted[i].start, 4096) + 4096;
> +	}
> +
> +	/*
> +	 * Keep a private fd open for pread() of symbol names.  Dup with
> +	 * O_CLOEXEC so children don't inherit it, and so that
> +	 * symsrc__destroy() can close the original regardless of whether
> +	 * it is a real file or a temporary debugdata extraction.
> +	 *
> +	 * If the dup fails (e.g. fd exhaustion), decline by returning 0
> +	 * without setting the index: the caller falls back to the eager
> +	 * loader so the DSO still gets symbols rather than going symbol-less.
> +	 */
> +	od->fd = fcntl(syms_ss->fd, F_DUPFD_CLOEXEC, 0);
> +	if (od->fd < 0) {
> +		free(od->sorted);
> +		free(od);
> +		return 0;
> +	}
> +	od->strtab_offset = strtab_offset;
> +	od->strtab_size = strshdr.sh_size;
> +	od->nr_sorted = count;
> +
> +	symbol__account_bytes(od->nr_alloc * sizeof(*od->sorted));
> +
> +	dso__set_ondemand(dso, od);
> +
> +	pr_debug("%s: on-demand index: %u symbols\n",
> +		 dso__long_name(dso), count);
> +
> +	return 1;
> +}
> +
> +/*
> + * Read a NUL-terminated symbol name from the file's string table at
> + * file offset @off.  The fast path uses a stack buffer; if the NUL is
> + * not found within it (names can exceed 1 KiB for template-heavy C++
> + * mangled names), grow a heap buffer geometrically from 4 KiB, doubling
> + * until the terminator is found or the strtab is exhausted.  This keeps
> + * a single long name cheap even when it sits near the start of a large
> + * (tens of MB) strtab, while bounding a corrupt/missing terminator by
> + * the remaining strtab size.
> + *
> + * Returns a pointer to the name (either @buf or a heap allocation) and
> + * sets *@to_free to the buffer that must be free()d (NULL for @buf).
> + * Returns NULL on read error or if no NUL terminator exists within the
> + * strtab bounds.
> + */

We have dso-cache APIs to read file data (dso__data_read_offset) and it
manages file descriptors so you don't need to worry about FD exhaustion.


> +static const char *ondemand_read_name(int fd, u64 strtab_offset,
> +				      u64 strtab_size, u64 name_off,
> +				      char *buf, size_t buflen,
> +				      char **to_free)
> +{
> +	ssize_t n;
> +	u64 remain;
> +	u64 file_off;
> +	size_t cap;
> +
> +	*to_free = NULL;
> +
> +	/* name_off is strtab-relative (the symbol's st_name). */
> +	if (name_off >= strtab_size)
> +		return NULL;
> +	file_off = strtab_offset + name_off;
> +	remain = strtab_size - name_off;
> +
> +	/*
> +	 * Fast path: stack buffer, expect the name to fit.  Cap at the
> +	 * remaining strtab so a missing terminator can't read past the
> +	 * section into adjacent file data.
> +	 */
> +	n = pread(fd, buf, min((u64)(buflen - 1), remain), file_off);
> +	if (n <= 0)
> +		return NULL;
> +	buf[n] = '\0';
> +	if (memchr(buf, '\0', n))
> +		return buf;
> +
> +	/*
> +	 * Slow path: the name is longer than buflen.  Grow a heap buffer
> +	 * geometrically, doubling until the terminator appears, so a long
> +	 * name costs O(name length), not O(remaining strtab size).
> +	 */
> +	cap = 4096;
> +	for (;;) {
> +		char *tmp;
> +		size_t want = cap;
> +
> +		if (want > remain)
> +			want = remain;
> +		if (want == 0)
> +			break;
> +
> +		tmp = *to_free ? realloc(*to_free, want + 1) : malloc(want + 1);
> +		if (!tmp) {
> +			free(*to_free);
> +			*to_free = NULL;
> +			return NULL;
> +		}
> +		*to_free = tmp;
> +
> +		n = pread(fd, *to_free, want, file_off);
> +		if (n <= 0) {
> +			free(*to_free);
> +			*to_free = NULL;
> +			return NULL;
> +		}
> +		(*to_free)[n] = '\0';
> +
> +		if (memchr(*to_free, '\0', n))
> +			return *to_free;
> +
> +		/*
> +		 * Read the whole remaining strtab (or hit EOF) with no
> +		 * terminator: corrupt file, bail instead of re-reading.
> +		 */
> +		if (want >= remain || (u64)n >= remain)
> +			break;
> +
> +		/* Avoid size_t overflow on absurdly large strtabs. */
> +		if (cap > SIZE_MAX / 2)
> +			break;
> +		cap *= 2;
> +	}
> +
> +	free(*to_free);
> +	*to_free = NULL;
> +	return NULL;
> +}
> +
> +struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr)
> +{
> +	struct dso_ondemand *od = dso__ondemand(dso);
> +	u32 lo, hi, mid;
> +	const char *name;
> +	char namebuf[1024];
> +	char *name_heap = NULL;
> +	char *demangled;
> +	struct symbol *s = NULL;
> +
> +	if (!od || !od->sorted || od->fd < 0)
> +		return NULL;
> +
> +	lo = 0;
> +	hi = od->nr_sorted;
> +	while (lo < hi) {
> +		mid = (lo + hi) / 2;
> +		if (addr < od->sorted[mid].start)
> +			hi = mid;
> +		else if (addr >= od->sorted[mid].end)
> +			lo = mid + 1;
> +		else
> +			goto found;
> +	}

bsearch()?

Thanks,
Namhyung

> +
> +	/* Not found */
> +	return NULL;
> +
> +found:
> +	/*
> +	 * Check the budget before doing any name I/O or demangling, so an
> +	 * over-budget DSO stops paying pread+demangle on every later miss.
> +	 */
> +	if (symbol_conf.max_symbol_bytes &&
> +	    symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
> +		symbol_budget_warning();
> +		return NULL;
> +	}
> +
> +	name = ondemand_read_name(od->fd, od->strtab_offset, od->strtab_size,
> +				  od->sorted[mid].name_off,
> +				  namebuf, sizeof(namebuf), &name_heap);
> +	if (!name)
> +		return NULL;
> +
> +	demangled = dso__demangle_sym(dso, 0, name);
> +	if (demangled)
> +		name = demangled;
> +
> +	s = symbol__new(od->sorted[mid].start,
> +			od->sorted[mid].end - od->sorted[mid].start,
> +			od->sorted[mid].binding,
> +			od->sorted[mid].type, name);
> +	free(demangled);
> +	free(name_heap);
> +	if (s)
> +		__symbols__insert(dso__symbols(dso), s);
> +	return s;
> +}

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory
  2026-09-17  6:53   ` Namhyung Kim
@ 2026-09-19 18:37     ` Alireza Haghdoost
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-19 18:37 UTC (permalink / raw)
  To: Namhyung Kim
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Ian Rogers,
	Adrian Hunter, James Clark, Andrii Nakryiko, Alexei Starovoitov,
	linux-perf-users, linux-kernel

> > The counter includes every symbol__new() allocation, but this patch
> > enforces the limit only in the ELF loader, which is the source of the
> > unbounded memory growth addressed here. In this path, reaching the limit
> > can safely produce [unknown] symbols. Other loaders currently treat a
> > failed symbol allocation as an error. Capping those paths would therefore
> > require separate changes whose complexity may outweigh the potential
> > memory savings.
>
> What's the other loaders you meant?
>
> I don't see it produces [unknown] symbols.  Do you mean it just stops
> loading symbols when it hits the limit?
>

The other loaders are kallsyms, JIT/perf-map, and libbfd. Their
allocations are counted by symbol__new(), but the current patch
enforces the limit only in the libelf loader. I will name those paths
explicitly in v2 of this patch series.

The loader stops adding ELF symbols once the budget is reached and
warns once. Later unresolved IPs are printed as [unknown] by
symbol_fprintf.c:40 when sym is NULL.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading
  2026-09-17  7:24   ` Namhyung Kim
@ 2026-09-19 23:22     ` Alireza Haghdoost
  0 siblings, 0 replies; 23+ messages in thread
From: Alireza Haghdoost @ 2026-09-19 23:22 UTC (permalink / raw)
  To: Namhyung Kim
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Ian Rogers,
	Adrian Hunter, James Clark, Andrii Nakryiko, Alexei Starovoitov,
	linux-perf-users, linux-kernel

> > +static void dso__clip_ondemand_symbols_at(struct dso *dso, u64 addr)
> > +{
> > +     struct dso_ondemand *od = dso__ondemand(dso);
> > +     struct sym_idx *sym;
> > +     u32 lo = 0, hi, mid;
> > +
> > +     if (!od)
> > +             return;
> > +
> > +     hi = od->nr_sorted;
> > +     while (lo < hi) {
> > +             mid = (lo + hi) / 2;
> > +             if (od->sorted[mid].start < addr)
> > +                     lo = mid + 1;
> > +             else
> > +                     hi = mid;
> > +     }
>
> Why not use bsearch()?
>

This search is a lower-bound (bisect_left) operation. I could not find
an ISO C or glibc API that provides that operation.
In patch series v2, I will refactor this search into a small
sym_idx__lower_bound() helper. The clipping helper (symbol-elf.c:626)
and the exact-start IRELATIVE lookup (symbol-elf.c:2176) will use it,
while ordinary non-overlapping interval lookups will use bsearch().


> >  int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
> >  {
> >       uint32_t idx;
> > @@ -623,6 +650,8 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
> >       if (!elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL))
> >               return 0;
> >
> > +     dso__clip_ondemand_symbols_at(dso, shdr_plt.sh_offset);
>
> Why is this needed?
>

Lazy address lookup uses the index, not the rb-tree. Without clipping
the index at .plt, a preceding fill-forward range can claim PLT IPs
before dso__synthesize_plt_symbols() inserts the PLT symbols. This is
the same truncation the eager path already does on the rb-tree
(symbol-elf.c:680). I will add that explanation at the call site in
v2.

> > +
> > +     /* Alias dedup: keep only the best symbol for each start address */
> > +     if (!symbol_conf.allow_aliases) {
>
> Probably better to factor out the dedup logic into a function.
>

I will factor out the duplicate-selection policy into
symbol__choose_best() and use it in both the eager and lazy paths. The
surrounding dedup loops remain separate because the eager path uses
rb-tree struct symbol objects while the lazy path uses compact index
entries before materialization.

> > +             u32 out = 0;
> > +
> > +             for (i = 0; i < count; i++) {
> > +                     u32 best = i;
> > +                     const char *na = NULL, *nb;
> > +                     char *da = NULL, *db;
> > +
> > +                     /* name_off is the strtab index (st_name) */
> > +                     na = elf_strptr(elf, shdr.sh_link,
> > +                                     od->sorted[best].name_off);
> > +                     if (na) {
> > +                             da = dso__demangle_sym(dso, 0, na);
> > +                             if (da)
> > +                                     na = da;
> > +                     }
> > +
> > +                     /* Find the best among all entries with this start */
> > +                     for (j = i + 1; j < count &&
> > +                          od->sorted[j].start == od->sorted[i].start; j++) {
> > +                             nb = elf_strptr(elf, shdr.sh_link,
> > +                                             od->sorted[j].name_off);
> > +                             if (!na || !nb)
> > +                                     continue;
> > +
> > +                             /* Demangle for comparison, matching eager path */
> > +                             db = dso__demangle_sym(dso, 0, nb);
> > +                             if (db)
> > +                                     nb = db;
> > +
> > +                             /* od->sorted[].end holds st_size at this point */
> > +                             if (choose_best_symbol_raw(
> > +                                         od->sorted[best].end,
> > +                                         od->sorted[best].type,
> > +                                         od->sorted[best].binding, na,
> > +                                         od->sorted[j].end,
> > +                                         od->sorted[j].type,
> > +                                         od->sorted[j].binding, nb) == SYMBOL_B) {
> > +                                     best = j;
> > +                                     free(da);
> > +                                     da = db;
> > +                                     na = nb;
> > +                             } else {
> > +                                     free(db);
> > +                             }
> > +                     }
> > +
> > +                     free(da);
> > +                     od->sorted[out++] = od->sorted[best];
> > +                     i = j - 1; /* skip past all aliases of this start */
> > +             }
> > +
> > +             if (out < count) {
> > +                     struct sym_idx *shrunk;
> > +
> > +                     shrunk = realloc(od->sorted, out * sizeof(*od->sorted));
> > +                     if (shrunk) {
> > +                             od->sorted = shrunk;
> > +                             od->nr_alloc = out;
> > +                     }
> > +             }
> > +             count = out;
> > +     }
> > +
> > +     /* Convert st_size to end addresses */
> > +     for (i = 0; i < count; i++) {
> > +             u64 size = od->sorted[i].end; /* was st_size */
> > +
> > +             if (size > 0)
> > +                     od->sorted[i].end = od->sorted[i].start + size;
> > +             else if (i + 1 < count)
> > +                     od->sorted[i].end = od->sorted[i + 1].start;
> > +             else
> > +                     /* Match symbols__fixup_end's last-symbol formula. */
> > +                     od->sorted[i].end = roundup(od->sorted[i].start, 4096) + 4096;
> > +     }
> > +
> > +     /*
> > +      * Keep a private fd open for pread() of symbol names.  Dup with
> > +      * O_CLOEXEC so children don't inherit it, and so that
> > +      * symsrc__destroy() can close the original regardless of whether
> > +      * it is a real file or a temporary debugdata extraction.
> > +      *
> > +      * If the dup fails (e.g. fd exhaustion), decline by returning 0
> > +      * without setting the index: the caller falls back to the eager
> > +      * loader so the DSO still gets symbols rather than going symbol-less.
> > +      */
> > +     od->fd = fcntl(syms_ss->fd, F_DUPFD_CLOEXEC, 0);
> > +     if (od->fd < 0) {
> > +             free(od->sorted);
> > +             free(od);
> > +             return 0;
> > +     }
> > +     od->strtab_offset = strtab_offset;
> > +     od->strtab_size = strshdr.sh_size;
> > +     od->nr_sorted = count;
> > +
> > +     symbol__account_bytes(od->nr_alloc * sizeof(*od->sorted));
> > +
> > +     dso__set_ondemand(dso, od);
> > +
> > +     pr_debug("%s: on-demand index: %u symbols\n",
> > +              dso__long_name(dso), count);
> > +
> > +     return 1;
> > +}
> > +
> > +/*
> > + * Read a NUL-terminated symbol name from the file's string table at
> > + * file offset @off.  The fast path uses a stack buffer; if the NUL is
> > + * not found within it (names can exceed 1 KiB for template-heavy C++
> > + * mangled names), grow a heap buffer geometrically from 4 KiB, doubling
> > + * until the terminator is found or the strtab is exhausted.  This keeps
> > + * a single long name cheap even when it sits near the start of a large
> > + * (tens of MB) strtab, while bounding a corrupt/missing terminator by
> > + * the remaining strtab size.
> > + *
> > + * Returns a pointer to the name (either @buf or a heap allocation) and
> > + * sets *@to_free to the buffer that must be free()d (NULL for @buf).
> > + * Returns NULL on read error or if no NUL terminator exists within the
> > + * strtab bounds.
> > + */
>
> We have dso-cache APIs to read file data (dso__data_read_offset) and it
> manages file descriptors so you don't need to worry about FD exhaustion.
>

Thanks, I will use dso__data_read_offset() in v2 and remove the
per-index retained fd. I will configure the data-cache reads with the
same symbol-file path and namespace used to build the index, so
split-debuginfo string-table offsets are not read from the runtime
image.


> > +struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr)
> > +{
> > +     struct dso_ondemand *od = dso__ondemand(dso);
> > +     u32 lo, hi, mid;
> > +     const char *name;
> > +     char namebuf[1024];
> > +     char *name_heap = NULL;
> > +     char *demangled;
> > +     struct symbol *s = NULL;
> > +
> > +     if (!od || !od->sorted || od->fd < 0)
> > +             return NULL;
> > +
> > +     lo = 0;
> > +     hi = od->nr_sorted;
> > +     while (lo < hi) {
> > +             mid = (lo + hi) / 2;
> > +             if (addr < od->sorted[mid].start)
> > +                     hi = mid;
> > +             else if (addr >= od->sorted[mid].end)
> > +                     lo = mid + 1;
> > +             else
> > +                     goto found;
> > +     }
>
> bsearch()?
>

Yes. In v2, the default deduplicated index has non-overlapping ranges,
so this lookup will use bsearch() with an interval comparator.
With --allow-aliases, two symbols can cover the same address.
bsearch() then cannot reliably choose which alias to return, so that
uncommon case keeps the explicit search.

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2026-09-19 23:22 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-15 18:42 [PATCH 0/4] perf script: Bounded and lazy symbol loading Alireza Haghdoost
2026-09-15 18:42 ` Alireza Haghdoost via B4 Relay
2026-09-15 18:42 ` [PATCH 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard Alireza Haghdoost
2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
2026-09-15 18:53   ` sashiko-bot
2026-09-15 19:04     ` Alireza Haghdoost
2026-09-17  0:30       ` Namhyung Kim
2026-09-15 18:42 ` [PATCH 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory Alireza Haghdoost
2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
2026-09-15 19:01   ` sashiko-bot
2026-09-16 16:15     ` Alireza Haghdoost
2026-09-17  6:53   ` Namhyung Kim
2026-09-19 18:37     ` Alireza Haghdoost
2026-09-15 18:42 ` [PATCH 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading Alireza Haghdoost
2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
2026-09-15 18:54   ` sashiko-bot
2026-09-16 16:18     ` Alireza Haghdoost
2026-09-17  7:24   ` Namhyung Kim
2026-09-19 23:22     ` Alireza Haghdoost
2026-09-15 18:42 ` [PATCH 4/4] perf script: Document and test --lazy-load-symbols and --max-symbol-bytes Alireza Haghdoost
2026-09-15 18:42   ` Alireza Haghdoost via B4 Relay
2026-09-15 18:50   ` sashiko-bot
2026-09-16 16:21     ` Alireza Haghdoost

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.