[PATCH v5 0/8] perf: support specify vdso path in cmdline

[PATCH v5 0/8] perf: support specify vdso path in cmdline

[Changbin Du]

The vdso dumped from process memory (in buildid-cache) lacks debugging
info. To annotate vdso symbols with source lines we need a debugging
version.

For x86, we can find them from your local build as
'arch/x86/entry/vdso/vdso{32,64}.so.dbg'. Or they may resides in
'/lib/modules/<version>/vdso/vdso{32,64}.so' on Ubuntu. But notice that the
builid has to match. 

If user doesn't specify the path, perf will search them internally as long
as vmlinux when recording samples. The searched debugging vdso will add to
buildid cache.

Below samples are captured on my local build kernel. perf succesfully
find debugging version vdso and we can annotate with source without
specifying vdso path.

$ sudo perf record -a
$ sudo perf report --objdump=llvm-objdump

Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
__vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
Percent│       movq    -48(%rbp),%rsi
       │       testq   %rax,%rax
       │     ;               return vread_hvclock();
       │       movq    %rax,%rdx
       │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
       │     ↑ js      eb
       │     ↑ jmp     74
       │     ;               ts->tv_sec = vdso_ts->sec;
  0.02 │147:   leaq    2(%rbx),%rax
       │       shlq    $4, %rax
       │       addq    %r10,%rax
       │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
  9.38 │152:   movl    (%r10),%ecx

When doing cross platform analysis, we need to specify the vdso path if
we are interested in its symbols. At most two vdso can be given. Also you
can pack your buildid cache with perf-archive if the debugging vdso can be
found on the sampled machine.

$ sudo perf report --objdump=llvm-objdump \
      --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg

I also improved perf-buildid-cache command recognize vdso when adding files, then
place it at correct place.

v5:
  - Searching the vdso in record stage instead of report. So the debugging
    vdso will be in build-id cache. This is friendly for cross-machine analysis.
  - Improve perf-buildid-cache command recognize vdso when adding files
v4:
  - split the refactoring from the actual change.
v3:
  - update documentation.
v2:
  - now search vdso automatically as long as vmlinux, as suggested by Adrian.
  - remove change 'prefer symsrc_filename for filename'.


Changbin Du (8):
  perf: support specify vdso path in cmdline
  perf: disasm: refactor function dso__disassemble_filename
  perf: disasm: use build_id_path if fallback failed
  perf: build-id: name debugging vdso as "debug"
  perf: symbol: generalize vmlinux path searching
  perf: build-id: try to search debugging vdso and add to cache
  perf: disasm: prefer debugging files in build-id cache
  perf buildid-cache: recognize vdso when adding files

 tools/perf/Documentation/perf-annotate.txt |   3 +
 tools/perf/Documentation/perf-c2c.txt      |   3 +
 tools/perf/Documentation/perf-inject.txt   |   3 +
 tools/perf/Documentation/perf-report.txt   |   3 +
 tools/perf/Documentation/perf-script.txt   |   3 +
 tools/perf/Documentation/perf-top.txt      |   3 +
 tools/perf/builtin-annotate.c              |   2 +
 tools/perf/builtin-buildid-cache.c         |  26 ++-
 tools/perf/builtin-c2c.c                   |   2 +
 tools/perf/builtin-inject.c                |   2 +
 tools/perf/builtin-report.c                |   2 +
 tools/perf/builtin-script.c                |   2 +
 tools/perf/builtin-top.c                   |   2 +
 tools/perf/util/build-id.c                 |  57 +++++-
 tools/perf/util/disasm.c                   | 131 ++++++++-----
 tools/perf/util/machine.c                  |   4 +-
 tools/perf/util/symbol.c                   | 209 ++++++++++++++++-----
 tools/perf/util/symbol.h                   |   9 +-
 tools/perf/util/symbol_conf.h              |   5 +
 19 files changed, 359 insertions(+), 112 deletions(-)

-- 
2.34.1

[PATCH v5 1/8] perf: support specify vdso path in cmdline

[Changbin Du]

The vdso dumped from process memory (in buildid-cache) lacks debugging
info. To annotate vdso symbols with source lines we need specify a
debugging version.

For x86, we can find them from your local build as
arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
/lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
the buildid has to match.

$ sudo perf record -a
$ sudo perf report --objdump=llvm-objdump \
  --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg

Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
__vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
Percent│       movq    -48(%rbp),%rsi
       │       testq   %rax,%rax
       │     ;               return vread_hvclock();
       │       movq    %rax,%rdx
       │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
       │     ↑ js      eb
       │     ↑ jmp     74
       │     ;               ts->tv_sec = vdso_ts->sec;
  0.02 │147:   leaq    2(%rbx),%rax
       │       shlq    $4, %rax
       │       addq    %r10,%rax
       │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
  9.38 │152:   movl    (%r10),%ecx

When doing cross platform analysis, we also need specify the vdso path if
we are interested in its symbols.

v2: update documentation.

Signed-off-by: Changbin Du <changbin.du@huawei.com>
---
 tools/perf/Documentation/perf-annotate.txt |  3 +
 tools/perf/Documentation/perf-c2c.txt      |  3 +
 tools/perf/Documentation/perf-inject.txt   |  3 +
 tools/perf/Documentation/perf-report.txt   |  3 +
 tools/perf/Documentation/perf-script.txt   |  3 +
 tools/perf/Documentation/perf-top.txt      |  3 +
 tools/perf/builtin-annotate.c              |  2 +
 tools/perf/builtin-c2c.c                   |  2 +
 tools/perf/builtin-inject.c                |  2 +
 tools/perf/builtin-report.c                |  2 +
 tools/perf/builtin-script.c                |  2 +
 tools/perf/builtin-top.c                   |  2 +
 tools/perf/util/disasm.c                   |  7 +-
 tools/perf/util/symbol.c                   | 82 +++++++++++++++++++++-
 tools/perf/util/symbol_conf.h              |  5 ++
 15 files changed, 119 insertions(+), 5 deletions(-)

diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt
index b95524bea021..4b6692f9a793 100644
--- a/tools/perf/Documentation/perf-annotate.txt
+++ b/tools/perf/Documentation/perf-annotate.txt
@@ -58,6 +58,9 @@ OPTIONS
 --ignore-vmlinux::
 	Ignore vmlinux files.
 
+--vdso=<vdso1[,vdso2]>::
+	Specify vdso pathnames. You can specify up to two for multiarch-support.
+
 --itrace::
 	Options for decoding instruction tracing data. The options are:
 
diff --git a/tools/perf/Documentation/perf-c2c.txt b/tools/perf/Documentation/perf-c2c.txt
index 856f0dfb8e5a..7c07efca7542 100644
--- a/tools/perf/Documentation/perf-c2c.txt
+++ b/tools/perf/Documentation/perf-c2c.txt
@@ -71,6 +71,9 @@ REPORT OPTIONS
 --vmlinux=<file>::
 	vmlinux pathname
 
+--vdso=<vdso1[,vdso2]>::
+	Specify vdso pathnames. You can specify up to two for multiarch-support.
+
 -v::
 --verbose::
 	Be more verbose (show counter open errors, etc).
diff --git a/tools/perf/Documentation/perf-inject.txt b/tools/perf/Documentation/perf-inject.txt
index c972032f4ca0..3c88967b4c7f 100644
--- a/tools/perf/Documentation/perf-inject.txt
+++ b/tools/perf/Documentation/perf-inject.txt
@@ -62,6 +62,9 @@ OPTIONS
 --kallsyms=<file>::
 	kallsyms pathname
 
+--vdso=<vdso1[,vdso2]>::
+	Specify vdso pathnames. You can specify up to two for multiarch-support.
+
 --itrace::
 	Decode Instruction Tracing data, replacing it with synthesized events.
 	Options are:
diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt
index d2b1593ef700..8a3ba5f74cac 100644
--- a/tools/perf/Documentation/perf-report.txt
+++ b/tools/perf/Documentation/perf-report.txt
@@ -345,6 +345,9 @@ OPTIONS
         Load module symbols. WARNING: This should only be used with -k and
         a LIVE kernel.
 
+--vdso=<vdso1[,vdso2]>::
+	Specify vdso pathnames. You can specify up to two for multiarch-support.
+
 -f::
 --force::
         Don't do ownership validation.
diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index ff086ef05a0c..48f9974ca4c5 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -296,6 +296,9 @@ OPTIONS
 --kallsyms=<file>::
         kallsyms pathname
 
+--vdso=<vdso1[,vdso2]>::
+	Specify vdso pathnames. You can specify up to two for multiarch-support.
+
 --symfs=<directory>::
         Look for files with symbols relative to this directory.
 
diff --git a/tools/perf/Documentation/perf-top.txt b/tools/perf/Documentation/perf-top.txt
index a754875fa5bb..1cf3ee16a0c1 100644
--- a/tools/perf/Documentation/perf-top.txt
+++ b/tools/perf/Documentation/perf-top.txt
@@ -76,6 +76,9 @@ Default is to monitor all CPUS.
 --kallsyms=<file>::
 	kallsyms pathname
 
+--vdso=<vdso1[,vdso2]>::
+	Specify vdso pathnames. You can specify up to two for multiarch-support.
+
 -m <pages>::
 --mmap-pages=<pages>::
 	Number of mmap data pages (must be a power of two) or size
diff --git a/tools/perf/builtin-annotate.c b/tools/perf/builtin-annotate.c
index 50d2fb222d48..ff466882065d 100644
--- a/tools/perf/builtin-annotate.c
+++ b/tools/perf/builtin-annotate.c
@@ -742,6 +742,8 @@ int cmd_annotate(int argc, const char **argv)
 		   "file", "vmlinux pathname"),
 	OPT_BOOLEAN('m', "modules", &symbol_conf.use_modules,
 		    "load module symbols - WARNING: use only with -k and LIVE kernel"),
+	OPT_CALLBACK(0, "vdso", NULL, "vdso1[,vdso2]", "vdso pathnames",
+		     parse_vdso_pathnames),
 	OPT_BOOLEAN('l', "print-line", &annotate_opts.print_lines,
 		    "print matching source lines (may be slow)"),
 	OPT_BOOLEAN('P', "full-paths", &annotate_opts.full_path,
diff --git a/tools/perf/builtin-c2c.c b/tools/perf/builtin-c2c.c
index c157bd31f2e5..4764f9139661 100644
--- a/tools/perf/builtin-c2c.c
+++ b/tools/perf/builtin-c2c.c
@@ -3018,6 +3018,8 @@ static int perf_c2c__report(int argc, const char **argv)
 	const struct option options[] = {
 	OPT_STRING('k', "vmlinux", &symbol_conf.vmlinux_name,
 		   "file", "vmlinux pathname"),
+	OPT_CALLBACK(0, "vdso", NULL, "vdso1[,vdso2]", "vdso pathnames",
+		     parse_vdso_pathnames),
 	OPT_STRING('i', "input", &input_name, "file",
 		   "the input file to process"),
 	OPT_INCR('N', "node-info", &c2c.node_info,
diff --git a/tools/perf/builtin-inject.c b/tools/perf/builtin-inject.c
index a212678d47be..e774e83d0a0f 100644
--- a/tools/perf/builtin-inject.c
+++ b/tools/perf/builtin-inject.c
@@ -2247,6 +2247,8 @@ int cmd_inject(int argc, const char **argv)
 			    "don't load vmlinux even if found"),
 		OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name, "file",
 			   "kallsyms pathname"),
+		OPT_CALLBACK(0, "vdso", NULL, "vdso1[,vdso2]", "vdso pathnames",
+		     parse_vdso_pathnames),
 		OPT_BOOLEAN('f', "force", &data.force, "don't complain, do it"),
 		OPT_CALLBACK_OPTARG(0, "itrace", &inject.itrace_synth_opts,
 				    NULL, "opts", "Instruction Tracing options\n"
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 69618fb0110b..a64b48460dce 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1324,6 +1324,8 @@ int cmd_report(int argc, const char **argv)
                     "don't load vmlinux even if found"),
 	OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name,
 		   "file", "kallsyms pathname"),
+	OPT_CALLBACK(0, "vdso", NULL, "vdso1[,vdso2]", "vdso pathnames",
+		     parse_vdso_pathnames),
 	OPT_BOOLEAN('f', "force", &symbol_conf.force, "don't complain, do it"),
 	OPT_BOOLEAN('m', "modules", &symbol_conf.use_modules,
 		    "load module symbols - WARNING: use only with -k and LIVE kernel"),
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index c16224b1fef3..2e358922a8d1 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -3965,6 +3965,8 @@ int cmd_script(int argc, const char **argv)
 		   "file", "vmlinux pathname"),
 	OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name,
 		   "file", "kallsyms pathname"),
+	OPT_CALLBACK(0, "vdso", NULL, "vdso1[,vdso2]", "vdso pathnames",
+		     parse_vdso_pathnames),
 	OPT_BOOLEAN('G', "hide-call-graph", &no_callchain,
 		    "When printing symbols do not display call chain"),
 	OPT_CALLBACK(0, "symfs", NULL, "directory",
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 1d6aef51c122..a3cce4e76eb9 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -1479,6 +1479,8 @@ int cmd_top(int argc, const char **argv)
 		   "file", "kallsyms pathname"),
 	OPT_BOOLEAN('K', "hide_kernel_symbols", &top.hide_kernel_symbols,
 		    "hide kernel symbols"),
+	OPT_CALLBACK(0, "vdso", NULL, "vdso1[,vdso2]", "vdso pathnames",
+		     parse_vdso_pathnames),
 	OPT_CALLBACK('m', "mmap-pages", &opts->mmap_pages, "pages",
 		     "number of mmap data pages", evlist__parse_mmap_pages),
 	OPT_INTEGER('r', "realtime", &top.realtime_prio,
diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
index 72aec8f61b94..8d9d78d3150c 100644
--- a/tools/perf/util/disasm.c
+++ b/tools/perf/util/disasm.c
@@ -16,6 +16,7 @@
 #include "debug.h"
 #include "disasm.h"
 #include "dso.h"
+#include "vdso.h"
 #include "env.h"
 #include "evsel.h"
 #include "map.h"
@@ -1126,7 +1127,7 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 	if (pos && strlen(pos) < SBUILD_ID_SIZE - 2)
 		dirname(build_id_path);
 
-	if (dso__is_kcore(dso))
+	if (dso__is_kcore(dso) || dso__is_vdso(dso))
 		goto fallback;
 
 	len = readlink(build_id_path, linkname, sizeof(linkname) - 1);
@@ -1134,7 +1135,7 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 		goto fallback;
 
 	linkname[len] = '\0';
-	if (strstr(linkname, DSO__NAME_KALLSYMS) ||
+	if (strstr(linkname, DSO__NAME_KALLSYMS) || strstr(linkname, DSO__NAME_VDSO) ||
 		access(filename, R_OK)) {
 fallback:
 		/*
@@ -1142,7 +1143,7 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 		 * cache, or is just a kallsyms file, well, lets hope that this
 		 * DSO is the same as when 'perf record' ran.
 		 */
-		if (dso__kernel(dso) && dso__long_name(dso)[0] == '/')
+		if ((dso__kernel(dso) || dso__is_vdso(dso)) && dso__long_name(dso)[0] == '/')
 			snprintf(filename, filename_size, "%s", dso__long_name(dso));
 		else
 			__symbol__join_symfs(filename, filename_size, dso__long_name(dso));
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 9e5940b5bc59..a90c647d37e1 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -19,6 +19,7 @@
 #include "build-id.h"
 #include "cap.h"
 #include "dso.h"
+#include "vdso.h"
 #include "util.h" // lsdir()
 #include "debug.h"
 #include "event.h"
@@ -44,6 +45,7 @@
 
 static int dso__load_kernel_sym(struct dso *dso, struct map *map);
 static int dso__load_guest_kernel_sym(struct dso *dso, struct map *map);
+static int dso__load_vdso_sym(struct dso *dso, struct map *map);
 static bool symbol__is_idle(const char *name);
 
 int vmlinux_path__nr_entries;
@@ -1833,6 +1835,12 @@ int dso__load(struct dso *dso, struct map *map)
 		goto out;
 	}
 
+	if (dso__is_vdso(dso)) {
+		ret = dso__load_vdso_sym(dso, map);
+		if (ret > 0)
+			goto out;
+	}
+
 	dso__set_adjust_symbols(dso, false);
 
 	if (perfmap) {
@@ -2017,12 +2025,14 @@ int dso__load_vmlinux(struct dso *dso, struct map *map,
 		dso__set_binary_type(dso, DSO_BINARY_TYPE__VMLINUX);
 
 	err = dso__load_sym(dso, map, &ss, &ss, 0);
-	symsrc__destroy(&ss);
-
 	if (err > 0) {
 		dso__set_loaded(dso);
 		pr_debug("Using %s for symbols\n", symfs_vmlinux);
+
+		if (symsrc__has_symtab(&ss) && !dso__symsrc_filename(dso))
+			dso__set_symsrc_filename(dso, strdup(symfs_vmlinux));
 	}
+	symsrc__destroy(&ss);
 
 	return err;
 }
@@ -2349,6 +2359,74 @@ static int vmlinux_path__init(struct perf_env *env)
 	return -1;
 }
 
+int parse_vdso_pathnames(const struct option *opt __maybe_unused,
+			 const char *arg, int unset __maybe_unused)
+{
+	char *tmp, *tok, *str = strdup(arg);
+	unsigned int i = 0;
+
+	for (tok = strtok_r(str, ",", &tmp); tok && i < ARRAY_SIZE(symbol_conf.vdso_name);
+	     tok = strtok_r(NULL, ",", &tmp)) {
+		symbol_conf.vdso_name[i++] = strdup(tok);
+	}
+
+	free(str);
+	return 0;
+}
+
+static int dso__load_vdso(struct dso *dso, struct map *map,
+			  const char *vdso)
+{
+	int err = -1;
+	struct symsrc ss;
+	char symfs_vdso[PATH_MAX];
+
+	if (vdso[0] == '/')
+		snprintf(symfs_vdso, sizeof(symfs_vdso), "%s", vdso);
+	else
+		symbol__join_symfs(symfs_vdso, vdso);
+
+	if (symsrc__init(&ss, dso, symfs_vdso, DSO_BINARY_TYPE__SYSTEM_PATH_DSO))
+		return -1;
+
+	/*
+	 * dso__load_sym() may copy 'dso' which will result in the copies having
+	 * an incorrect long name unless we set it here first.
+	 */
+	dso__set_long_name(dso, vdso, false);
+	dso__set_binary_type(dso, DSO_BINARY_TYPE__SYSTEM_PATH_DSO);
+
+	err = dso__load_sym(dso, map, &ss, &ss, 0);
+	if (err > 0) {
+		dso__set_loaded(dso);
+		pr_debug("Using %s for %s symbols\n", symfs_vdso, dso__short_name(dso));
+
+		if (symsrc__has_symtab(&ss) && !dso__symsrc_filename(dso))
+			dso__set_symsrc_filename(dso, strdup(symfs_vdso));
+	}
+	symsrc__destroy(&ss);
+
+	return err;
+}
+
+static int dso__load_vdso_sym(struct dso *dso, struct map *map)
+{
+	int ret;
+
+	if (!dso__is_vdso(dso))
+		return -1;
+
+	for (unsigned int i = 0; i < ARRAY_SIZE(symbol_conf.vdso_name); i++) {
+		if (symbol_conf.vdso_name[i] != NULL) {
+			ret = dso__load_vdso(dso, map, symbol_conf.vdso_name[i]);
+			if (ret > 0)
+				return ret;
+		}
+	}
+
+	return -1;
+}
+
 int setup_list(struct strlist **list, const char *list_str,
 		      const char *list_name)
 {
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index c114bbceef40..108356e3c981 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -3,6 +3,7 @@
 #define __PERF_SYMBOL_CONF 1
 
 #include <stdbool.h>
+#include <subcmd/parse-options.h>
 
 struct strlist;
 struct intlist;
@@ -55,6 +56,7 @@ struct symbol_conf {
 	const char	*default_guest_vmlinux_name,
 			*default_guest_kallsyms,
 			*default_guest_modules;
+	const char	*vdso_name[2];
 	const char	*guestmount;
 	const char	*dso_list_str,
 			*comm_list_str,
@@ -85,4 +87,7 @@ struct symbol_conf {
 
 extern struct symbol_conf symbol_conf;
 
+int parse_vdso_pathnames(const struct option *opt __maybe_unused,
+			 const char *arg, int unset __maybe_unused);
+
 #endif // __PERF_SYMBOL_CONF
-- 
2.34.1

Re: [PATCH v5 1/8] perf: support specify vdso path in cmdline

[Adrian Hunter]

On 2/07/24 07:18, Changbin Du wrote:
> The vdso dumped from process memory (in buildid-cache) lacks debugging
> info. To annotate vdso symbols with source lines we need specify a
> debugging version.
> 
> For x86, we can find them from your local build as
> arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
> /lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
> the buildid has to match.
> 
> $ sudo perf record -a
> $ sudo perf report --objdump=llvm-objdump \
>   --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
> 
> Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
> __vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
> Percent│       movq    -48(%rbp),%rsi
>        │       testq   %rax,%rax
>        │     ;               return vread_hvclock();
>        │       movq    %rax,%rdx
>        │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
>        │     ↑ js      eb
>        │     ↑ jmp     74
>        │     ;               ts->tv_sec = vdso_ts->sec;
>   0.02 │147:   leaq    2(%rbx),%rax
>        │       shlq    $4, %rax
>        │       addq    %r10,%rax
>        │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
>   9.38 │152:   movl    (%r10),%ecx
> 
> When doing cross platform analysis, we also need specify the vdso path if
> we are interested in its symbols.
> 

Just realized this is about objdump.  Sorry for not getting that
earlier.

Like perf tools, objdump follows the paradigm of attempting to
locate and use debug info transparently.  However, objdump looks
at the installed debug info in /usr/lib/debug/.build-id/

For example, if the debug file is copied (or linked) there, then
objdump or llvm-objdump will find it:

$ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20

~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64

Disassembly of section .text:

00000000000006d0 <.text>:
    6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
    6d7: eb 19                         jmp     0x6f2 <.text+0x22>
    6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
    6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
    6e7: 0f 01 f9                      rdtscp
    6ea: 66 90                         nop
    6ec: 8b 0f                         movl    (%rdi), %ecx
    6ee: 39 ce                         cmpl    %ecx, %esi
    6f0: 74 0e                         je      0x700 <.text+0x30>
    6f2: 8b 37                         movl    (%rdi), %esi
    6f4: 85 f6                         testl   %esi, %esi
    6f6: 75 e1                         jne     0x6d9 <.text+0x9>
    6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
    6ff: c3                            retq
$ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
$ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20                                                       

~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64

Disassembly of section .text:

00000000000006d0 <vread_hvclock>:
; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
    6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
    6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
;               scale = READ_ONCE(tsc_pg->tsc_scale);
    6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
;               offset = READ_ONCE(tsc_pg->tsc_offset);
    6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
;       asm volatile(ALTERNATIVE_2("rdtsc",
    6e7: 0f 31                         rdtsc
    6e9: 90                            nop
    6ea: 90                            nop
    6eb: 90                            nop
;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
    6ec: 8b 0f                         movl    (%rdi), %ecx
$

Would that solve your problem?

Notably, later versions of llvm-objdump have an option
--debug-file-directory which makes it possible to
have the debug info in a directory owned by the user,
for example:

$ sudo rm /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
$ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20

~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64

Disassembly of section .text:

00000000000006d0 <.text>:
    6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
    6d7: eb 19                         jmp     0x6f2 <.text+0x22>
    6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
    6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
    6e7: 0f 01 f9                      rdtscp
    6ea: 66 90                         nop
    6ec: 8b 0f                         movl    (%rdi), %ecx
    6ee: 39 ce                         cmpl    %ecx, %esi
    6f0: 74 0e                         je      0x700 <.text+0x30>
    6f2: 8b 37                         movl    (%rdi), %esi
    6f4: 85 f6                         testl   %esi, %esi
    6f6: 75 e1                         jne     0x6d9 <.text+0x9>
    6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
    6ff: c3                            retq
$ mkdir -p /tmp/debug/.build-id/cf/
$ ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /tmp/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug        
$ llvm-objdump-18 --debug-file-directory /tmp/debug -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20

~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64

Disassembly of section .text:

00000000000006d0 <vread_hvclock>:
; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
    6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
    6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
;               scale = READ_ONCE(tsc_pg->tsc_scale);
    6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
;               offset = READ_ONCE(tsc_pg->tsc_offset);
    6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
;       asm volatile(ALTERNATIVE_2("rdtsc",
    6e7: 0f 31                         rdtsc
    6e9: 90                            nop
    6ea: 90                            nop
    6eb: 90                            nop
;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
    6ec: 8b 0f                         movl    (%rdi), %ecx
$

Re: [PATCH v5 1/8] perf: support specify vdso path in cmdline

[duchangbin]

On Thu, Jul 18, 2024 at 08:02:16PM +0300, Adrian Hunter wrote:
> On 2/07/24 07:18, Changbin Du wrote:
> > The vdso dumped from process memory (in buildid-cache) lacks debugging
> > info. To annotate vdso symbols with source lines we need specify a
> > debugging version.
> > 
> > For x86, we can find them from your local build as
> > arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
> > /lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
> > the buildid has to match.
> > 
> > $ sudo perf record -a
> > $ sudo perf report --objdump=llvm-objdump \
> >   --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
> > 
> > Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
> > __vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
> > Percent│       movq    -48(%rbp),%rsi
> >        │       testq   %rax,%rax
> >        │     ;               return vread_hvclock();
> >        │       movq    %rax,%rdx
> >        │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
> >        │     ↑ js      eb
> >        │     ↑ jmp     74
> >        │     ;               ts->tv_sec = vdso_ts->sec;
> >   0.02 │147:   leaq    2(%rbx),%rax
> >        │       shlq    $4, %rax
> >        │       addq    %r10,%rax
> >        │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
> >   9.38 │152:   movl    (%r10),%ecx
> > 
> > When doing cross platform analysis, we also need specify the vdso path if
> > we are interested in its symbols.
> > 
> 
> Just realized this is about objdump.  Sorry for not getting that
> earlier.
> 
> Like perf tools, objdump follows the paradigm of attempting to
> locate and use debug info transparently.  However, objdump looks
> at the installed debug info in /usr/lib/debug/.build-id/
> 
> For example, if the debug file is copied (or linked) there, then
> objdump or llvm-objdump will find it:
> 
> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
> 
> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> 
> Disassembly of section .text:
> 
> 00000000000006d0 <.text>:
>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
>     6d7: eb 19                         jmp     0x6f2 <.text+0x22>
>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
>     6e7: 0f 01 f9                      rdtscp
>     6ea: 66 90                         nop
>     6ec: 8b 0f                         movl    (%rdi), %ecx
>     6ee: 39 ce                         cmpl    %ecx, %esi
>     6f0: 74 0e                         je      0x700 <.text+0x30>
>     6f2: 8b 37                         movl    (%rdi), %esi
>     6f4: 85 f6                         testl   %esi, %esi
>     6f6: 75 e1                         jne     0x6d9 <.text+0x9>
>     6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
>     6ff: c3                            retq
> $ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20                                                       
> 
> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> 
> Disassembly of section .text:
> 
> 00000000000006d0 <vread_hvclock>:
> ; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
>     6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
> ;               scale = READ_ONCE(tsc_pg->tsc_scale);
>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
> ;               offset = READ_ONCE(tsc_pg->tsc_offset);
>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
> ;       asm volatile(ALTERNATIVE_2("rdtsc",
>     6e7: 0f 31                         rdtsc
>     6e9: 90                            nop
>     6ea: 90                            nop
>     6eb: 90                            nop
> ;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
>     6ec: 8b 0f                         movl    (%rdi), %ecx
> $
> 
> Would that solve your problem?
> 
> Notably, later versions of llvm-objdump have an option
> --debug-file-directory which makes it possible to
> have the debug info in a directory owned by the user,
> for example:
> 
> $ sudo rm /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
> 
> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> 
> Disassembly of section .text:
> 
> 00000000000006d0 <.text>:
>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
>     6d7: eb 19                         jmp     0x6f2 <.text+0x22>
>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
>     6e7: 0f 01 f9                      rdtscp
>     6ea: 66 90                         nop
>     6ec: 8b 0f                         movl    (%rdi), %ecx
>     6ee: 39 ce                         cmpl    %ecx, %esi
>     6f0: 74 0e                         je      0x700 <.text+0x30>
>     6f2: 8b 37                         movl    (%rdi), %esi
>     6f4: 85 f6                         testl   %esi, %esi
>     6f6: 75 e1                         jne     0x6d9 <.text+0x9>
>     6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
>     6ff: c3                            retq
> $ mkdir -p /tmp/debug/.build-id/cf/
> $ ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /tmp/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug        
> $ llvm-objdump-18 --debug-file-directory /tmp/debug -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
> 
> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> 
> Disassembly of section .text:
> 
> 00000000000006d0 <vread_hvclock>:
> ; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
>     6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
> ;               scale = READ_ONCE(tsc_pg->tsc_scale);
>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
> ;               offset = READ_ONCE(tsc_pg->tsc_offset);
>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
> ;       asm volatile(ALTERNATIVE_2("rdtsc",
>     6e7: 0f 31                         rdtsc
>     6e9: 90                            nop
>     6ea: 90                            nop
>     6eb: 90                            nop
> ;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
>     6ec: 8b 0f                         movl    (%rdi), %ecx
> $
>
Yes, not only llvm-objdump. I can see other tools also can handle this and even
can download from online by Debuginfod.
  - perf can search debugging info there if HAVE_DEBUGINFOD_SUPPORT.
  - gdb can search debugging info there and download via Debuginfod if not exist.

While above requires debugging files are placed correctly in
/usr/lib/debug/.build-id/. For vdso mostly you need to copy it manully.

The target of this series is to handle it transparently, espacially for locally
built kernels.

> 
> 

-- 
Cheers,
Changbin Du

Re: [PATCH v5 1/8] perf: support specify vdso path in cmdline

[Adrian Hunter]

On 22/07/24 15:19, duchangbin wrote:
> On Thu, Jul 18, 2024 at 08:02:16PM +0300, Adrian Hunter wrote:
>> On 2/07/24 07:18, Changbin Du wrote:
>>> The vdso dumped from process memory (in buildid-cache) lacks debugging
>>> info. To annotate vdso symbols with source lines we need specify a
>>> debugging version.
>>>
>>> For x86, we can find them from your local build as
>>> arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
>>> /lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
>>> the buildid has to match.
>>>
>>> $ sudo perf record -a
>>> $ sudo perf report --objdump=llvm-objdump \
>>>   --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
>>>
>>> Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
>>> __vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
>>> Percent│       movq    -48(%rbp),%rsi
>>>        │       testq   %rax,%rax
>>>        │     ;               return vread_hvclock();
>>>        │       movq    %rax,%rdx
>>>        │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
>>>        │     ↑ js      eb
>>>        │     ↑ jmp     74
>>>        │     ;               ts->tv_sec = vdso_ts->sec;
>>>   0.02 │147:   leaq    2(%rbx),%rax
>>>        │       shlq    $4, %rax
>>>        │       addq    %r10,%rax
>>>        │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
>>>   9.38 │152:   movl    (%r10),%ecx
>>>
>>> When doing cross platform analysis, we also need specify the vdso path if
>>> we are interested in its symbols.
>>>
>>
>> Just realized this is about objdump.  Sorry for not getting that
>> earlier.
>>
>> Like perf tools, objdump follows the paradigm of attempting to
>> locate and use debug info transparently.  However, objdump looks
>> at the installed debug info in /usr/lib/debug/.build-id/
>>
>> For example, if the debug file is copied (or linked) there, then
>> objdump or llvm-objdump will find it:
>>
>> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
>>
>> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
>>
>> Disassembly of section .text:
>>
>> 00000000000006d0 <.text>:
>>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
>>     6d7: eb 19                         jmp     0x6f2 <.text+0x22>
>>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
>>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
>>     6e7: 0f 01 f9                      rdtscp
>>     6ea: 66 90                         nop
>>     6ec: 8b 0f                         movl    (%rdi), %ecx
>>     6ee: 39 ce                         cmpl    %ecx, %esi
>>     6f0: 74 0e                         je      0x700 <.text+0x30>
>>     6f2: 8b 37                         movl    (%rdi), %esi
>>     6f4: 85 f6                         testl   %esi, %esi
>>     6f6: 75 e1                         jne     0x6d9 <.text+0x9>
>>     6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
>>     6ff: c3                            retq
>> $ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
>> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20                                                       
>>
>> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
>>
>> Disassembly of section .text:
>>
>> 00000000000006d0 <vread_hvclock>:
>> ; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
>>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
>>     6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
>> ;               scale = READ_ONCE(tsc_pg->tsc_scale);
>>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
>> ;               offset = READ_ONCE(tsc_pg->tsc_offset);
>>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
>> ;       asm volatile(ALTERNATIVE_2("rdtsc",
>>     6e7: 0f 31                         rdtsc
>>     6e9: 90                            nop
>>     6ea: 90                            nop
>>     6eb: 90                            nop
>> ;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
>>     6ec: 8b 0f                         movl    (%rdi), %ecx
>> $
>>
>> Would that solve your problem?
>>
>> Notably, later versions of llvm-objdump have an option
>> --debug-file-directory which makes it possible to
>> have the debug info in a directory owned by the user,
>> for example:
>>
>> $ sudo rm /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
>> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
>>
>> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
>>
>> Disassembly of section .text:
>>
>> 00000000000006d0 <.text>:
>>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
>>     6d7: eb 19                         jmp     0x6f2 <.text+0x22>
>>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
>>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
>>     6e7: 0f 01 f9                      rdtscp
>>     6ea: 66 90                         nop
>>     6ec: 8b 0f                         movl    (%rdi), %ecx
>>     6ee: 39 ce                         cmpl    %ecx, %esi
>>     6f0: 74 0e                         je      0x700 <.text+0x30>
>>     6f2: 8b 37                         movl    (%rdi), %esi
>>     6f4: 85 f6                         testl   %esi, %esi
>>     6f6: 75 e1                         jne     0x6d9 <.text+0x9>
>>     6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
>>     6ff: c3                            retq
>> $ mkdir -p /tmp/debug/.build-id/cf/
>> $ ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /tmp/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug        
>> $ llvm-objdump-18 --debug-file-directory /tmp/debug -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
>>
>> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
>>
>> Disassembly of section .text:
>>
>> 00000000000006d0 <vread_hvclock>:
>> ; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
>>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
>>     6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
>> ;               scale = READ_ONCE(tsc_pg->tsc_scale);
>>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
>> ;               offset = READ_ONCE(tsc_pg->tsc_offset);
>>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
>> ;       asm volatile(ALTERNATIVE_2("rdtsc",
>>     6e7: 0f 31                         rdtsc
>>     6e9: 90                            nop
>>     6ea: 90                            nop
>>     6eb: 90                            nop
>> ;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
>>     6ec: 8b 0f                         movl    (%rdi), %ecx
>> $
>>
> Yes, not only llvm-objdump. I can see other tools also can handle this and even
> can download from online by Debuginfod.
>   - perf can search debugging info there if HAVE_DEBUGINFOD_SUPPORT.
>   - gdb can search debugging info there and download via Debuginfod if not exist.
> 
> While above requires debugging files are placed correctly in
> /usr/lib/debug/.build-id/. For vdso mostly you need to copy it manully.
> 
> The target of this series is to handle it transparently, espacially for locally
> built kernels.

Passing the file name to perf tool doesn't seem transparent.

For locally built kernels, maybe there should be a make target for it.
There seems to be "vdso_install" for the stripped binary.

Re: [PATCH v5 1/8] perf: support specify vdso path in cmdline

[duchangbin]

On Mon, Jul 22, 2024 at 04:15:56PM +0300, Adrian Hunter wrote:
> On 22/07/24 15:19, duchangbin wrote:
> > On Thu, Jul 18, 2024 at 08:02:16PM +0300, Adrian Hunter wrote:
> >> On 2/07/24 07:18, Changbin Du wrote:
> >>> The vdso dumped from process memory (in buildid-cache) lacks debugging
> >>> info. To annotate vdso symbols with source lines we need specify a
> >>> debugging version.
> >>>
> >>> For x86, we can find them from your local build as
> >>> arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
> >>> /lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
> >>> the buildid has to match.
> >>>
> >>> $ sudo perf record -a
> >>> $ sudo perf report --objdump=llvm-objdump \
> >>>   --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
> >>>
> >>> Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
> >>> __vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
> >>> Percent│       movq    -48(%rbp),%rsi
> >>>        │       testq   %rax,%rax
> >>>        │     ;               return vread_hvclock();
> >>>        │       movq    %rax,%rdx
> >>>        │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
> >>>        │     ↑ js      eb
> >>>        │     ↑ jmp     74
> >>>        │     ;               ts->tv_sec = vdso_ts->sec;
> >>>   0.02 │147:   leaq    2(%rbx),%rax
> >>>        │       shlq    $4, %rax
> >>>        │       addq    %r10,%rax
> >>>        │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
> >>>   9.38 │152:   movl    (%r10),%ecx
> >>>
> >>> When doing cross platform analysis, we also need specify the vdso path if
> >>> we are interested in its symbols.
> >>>
> >>
> >> Just realized this is about objdump.  Sorry for not getting that
> >> earlier.
> >>
> >> Like perf tools, objdump follows the paradigm of attempting to
> >> locate and use debug info transparently.  However, objdump looks
> >> at the installed debug info in /usr/lib/debug/.build-id/
> >>
> >> For example, if the debug file is copied (or linked) there, then
> >> objdump or llvm-objdump will find it:
> >>
> >> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
> >>
> >> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> >>
> >> Disassembly of section .text:
> >>
> >> 00000000000006d0 <.text>:
> >>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
> >>     6d7: eb 19                         jmp     0x6f2 <.text+0x22>
> >>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
> >>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
> >>     6e7: 0f 01 f9                      rdtscp
> >>     6ea: 66 90                         nop
> >>     6ec: 8b 0f                         movl    (%rdi), %ecx
> >>     6ee: 39 ce                         cmpl    %ecx, %esi
> >>     6f0: 74 0e                         je      0x700 <.text+0x30>
> >>     6f2: 8b 37                         movl    (%rdi), %esi
> >>     6f4: 85 f6                         testl   %esi, %esi
> >>     6f6: 75 e1                         jne     0x6d9 <.text+0x9>
> >>     6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
> >>     6ff: c3                            retq
> >> $ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
> >> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20                                                       
> >>
> >> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> >>
> >> Disassembly of section .text:
> >>
> >> 00000000000006d0 <vread_hvclock>:
> >> ; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
> >>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
> >>     6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
> >> ;               scale = READ_ONCE(tsc_pg->tsc_scale);
> >>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
> >> ;               offset = READ_ONCE(tsc_pg->tsc_offset);
> >>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
> >> ;       asm volatile(ALTERNATIVE_2("rdtsc",
> >>     6e7: 0f 31                         rdtsc
> >>     6e9: 90                            nop
> >>     6ea: 90                            nop
> >>     6eb: 90                            nop
> >> ;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
> >>     6ec: 8b 0f                         movl    (%rdi), %ecx
> >> $
> >>
> >> Would that solve your problem?
> >>
> >> Notably, later versions of llvm-objdump have an option
> >> --debug-file-directory which makes it possible to
> >> have the debug info in a directory owned by the user,
> >> for example:
> >>
> >> $ sudo rm /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
> >> $ llvm-objdump-18 -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
> >>
> >> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> >>
> >> Disassembly of section .text:
> >>
> >> 00000000000006d0 <.text>:
> >>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000
> >>     6d7: eb 19                         jmp     0x6f2 <.text+0x22>
> >>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008
> >>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010
> >>     6e7: 0f 01 f9                      rdtscp
> >>     6ea: 66 90                         nop
> >>     6ec: 8b 0f                         movl    (%rdi), %ecx
> >>     6ee: 39 ce                         cmpl    %ecx, %esi
> >>     6f0: 74 0e                         je      0x700 <.text+0x30>
> >>     6f2: 8b 37                         movl    (%rdi), %esi
> >>     6f4: 85 f6                         testl   %esi, %esi
> >>     6f6: 75 e1                         jne     0x6d9 <.text+0x9>
> >>     6f8: 48 c7 c0 ff ff ff ff          movq    $-0x1, %rax
> >>     6ff: c3                            retq
> >> $ mkdir -p /tmp/debug/.build-id/cf/
> >> $ ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /tmp/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug        
> >> $ llvm-objdump-18 --debug-file-directory /tmp/debug -dS ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso | head -20
> >>
> >> ~/.debug/[vdso]/cf702469f4637840fd6ba1a8d8a628ff83253d04/vdso:      file format elf64-x86-64
> >>
> >> Disassembly of section .text:
> >>
> >> 00000000000006d0 <vread_hvclock>:
> >> ; hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
> >>     6d0: 48 8d 3d 29 d9 ff ff          leaq    -0x26d7(%rip), %rdi     # 0xffffffffffffe000 <hvclock_page>
> >>     6d7: eb 19                         jmp     0x6f2 <vread_hvclock+0x22>
> >> ;               scale = READ_ONCE(tsc_pg->tsc_scale);
> >>     6d9: 4c 8b 0d 28 d9 ff ff          movq    -0x26d8(%rip), %r9      # 0xffffffffffffe008 <hvclock_page+0x8>
> >> ;               offset = READ_ONCE(tsc_pg->tsc_offset);
> >>     6e0: 4c 8b 05 29 d9 ff ff          movq    -0x26d7(%rip), %r8      # 0xffffffffffffe010 <hvclock_page+0x10>
> >> ;       asm volatile(ALTERNATIVE_2("rdtsc",
> >>     6e7: 0f 31                         rdtsc
> >>     6e9: 90                            nop
> >>     6ea: 90                            nop
> >>     6eb: 90                            nop
> >> ;       } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
> >>     6ec: 8b 0f                         movl    (%rdi), %ecx
> >> $
> >>
> > Yes, not only llvm-objdump. I can see other tools also can handle this and even
> > can download from online by Debuginfod.
> >   - perf can search debugging info there if HAVE_DEBUGINFOD_SUPPORT.
> >   - gdb can search debugging info there and download via Debuginfod if not exist.
> > 
> > While above requires debugging files are placed correctly in
> > /usr/lib/debug/.build-id/. For vdso mostly you need to copy it manully.
> > 
> > The target of this series is to handle it transparently, espacially for locally
> > built kernels.
> 
> Passing the file name to perf tool doesn't seem transparent.
>
This is an alternative way to find debugging file if the file is not in
pre-defined paths. This is how we handle vmlinux with '--vmlinux' option.

> For locally built kernels, maybe there should be a make target for it.
> There seems to be "vdso_install" for the stripped binary.
>
I don't know whether installing debugging files to /usr/lib/debug/.build-id/ is
a good scheme. For me, I really do not want too many debugging files occupy my
disk space. (If so, why only install vdso there, right?)

> 
> 

-- 
Cheers,
Changbin Du

[PATCH v5 2/8] perf: disasm: refactor function dso__disassemble_filename

[Changbin Du]

To make change easy, this change refactors the dso__disassemble_filename()
function by extracting two functions read_buildid_linkname() and
fallback_filename().

Signed-off-by: Changbin Du <changbin.du@huawei.com>

---
v2: split refactoring from logical change.
---
 tools/perf/util/disasm.c | 117 +++++++++++++++++++++++----------------
 1 file changed, 70 insertions(+), 47 deletions(-)

diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
index 8d9d78d3150c..c4ab98c0eaa1 100644
--- a/tools/perf/util/disasm.c
+++ b/tools/perf/util/disasm.c
@@ -1092,28 +1092,12 @@ int symbol__strerror_disassemble(struct map_symbol *ms, int errnum, char *buf, s
 	return 0;
 }
 
-static int dso__disassemble_filename(struct dso *dso, char *filename, size_t filename_size)
+static int read_buildid_linkname(char *filename, char *linkname, size_t linkname_size)
 {
-	char linkname[PATH_MAX];
-	char *build_id_filename;
 	char *build_id_path = NULL;
 	char *pos;
 	int len;
 
-	if (dso__symtab_type(dso) == DSO_BINARY_TYPE__KALLSYMS &&
-	    !dso__is_kcore(dso))
-		return SYMBOL_ANNOTATE_ERRNO__NO_VMLINUX;
-
-	build_id_filename = dso__build_id_filename(dso, NULL, 0, false);
-	if (build_id_filename) {
-		__symbol__join_symfs(filename, filename_size, build_id_filename);
-		free(build_id_filename);
-	} else {
-		if (dso__has_build_id(dso))
-			return ENOMEM;
-		goto fallback;
-	}
-
 	build_id_path = strdup(filename);
 	if (!build_id_path)
 		return ENOMEM;
@@ -1127,41 +1111,80 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 	if (pos && strlen(pos) < SBUILD_ID_SIZE - 2)
 		dirname(build_id_path);
 
-	if (dso__is_kcore(dso) || dso__is_vdso(dso))
-		goto fallback;
-
-	len = readlink(build_id_path, linkname, sizeof(linkname) - 1);
-	if (len < 0)
-		goto fallback;
+	len = readlink(build_id_path, linkname, linkname_size);
+	if (len < 0) {
+		free(build_id_path);
+		return -1;
+	}
 
 	linkname[len] = '\0';
-	if (strstr(linkname, DSO__NAME_KALLSYMS) || strstr(linkname, DSO__NAME_VDSO) ||
-		access(filename, R_OK)) {
-fallback:
-		/*
-		 * If we don't have build-ids or the build-id file isn't in the
-		 * cache, or is just a kallsyms file, well, lets hope that this
-		 * DSO is the same as when 'perf record' ran.
-		 */
-		if ((dso__kernel(dso) || dso__is_vdso(dso)) && dso__long_name(dso)[0] == '/')
-			snprintf(filename, filename_size, "%s", dso__long_name(dso));
-		else
-			__symbol__join_symfs(filename, filename_size, dso__long_name(dso));
-
-		mutex_lock(dso__lock(dso));
-		if (access(filename, R_OK) && errno == ENOENT && dso__nsinfo(dso)) {
-			char *new_name = dso__filename_with_chroot(dso, filename);
-			if (new_name) {
-				strlcpy(filename, new_name, filename_size);
-				free(new_name);
-			}
+	free(build_id_path);
+	return 0;
+}
+
+static int fallback_filename(struct dso *dso, char *filename, size_t filename_size)
+{
+	char filepath[PATH_MAX];
+
+	/*
+	 * If we don't have build-ids or the build-id file isn't in the
+	 * cache, or is just a kallsyms file, well, lets hope that this
+	 * DSO is the same as when 'perf record' ran.
+	 */
+	if ((dso__kernel(dso) || dso__is_vdso(dso)) && dso__long_name(dso)[0] == '/')
+		snprintf(filepath, sizeof(filepath), "%s", dso__long_name(dso));
+	else
+		__symbol__join_symfs(filepath, sizeof(filepath), dso__long_name(dso));
+
+	mutex_lock(dso__lock(dso));
+	if (access(filepath, R_OK) && errno == ENOENT && dso__nsinfo(dso)) {
+		char *new_name = dso__filename_with_chroot(dso, filepath);
+		if (new_name) {
+			strlcpy(filepath, new_name, sizeof(filepath));
+			free(new_name);
 		}
-		mutex_unlock(dso__lock(dso));
-	} else if (dso__binary_type(dso) == DSO_BINARY_TYPE__NOT_FOUND) {
-		dso__set_binary_type(dso, DSO_BINARY_TYPE__BUILD_ID_CACHE);
 	}
+	mutex_unlock(dso__lock(dso));
 
-	free(build_id_path);
+	if (access(filepath, R_OK) && errno == ENOENT)
+		return ENOENT;
+
+	snprintf(filename, filename_size, "%s", filepath);
+	return 0;
+}
+
+static int dso__disassemble_filename(struct dso *dso, char *filename, size_t filename_size)
+{
+	char linkname[PATH_MAX];
+	char *build_id_filename;
+
+	if (dso__symtab_type(dso) == DSO_BINARY_TYPE__KALLSYMS &&
+	    !dso__is_kcore(dso))
+		return SYMBOL_ANNOTATE_ERRNO__NO_VMLINUX;
+
+	build_id_filename = dso__build_id_filename(dso, NULL, 0, false);
+	if (build_id_filename) {
+		__symbol__join_symfs(filename, filename_size, build_id_filename);
+		free(build_id_filename);
+	} else {
+		if (dso__has_build_id(dso))
+			return ENOMEM;
+		return fallback_filename(dso, filename, filename_size);
+	}
+
+	if (access(filename, R_OK))
+		return fallback_filename(dso, filename, filename_size);
+
+	if (dso__is_kcore(dso) || dso__is_vdso(dso))
+		return fallback_filename(dso, filename, filename_size);
+
+	if (read_buildid_linkname(filename, linkname, sizeof(linkname) - 1) ||
+	    strstr(linkname, DSO__NAME_KALLSYMS) || strstr(linkname, DSO__NAME_VDSO)) {
+		return fallback_filename(dso, filename, filename_size);
+	}
+
+	if (dso__binary_type(dso) == DSO_BINARY_TYPE__NOT_FOUND)
+		dso__set_binary_type(dso, DSO_BINARY_TYPE__BUILD_ID_CACHE);
 	return 0;
 }
 
-- 
2.34.1

[PATCH v5 3/8] perf: disasm: use build_id_path if fallback failed

[Changbin Du]

If we can not fallback for special dso (vmlinx and vdso), use the
build_id_path found previously.

Signed-off-by: Changbin Du <changbin.du@huawei.com>
---
 tools/perf/util/disasm.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
index c4ab98c0eaa1..f4e94ff37e50 100644
--- a/tools/perf/util/disasm.c
+++ b/tools/perf/util/disasm.c
@@ -1176,15 +1176,21 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 		return fallback_filename(dso, filename, filename_size);
 
 	if (dso__is_kcore(dso) || dso__is_vdso(dso))
-		return fallback_filename(dso, filename, filename_size);
+		goto fallback;
 
-	if (read_buildid_linkname(filename, linkname, sizeof(linkname) - 1) ||
-	    strstr(linkname, DSO__NAME_KALLSYMS) || strstr(linkname, DSO__NAME_VDSO)) {
-		return fallback_filename(dso, filename, filename_size);
+	if (!read_buildid_linkname(filename, linkname, sizeof(linkname) - 1) &&
+	    (!strstr(linkname, DSO__NAME_KALLSYMS) && !strstr(linkname, DSO__NAME_VDSO))) {
+		/* It's not kallsysms or vdso, use build_id path found above */
+		goto out;
 	}
 
-	if (dso__binary_type(dso) == DSO_BINARY_TYPE__NOT_FOUND)
-		dso__set_binary_type(dso, DSO_BINARY_TYPE__BUILD_ID_CACHE);
+fallback:
+	if (fallback_filename(dso, filename, filename_size)) {
+		/* if fallback failed, use build_id path found above */
+out:
+		if (dso__binary_type(dso) == DSO_BINARY_TYPE__NOT_FOUND)
+			dso__set_binary_type(dso, DSO_BINARY_TYPE__BUILD_ID_CACHE);
+	}
 	return 0;
 }
 
-- 
2.34.1

[PATCH v5 4/8] perf: build-id: name debugging vdso as "debug"

[Changbin Du]

As normal objects, we will add debugging vdso elf to build-id cache later.
Here we name the debugging one as "debug".

Signed-off-by: Changbin Du <changbin.du@huawei.com>
---
 tools/perf/util/build-id.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
index 83a1581e8cf1..15530af2bad9 100644
--- a/tools/perf/util/build-id.c
+++ b/tools/perf/util/build-id.c
@@ -259,8 +259,8 @@ static bool build_id_cache__valid_id(char *sbuild_id)
 static const char *build_id_cache__basename(bool is_kallsyms, bool is_vdso,
 					    bool is_debug)
 {
-	return is_kallsyms ? "kallsyms" : (is_vdso ? "vdso" : (is_debug ?
-	    "debug" : "elf"));
+	return is_kallsyms ? "kallsyms" : (is_debug ? "debug" : (is_vdso ?
+		"vdso" : "elf"));
 }
 
 char *__dso__build_id_filename(const struct dso *dso, char *bf, size_t size,
-- 
2.34.1

Re: [PATCH v5 4/8] perf: build-id: name debugging vdso as "debug"

[Adrian Hunter]

On 2/07/24 07:18, Changbin Du wrote:
> As normal objects, we will add debugging vdso elf to build-id cache later.
> Here we name the debugging one as "debug".
> 
> Signed-off-by: Changbin Du <changbin.du@huawei.com>
> ---
>  tools/perf/util/build-id.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
> index 83a1581e8cf1..15530af2bad9 100644
> --- a/tools/perf/util/build-id.c
> +++ b/tools/perf/util/build-id.c
> @@ -259,8 +259,8 @@ static bool build_id_cache__valid_id(char *sbuild_id)
>  static const char *build_id_cache__basename(bool is_kallsyms, bool is_vdso,
>  					    bool is_debug)
>  {
> -	return is_kallsyms ? "kallsyms" : (is_vdso ? "vdso" : (is_debug ?
> -	    "debug" : "elf"));
> +	return is_kallsyms ? "kallsyms" : (is_debug ? "debug" : (is_vdso ?
> +		"vdso" : "elf"));
>  }
>  
>  char *__dso__build_id_filename(const struct dso *dso, char *bf, size_t size,

To actually add "debug", this also needs:

diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
index 15530af2bad9..b5bd02a1ad0f 100644
--- a/tools/perf/util/build-id.c
+++ b/tools/perf/util/build-id.c
@@ -701,7 +701,7 @@ build_id_cache__add(const char *sbuild_id, const char *name, const char *realnam
 	 * file itself may not be very useful to users of our tools without a
 	 * symtab.
 	 */
-	if (!is_kallsyms && !is_vdso &&
+	if (!is_kallsyms &&
 	    strncmp(".ko", name + strlen(name) - 3, 3)) {
 		debugfile = build_id_cache__find_debug(sbuild_id, nsi, root_dir);
 		if (debugfile) {



With that perf will populated the "debug" entry in the build-id cache.
Currently, when adding to the build-id cache, perf only looks in
/usr/lib/debug/.build-id (refer build_id_cache__find_debug()), for
example:


$ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
$ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
total 8
-rw-r--r-- 1 ahunter ahunter    0 Jul 18 13:33 probes
-rw------- 1 ahunter ahunter 8192 Jul 18 13:33 vdso
$ perf record uname
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.010 MB perf.data (2 samples) ]
$ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
total 40
-rwxrwxr-x 2 ahunter ahunter 32760 May 27 17:42 debug
-rw-r--r-- 1 ahunter ahunter     0 Jul 18 13:33 probes
-rw------- 1 ahunter ahunter  8192 Jul 18 13:33 vdso


Note, perf will anyway find the debug object in /usr/lib/debug/.build-id
so the benefit is if perf-archive is used to copy from the buildid-cache
to take to another machine.

Re: [PATCH v5 4/8] perf: build-id: name debugging vdso as "debug"

[duchangbin]

On Thu, Jul 18, 2024 at 08:03:07PM +0300, Adrian Hunter wrote:
> On 2/07/24 07:18, Changbin Du wrote:
> > As normal objects, we will add debugging vdso elf to build-id cache later.
> > Here we name the debugging one as "debug".
> > 
> > Signed-off-by: Changbin Du <changbin.du@huawei.com>
> > ---
> >  tools/perf/util/build-id.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
> > index 83a1581e8cf1..15530af2bad9 100644
> > --- a/tools/perf/util/build-id.c
> > +++ b/tools/perf/util/build-id.c
> > @@ -259,8 +259,8 @@ static bool build_id_cache__valid_id(char *sbuild_id)
> >  static const char *build_id_cache__basename(bool is_kallsyms, bool is_vdso,
> >  					    bool is_debug)
> >  {
> > -	return is_kallsyms ? "kallsyms" : (is_vdso ? "vdso" : (is_debug ?
> > -	    "debug" : "elf"));
> > +	return is_kallsyms ? "kallsyms" : (is_debug ? "debug" : (is_vdso ?
> > +		"vdso" : "elf"));
> >  }
> >  
> >  char *__dso__build_id_filename(const struct dso *dso, char *bf, size_t size,
> 
> To actually add "debug", this also needs:
> 
> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
> index 15530af2bad9..b5bd02a1ad0f 100644
> --- a/tools/perf/util/build-id.c
> +++ b/tools/perf/util/build-id.c
> @@ -701,7 +701,7 @@ build_id_cache__add(const char *sbuild_id, const char *name, const char *realnam
>  	 * file itself may not be very useful to users of our tools without a
>  	 * symtab.
>  	 */
> -	if (!is_kallsyms && !is_vdso &&
> +	if (!is_kallsyms &&
>  	    strncmp(".ko", name + strlen(name) - 3, 3)) {
>  		debugfile = build_id_cache__find_debug(sbuild_id, nsi, root_dir);
>  		if (debugfile) {
> 
> 
> 
This is done by later patch named "perf: build-id: try to search debugging vdso
and add to cache". I split the changes into two patches.

> With that perf will populated the "debug" entry in the build-id cache.
> Currently, when adding to the build-id cache, perf only looks in
> /usr/lib/debug/.build-id (refer build_id_cache__find_debug()), for
> example:
> 
> 
> $ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
> $ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
> total 8
> -rw-r--r-- 1 ahunter ahunter    0 Jul 18 13:33 probes
> -rw------- 1 ahunter ahunter 8192 Jul 18 13:33 vdso
> $ perf record uname
> Linux
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.010 MB perf.data (2 samples) ]
> $ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
> total 40
> -rwxrwxr-x 2 ahunter ahunter 32760 May 27 17:42 debug
> -rw-r--r-- 1 ahunter ahunter     0 Jul 18 13:33 probes
> -rw------- 1 ahunter ahunter  8192 Jul 18 13:33 vdso
> 
> 
> Note, perf will anyway find the debug object in /usr/lib/debug/.build-id
> so the benefit is if perf-archive is used to copy from the buildid-cache
> to take to another machine.
> 
> 

-- 
Cheers,
Changbin Du

Re: [PATCH v5 4/8] perf: build-id: name debugging vdso as "debug"

[Adrian Hunter]

On 22/07/24 15:08, duchangbin wrote:
> On Thu, Jul 18, 2024 at 08:03:07PM +0300, Adrian Hunter wrote:
>> On 2/07/24 07:18, Changbin Du wrote:
>>> As normal objects, we will add debugging vdso elf to build-id cache later.
>>> Here we name the debugging one as "debug".
>>>
>>> Signed-off-by: Changbin Du <changbin.du@huawei.com>
>>> ---
>>>  tools/perf/util/build-id.c | 4 ++--
>>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
>>> index 83a1581e8cf1..15530af2bad9 100644
>>> --- a/tools/perf/util/build-id.c
>>> +++ b/tools/perf/util/build-id.c
>>> @@ -259,8 +259,8 @@ static bool build_id_cache__valid_id(char *sbuild_id)
>>>  static const char *build_id_cache__basename(bool is_kallsyms, bool is_vdso,
>>>  					    bool is_debug)
>>>  {
>>> -	return is_kallsyms ? "kallsyms" : (is_vdso ? "vdso" : (is_debug ?
>>> -	    "debug" : "elf"));
>>> +	return is_kallsyms ? "kallsyms" : (is_debug ? "debug" : (is_vdso ?
>>> +		"vdso" : "elf"));
>>>  }
>>>  
>>>  char *__dso__build_id_filename(const struct dso *dso, char *bf, size_t size,
>>
>> To actually add "debug", this also needs:
>>
>> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
>> index 15530af2bad9..b5bd02a1ad0f 100644
>> --- a/tools/perf/util/build-id.c
>> +++ b/tools/perf/util/build-id.c
>> @@ -701,7 +701,7 @@ build_id_cache__add(const char *sbuild_id, const char *name, const char *realnam
>>  	 * file itself may not be very useful to users of our tools without a
>>  	 * symtab.
>>  	 */
>> -	if (!is_kallsyms && !is_vdso &&
>> +	if (!is_kallsyms &&
>>  	    strncmp(".ko", name + strlen(name) - 3, 3)) {
>>  		debugfile = build_id_cache__find_debug(sbuild_id, nsi, root_dir);
>>  		if (debugfile) {
>>
>>
>>
> This is done by later patch named "perf: build-id: try to search debugging vdso
> and add to cache". I split the changes into two patches.

With the above, the split is more functionally logical:
	1. Add support for build-id cache vdso debug
	2. For vdso, extend build_id_cache__find_debug() to find
	local kernel build files

> 
>> With that perf will populated the "debug" entry in the build-id cache.
>> Currently, when adding to the build-id cache, perf only looks in
>> /usr/lib/debug/.build-id (refer build_id_cache__find_debug()), for
>> example:
>>
>>
>> $ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
>> $ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
>> total 8
>> -rw-r--r-- 1 ahunter ahunter    0 Jul 18 13:33 probes
>> -rw------- 1 ahunter ahunter 8192 Jul 18 13:33 vdso
>> $ perf record uname
>> Linux
>> [ perf record: Woken up 1 times to write data ]
>> [ perf record: Captured and wrote 0.010 MB perf.data (2 samples) ]
>> $ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
>> total 40
>> -rwxrwxr-x 2 ahunter ahunter 32760 May 27 17:42 debug
>> -rw-r--r-- 1 ahunter ahunter     0 Jul 18 13:33 probes
>> -rw------- 1 ahunter ahunter  8192 Jul 18 13:33 vdso
>>
>>
>> Note, perf will anyway find the debug object in /usr/lib/debug/.build-id
>> so the benefit is if perf-archive is used to copy from the buildid-cache
>> to take to another machine.
>>
>>
> 

Re: [PATCH v5 4/8] perf: build-id: name debugging vdso as "debug"

[duchangbin]

On Mon, Jul 22, 2024 at 04:21:46PM +0300, Adrian Hunter wrote:
> On 22/07/24 15:08, duchangbin wrote:
> > On Thu, Jul 18, 2024 at 08:03:07PM +0300, Adrian Hunter wrote:
> >> On 2/07/24 07:18, Changbin Du wrote:
> >>> As normal objects, we will add debugging vdso elf to build-id cache later.
> >>> Here we name the debugging one as "debug".
> >>>
> >>> Signed-off-by: Changbin Du <changbin.du@huawei.com>
> >>> ---
> >>>  tools/perf/util/build-id.c | 4 ++--
> >>>  1 file changed, 2 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
> >>> index 83a1581e8cf1..15530af2bad9 100644
> >>> --- a/tools/perf/util/build-id.c
> >>> +++ b/tools/perf/util/build-id.c
> >>> @@ -259,8 +259,8 @@ static bool build_id_cache__valid_id(char *sbuild_id)
> >>>  static const char *build_id_cache__basename(bool is_kallsyms, bool is_vdso,
> >>>  					    bool is_debug)
> >>>  {
> >>> -	return is_kallsyms ? "kallsyms" : (is_vdso ? "vdso" : (is_debug ?
> >>> -	    "debug" : "elf"));
> >>> +	return is_kallsyms ? "kallsyms" : (is_debug ? "debug" : (is_vdso ?
> >>> +		"vdso" : "elf"));
> >>>  }
> >>>  
> >>>  char *__dso__build_id_filename(const struct dso *dso, char *bf, size_t size,
> >>
> >> To actually add "debug", this also needs:
> >>
> >> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
> >> index 15530af2bad9..b5bd02a1ad0f 100644
> >> --- a/tools/perf/util/build-id.c
> >> +++ b/tools/perf/util/build-id.c
> >> @@ -701,7 +701,7 @@ build_id_cache__add(const char *sbuild_id, const char *name, const char *realnam
> >>  	 * file itself may not be very useful to users of our tools without a
> >>  	 * symtab.
> >>  	 */
> >> -	if (!is_kallsyms && !is_vdso &&
> >> +	if (!is_kallsyms &&
> >>  	    strncmp(".ko", name + strlen(name) - 3, 3)) {
> >>  		debugfile = build_id_cache__find_debug(sbuild_id, nsi, root_dir);
> >>  		if (debugfile) {
> >>
> >>
> >>
> > This is done by later patch named "perf: build-id: try to search debugging vdso
> > and add to cache". I split the changes into two patches.
> 
> With the above, the split is more functionally logical:
> 	1. Add support for build-id cache vdso debug
> 	2. For vdso, extend build_id_cache__find_debug() to find
> 	local kernel build files
>
Sounds good for me.

> > 
> >> With that perf will populated the "debug" entry in the build-id cache.
> >> Currently, when adding to the build-id cache, perf only looks in
> >> /usr/lib/debug/.build-id (refer build_id_cache__find_debug()), for
> >> example:
> >>
> >>
> >> $ sudo ln -s /lib/modules/6.9.2-local/build/arch/x86/entry/vdso/vdso64.so.dbg /usr/lib/debug/.build-id/cf/702469f4637840fd6ba1a8d8a628ff83253d04.debug
> >> $ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
> >> total 8
> >> -rw-r--r-- 1 ahunter ahunter    0 Jul 18 13:33 probes
> >> -rw------- 1 ahunter ahunter 8192 Jul 18 13:33 vdso
> >> $ perf record uname
> >> Linux
> >> [ perf record: Woken up 1 times to write data ]
> >> [ perf record: Captured and wrote 0.010 MB perf.data (2 samples) ]
> >> $ ls -l ~/.debug/\[vdso\]/cf702469f4637840fd6ba1a8d8a628ff83253d04/
> >> total 40
> >> -rwxrwxr-x 2 ahunter ahunter 32760 May 27 17:42 debug
> >> -rw-r--r-- 1 ahunter ahunter     0 Jul 18 13:33 probes
> >> -rw------- 1 ahunter ahunter  8192 Jul 18 13:33 vdso
> >>
> >>
> >> Note, perf will anyway find the debug object in /usr/lib/debug/.build-id
> >> so the benefit is if perf-archive is used to copy from the buildid-cache
> >> to take to another machine.
> >>
> >>
> > 
> 
> 

-- 
Cheers,
Changbin Du

[PATCH v5 5/8] perf: symbol: generalize vmlinux path searching

[Changbin Du]

This generalizes the vmlinux path searching logic. Later we will add
another instance for vdso.

The search pattern is described by struct dso_filename_pattern, and the
formatted paths are hold in struct dso_filename_paths.

Signed-off-by: Changbin Du <changbin.du@huawei.com>
---
 tools/perf/util/machine.c |   4 +-
 tools/perf/util/symbol.c  | 112 +++++++++++++++++++++-----------------
 tools/perf/util/symbol.h  |   8 ++-
 3 files changed, 70 insertions(+), 54 deletions(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 8477edefc299..68315520f15b 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -896,9 +896,9 @@ size_t machine__fprintf_vmlinux_path(struct machine *machine, FILE *fp)
 			printed += fprintf(fp, "[0] %s\n", filename);
 	}
 
-	for (i = 0; i < vmlinux_path__nr_entries; ++i) {
+	for (i = 0; i < vmlinux_paths.nr_entries; ++i) {
 		printed += fprintf(fp, "[%d] %s\n", i + dso__has_build_id(kdso),
-				   vmlinux_path[i]);
+				   vmlinux_paths.paths[i]);
 	}
 	return printed;
 }
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index a90c647d37e1..83e5c3807a2c 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -48,8 +48,7 @@ static int dso__load_guest_kernel_sym(struct dso *dso, struct map *map);
 static int dso__load_vdso_sym(struct dso *dso, struct map *map);
 static bool symbol__is_idle(const char *name);
 
-int vmlinux_path__nr_entries;
-char **vmlinux_path;
+struct dso_filename_paths vmlinux_paths;
 
 struct symbol_conf symbol_conf = {
 	.nanosecs		= false,
@@ -2043,10 +2042,10 @@ int dso__load_vmlinux_path(struct dso *dso, struct map *map)
 	char *filename = NULL;
 
 	pr_debug("Looking at the vmlinux_path (%d entries long)\n",
-		 vmlinux_path__nr_entries + 1);
+		 vmlinux_paths.nr_entries + 1);
 
-	for (i = 0; i < vmlinux_path__nr_entries; ++i) {
-		err = dso__load_vmlinux(dso, map, vmlinux_path[i], false);
+	for (i = 0; i < vmlinux_paths.nr_entries; ++i) {
+		err = dso__load_vmlinux(dso, map, vmlinux_paths.paths[i], false);
 		if (err > 0)
 			goto out;
 	}
@@ -2210,7 +2209,7 @@ static int dso__load_kernel_sym(struct dso *dso, struct map *map)
 			return err;
 	}
 
-	if (!symbol_conf.ignore_vmlinux && vmlinux_path != NULL) {
+	if (!symbol_conf.ignore_vmlinux && vmlinux_paths.paths != NULL) {
 		err = dso__load_vmlinux_path(dso, map);
 		if (err > 0)
 			return err;
@@ -2285,57 +2284,55 @@ static int dso__load_guest_kernel_sym(struct dso *dso, struct map *map)
 	return err;
 }
 
-static void vmlinux_path__exit(void)
-{
-	while (--vmlinux_path__nr_entries >= 0)
-		zfree(&vmlinux_path[vmlinux_path__nr_entries]);
-	vmlinux_path__nr_entries = 0;
-
-	zfree(&vmlinux_path);
-}
-
-static const char * const vmlinux_paths[] = {
-	"vmlinux",
-	"/boot/vmlinux"
+struct dso_filename_pattern {
+	const char *pattern;
+	/*
+	 * 0 for matching directly,
+	 * 1 for matching by kernel_version,
+	 * 2 for matching by kernel_version + arch.
+	 */
+	int match_type;
 };
 
-static const char * const vmlinux_paths_upd[] = {
-	"/boot/vmlinux-%s",
-	"/usr/lib/debug/boot/vmlinux-%s",
-	"/lib/modules/%s/build/vmlinux",
-	"/usr/lib/debug/lib/modules/%s/vmlinux",
-	"/usr/lib/debug/boot/vmlinux-%s.debug"
+struct dso_filename_pattern vmlinux_patterns[] = {
+	{"vmlinux", 0},
+	{"/boot/vmlinux", 0},
+	{"/boot/vmlinux-%s", 1},
+	{"/usr/lib/debug/boot/vmlinux-%s", 1},
+	{"/lib/modules/%s/build/vmlinux", 1},
+	{"/usr/lib/debug/lib/modules/%s/vmlinux", 1},
+	{"/usr/lib/debug/boot/vmlinux-%s.debug", 1},
 };
 
-static int vmlinux_path__add(const char *new_entry)
+static int dso_filename_path__add(struct dso_filename_paths *paths, const char *new_entry)
 {
-	vmlinux_path[vmlinux_path__nr_entries] = strdup(new_entry);
-	if (vmlinux_path[vmlinux_path__nr_entries] == NULL)
+	paths->paths[paths->nr_entries] = strdup(new_entry);
+	if (paths->paths[paths->nr_entries] == NULL)
 		return -1;
-	++vmlinux_path__nr_entries;
+	++paths->nr_entries;
 
 	return 0;
 }
 
-static int vmlinux_path__init(struct perf_env *env)
+static void dso_filename_path__exit(struct dso_filename_paths *paths)
 {
-	struct utsname uts;
-	char bf[PATH_MAX];
-	char *kernel_version;
-	unsigned int i;
+	while (--paths->nr_entries >= 0)
+		zfree(&paths->paths[paths->nr_entries]);
+	paths->nr_entries = 0;
 
-	vmlinux_path = malloc(sizeof(char *) * (ARRAY_SIZE(vmlinux_paths) +
-			      ARRAY_SIZE(vmlinux_paths_upd)));
-	if (vmlinux_path == NULL)
-		return -1;
-
-	for (i = 0; i < ARRAY_SIZE(vmlinux_paths); i++)
-		if (vmlinux_path__add(vmlinux_paths[i]) < 0)
-			goto out_fail;
+	zfree(&paths->paths);
+}
 
-	/* only try kernel version if no symfs was given */
-	if (symbol_conf.symfs[0] != 0)
-		return 0;
+static int dso_filename_path__init(struct dso_filename_paths *paths,
+				   struct dso_filename_pattern *patterns,
+				   int nr_patterns,
+				   struct perf_env *env)
+{
+	struct utsname uts;
+	char bf[PATH_MAX];
+	const char *kernel_version;
+	const char *arch = perf_env__arch(env);
+	int i;
 
 	if (env) {
 		kernel_version = env->os_release;
@@ -2346,16 +2343,28 @@ static int vmlinux_path__init(struct perf_env *env)
 		kernel_version = uts.release;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(vmlinux_paths_upd); i++) {
-		snprintf(bf, sizeof(bf), vmlinux_paths_upd[i], kernel_version);
-		if (vmlinux_path__add(bf) < 0)
+	paths->paths = malloc(sizeof(char *) * nr_patterns);
+	if (paths->paths == NULL)
+		return -1;
+
+	for (i = 0; i < nr_patterns; i++) {
+		if (patterns[i].match_type == 0)
+			strlcpy(bf, patterns[i].pattern, sizeof(bf));
+		else if (symbol_conf.symfs[0] == 0) {
+			/* only try kernel version if no symfs was given */
+			if (patterns[i].match_type == 1)
+				snprintf(bf, sizeof(bf), patterns[i].pattern, kernel_version);
+			else if (patterns[i].match_type == 2)
+				snprintf(bf, sizeof(bf), patterns[i].pattern, kernel_version, arch);
+		}
+		if (dso_filename_path__add(paths, bf) < 0)
 			goto out_fail;
 	}
 
 	return 0;
 
 out_fail:
-	vmlinux_path__exit();
+	dso_filename_path__exit(paths);
 	return -1;
 }
 
@@ -2551,8 +2560,11 @@ int symbol__init(struct perf_env *env)
 
 	symbol__elf_init();
 
-	if (symbol_conf.try_vmlinux_path && vmlinux_path__init(env) < 0)
+	if (symbol_conf.try_vmlinux_path &&
+	    dso_filename_path__init(&vmlinux_paths, vmlinux_patterns,
+				    ARRAY_SIZE(vmlinux_patterns), env) < 0) {
 		return -1;
+	}
 
 	if (symbol_conf.field_sep && *symbol_conf.field_sep == '.') {
 		pr_err("'.' is the only non valid --field-separator argument\n");
@@ -2629,7 +2641,7 @@ void symbol__exit(void)
 	intlist__delete(symbol_conf.tid_list);
 	intlist__delete(symbol_conf.pid_list);
 	intlist__delete(symbol_conf.addr_list);
-	vmlinux_path__exit();
+	dso_filename_path__exit(&vmlinux_paths);
 	symbol_conf.sym_list = symbol_conf.dso_list = symbol_conf.comm_list = NULL;
 	symbol_conf.bt_stop_list = NULL;
 	symbol_conf.initialized = false;
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 3fb5d146d9b1..30056884945b 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -101,8 +101,12 @@ static inline int __symbol__join_symfs(char *bf, size_t size, const char *path)
 
 #define symbol__join_symfs(bf, path) __symbol__join_symfs(bf, sizeof(bf), path)
 
-extern int vmlinux_path__nr_entries;
-extern char **vmlinux_path;
+struct dso_filename_paths {
+	int nr_entries;
+	char **paths;
+};
+
+extern struct dso_filename_paths vmlinux_paths;
 
 static inline void *symbol__priv(struct symbol *sym)
 {
-- 
2.34.1

[PATCH v5 6/8] perf: build-id: try to search debugging vdso and add to cache

[Changbin Du]

Just like vmlinux, try to search vdso in predefined paths when collecting
build-ids. The searched paths usually have debugging info.

For example, the vdso can be found in
/lib/modules/<version>/build/arch/x86/entry/vdso/vdso*.so.dbg for local
build on x86.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Changbin Du <changbin.du@huawei.com>

---
v2:
  - Searching the vdso in record stage instead of report. So the debugging
    vdso will be in build-id cache. This is friendly for cross-machine
    analysis.
---
 tools/perf/util/build-id.c | 53 +++++++++++++++++++++++++++++++++-----
 tools/perf/util/symbol.c   | 17 ++++++++++++
 tools/perf/util/symbol.h   |  1 +
 3 files changed, 64 insertions(+), 7 deletions(-)

diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
index 15530af2bad9..f40de2d9843a 100644
--- a/tools/perf/util/build-id.c
+++ b/tools/perf/util/build-id.c
@@ -593,9 +593,8 @@ static int build_id_cache__add_sdt_cache(const char *sbuild_id,
 #define build_id_cache__add_sdt_cache(sbuild_id, realname, nsi) (0)
 #endif
 
-static char *build_id_cache__find_debug(const char *sbuild_id,
-					struct nsinfo *nsi,
-					const char *root_dir)
+static char *build_id_cache__find_debug_normal(const char *sbuild_id,
+				struct nsinfo *nsi, const char *root_dir)
 {
 	const char *dirname = "/usr/lib/debug/.build-id/";
 	char *realname = NULL;
@@ -646,6 +645,47 @@ static char *build_id_cache__find_debug(const char *sbuild_id,
 	return realname;
 }
 
+static char *build_id_cache__find_debug_vdso(const char *sbuild_id)
+{
+	char sbuild_id_tmp[SBUILD_ID_SIZE];
+	struct build_id bid;
+	int i, ret = 0;
+
+	if (!vdso_paths.paths)
+		return NULL;
+
+	pr_debug("Looking at the vdso_path (%d entries long)\n",
+		 vdso_paths.nr_entries + 1);
+
+	for (i = 0; i < vdso_paths.nr_entries; ++i) {
+		ret = filename__read_build_id(vdso_paths.paths[i], &bid);
+		if (ret < 0)
+			continue;
+
+		build_id__sprintf(&bid, sbuild_id_tmp);
+		if (!strcmp(sbuild_id, sbuild_id_tmp)) {
+			pr_debug("Found debugging vdso %s\n", vdso_paths.paths[i]);
+			return strdup(vdso_paths.paths[i]);
+		}
+	}
+
+	return NULL;
+}
+
+static char *build_id_cache__find_debug(const char *sbuild_id,
+					struct nsinfo *nsi,
+					bool is_vdso,
+					const char *root_dir)
+{
+	char *name = NULL;
+
+	if (is_vdso)
+		name = build_id_cache__find_debug_vdso(sbuild_id);
+	else
+		name = build_id_cache__find_debug_normal(sbuild_id, nsi, root_dir);
+	return name;
+}
+
 int
 build_id_cache__add(const char *sbuild_id, const char *name, const char *realname,
 		    struct nsinfo *nsi, bool is_kallsyms, bool is_vdso,
@@ -701,13 +741,12 @@ build_id_cache__add(const char *sbuild_id, const char *name, const char *realnam
 	 * file itself may not be very useful to users of our tools without a
 	 * symtab.
 	 */
-	if (!is_kallsyms && !is_vdso &&
-	    strncmp(".ko", name + strlen(name) - 3, 3)) {
-		debugfile = build_id_cache__find_debug(sbuild_id, nsi, root_dir);
+	if (!is_kallsyms && strncmp(".ko", name + strlen(name) - 3, 3)) {
+		debugfile = build_id_cache__find_debug(sbuild_id, nsi, is_vdso, root_dir);
 		if (debugfile) {
 			zfree(&filename);
 			if (asprintf(&filename, "%s/%s", dir_name,
-			    build_id_cache__basename(false, false, true)) < 0) {
+			    build_id_cache__basename(false, is_vdso, true)) < 0) {
 				filename = NULL;
 				goto out_free;
 			}
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 83e5c3807a2c..6ad3a057f727 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -49,6 +49,7 @@ static int dso__load_vdso_sym(struct dso *dso, struct map *map);
 static bool symbol__is_idle(const char *name);
 
 struct dso_filename_paths vmlinux_paths;
+struct dso_filename_paths vdso_paths;
 
 struct symbol_conf symbol_conf = {
 	.nanosecs		= false,
@@ -2304,6 +2305,16 @@ struct dso_filename_pattern vmlinux_patterns[] = {
 	{"/usr/lib/debug/boot/vmlinux-%s.debug", 1},
 };
 
+struct dso_filename_pattern vdso_patterns[] = {
+	{"/lib/modules/%s/vdso/vdso.so", 1},
+	{"/lib/modules/%s/vdso/vdso64.so", 1},
+	{"/lib/modules/%s/vdso/vdso32.so", 1},
+	{"/lib/modules/%s/build/arch/%s/vdso/vdso.so.dbg", 2},
+	{"/lib/modules/%s/build/arch/%s/kernel/vdso/vdso.so.dbg", 2},
+	{"/lib/modules/%s/build/arch/%s/entry/vdso/vdso32.so.dbg", 2},
+	{"/lib/modules/%s/build/arch/%s/entry/vdso/vdso64.so.dbg", 2},
+};
+
 static int dso_filename_path__add(struct dso_filename_paths *paths, const char *new_entry)
 {
 	paths->paths[paths->nr_entries] = strdup(new_entry);
@@ -2566,6 +2577,11 @@ int symbol__init(struct perf_env *env)
 		return -1;
 	}
 
+	if (dso_filename_path__init(&vdso_paths, vdso_patterns,
+				    ARRAY_SIZE(vdso_patterns), env) < 0) {
+		return -1;
+	}
+
 	if (symbol_conf.field_sep && *symbol_conf.field_sep == '.') {
 		pr_err("'.' is the only non valid --field-separator argument\n");
 		return -1;
@@ -2642,6 +2658,7 @@ void symbol__exit(void)
 	intlist__delete(symbol_conf.pid_list);
 	intlist__delete(symbol_conf.addr_list);
 	dso_filename_path__exit(&vmlinux_paths);
+	dso_filename_path__exit(&vdso_paths);
 	symbol_conf.sym_list = symbol_conf.dso_list = symbol_conf.comm_list = NULL;
 	symbol_conf.bt_stop_list = NULL;
 	symbol_conf.initialized = false;
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 30056884945b..08c339594d4e 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -107,6 +107,7 @@ struct dso_filename_paths {
 };
 
 extern struct dso_filename_paths vmlinux_paths;
+extern struct dso_filename_paths vdso_paths;
 
 static inline void *symbol__priv(struct symbol *sym)
 {
-- 
2.34.1

[PATCH v5 7/8] perf: disasm: prefer debugging files in build-id cache

[Changbin Du]

The build-id cache might have both debugging and non-debugging files. Here
we prefer the debugging version for annotation.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Changbin Du <changbin.du@huawei.com>
---
 tools/perf/util/disasm.c | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
index f4e94ff37e50..ecfd6a03713d 100644
--- a/tools/perf/util/disasm.c
+++ b/tools/perf/util/disasm.c
@@ -1162,18 +1162,25 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 	    !dso__is_kcore(dso))
 		return SYMBOL_ANNOTATE_ERRNO__NO_VMLINUX;
 
-	build_id_filename = dso__build_id_filename(dso, NULL, 0, false);
-	if (build_id_filename) {
-		__symbol__join_symfs(filename, filename_size, build_id_filename);
-		free(build_id_filename);
-	} else {
-		if (dso__has_build_id(dso))
-			return ENOMEM;
-		return fallback_filename(dso, filename, filename_size);
-	}
+	/* Prefer debugging file if exists, otherwise non-debugging one is used. */
+	for (int i = 0; i < 2; i++) {
+		build_id_filename = dso__build_id_filename(dso, NULL, 0, !i);
+		if (build_id_filename) {
+			__symbol__join_symfs(filename, filename_size, build_id_filename);
+			free(build_id_filename);
+		} else {
+			if (dso__has_build_id(dso))
+				return ENOMEM;
+			return fallback_filename(dso, filename, filename_size);
+		}
 
-	if (access(filename, R_OK))
-		return fallback_filename(dso, filename, filename_size);
+		if (!access(filename, R_OK))
+			break;
+		else if (i != 0) {
+			/* nor debugging or non-debugging is found */
+			return fallback_filename(dso, filename, filename_size);
+		}
+	}
 
 	if (dso__is_kcore(dso) || dso__is_vdso(dso))
 		goto fallback;
-- 
2.34.1

[PATCH v5 8/8] perf buildid-cache: recognize vdso when adding files

[Changbin Du]

Identify vdso by file name matching. The vdso objects have name
as vdso[32,64].so[.dbg].

$ perf buildid-cache -a /work/linux/arch/x86/entry/vdso/vdso64.so.dbg

Without this change, adding vdso using above command actually will never
be used.

Signed-off-by: Changbin Du <changbin.du@huawei.com>
---
 tools/perf/builtin-buildid-cache.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/tools/perf/builtin-buildid-cache.c b/tools/perf/builtin-buildid-cache.c
index b0511d16aeb6..8edea9044a65 100644
--- a/tools/perf/builtin-buildid-cache.c
+++ b/tools/perf/builtin-buildid-cache.c
@@ -172,6 +172,30 @@ static int build_id_cache__add_kcore(const char *filename, bool force)
 	return 0;
 }
 
+static bool filename_is_vdso(const char *filename)
+{
+	char *fname, *bname;
+	static const char * const vdso_names[] = {
+		"vdso.so", "vdso32.so", "vdso64.so", "vdsox32.so"
+	};
+
+	fname = strdup(filename);
+	if (!fname) {
+		pr_err("no mememory\n");
+		return false;
+	}
+
+	bname = basename(fname);
+	if (!bname)
+		return false;
+
+	for (unsigned int i = 0; i < ARRAY_SIZE(vdso_names); i++) {
+		if (!strncmp(bname, vdso_names[i], strlen(vdso_names[i])))
+			return true;
+	}
+	return false;
+}
+
 static int build_id_cache__add_file(const char *filename, struct nsinfo *nsi)
 {
 	char sbuild_id[SBUILD_ID_SIZE];
@@ -189,7 +213,7 @@ static int build_id_cache__add_file(const char *filename, struct nsinfo *nsi)
 
 	build_id__sprintf(&bid, sbuild_id);
 	err = build_id_cache__add_s(sbuild_id, filename, nsi,
-				    false, false);
+				    false, filename_is_vdso(filename));
 	pr_debug("Adding %s %s: %s\n", sbuild_id, filename,
 		 err ? "FAIL" : "Ok");
 	return err;
-- 
2.34.1

Re: [PATCH v5 8/8] perf buildid-cache: recognize vdso when adding files

[Namhyung Kim]

Hello,

On Tue, Jul 02, 2024 at 12:18:37PM +0800, Changbin Du wrote:
> Identify vdso by file name matching. The vdso objects have name
> as vdso[32,64].so[.dbg].
> 
> $ perf buildid-cache -a /work/linux/arch/x86/entry/vdso/vdso64.so.dbg
> 
> Without this change, adding vdso using above command actually will never
> be used.

Can we handle /tmp/perf-vdso.so-XXXXXX too?

Thanks,
Namhyung

> 
> Signed-off-by: Changbin Du <changbin.du@huawei.com>
> ---
>  tools/perf/builtin-buildid-cache.c | 26 +++++++++++++++++++++++++-
>  1 file changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/builtin-buildid-cache.c b/tools/perf/builtin-buildid-cache.c
> index b0511d16aeb6..8edea9044a65 100644
> --- a/tools/perf/builtin-buildid-cache.c
> +++ b/tools/perf/builtin-buildid-cache.c
> @@ -172,6 +172,30 @@ static int build_id_cache__add_kcore(const char *filename, bool force)
>  	return 0;
>  }
>  
> +static bool filename_is_vdso(const char *filename)
> +{
> +	char *fname, *bname;
> +	static const char * const vdso_names[] = {
> +		"vdso.so", "vdso32.so", "vdso64.so", "vdsox32.so"
> +	};
> +
> +	fname = strdup(filename);
> +	if (!fname) {
> +		pr_err("no mememory\n");
> +		return false;
> +	}
> +
> +	bname = basename(fname);
> +	if (!bname)
> +		return false;
> +
> +	for (unsigned int i = 0; i < ARRAY_SIZE(vdso_names); i++) {
> +		if (!strncmp(bname, vdso_names[i], strlen(vdso_names[i])))
> +			return true;
> +	}
> +	return false;
> +}
> +
>  static int build_id_cache__add_file(const char *filename, struct nsinfo *nsi)
>  {
>  	char sbuild_id[SBUILD_ID_SIZE];
> @@ -189,7 +213,7 @@ static int build_id_cache__add_file(const char *filename, struct nsinfo *nsi)
>  
>  	build_id__sprintf(&bid, sbuild_id);
>  	err = build_id_cache__add_s(sbuild_id, filename, nsi,
> -				    false, false);
> +				    false, filename_is_vdso(filename));
>  	pr_debug("Adding %s %s: %s\n", sbuild_id, filename,
>  		 err ? "FAIL" : "Ok");
>  	return err;
> -- 
> 2.34.1
> 

Re: [PATCH v5 8/8] perf buildid-cache: recognize vdso when adding files

[duchangbin]

Hi, Namhyung,
On Wed, Jul 03, 2024 at 04:18:26PM -0700, Namhyung Kim wrote:
> Hello,
> 
> On Tue, Jul 02, 2024 at 12:18:37PM +0800, Changbin Du wrote:
> > Identify vdso by file name matching. The vdso objects have name
> > as vdso[32,64].so[.dbg].
> > 
> > $ perf buildid-cache -a /work/linux/arch/x86/entry/vdso/vdso64.so.dbg
> > 
> > Without this change, adding vdso using above command actually will never
> > be used.
> 
> Can we handle /tmp/perf-vdso.so-XXXXXX too?
>
This temporary path is already handled by perf_session__cache_build_ids()->...->dso__cache_build_id().

> Thanks,
> Namhyung
> 
> > 
> > Signed-off-by: Changbin Du <changbin.du@huawei.com>
> > ---
> >  tools/perf/builtin-buildid-cache.c | 26 +++++++++++++++++++++++++-
> >  1 file changed, 25 insertions(+), 1 deletion(-)
> > 
> > diff --git a/tools/perf/builtin-buildid-cache.c b/tools/perf/builtin-buildid-cache.c
> > index b0511d16aeb6..8edea9044a65 100644
> > --- a/tools/perf/builtin-buildid-cache.c
> > +++ b/tools/perf/builtin-buildid-cache.c
> > @@ -172,6 +172,30 @@ static int build_id_cache__add_kcore(const char *filename, bool force)
> >  	return 0;
> >  }
> >  
> > +static bool filename_is_vdso(const char *filename)
> > +{
> > +	char *fname, *bname;
> > +	static const char * const vdso_names[] = {
> > +		"vdso.so", "vdso32.so", "vdso64.so", "vdsox32.so"
> > +	};
> > +
> > +	fname = strdup(filename);
> > +	if (!fname) {
> > +		pr_err("no mememory\n");
> > +		return false;
> > +	}
> > +
> > +	bname = basename(fname);
> > +	if (!bname)
> > +		return false;
> > +
> > +	for (unsigned int i = 0; i < ARRAY_SIZE(vdso_names); i++) {
> > +		if (!strncmp(bname, vdso_names[i], strlen(vdso_names[i])))
> > +			return true;
> > +	}
> > +	return false;
> > +}
> > +
> >  static int build_id_cache__add_file(const char *filename, struct nsinfo *nsi)
> >  {
> >  	char sbuild_id[SBUILD_ID_SIZE];
> > @@ -189,7 +213,7 @@ static int build_id_cache__add_file(const char *filename, struct nsinfo *nsi)
> >  
> >  	build_id__sprintf(&bid, sbuild_id);
> >  	err = build_id_cache__add_s(sbuild_id, filename, nsi,
> > -				    false, false);
> > +				    false, filename_is_vdso(filename));
> >  	pr_debug("Adding %s %s: %s\n", sbuild_id, filename,
> >  		 err ? "FAIL" : "Ok");
> >  	return err;
> > -- 
> > 2.34.1
> > 
> 

-- 
Cheers,
Changbin Du

Re: [PATCH v5 8/8] perf buildid-cache: recognize vdso when adding files

[kernel test robot]

Hello,

kernel test robot noticed "perf-sanity-tests.build_id_cache_operations.fail" on:

commit: f715cdcabbfb1d34e86b0a2cbb700d690308d724 ("[PATCH v5 8/8] perf buildid-cache: recognize vdso when adding files")
url: https://github.com/intel-lab-lkp/linux/commits/Changbin-Du/perf-support-specify-vdso-path-in-cmdline/20240702-122209
base: https://git.kernel.org/cgit/linux/kernel/git/perf/perf-tools-next.git perf-tools-next
patch link: https://lore.kernel.org/all/20240702041837.5306-9-changbin.du@huawei.com/
patch subject: [PATCH v5 8/8] perf buildid-cache: recognize vdso when adding files

in testcase: perf-sanity-tests
version: 
with following parameters:

	perf_compiler: gcc



compiler: gcc-13
test machine: 224 threads 2 sockets Intel(R) Xeon(R) Platinum 8480+ (Sapphire Rapids) with 256G memory

(please refer to attached dmesg/kmsg for entire log/backtrace)



If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202407242254.98492680-oliver.sang@intel.com



2024-07-18 05:11:32 sudo /usr/src/linux-perf-x86_64-rhel-8.3-bpf-f715cdcabbfb1d34e86b0a2cbb700d690308d724/tools/perf/perf test 78
 78: build id cache operations                                       : FAILED!



The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20240724/202407242254.98492680-oliver.sang@intel.com



-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH v5 0/8] perf: support specify vdso path in cmdline

[duchangbin]

Hello, are there any more comments about this revision?

On Tue, Jul 02, 2024 at 12:18:29PM +0800, Changbin Du wrote:
> The vdso dumped from process memory (in buildid-cache) lacks debugging
> info. To annotate vdso symbols with source lines we need a debugging
> version.
> 
> For x86, we can find them from your local build as
> 'arch/x86/entry/vdso/vdso{32,64}.so.dbg'. Or they may resides in
> '/lib/modules/<version>/vdso/vdso{32,64}.so' on Ubuntu. But notice that the
> builid has to match. 
> 
> If user doesn't specify the path, perf will search them internally as long
> as vmlinux when recording samples. The searched debugging vdso will add to
> buildid cache.
> 
> Below samples are captured on my local build kernel. perf succesfully
> find debugging version vdso and we can annotate with source without
> specifying vdso path.
> 
> $ sudo perf record -a
> $ sudo perf report --objdump=llvm-objdump
> 
> Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
> __vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
> Percent│       movq    -48(%rbp),%rsi
>        │       testq   %rax,%rax
>        │     ;               return vread_hvclock();
>        │       movq    %rax,%rdx
>        │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
>        │     ↑ js      eb
>        │     ↑ jmp     74
>        │     ;               ts->tv_sec = vdso_ts->sec;
>   0.02 │147:   leaq    2(%rbx),%rax
>        │       shlq    $4, %rax
>        │       addq    %r10,%rax
>        │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
>   9.38 │152:   movl    (%r10),%ecx
> 
> When doing cross platform analysis, we need to specify the vdso path if
> we are interested in its symbols. At most two vdso can be given. Also you
> can pack your buildid cache with perf-archive if the debugging vdso can be
> found on the sampled machine.
> 
> $ sudo perf report --objdump=llvm-objdump \
>       --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
> 
> I also improved perf-buildid-cache command recognize vdso when adding files, then
> place it at correct place.
> 
> v5:
>   - Searching the vdso in record stage instead of report. So the debugging
>     vdso will be in build-id cache. This is friendly for cross-machine analysis.
>   - Improve perf-buildid-cache command recognize vdso when adding files
> v4:
>   - split the refactoring from the actual change.
> v3:
>   - update documentation.
> v2:
>   - now search vdso automatically as long as vmlinux, as suggested by Adrian.
>   - remove change 'prefer symsrc_filename for filename'.
> 
> 
> Changbin Du (8):
>   perf: support specify vdso path in cmdline
>   perf: disasm: refactor function dso__disassemble_filename
>   perf: disasm: use build_id_path if fallback failed
>   perf: build-id: name debugging vdso as "debug"
>   perf: symbol: generalize vmlinux path searching
>   perf: build-id: try to search debugging vdso and add to cache
>   perf: disasm: prefer debugging files in build-id cache
>   perf buildid-cache: recognize vdso when adding files
> 
>  tools/perf/Documentation/perf-annotate.txt |   3 +
>  tools/perf/Documentation/perf-c2c.txt      |   3 +
>  tools/perf/Documentation/perf-inject.txt   |   3 +
>  tools/perf/Documentation/perf-report.txt   |   3 +
>  tools/perf/Documentation/perf-script.txt   |   3 +
>  tools/perf/Documentation/perf-top.txt      |   3 +
>  tools/perf/builtin-annotate.c              |   2 +
>  tools/perf/builtin-buildid-cache.c         |  26 ++-
>  tools/perf/builtin-c2c.c                   |   2 +
>  tools/perf/builtin-inject.c                |   2 +
>  tools/perf/builtin-report.c                |   2 +
>  tools/perf/builtin-script.c                |   2 +
>  tools/perf/builtin-top.c                   |   2 +
>  tools/perf/util/build-id.c                 |  57 +++++-
>  tools/perf/util/disasm.c                   | 131 ++++++++-----
>  tools/perf/util/machine.c                  |   4 +-
>  tools/perf/util/symbol.c                   | 209 ++++++++++++++++-----
>  tools/perf/util/symbol.h                   |   9 +-
>  tools/perf/util/symbol_conf.h              |   5 +
>  19 files changed, 359 insertions(+), 112 deletions(-)
> 
> -- 
> 2.34.1
> 

-- 
Cheers,
Changbin Du