* Re: [BUG] reporting stuck on s390 in dso__split_kallsyms_for_kcore
2015-10-26 13:01 [BUG] reporting stuck on s390 in dso__split_kallsyms_for_kcore Jiri Olsa
@ 2015-10-26 13:22 ` Adrian Hunter
2015-10-26 13:51 ` [PATCH] perf tools: Fix endless loop " Jiri Olsa
0 siblings, 1 reply; 4+ messages in thread
From: Adrian Hunter @ 2015-10-26 13:22 UTC (permalink / raw)
To: Jiri Olsa
Cc: acme, Peter Zijlstra, Ingo Molnar, Namhyung Kim, Kan Liang,
Andi Kleen, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 5410 bytes --]
On 26/10/15 15:01, Jiri Olsa wrote:
> hi,
> I'm getting stuck buildid-list command on s390
>
> seems like the kcore code gets stuck with inseting
> into rbtree while iterating it..
>
> I was able to fix it with patch below, bu I'm not sure it's the
> correct fix because the kcore maps magic is beyond me so far ;-)
>
> please check attached backtrace and patch
Your fix looks correct to me.
>
>
> (gdb) r buildid-list -i perf.data --with-hits
By the way, there are problems with buildid-list, I have the patches
attached but haven't had a chance to send them. If you find are missing
buildids you might want to try them.
> Starting program: /root/linux/tools/perf/./perf buildid-list -i perf.data --with-hits
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> Detaching after fork from child process 56723.
> ^C
> Program received signal SIGINT, Interrupt.
> rb_next (node=0x81573930, node@entry=0x81551840) at ../lib/rbtree.c:451
> 451 node=node->rb_left;
> Missing separate debuginfos, use: debuginfo-install audit-libs-2.4.1-5.el7.s390x bzip2-libs-1.0.6-13.el7.s390x elfutils-libelf-0.163-3.el7.s390x elfutils-libs-0.163-3.el7.s390x glibc-2.17-105.el7.s390x nss-softokn-freebl-3.16.2.3-13.el7_1.s390x perl-libs-5.16.3-286.el7.s390x python-libs-2.7.5-34.el7.s390x slang-2.2.4-11.el7.s390x xz-libs-5.1.2-12alpha.el7.s390x zlib-1.2.7-15.el7.s390x
> (gdb) bt
> #0 rb_next (node=0x81573930, node@entry=0x81551840) at ../lib/rbtree.c:451
> #1 0x00000000800a7da0 in dso__split_kallsyms_for_kcore (dso=dso@entry=0x812e69c0, map=map@entry=0x812e6fb0, filter=filter@entry=0x0)
> at util/symbol.c:668
> #2 0x00000000800a9ac6 in dso__load_kallsyms (dso=dso@entry=0x812e69c0, filename=filename@entry=0x81353030 "/proc/kallsyms",
> map=map@entry=0x812e6fb0, filter=filter@entry=0x0) at util/symbol.c:1289
> #3 0x00000000800aa0c4 in dso__load_kernel_sym (dso=dso@entry=0x812e69c0, map=map@entry=0x812e6fb0, filter=filter@entry=0x0)
> at util/symbol.c:1783
> #4 0x00000000800aa208 in dso__load (dso=0x812e69c0, map=map@entry=0x812e6fb0, filter=filter@entry=0x0) at util/symbol.c:1420
> #5 0x00000000800bb2aa in map__load (map=0x812e6fb0, filter=0x0) at util/map.c:289
> #6 0x0000000080087c9e in thread__find_addr_map (thread=<optimized out>, cpumode=cpumode@entry=1 '\001', type=type@entry=MAP__FUNCTION,
> addr=<optimized out>, al=al@entry=0x3ffffffe118) at util/event.c:969
> #7 0x0000000080081908 in build_id__mark_dso_hit (tool=<optimized out>, event=0x3fffd5aa568, sample=0x3ffffffe410, evsel=<optimized out>,
> machine=<optimized out>) at util/build-id.c:41
> #8 0x00000000800bdfec in perf_evlist__deliver_sample (evlist=evlist@entry=0x812e5d60,
> tool=tool@entry=0x8024b240 <build_id__mark_dso_hit_ops>, event=event@entry=0x3fffd5aa568, sample=sample@entry=0x3ffffffe410,
> evsel=evsel@entry=0x812e6770, machine=0x812e5ab8) at util/session.c:1039
> #9 0x00000000800be156 in machines__deliver_event (machines=machines@entry=0x812e5ab8, evlist=0x812e5d60,
> event=event@entry=0x3fffd5aa568, sample=sample@entry=0x3ffffffe410, tool=tool@entry=0x8024b240 <build_id__mark_dso_hit_ops>,
> file_offset=136552) at util/session.c:1076
> #10 0x00000000800be3f2 in perf_session__deliver_event (session=session@entry=0x812e59e0, event=event@entry=0x3fffd5aa568,
> sample=sample@entry=0x3ffffffe410, tool=tool@entry=0x8024b240 <build_id__mark_dso_hit_ops>, file_offset=<optimized out>)
> at util/session.c:1133
> #11 0x00000000800c0146 in perf_session__process_event (session=session@entry=0x812e59e0, event=event@entry=0x3fffd5aa568,
> file_offset=file_offset@entry=136552) at util/session.c:1298
> #12 0x00000000800c0746 in __perf_session__process_events (session=session@entry=0x812e59e0, data_offset=<optimized out>,
> data_size=<optimized out>, file_size=1164232, file_size@entry=1166128) at util/session.c:1633
> #13 0x00000000800c0c2c in perf_session__process_events (session=session@entry=0x812e59e0) at util/session.c:1683
> #14 0x000000008002ae5e in perf_session__list_build_ids (force=<optimized out>, with_hits=true) at builtin-buildid-list.c:82
> #15 0x000000008002b078 in cmd_buildid_list (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>)
> at builtin-buildid-list.c:115
> #16 0x000000008007c71a in run_builtin (p=p@entry=0x8024aab8 <commands+24>, argc=argc@entry=4, argv=0x3fffffff090) at perf.c:385
> #17 0x000000008007c95e in handle_internal_command (argc=<optimized out>, argv=<optimized out>) at perf.c:445
> #18 0x000000008007c9e4 in run_argv (argcp=argcp@entry=0x3ffffffedd4, argv=argv@entry=0x3ffffffedc8) at perf.c:489
> #19 0x000000008007cca2 in main (argc=4, argv=0x3fffffff090) at perf.c:606
>
>
> thanks,
> jirka
>
>
> ---
> diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
> index e7bf0c4..b0d2fb2 100644
> --- a/tools/perf/util/symbol.c
> +++ b/tools/perf/util/symbol.c
> @@ -680,7 +680,7 @@ static int dso__split_kallsyms_for_kcore(struct dso *dso, struct map *map,
> pos->start -= curr_map->start - curr_map->pgoff;
> if (pos->end)
> pos->end -= curr_map->start - curr_map->pgoff;
> - if (curr_map != map) {
> + if (curr_map->dso != map->dso) {
> rb_erase_init(&pos->rb_node, root);
> symbols__insert(
> &curr_map->dso->symbols[curr_map->type],
>
[-- Attachment #2: 0010-perf-tools-Fix-dso-lookup-by-long-name-and-missing-b.patch --]
[-- Type: text/x-patch, Size: 4404 bytes --]
>From 23033db810aa8416069e68b0389eac0a33b6bb3d Mon Sep 17 00:00:00 2001
From: Adrian Hunter <adrian.hunter@intel.com>
Date: Sat, 24 Oct 2015 15:00:45 +0300
Subject: [PATCH 10/13] perf tools: Fix dso lookup by long name and missing
buildids
Commit 4598a0a6d22f ("perf symbols: Improve DSO long names lookup speed with rbtree")
Added a tree to lookup dsos by long name. That tree gets corrupted
whenever a dso long name is changed because the tree is not updated.
One effect of that is buildid-list does not work with the 'with-hits'
option because dso lookup fails and results in two structs for the same
dso. The first has the buildid but no hits, the second has hits but no
buildid. e.g.
Before:
$ tools/perf/perf record ls
arch certs CREDITS Documentation firmware include ipc Kconfig lib Makefile net REPORTING-BUGS scripts sound usr
block COPYING crypto drivers fs init Kbuild kernel MAINTAINERS mm README samples security tools virt
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.012 MB perf.data (11 samples) ]
$ tools/perf/perf buildid-list
574da826c66538a8d9060d393a8866289bd06005 [kernel.kallsyms]
30c94dc66a1fe95180c3d68d2b89e576d5ae213c /lib/x86_64-linux-gnu/libc-2.19.so
$ tools/perf/perf buildid-list -H
574da826c66538a8d9060d393a8866289bd06005 [kernel.kallsyms]
0000000000000000000000000000000000000000 /lib/x86_64-linux-gnu/libc-2.19.so
After:
$ tools/perf/perf buildid-list -H
574da826c66538a8d9060d393a8866289bd06005 [kernel.kallsyms]
30c94dc66a1fe95180c3d68d2b89e576d5ae213c /lib/x86_64-linux-gnu/libc-2.19.so
The fix is to record the root of the tree on the dso so that
dso__set_long_name() can update the tree when the long name changes.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
tools/perf/util/dso.c | 17 +++++++++++++++++
tools/perf/util/dso.h | 1 +
tools/perf/util/machine.c | 1 +
3 files changed, 19 insertions(+)
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 7c0c08386a1d..425df5c86c9c 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -933,6 +933,7 @@ static struct dso *__dso__findlink_by_longname(struct rb_root *root,
/* Add new node and rebalance tree */
rb_link_node(&dso->rb_node, parent, p);
rb_insert_color(&dso->rb_node, root);
+ dso->root = root;
}
return NULL;
}
@@ -945,15 +946,30 @@ static inline struct dso *__dso__find_by_longname(struct rb_root *root,
void dso__set_long_name(struct dso *dso, const char *name, bool name_allocated)
{
+ struct rb_root *root = dso->root;
+
if (name == NULL)
return;
if (dso->long_name_allocated)
free((char *)dso->long_name);
+ if (root) {
+ rb_erase(&dso->rb_node, root);
+ /*
+ * __dso__findlink_by_longname() isn't guaranteed to add it
+ * back, so a clean removal is required here.
+ */
+ RB_CLEAR_NODE(&dso->rb_node);
+ dso->root = NULL;
+ }
+
dso->long_name = name;
dso->long_name_len = strlen(name);
dso->long_name_allocated = name_allocated;
+
+ if (root)
+ __dso__findlink_by_longname(root, dso, NULL);
}
void dso__set_short_name(struct dso *dso, const char *name, bool name_allocated)
@@ -1046,6 +1062,7 @@ struct dso *dso__new(const char *name)
dso->kernel = DSO_TYPE_USER;
dso->needs_swap = DSO_SWAP__UNSET;
RB_CLEAR_NODE(&dso->rb_node);
+ dso->root = NULL;
INIT_LIST_HEAD(&dso->node);
INIT_LIST_HEAD(&dso->data.open_entry);
pthread_mutex_init(&dso->lock, NULL);
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index fc8db9c764ac..45ec4d0a50ed 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -135,6 +135,7 @@ struct dso {
pthread_mutex_t lock;
struct list_head node;
struct rb_node rb_node; /* rbtree node sorted by long name */
+ struct rb_root *root; /* root of rbtree that rb_node is in */
struct rb_root symbols[MAP__NR_TYPES];
struct rb_root symbol_names[MAP__NR_TYPES];
struct {
diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 5ef90be2a249..8b303ff20289 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -91,6 +91,7 @@ static void dsos__purge(struct dsos *dsos)
list_for_each_entry_safe(pos, n, &dsos->head, node) {
RB_CLEAR_NODE(&pos->rb_node);
+ pos->root = NULL;
list_del_init(&pos->node);
dso__put(pos);
}
--
1.9.1
[-- Attachment #3: 0011-perf-buildid-list-Requires-ordered-events.patch --]
[-- Type: text/x-patch, Size: 1095 bytes --]
>From a0c4be2534707d074c19f9f371aec608b0953c1c Mon Sep 17 00:00:00 2001
From: Adrian Hunter <adrian.hunter@intel.com>
Date: Mon, 26 Oct 2015 11:40:39 +0200
Subject: [PATCH 11/13] perf buildid-list: Requires ordered events
perf buildid-list processes events to determine hits
(i.e. with-hits option). That may not work if events
are not sorted in order. i.e. MMAP events must be
processed before the samples that depend on them so
that sample processing can 'hit' the DSO to which
the MMAP refers.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
tools/perf/util/build-id.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
index d909459fb54c..217b5a60e2ab 100644
--- a/tools/perf/util/build-id.c
+++ b/tools/perf/util/build-id.c
@@ -76,6 +76,7 @@ struct perf_tool build_id__mark_dso_hit_ops = {
.exit = perf_event__exit_del_thread,
.attr = perf_event__process_attr,
.build_id = perf_event__process_build_id,
+ .ordered_events = true,
};
int build_id__sprintf(const u8 *build_id, int len, char *bf)
--
1.9.1
^ permalink raw reply related [flat|nested] 4+ messages in thread