From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753651AbbJZNZo (ORCPT ); Mon, 26 Oct 2015 09:25:44 -0400 Received: from mga11.intel.com ([192.55.52.93]:15850 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752357AbbJZNZn (ORCPT ); Mon, 26 Oct 2015 09:25:43 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,201,1444719600"; d="scan'208,223";a="819618190" Subject: Re: [BUG] reporting stuck on s390 in dso__split_kallsyms_for_kcore To: Jiri Olsa References: <20151026130154.GA4400@krava.local> Cc: acme@kernel.org, Peter Zijlstra , Ingo Molnar , Namhyung Kim , Kan Liang , Andi Kleen , linux-kernel@vger.kernel.org From: Adrian Hunter Organization: Intel Finland Oy, Registered Address: PL 281, 00181 Helsinki, Business Identity Code: 0357606 - 4, Domiciled in Helsinki Message-ID: <562E2919.5000601@intel.com> Date: Mon, 26 Oct 2015 15:22:33 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151026130154.GA4400@krava.local> Content-Type: multipart/mixed; boundary="------------030304050608050301080801" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a multi-part message in MIME format. --------------030304050608050301080801 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit On 26/10/15 15:01, Jiri Olsa wrote: > hi, > I'm getting stuck buildid-list command on s390 > > seems like the kcore code gets stuck with inseting > into rbtree while iterating it.. > > I was able to fix it with patch below, bu I'm not sure it's the > correct fix because the kcore maps magic is beyond me so far ;-) > > please check attached backtrace and patch Your fix looks correct to me. > > > (gdb) r buildid-list -i perf.data --with-hits By the way, there are problems with buildid-list, I have the patches attached but haven't had a chance to send them. If you find are missing buildids you might want to try them. > Starting program: /root/linux/tools/perf/./perf buildid-list -i perf.data --with-hits > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > Detaching after fork from child process 56723. > ^C > Program received signal SIGINT, Interrupt. > rb_next (node=0x81573930, node@entry=0x81551840) at ../lib/rbtree.c:451 > 451 node=node->rb_left; > Missing separate debuginfos, use: debuginfo-install audit-libs-2.4.1-5.el7.s390x bzip2-libs-1.0.6-13.el7.s390x elfutils-libelf-0.163-3.el7.s390x elfutils-libs-0.163-3.el7.s390x glibc-2.17-105.el7.s390x nss-softokn-freebl-3.16.2.3-13.el7_1.s390x perl-libs-5.16.3-286.el7.s390x python-libs-2.7.5-34.el7.s390x slang-2.2.4-11.el7.s390x xz-libs-5.1.2-12alpha.el7.s390x zlib-1.2.7-15.el7.s390x > (gdb) bt > #0 rb_next (node=0x81573930, node@entry=0x81551840) at ../lib/rbtree.c:451 > #1 0x00000000800a7da0 in dso__split_kallsyms_for_kcore (dso=dso@entry=0x812e69c0, map=map@entry=0x812e6fb0, filter=filter@entry=0x0) > at util/symbol.c:668 > #2 0x00000000800a9ac6 in dso__load_kallsyms (dso=dso@entry=0x812e69c0, filename=filename@entry=0x81353030 "/proc/kallsyms", > map=map@entry=0x812e6fb0, filter=filter@entry=0x0) at util/symbol.c:1289 > #3 0x00000000800aa0c4 in dso__load_kernel_sym (dso=dso@entry=0x812e69c0, map=map@entry=0x812e6fb0, filter=filter@entry=0x0) > at util/symbol.c:1783 > #4 0x00000000800aa208 in dso__load (dso=0x812e69c0, map=map@entry=0x812e6fb0, filter=filter@entry=0x0) at util/symbol.c:1420 > #5 0x00000000800bb2aa in map__load (map=0x812e6fb0, filter=0x0) at util/map.c:289 > #6 0x0000000080087c9e in thread__find_addr_map (thread=, cpumode=cpumode@entry=1 '\001', type=type@entry=MAP__FUNCTION, > addr=, al=al@entry=0x3ffffffe118) at util/event.c:969 > #7 0x0000000080081908 in build_id__mark_dso_hit (tool=, event=0x3fffd5aa568, sample=0x3ffffffe410, evsel=, > machine=) at util/build-id.c:41 > #8 0x00000000800bdfec in perf_evlist__deliver_sample (evlist=evlist@entry=0x812e5d60, > tool=tool@entry=0x8024b240 , event=event@entry=0x3fffd5aa568, sample=sample@entry=0x3ffffffe410, > evsel=evsel@entry=0x812e6770, machine=0x812e5ab8) at util/session.c:1039 > #9 0x00000000800be156 in machines__deliver_event (machines=machines@entry=0x812e5ab8, evlist=0x812e5d60, > event=event@entry=0x3fffd5aa568, sample=sample@entry=0x3ffffffe410, tool=tool@entry=0x8024b240 , > file_offset=136552) at util/session.c:1076 > #10 0x00000000800be3f2 in perf_session__deliver_event (session=session@entry=0x812e59e0, event=event@entry=0x3fffd5aa568, > sample=sample@entry=0x3ffffffe410, tool=tool@entry=0x8024b240 , file_offset=) > at util/session.c:1133 > #11 0x00000000800c0146 in perf_session__process_event (session=session@entry=0x812e59e0, event=event@entry=0x3fffd5aa568, > file_offset=file_offset@entry=136552) at util/session.c:1298 > #12 0x00000000800c0746 in __perf_session__process_events (session=session@entry=0x812e59e0, data_offset=, > data_size=, file_size=1164232, file_size@entry=1166128) at util/session.c:1633 > #13 0x00000000800c0c2c in perf_session__process_events (session=session@entry=0x812e59e0) at util/session.c:1683 > #14 0x000000008002ae5e in perf_session__list_build_ids (force=, with_hits=true) at builtin-buildid-list.c:82 > #15 0x000000008002b078 in cmd_buildid_list (argc=, argv=, prefix=) > at builtin-buildid-list.c:115 > #16 0x000000008007c71a in run_builtin (p=p@entry=0x8024aab8 , argc=argc@entry=4, argv=0x3fffffff090) at perf.c:385 > #17 0x000000008007c95e in handle_internal_command (argc=, argv=) at perf.c:445 > #18 0x000000008007c9e4 in run_argv (argcp=argcp@entry=0x3ffffffedd4, argv=argv@entry=0x3ffffffedc8) at perf.c:489 > #19 0x000000008007cca2 in main (argc=4, argv=0x3fffffff090) at perf.c:606 > > > thanks, > jirka > > > --- > diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c > index e7bf0c4..b0d2fb2 100644 > --- a/tools/perf/util/symbol.c > +++ b/tools/perf/util/symbol.c > @@ -680,7 +680,7 @@ static int dso__split_kallsyms_for_kcore(struct dso *dso, struct map *map, > pos->start -= curr_map->start - curr_map->pgoff; > if (pos->end) > pos->end -= curr_map->start - curr_map->pgoff; > - if (curr_map != map) { > + if (curr_map->dso != map->dso) { > rb_erase_init(&pos->rb_node, root); > symbols__insert( > &curr_map->dso->symbols[curr_map->type], > --------------030304050608050301080801 Content-Type: text/x-patch; name="0010-perf-tools-Fix-dso-lookup-by-long-name-and-missing-b.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0010-perf-tools-Fix-dso-lookup-by-long-name-and-missing-b.pa"; filename*1="tch" >>From 23033db810aa8416069e68b0389eac0a33b6bb3d Mon Sep 17 00:00:00 2001 From: Adrian Hunter Date: Sat, 24 Oct 2015 15:00:45 +0300 Subject: [PATCH 10/13] perf tools: Fix dso lookup by long name and missing buildids Commit 4598a0a6d22f ("perf symbols: Improve DSO long names lookup speed with rbtree") Added a tree to lookup dsos by long name. That tree gets corrupted whenever a dso long name is changed because the tree is not updated. One effect of that is buildid-list does not work with the 'with-hits' option because dso lookup fails and results in two structs for the same dso. The first has the buildid but no hits, the second has hits but no buildid. e.g. Before: $ tools/perf/perf record ls arch certs CREDITS Documentation firmware include ipc Kconfig lib Makefile net REPORTING-BUGS scripts sound usr block COPYING crypto drivers fs init Kbuild kernel MAINTAINERS mm README samples security tools virt [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (11 samples) ] $ tools/perf/perf buildid-list 574da826c66538a8d9060d393a8866289bd06005 [kernel.kallsyms] 30c94dc66a1fe95180c3d68d2b89e576d5ae213c /lib/x86_64-linux-gnu/libc-2.19.so $ tools/perf/perf buildid-list -H 574da826c66538a8d9060d393a8866289bd06005 [kernel.kallsyms] 0000000000000000000000000000000000000000 /lib/x86_64-linux-gnu/libc-2.19.so After: $ tools/perf/perf buildid-list -H 574da826c66538a8d9060d393a8866289bd06005 [kernel.kallsyms] 30c94dc66a1fe95180c3d68d2b89e576d5ae213c /lib/x86_64-linux-gnu/libc-2.19.so The fix is to record the root of the tree on the dso so that dso__set_long_name() can update the tree when the long name changes. Signed-off-by: Adrian Hunter --- tools/perf/util/dso.c | 17 +++++++++++++++++ tools/perf/util/dso.h | 1 + tools/perf/util/machine.c | 1 + 3 files changed, 19 insertions(+) diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c index 7c0c08386a1d..425df5c86c9c 100644 --- a/tools/perf/util/dso.c +++ b/tools/perf/util/dso.c @@ -933,6 +933,7 @@ static struct dso *__dso__findlink_by_longname(struct rb_root *root, /* Add new node and rebalance tree */ rb_link_node(&dso->rb_node, parent, p); rb_insert_color(&dso->rb_node, root); + dso->root = root; } return NULL; } @@ -945,15 +946,30 @@ static inline struct dso *__dso__find_by_longname(struct rb_root *root, void dso__set_long_name(struct dso *dso, const char *name, bool name_allocated) { + struct rb_root *root = dso->root; + if (name == NULL) return; if (dso->long_name_allocated) free((char *)dso->long_name); + if (root) { + rb_erase(&dso->rb_node, root); + /* + * __dso__findlink_by_longname() isn't guaranteed to add it + * back, so a clean removal is required here. + */ + RB_CLEAR_NODE(&dso->rb_node); + dso->root = NULL; + } + dso->long_name = name; dso->long_name_len = strlen(name); dso->long_name_allocated = name_allocated; + + if (root) + __dso__findlink_by_longname(root, dso, NULL); } void dso__set_short_name(struct dso *dso, const char *name, bool name_allocated) @@ -1046,6 +1062,7 @@ struct dso *dso__new(const char *name) dso->kernel = DSO_TYPE_USER; dso->needs_swap = DSO_SWAP__UNSET; RB_CLEAR_NODE(&dso->rb_node); + dso->root = NULL; INIT_LIST_HEAD(&dso->node); INIT_LIST_HEAD(&dso->data.open_entry); pthread_mutex_init(&dso->lock, NULL); diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h index fc8db9c764ac..45ec4d0a50ed 100644 --- a/tools/perf/util/dso.h +++ b/tools/perf/util/dso.h @@ -135,6 +135,7 @@ struct dso { pthread_mutex_t lock; struct list_head node; struct rb_node rb_node; /* rbtree node sorted by long name */ + struct rb_root *root; /* root of rbtree that rb_node is in */ struct rb_root symbols[MAP__NR_TYPES]; struct rb_root symbol_names[MAP__NR_TYPES]; struct { diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index 5ef90be2a249..8b303ff20289 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -91,6 +91,7 @@ static void dsos__purge(struct dsos *dsos) list_for_each_entry_safe(pos, n, &dsos->head, node) { RB_CLEAR_NODE(&pos->rb_node); + pos->root = NULL; list_del_init(&pos->node); dso__put(pos); } -- 1.9.1 --------------030304050608050301080801 Content-Type: text/x-patch; name="0011-perf-buildid-list-Requires-ordered-events.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0011-perf-buildid-list-Requires-ordered-events.patch" >>From a0c4be2534707d074c19f9f371aec608b0953c1c Mon Sep 17 00:00:00 2001 From: Adrian Hunter Date: Mon, 26 Oct 2015 11:40:39 +0200 Subject: [PATCH 11/13] perf buildid-list: Requires ordered events perf buildid-list processes events to determine hits (i.e. with-hits option). That may not work if events are not sorted in order. i.e. MMAP events must be processed before the samples that depend on them so that sample processing can 'hit' the DSO to which the MMAP refers. Signed-off-by: Adrian Hunter --- tools/perf/util/build-id.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c index d909459fb54c..217b5a60e2ab 100644 --- a/tools/perf/util/build-id.c +++ b/tools/perf/util/build-id.c @@ -76,6 +76,7 @@ struct perf_tool build_id__mark_dso_hit_ops = { .exit = perf_event__exit_del_thread, .attr = perf_event__process_attr, .build_id = perf_event__process_build_id, + .ordered_events = true, }; int build_id__sprintf(const u8 *build_id, int len, char *bf) -- 1.9.1 --------------030304050608050301080801--