From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933615AbcDLNix (ORCPT ); Tue, 12 Apr 2016 09:38:53 -0400 Received: from mail.kernel.org ([198.145.29.136]:42526 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932221AbcDLNiw (ORCPT ); Tue, 12 Apr 2016 09:38:52 -0400 Date: Tue, 12 Apr 2016 10:38:47 -0300 From: Arnaldo Carvalho de Melo To: Namhyung Kim Cc: Jiri Olsa , David Ahern , Peter Zijlstra , Milian Wolff , Wang Nan , Linux Kernel Mailing List Subject: zeros in frame pointer callchains Message-ID: <20160412133847.GC10972@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Url: http://acmel.wordpress.com User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, Namhyung, while working on supporting callchains in 'perf trace' I noticed something I think we discussed at some point, zeros in frame pointer callchains... Oh well figured it out, its probably userspace not having frame pointers, grrr anyway, keeping the rest of the message, for reference, but now I think we need to have some expressiveness to say that for that particular event we want to use perf_event_attr.exclude_callchain_user... First add a special probe point that 'perf trace' uses, if present: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Then lets ask to see just when this thing hits, no syscall printing whatsoever ('trace' == 'perf trace'), sometimes just a few zeros at the top, sometimes a long series, need to investigate how to chop those off in the kernel, not to use the ring buffer needlessly, will start chopping these in userspace (in perf trace): # trace --no-sys --ev probe:vfs_getname/call-graph=fp/ 34033.533 probe:vfs_getname:(ffffffff8124c06c) pathname="/usr/share/pixmaps") getname_flags+0xfe20009d ([kernel.kallsyms]) user_path_at_empty+0xfe200023 ([kernel.kallsyms]) vfs_fstatat+0xfe200066 ([kernel.kallsyms]) SYSC_newstat+0xfe20002e ([kernel.kallsyms]) sys_newstat+0xfe20000e ([kernel.kallsyms]) entry_SYSCALL_64_fastpath+0xfe20001a ([kernel.kallsyms]) __xstat64+0xffff011e90b98015 (/usr/lib64/libc-2.22.so) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) 36001.681 probe:vfs_getname:(ffffffff8124c06c) pathname="/var/tmp") getname_flags+0xfe20009d ([kernel.kallsyms]) user_path_at_empty+0xfe200023 ([kernel.kallsyms]) vfs_fstatat+0xfe200066 ([kernel.kallsyms]) SYSC_newstat+0xfe20002e ([kernel.kallsyms]) sys_newstat+0xfe20000e ([kernel.kallsyms]) entry_SYSCALL_64_fastpath+0xfe20001a ([kernel.kallsyms]) __xstat64+0xffff00b816a28015 (/usr/lib64/libc-2.22.so) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) 36001.696 probe:vfs_getname:(ffffffff8124c06c) pathname="/var/tmp") getname_flags+0xfe20009d ([kernel.kallsyms]) user_path_at_empty+0xfe200023 ([kernel.kallsyms]) sys_access+0xfe2000b4 ([kernel.kallsyms]) entry_SYSCALL_64_fastpath+0xfe20001a ([kernel.kallsyms]) __GI___access+0xffff00b816a28007 (/usr/lib64/libc-2.22.so) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) [0] ([unknown]) 8254.597 probe:vfs_getname:(ffffffff8124c06c) pathname="/etc/localtime") getname_flags+0xfe20009d ([kernel.kallsyms]) user_path_at_empty+0xfe200023 ([kernel.kallsyms]) vfs_fstatat+0xfe200066 ([kernel.kallsyms]) SYSC_newstat+0xfe20002e ([kernel.kallsyms]) sys_newstat+0xfe20000e ([kernel.kallsyms]) entry_SYSCALL_64_fastpath+0xfe20001a ([kernel.kallsyms]) __xstat64+0xffff0104f5696015 (/usr/lib64/libc-2.22.so) [0] ([unknown]) [0] ([unknown]) 9001.567 probe:vfs_getname:(ffffffff8124c06c) pathname="/etc/firewalld/ipsets") getname_flags+0xfe20009d ([kernel.kallsyms]) user_path_at_empty+0xfe200023 ([kernel.kallsyms]) sys_inotify_add_watch+0xfe200098 ([kernel.kallsyms]) entry_SYSCALL_64_fastpath+0xfe20001a ([kernel.kallsyms]) inotify_add_watch+0xffff008bb9432007 (/usr/lib64/libc-2.22.so) 9450.181 probe:vfs_getname:(ffffffff8124c06c) pathname="/root") getname_flags+0xfe20009d ([kernel.kallsyms]) user_path_at_empty+0xfe200023 ([kernel.kallsyms]) sys_inotify_add_watch+0xfe200098 ([kernel.kallsyms]) entry_SYSCALL_64_fastpath+0xfe20001a ([kernel.kallsyms]) inotify_add_watch+0xffff01b7aa4f2007 (/usr/lib64/libc-2.22.so)