From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965393Ab0COQXd (ORCPT ); Mon, 15 Mar 2010 12:23:33 -0400 Received: from mail-fx0-f219.google.com ([209.85.220.219]:62601 "EHLO mail-fx0-f219.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965361Ab0COQXb (ORCPT ); Mon, 15 Mar 2010 12:23:31 -0400 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=wOpjIyT9NO073TS+psogyILgW1XcMpFpvVPvlg5Z0xBUxG7ZElt/jdAbZgPNdlJqxs 9h05CXpQvHxTM9hwO7Nbhw6usu+6TRHQIqSE0+CM3gSWKBZjc5N3pVJrMUjM3xyf7nLf nmKXmtQ/cZdGi4y1yF69WnRethIcZna4SDOAw= Message-ID: <4B9E5EFF.1080308@gmail.com> Date: Mon, 15 Mar 2010 18:23:27 +0200 From: =?UTF-8?B?VMO2csO2ayBFZHdpbg==?= User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20091109) MIME-Version: 1.0 To: Ingo Molnar CC: Thomas Gleixner , "H. Peter Anvin" , Peter Zijlstra , Paul Mackerras , x86@kernel.org, linux-kernel@vger.kernel.org Subject: Re: fix callgraphs of 32-bit processes on 64-bit kernels. References: <1268667260-5505-1-git-send-email-edwintorok@gmail.com> In-Reply-To: <1268667260-5505-1-git-send-email-edwintorok@gmail.com> X-Enigmail-Version: 0.95.0 OpenPGP: id=5379965D Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/15/2010 05:34 PM, Török Edwin wrote: > Hi, > > Callgraph profiling 32-bit apps on a 64-bit kernel doesn't work. > The reason is that perf_callchain_user tries to read a stackframe with 64-bit > pointers, which is wrong for a 32-bit process. > > This patch fixes that, and I am almost able to get nice callgraph profiles > from 32-bit apps now! (except for some problems with perf itself when tracing > kernel modules, see [1]) > > Page-faults can be traced nicely (sid-ia32 is a 32-bit chroot): > > $ sudo perf record -e page-faults -f -g /home/edwin/sid-ia32/usr/bin/glxgears > $ sudo perf report > ... > 45.33% libc-2.10.2.so [.] __GI_memcpy > | > --- __GI_memcpy > _mesa_BufferDataARB > _mesa_meta_Clear > radeonUserClear > r700Clear > _mesa_Clear > 0x8049367 > 0x804a6ba > __libc_start_main > 0x8049111 > > 16.96% libc-2.10.2.so [.] __GI_memset > | > --- __GI_memset > _tnl_init_vertices > _swsetup_CreateContext > r600CreateContext > driCreateNewContext > dri2CreateNewContext > 0xf77ab7dd > 0xf7783c67 > 0xf778514c > 0x804974f > 0x804a33d > __libc_start_main > 0x8049111 > > And CPU cycles can be traced too in userspace: > $ sudo perf record -f -g /home/edwin/sid-ia32/usr/bin/glxgears > $ sudo perf report --sort comm,dso > [...] > 44.97% glxgears r600_dri.so > | > |--5.85%-- r700SendSPIState > | radeonEmitState > | r700DrawPrims > | | > | |--95.45%-- vbo_save_playback_vertex_list > | | execute_list > | | _mesa_CallList > | | neutral_CallList > | | | > | | |--38.10%-- 0x80494a8 > | | | 0x804a6ba > | | | __libc_start_main > | | | 0x8049111 > [....] > 40.00% glxgears [kernel] > | > |--3.14%-- copy_user_generic_string > | | > | |--71.70%-- 0xffffffffa01b4493 > | | 0xffffffffa01b7c0b > | | 0xffffffffa018b45b > | | 0xffffffffa00ca927 > | | 0xffffffffa01c524e > | | compat_sys_ioctl > | | sysenter_dispatch > | | 0xf77ca430 > | | drmCommandWriteRead > | | 0xf74d7ab5 > | | 0xf74d89a4 > | | rcommonFlushCmdBufLocked > | | rcommonFlushCmdBuf > | | radeonFlush > | | _mesa_flush > | | _mesa_Flush > | | 0xf775f270 > | | 0x804a6d5 > | | __libc_start_main > | | 0x8049111 > | | > | |--15.09%-- 0xffffffffa01c524e > | | compat_sys_ioctl > | | sysenter_dispatch > | | 0xf77ca430 > | | drmCommandWriteRead > > [1] But there is a problem with the perf tool: it can't trace addresses in > kernel modules. This is a problem regardless if the traced app is 32-bit or > 64-bit; and regardless if I do callgraph profiling or not. > See the above trace, where the kernel addresses have all ffffffffa0* without a > symbol name. > > If I look at /proc/kallsyms I can guess the symbols, for example > 0xffffffffa01b4493 is probably this one: > ffffffffa01b4411 t r600_cs_packet_parse [radeon] > > If I record/report without callgraph its the same problem: > [...] > 24.01% glxgears [kernel] [k] 0xffffffffa01b4ee9 > 3.96% glxgears libdrm_radeon.so.1.0.0 [.] cs_gem_write_reloc > 3.53% glxgears r600_dri.so [.] r700SendSPIState > 2.77% glxgears r600_dri.so [.] r700DrawPrims > 1.99% glxgears r600_dri.so [.] r700SendVSConsts > > Kernel symbol for 0xffffffffa01b4ee9 is not shown, I can guess it is this one > (hey it was an exact match!): > ffffffffa01b4ee9 t r600_packet3_check [radeon] > > It would be good if perf knew how to lookup symbols in kernel modules! BTW perf report -m -k /home/edwin/builds/linux-2.6/vmlinux doesn't show the symbols either. > > Best regards, > --Edwin