From: Namhyung Kim <namhyung@kernel.org>
To: 沈郑东 <shenzhengdong23@mails.ucas.ac.cn>
Cc: 吴雨薇 <wuyuwei221@mails.ucas.ac.cn>,
linux-perf-users@vger.kernel.org, 于佳耕老师 <jiageng08@iscas.ac.cn>,
yangwang@iscas.ac.cn
Subject: Re: Re: Kernel functions showing only addresses in `perf` output
Date: Sun, 28 Jul 2024 18:34:38 -0700 [thread overview]
Message-ID: <Zqbxrpnt_7wZoscu@google.com> (raw)
In-Reply-To: <5dfad01b.be01.190ecf305c3.Coremail.shenzhengdong23@mails.ucas.ac.cn>
On Fri, Jul 26, 2024 at 10:51:08AM +0800, 沈郑东 wrote:
>
>
>
> > -----原始邮件-----
> > 发件人: "Namhyung Kim" <namhyung@kernel.org>
> > 发送时间: 2024-07-26 09:37:05 (星期五)
> > 收件人: 吴雨薇 <wuyuwei221@mails.ucas.ac.cn>
> > 抄送: linux-perf-users@vger.kernel.org, 于佳耕老师 <jiageng08@iscas.ac.cn>, shenzhengdong23@mails.ucas.ac.cn
> > 主题: Re: Kernel functions showing only addresses in `perf` output
> >
> > Hello,
> >
> > On Wed, Jul 24, 2024 at 07:11:27PM +0800, 吴雨薇 wrote:
> > > Kernel functions are only showing addresses and not symbols in the sampling output from the `perf` tool.
> > >
> > > Environment Information
> > > - Hardware: SG2042
> > > - OS Version: openEuler 23.09
> > > - Kernel Version: Linux openeuler-riscv64 6.6.0+ #1 SMP Wed Apr 3 15:53:06 CST 2024 riscv64 riscv64 riscv64 GNU/Linux
> > > - Perf Version: 6.6.g04f0a6491967
> > > - Kernel Compilation Options:
> > > The following kernel configuration options have been enabled:
> > > CONFIG_PERF_EVENTS
> > > CONFIG_KALLSYMS
> > > CONFIG_KALLSYMS_ALL
> > > CONFIG_FRAME_POINTER
> > > CONFIG_STACKTRACE
> > > CONFIG_DEBUG_INFO
> > >
> > > CONFIG_DEBUG_KERNEL
> > >
> > >
> > >
> > > Steps to Reproduce
> > >
> > > 1. Download Kernel Source Code:
> > > Clone the kernel source code repository on the Lichee Pi 4A board
> > >
> > > 2. Install Dependencies:
> > > Install all necessary development libraries and tools required by `perf`:
> > > sudo yum install elfutils-libelf-devel -y
> > > sudo yum install perl-devel -y
> > > sudo yum install numactl-devel -y
> > > sudo yum install libbabeltrace-devel -y
> > > sudo yum install binutils-devel -y
> > > sudo yum install genwqe-zlib-static -y
> > > sudo yum install elfutils-devel -y
> > > sudo yum install slang-devel -y
> > > sudo yum install zstd-devel -y
> > > sudo yum install xz-devel -y
> > > sudo yum install libcap-devel -y
> > > sudo yum install audit-devel audit-libs -y
> > > sudo yum install openssl-devel -y
> > > sudo yum install gtk2-devel -y
> > > sudo yum install systemtap-sdt-devel -y
> > > sudo yum install libtraceevent-devel -y
> > > sudo yum install libpfm -y
> > > sudo yum install ncurses-devel -y
> > > sudo yum install java-1.8.0-openjdk-devel -y
> > >
> > > 3. Compile `perf`:
> > > cd riscv-kernel/tools/perf
> > > make ARCH=riscv WERROR=0 -j4
> > >
> > > 4. Install `perf`:
> > > sudo cp ./perf /usr/bin/
> > >
> > > 5. Download `hackbench`
> > >
> > > 6. Compile `hackbench`:
> > > gcc -static -O0 -g -fno-omit-frame-pointer hackbench.c -o hackbench -lpthread
> > >
> > > 7. Run Tests with `perf`:
> > > Record performance data with `perf` and generate the report:
> > > sudo perf record -e cpu-cycles -g ./hackbench
> > > sudo perf report --no-children -G > perf_hackbench.txt
> > >
> > >
> > > Issue
> > > In the `perf_hackbench.txt` output, you can see that kernel functions show addresses, while user mode functions show function symbol names
> > > 96.22% hackbench [unknown] [k] 0xffffffff80cfb2ac
> > >
> > > Potential Cause：
> > > It seems that `perf` might be failing to resolve addresses to symbols because it cannot find the addresses in `/proc/kallsyms`.
> >
> > Can you please run `perf script --show-mmap-events` ?
> > I'd like to see if it has proper mappings in the data file.
> >
> > Thanks,
> > Namhyung
> Hello, I have run 'perf script --show-mmap-eventd' again. The results are attached.
Thanks, I saw the first few lines are like below:
swapper 0 0.000000: PERF_RECORD_MMAP -1/0: [0xffffffff80002000(0x2570600) @ 0xffffffff80002000]: x [kernel.kallsyms]_text
swapper 0 0.000000: PERF_RECORD_MMAP -1/0: [0xffffffff029d7000(0x1e000) @ 0]: x /lib/modules/6.6.0-27.0.0.31.oe2403.riscv64/kernel/net/netfilter/nfnetlink.ko.xz
swapper 0 0.000000: PERF_RECORD_MMAP -1/0: [0xffffffff029f8000(0x185000) @ 0]: x /lib/modules/6.6.0-27.0.0.31.oe2403.riscv64/kernel/fs/fuse/fuse.ko.xz
swapper 0 0.000000: PERF_RECORD_MMAP -1/0: [0xffffffff02b81000(0x18000) @ 0]: x /lib/modules/6.6.0-27.0.0.31.oe2403.riscv64/kernel/drivers/uio/uio.ko.xz
swapper 0 0.000000: PERF_RECORD_MMAP -1/0: [0xffffffff02b99000(0x9000) @ 0]: x /lib/modules/6.6.0-27.0.0.31.oe2403.riscv64/kernel/drivers/uio/uio_pdrv_genirq.ko.xz
swapper 0 0.000000: PERF_RECORD_MMAP -1/0: [0xffffffff02b9e000(0x25000) @ 0]: x /lib/modules/6.6.0-27.0.0.31.oe2403.riscv64/kernel/drivers/rtc/rtc-ds1307.ko.xz
...
It looks good, so lines like below should find the [kernel.kallsyms]
mapping.
0.29% hackbench [unknown] [k] 0xffffffff800555c0
0.07% hackbench [unknown] [k] 0xffffffff80cfc74c
0.06% hackbench [unknown] [k] 0xffffffff802afde8
0.06% hackbench [unknown] [k] 0xffffffff80cfafbe
0.05% hackbench [unknown] [k] 0xffffffff802d8a36
Maybe it's because the module addresses are below the kernel mapping and
IIRC ARM had a similar issue and we have the fix now. Have you tried
the latest version of perf tool? I guess you just used v6.6..
Thanks,
Namhyung
prev parent reply other threads:[~2024-07-29 1:34 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-07-24 11:11 Kernel functions showing only addresses in `perf` output 吴雨薇
2024-07-26 1:37 ` Namhyung Kim
2024-07-26 2:51 ` 沈郑东
2024-07-29 1:34 ` Namhyung Kim [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Zqbxrpnt_7wZoscu@google.com \
--to=namhyung@kernel.org \
--cc=jiageng08@iscas.ac.cn \
--cc=linux-perf-users@vger.kernel.org \
--cc=shenzhengdong23@mails.ucas.ac.cn \
--cc=wuyuwei221@mails.ucas.ac.cn \
--cc=yangwang@iscas.ac.cn \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.