* BUG: perf kvm top --callgraph not showing callgraph
@ 2013-06-14 1:04 Cody P Schafer
2013-06-14 1:10 ` David Ahern
0 siblings, 1 reply; 6+ messages in thread
From: Cody P Schafer @ 2013-06-14 1:04 UTC (permalink / raw)
To: linux-perf-users
Hi all, I'm trying to use perf kvm to profile linux early boot.
However, perf kvm top always appears to behave as though it's in
non-callgraph mode.
I dug through the code a bit, and it looks as though
machine__resolve_challchain_sample() is getting called with an
ip_callchain consistently containing only 2 entries:
PERF_CONTEXT_KERNEL followed by PERF_CONTEXT_USER. This makes it
appear that the perf_sample is not getting any callchain info from the
kernel.
System setup:
I'm running qemu as follows:
$ qemu-system-x86_64 -M pc -kernel ./bzImage -drive
file=./rootfs.ext2,if=virtio -append "root=/dev/vda console=ttyS0
earlyprintk=ttyS0 debug numa=fake=2 kmemleak=on extra_nr_node_ids=100"
-smp 8 -nographic -enable-kvm -s -m 64G
And perf kvm top as follows:
$ sudo ./perf kvm --guestvmlinux ~/vmlinux top --call-graph
The host `uname -a` is
Linux pizza 3.7.10-gentoo-r1 #1 SMP Thu Apr 25 15:11:18 PDT 2013
x86_64 Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz GenuineIntel
GNU/Linux
(And I've gotten the same results with 3.8.0-22-generic #33~precise1-Ubuntu).
The guest kernel is v3.10-rc5 with some additional patches mostly to mm/*
The version of the perf tool is 3.9.rc8.g25e33e (the version in
linus's tree today).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BUG: perf kvm top --callgraph not showing callgraph
2013-06-14 1:04 BUG: perf kvm top --callgraph not showing callgraph Cody P Schafer
@ 2013-06-14 1:10 ` David Ahern
2013-06-14 21:53 ` Cody P Schafer
0 siblings, 1 reply; 6+ messages in thread
From: David Ahern @ 2013-06-14 1:10 UTC (permalink / raw)
To: Cody P Schafer; +Cc: linux-perf-users
On 6/13/13 7:04 PM, Cody P Schafer wrote:
> Hi all, I'm trying to use perf kvm to profile linux early boot.
>
> However, perf kvm top always appears to behave as though it's in
> non-callgraph mode.
During event collection perf kernel side does not walk the guest
callchain. I believe it only collects IP.
David
>
> I dug through the code a bit, and it looks as though
> machine__resolve_challchain_sample() is getting called with an
> ip_callchain consistently containing only 2 entries:
> PERF_CONTEXT_KERNEL followed by PERF_CONTEXT_USER. This makes it
> appear that the perf_sample is not getting any callchain info from the
> kernel.
>
> System setup:
>
> I'm running qemu as follows:
>
> $ qemu-system-x86_64 -M pc -kernel ./bzImage -drive
> file=./rootfs.ext2,if=virtio -append "root=/dev/vda console=ttyS0
> earlyprintk=ttyS0 debug numa=fake=2 kmemleak=on extra_nr_node_ids=100"
> -smp 8 -nographic -enable-kvm -s -m 64G
>
> And perf kvm top as follows:
> $ sudo ./perf kvm --guestvmlinux ~/vmlinux top --call-graph
>
> The host `uname -a` is
> Linux pizza 3.7.10-gentoo-r1 #1 SMP Thu Apr 25 15:11:18 PDT 2013
> x86_64 Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz GenuineIntel
> GNU/Linux
>
> (And I've gotten the same results with 3.8.0-22-generic #33~precise1-Ubuntu).
>
> The guest kernel is v3.10-rc5 with some additional patches mostly to mm/*
>
> The version of the perf tool is 3.9.rc8.g25e33e (the version in
> linus's tree today).
> --
> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BUG: perf kvm top --callgraph not showing callgraph
2013-06-14 1:10 ` David Ahern
@ 2013-06-14 21:53 ` Cody P Schafer
2013-06-14 22:01 ` David Ahern
0 siblings, 1 reply; 6+ messages in thread
From: Cody P Schafer @ 2013-06-14 21:53 UTC (permalink / raw)
To: David Ahern; +Cc: linux-perf-users
On Thu, Jun 13, 2013 at 6:10 PM, David Ahern <dsahern@gmail.com> wrote:
> On 6/13/13 7:04 PM, Cody P Schafer wrote:
>>
>> Hi all, I'm trying to use perf kvm to profile linux early boot.
>>
>> However, perf kvm top always appears to behave as though it's in
>> non-callgraph mode.
>
>
> During event collection perf kernel side does not walk the guest callchain.
> I believe it only collects IP.
>
Some further investigation shows that the user unwind code could be
used, but [the following] is always false.
if (!((evsel->attr.sample_type & PERF_SAMPLE_REGS_USER) &&
(evsel->attr.sample_type & PERF_SAMPLE_STACK_USER)))
return 0;
There don't appear to be any users which _set_ PERF_SAMPLE_REGS_USER
or PERF_SAMPLE_STACK_USER, only ones that check it. Can we just use
__perf_evsel__set_sample_bit() twice and have this magically work?
So, a few questions:
- can we get perf to dump the user regs & stack for kvm guest kernels?
- does the in kernel stack unwinding _only_ trigger for host (ie:
real) kernel mode samples? If so, does this restriction always make
sense?
- How did all this user unwind code get triggered if no one is
setting PERF_SAMPLE_REGS_USER and PERF_SAMPLE_STACK_USER? Was it ever
triggered?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BUG: perf kvm top --callgraph not showing callgraph
2013-06-14 21:53 ` Cody P Schafer
@ 2013-06-14 22:01 ` David Ahern
2013-06-14 23:11 ` Cody P Schafer
0 siblings, 1 reply; 6+ messages in thread
From: David Ahern @ 2013-06-14 22:01 UTC (permalink / raw)
To: Cody P Schafer; +Cc: linux-perf-users
On 6/14/13 3:53 PM, Cody P Schafer wrote:
> On Thu, Jun 13, 2013 at 6:10 PM, David Ahern <dsahern@gmail.com> wrote:
>> On 6/13/13 7:04 PM, Cody P Schafer wrote:
>>>
>>> Hi all, I'm trying to use perf kvm to profile linux early boot.
>>>
>>> However, perf kvm top always appears to behave as though it's in
>>> non-callgraph mode.
>>
>>
>> During event collection perf kernel side does not walk the guest callchain.
>> I believe it only collects IP.
>>
>
> Some further investigation shows that the user unwind code could be
> used, but [the following] is always false.
> if (!((evsel->attr.sample_type & PERF_SAMPLE_REGS_USER) &&
> (evsel->attr.sample_type & PERF_SAMPLE_STACK_USER)))
> return 0;
>
> There don't appear to be any users which _set_ PERF_SAMPLE_REGS_USER
> or PERF_SAMPLE_STACK_USER, only ones that check it. Can we just use
> __perf_evsel__set_sample_bit() twice and have this magically work?
>
> So, a few questions:
> - can we get perf to dump the user regs & stack for kvm guest kernels?
> - does the in kernel stack unwinding _only_ trigger for host (ie:
> real) kernel mode samples? If so, does this restriction always make
> sense?
> - How did all this user unwind code get triggered if no one is
> setting PERF_SAMPLE_REGS_USER and PERF_SAMPLE_STACK_USER? Was it ever
> triggered?
>
I am under the impression the limitations are these 2 snippets in
arch/x86/kernel/cpu/perf_event.c:
oid
perf_callchain_kernel(struct perf_callchain_entry *entry, struct pt_regs
*regs)
{
if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
/* TODO: We don't support guest os callchain now */
return;
}
...
void
perf_callchain_user(struct perf_callchain_entry *entry, struct pt_regs
*regs)
{
struct stack_frame frame;
const void __user *fp;
if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
/* TODO: We don't support guest os callchain now */
return;
}
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BUG: perf kvm top --callgraph not showing callgraph
2013-06-14 22:01 ` David Ahern
@ 2013-06-14 23:11 ` Cody P Schafer
2013-06-14 23:23 ` David Ahern
0 siblings, 1 reply; 6+ messages in thread
From: Cody P Schafer @ 2013-06-14 23:11 UTC (permalink / raw)
To: David Ahern; +Cc: linux-perf-users
On Fri, Jun 14, 2013 at 3:01 PM, David Ahern <dsahern@gmail.com> wrote:
> On 6/14/13 3:53 PM, Cody P Schafer wrote:
>>
>> On Thu, Jun 13, 2013 at 6:10 PM, David Ahern <dsahern@gmail.com> wrote:
>>>
>>> On 6/13/13 7:04 PM, Cody P Schafer wrote:
>>>>
>>>> Hi all, I'm trying to use perf kvm to profile linux early boot.
>>>>
>>>> However, perf kvm top always appears to behave as though it's in
>>>> non-callgraph mode.
>>>
>>> During event collection perf kernel side does not walk the guest
>>> callchain.
>>> I believe it only collects IP.
>>>
>>
>> Some further investigation shows that the user unwind code could be
>> used, but [the following] is always false.
>> if (!((evsel->attr.sample_type & PERF_SAMPLE_REGS_USER) &&
>> (evsel->attr.sample_type & PERF_SAMPLE_STACK_USER)))
>> return 0;
>>
>> There don't appear to be any users which _set_ PERF_SAMPLE_REGS_USER
>> or PERF_SAMPLE_STACK_USER, only ones that check it. Can we just use
>> __perf_evsel__set_sample_bit() twice and have this magically work?
>>
>> So, a few questions:
>> - can we get perf to dump the user regs & stack for kvm guest kernels?
>> - does the in kernel stack unwinding _only_ trigger for host (ie:
>> real) kernel mode samples? If so, does this restriction always make
>> sense?
>> - How did all this user unwind code get triggered if no one is
>> setting PERF_SAMPLE_REGS_USER and PERF_SAMPLE_STACK_USER? Was it ever
>> triggered?
>>
>
> I am under the impression the limitations are these 2 snippets in
> arch/x86/kernel/cpu/perf_event.c:
Yep, that's disabling the in-kernel callchain generation (the in-perf
callchain code should still be an option without this). I wonder if
one could use perf_callchain_user() to find the callchain of the guest
kernel.
Does anyone know the particular changes that need to be done to
support kvm backtraces/the reasons it is specifically disabled?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BUG: perf kvm top --callgraph not showing callgraph
2013-06-14 23:11 ` Cody P Schafer
@ 2013-06-14 23:23 ` David Ahern
0 siblings, 0 replies; 6+ messages in thread
From: David Ahern @ 2013-06-14 23:23 UTC (permalink / raw)
To: Cody P Schafer; +Cc: linux-perf-users, KVM, Xiao Guangrong
[added kvm devel list and Xiao who implemented the original perf-kvm as
I recall]
On 6/14/13 5:11 PM, Cody P Schafer wrote:
>> I am under the impression the limitations are these 2 snippets in
>> >arch/x86/kernel/cpu/perf_event.c:
> Yep, that's disabling the in-kernel callchain generation (the in-perf
> callchain code should still be an option without this). I wonder if
> one could use perf_callchain_user() to find the callchain of the guest
> kernel.
>
> Does anyone know the particular changes that need to be done to
> support kvm backtraces/the reasons it is specifically disabled?
It's going to require some KVM changes I would think. e.g., for the IP:
unsigned long perf_instruction_pointer(struct pt_regs *regs)
{
if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
return perf_guest_cbs->get_guest_ip();
return regs->ip + code_segment_base(regs);
}
get_guest_ip comes from arch/x86/kvm/x86.c.
callchain walking starts in perf_callchain, kernel/events/callchain.c
For more context:
perf_prepare_sample()
data->callchain = perf_callchain(event, regs);
For guests you'll want regs from the guest which will need to be
supplied by kvm.
David
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-06-14 23:23 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-06-14 1:04 BUG: perf kvm top --callgraph not showing callgraph Cody P Schafer
2013-06-14 1:10 ` David Ahern
2013-06-14 21:53 ` Cody P Schafer
2013-06-14 22:01 ` David Ahern
2013-06-14 23:11 ` Cody P Schafer
2013-06-14 23:23 ` David Ahern
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).