[BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[Nikunj A. Dadhania]

From: Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com>

Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'

Was resulting in a segfault. For event type PERF_RECORD_MMAP,
event->ip.pid is being used in perf_session__find_machine_for_cpumode,
which is not correct.

event->ip.pid happens to be 0 in this case and results in returning a
NULL machine object. Finally, access to self->pid in
machine__mmap_name, results in a segfault later.

For PERF_RECORD_MMAP type, pass event->mmap.pid.

Signed-off-by: Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com>
---

 tools/perf/util/session.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index 9412e3b..00923cd 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -826,8 +826,16 @@ static struct machine *
 {
 	const u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
 
-	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL && perf_guest)
-		return perf_session__find_machine(session, event->ip.pid);
+	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL && perf_guest) {
+		u32 pid;
+
+		if (event->header.type == PERF_RECORD_MMAP)
+			pid = event->mmap.pid;
+		else
+			pid = event->ip.pid;
+
+		return perf_session__find_machine(session, pid);
+	}
 
 	return perf_session__find_host_machine(session);
 }

Re: [BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[David Ahern]

On 4/9/12 2:22 AM, Nikunj A. Dadhania wrote:
> From: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>
>
> Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'
>
> Was resulting in a segfault. For event type PERF_RECORD_MMAP,
> event->ip.pid is being used in perf_session__find_machine_for_cpumode,
> which is not correct.

I'm not getting the segfault with latest tip/perf/urgent, but I am 
getting a hang.

It's stuck on buildid:

(gdb) bt
#0  0x00007fabf53e79da in mmap64 () from /lib64/libc.so.6
#1  0x0000000000454656 in __perf_session__process_events 
(session=0x1e06110,
     data_offset=<value optimized out>, data_size=<value optimized out>, 
file_size=37328,
     tool=0x75db40) at util/session.c:1192
#2  0x00000000004102e6 in process_buildids (status=<value optimized 
out>, arg=0x75cc00)
     at builtin-record.c:320
#3  perf_record__exit (status=<value optimized out>, arg=0x75cc00) at 
builtin-record.c:333
#4  0x00007fabf53419c9 in exit () from /lib64/libc.so.6
#5  0x0000000000406b84 in handle_internal_command (argc=9, 
argv=0x7fff3f14d6c0) at perf.c:345
#6  run_argv (argc=9, argv=0x7fff3f14d6c0) at perf.c:389
#7  main (argc=9, argv=0x7fff3f14d6c0) at perf.c:487


Adding -B works around it and proves something is not right regarding 
buildid's and perf-kvm. I'll dig into it as time allows, but wanted to 
send this out in case Arnaldo understands the problem now.

David

>
> event->ip.pid happens to be 0 in this case and results in returning a
> NULL machine object. Finally, access to self->pid in
> machine__mmap_name, results in a segfault later.
>
> For PERF_RECORD_MMAP type, pass event->mmap.pid.
>
> Signed-off-by: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>
> ---
>
>   tools/perf/util/session.c |   12 ++++++++++--
>   1 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
> index 9412e3b..00923cd 100644
> --- a/tools/perf/util/session.c
> +++ b/tools/perf/util/session.c
> @@ -826,8 +826,16 @@ static struct machine *
>   {
>   	const u8 cpumode = event->header.misc&  PERF_RECORD_MISC_CPUMODE_MASK;
>
> -	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL&&  perf_guest)
> -		return perf_session__find_machine(session, event->ip.pid);
> +	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL&&  perf_guest) {
> +		u32 pid;
> +
> +		if (event->header.type == PERF_RECORD_MMAP)
> +			pid = event->mmap.pid;
> +		else
> +			pid = event->ip.pid;
> +
> +		return perf_session__find_machine(session, pid);
> +	}
>
>   	return perf_session__find_host_machine(session);
>   }
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[David Ahern]

On 4/9/12 2:22 AM, Nikunj A. Dadhania wrote:
> From: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>
>
> Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'
>
> Was resulting in a segfault. For event type PERF_RECORD_MMAP,
> event->ip.pid is being used in perf_session__find_machine_for_cpumode,
> which is not correct.
>
> event->ip.pid happens to be 0 in this case and results in returning a
> NULL machine object. Finally, access to self->pid in
> machine__mmap_name, results in a segfault later.

I have tried unsuccessfully to recreate the segfault. All of the samples 
I get are of type PERF_RECORD_SAMPLE. I do not see how an MMAP event can 
be generated in a guest context.

David


>
> For PERF_RECORD_MMAP type, pass event->mmap.pid.
>
> Signed-off-by: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>
> ---
>
>   tools/perf/util/session.c |   12 ++++++++++--
>   1 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
> index 9412e3b..00923cd 100644
> --- a/tools/perf/util/session.c
> +++ b/tools/perf/util/session.c
> @@ -826,8 +826,16 @@ static struct machine *
>   {
>   	const u8 cpumode = event->header.misc&  PERF_RECORD_MISC_CPUMODE_MASK;
>
> -	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL&&  perf_guest)
> -		return perf_session__find_machine(session, event->ip.pid);
> +	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL&&  perf_guest) {
> +		u32 pid;
> +
> +		if (event->header.type == PERF_RECORD_MMAP)
> +			pid = event->mmap.pid;
> +		else
> +			pid = event->ip.pid;
> +
> +		return perf_session__find_machine(session, pid);
> +	}
>
>   	return perf_session__find_host_machine(session);
>   }
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[Nikunj A Dadhania]

On Mon, 09 Apr 2012 16:59:50 -0600, David Ahern <dsahern@gmail.com> wrote:
> On 4/9/12 2:22 AM, Nikunj A. Dadhania wrote:
> > From: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>
> >
> > Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'
> >
> > Was resulting in a segfault. For event type PERF_RECORD_MMAP,
> > event->ip.pid is being used in perf_session__find_machine_for_cpumode,
> > which is not correct.
> >
> > event->ip.pid happens to be 0 in this case and results in returning a
> > NULL machine object. Finally, access to self->pid in
> > machine__mmap_name, results in a segfault later.
> 
> I have tried unsuccessfully to recreate the segfault. All of the samples 
> I get are of type PERF_RECORD_SAMPLE. I do not see how an MMAP event can 
> be generated in a guest context.
> 
I can think of two places where it can happen:

1) perf_event__synthesize_kernel_mmap

        [...]
 	mmap_name = machine__mmap_name(machine, name_buff, sizeof(name_buff));
	if (machine__is_host(machine)) {
           ...
	} else {
		event->header.misc = PERF_RECORD_MISC_GUEST_KERNEL;
		if (machine__is_default_guest(machine))
			filename = (char *) symbol_conf.default_guest_kallsyms;
		else {
			sprintf(path, "%s/proc/kallsyms", machine->root_dir);
			filename = path;
		}
	}
        [...]
	event->mmap.header.type = PERF_RECORD_MMAP;
        [...]
        event->mmap.pid   = machine->pid;        

2) perf_event__synthesize_modules

        [...]
	if (machine__is_host(machine))
		event->header.misc = PERF_RECORD_MISC_KERNEL;
	else
		event->header.misc = PERF_RECORD_MISC_GUEST_KERNEL;
        [...]             
	event->mmap.header.type = PERF_RECORD_MMAP;
        [...]
        event->mmap.pid   = machine->pid;

BTW, i am testing this on latest -tip(498c911).

In my case, I am able to recreate this everytime. And have verified that
the event->header.type that I hit here is of type PERF_RECORD_MMAP.

Let me know if you want me to test out some debug patches.

Regards,
Nikunj

Re: [BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[David Ahern]

On 4/9/12 9:40 PM, Nikunj A Dadhania wrote:
> I can think of two places where it can happen:
>
> 1) perf_event__synthesize_kernel_mmap
>
>          [...]
>   	mmap_name = machine__mmap_name(machine, name_buff, sizeof(name_buff));
> 	if (machine__is_host(machine)) {
>             ...
> 	} else {
> 		event->header.misc = PERF_RECORD_MISC_GUEST_KERNEL;
> 		if (machine__is_default_guest(machine))
> 			filename = (char *) symbol_conf.default_guest_kallsyms;
> 		else {
> 			sprintf(path, "%s/proc/kallsyms", machine->root_dir);
> 			filename = path;
> 		}
> 	}
>          [...]
> 	event->mmap.header.type = PERF_RECORD_MMAP;
>          [...]
>          event->mmap.pid   = machine->pid;
>
> 2) perf_event__synthesize_modules
>
>          [...]
> 	if (machine__is_host(machine))
> 		event->header.misc = PERF_RECORD_MISC_KERNEL;
> 	else
> 		event->header.misc = PERF_RECORD_MISC_GUEST_KERNEL;
>          [...]
> 	event->mmap.header.type = PERF_RECORD_MMAP;
>          [...]
>          event->mmap.pid   = machine->pid;
>
> BTW, i am testing this on latest -tip(498c911).
>
> In my case, I am able to recreate this everytime. And have verified that
> the event->header.type that I hit here is of type PERF_RECORD_MMAP.

I was thinking runtime, not initialization. Adding -o direct_io to the 
sshfs mount did the trick. Reading guest /proc files exposes the segfault.

David

Re: [BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[David Ahern]

On 4/9/12 2:22 AM, Nikunj A. Dadhania wrote:
> From: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>
>
> Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'
>
> Was resulting in a segfault. For event type PERF_RECORD_MMAP,
> event->ip.pid is being used in perf_session__find_machine_for_cpumode,
> which is not correct.
>
> event->ip.pid happens to be 0 in this case and results in returning a
> NULL machine object. Finally, access to self->pid in
> machine__mmap_name, results in a segfault later.
>
> For PERF_RECORD_MMAP type, pass event->mmap.pid.
>
> Signed-off-by: Nikunj A. Dadhania<nikunj@linux.vnet.ibm.com>

Reviewed-by: David Ahern <dsahern@gmail.com>
Tested-by: David Ahern <dsahern@gmail.com>

David

> ---
>
>   tools/perf/util/session.c |   12 ++++++++++--
>   1 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
> index 9412e3b..00923cd 100644
> --- a/tools/perf/util/session.c
> +++ b/tools/perf/util/session.c
> @@ -826,8 +826,16 @@ static struct machine *
>   {
>   	const u8 cpumode = event->header.misc&  PERF_RECORD_MISC_CPUMODE_MASK;
>
> -	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL&&  perf_guest)
> -		return perf_session__find_machine(session, event->ip.pid);
> +	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL&&  perf_guest) {
> +		u32 pid;
> +
> +		if (event->header.type == PERF_RECORD_MMAP)
> +			pid = event->mmap.pid;
> +		else
> +			pid = event->ip.pid;
> +
> +		return perf_session__find_machine(session, pid);
> +	}
>
>   	return perf_session__find_host_machine(session);
>   }
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [BUG PATCH] perf: kvm - finding struct machine fails for PERF_RECORD_MMAP

[Arnaldo Carvalho de Melo]

Em Tue, Apr 10, 2012 at 09:16:37AM -0600, David Ahern escreveu:
> On 4/9/12 2:22 AM, Nikunj A. Dadhania wrote:
> >Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'
> >
> >Was resulting in a segfault. For event type PERF_RECORD_MMAP,
> >event->ip.pid is being used in perf_session__find_machine_for_cpumode,
> >which is not correct.
> >
> >event->ip.pid happens to be 0 in this case and results in returning a
> >NULL machine object. Finally, access to self->pid in
> >machine__mmap_name, results in a segfault later.
> >
> >For PERF_RECORD_MMAP type, pass event->mmap.pid.
> 
> Reviewed-by: David Ahern <dsahern@gmail.com>
> Tested-by: David Ahern <dsahern@gmail.com>

Thanks guys, applied this to my perf/urgent branch.

- Arnaldo

[tip:perf/urgent] perf kvm: Finding struct machine fails for PERF_RECORD_MMAP

[tip-bot for Nikunj A. Dadhania]

Commit-ID:  7fb0a5ee8889488f7568ffddffeb66ddeb50917e
Gitweb:     http://git.kernel.org/tip/7fb0a5ee8889488f7568ffddffeb66ddeb50917e
Author:     Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com>
AuthorDate: Mon, 9 Apr 2012 13:52:23 +0530
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 11 Apr 2012 11:45:12 -0300

perf kvm: Finding struct machine fails for PERF_RECORD_MMAP

Running 'perf kvm --host --guest --guestmount /tmp/guestmount record -a -g -- sleep 2'

Was resulting in a segfault. For event type PERF_RECORD_MMAP,
event->ip.pid is being used in perf_session__find_machine_for_cpumode,
which is not correct.

The event->ip.pid field happens to be 0 in this case and results in
returning a NULL machine object. Finally, access to self->pid in
machine__mmap_name, results in a segfault later.

For PERF_RECORD_MMAP type, pass event->mmap.pid.

Signed-off-by: Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Tested-by: David Ahern <dsahern@gmail.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Cc: Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20120409081835.10576.22018.stgit@abhimanyu.in.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/session.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index 9412e3b..00923cd 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -826,8 +826,16 @@ static struct machine *
 {
 	const u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
 
-	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL && perf_guest)
-		return perf_session__find_machine(session, event->ip.pid);
+	if (cpumode == PERF_RECORD_MISC_GUEST_KERNEL && perf_guest) {
+		u32 pid;
+
+		if (event->header.type == PERF_RECORD_MMAP)
+			pid = event->mmap.pid;
+		else
+			pid = event->ip.pid;
+
+		return perf_session__find_machine(session, pid);
+	}
 
 	return perf_session__find_host_machine(session);
 }