[PATCH] perf kvm record/report: 'unprocessable sample' error while recording/reporting guest data

[PATCH] perf kvm record/report: 'unprocessable sample' error while recording/reporting guest data

[Ravi Bangoria]

While recording guest samples in host using perf kvm record, it will
populate unprocessable sample error, though samples will be recorded
properly. While generating report using perf kvm report, no samples will
be processed and same error will populate. We have seen this behaviour
with upstream perf(4.4-rc3) on x86 and ppc64 hardware.

Reason behind this failure is, when it tries to fetch machine from rb_tree
of machines, it fails. As a part of tracing a bug, we figured out that this
code was incorrectly refactored in commit 54245fdc357613633954bfd38cffb71cb9def067
("perf session: Remove wrappers to machines__find")

This patch will change the functionality such that if it can't fetch
machine in first trial, it will create one node of machine and add that to
rb_tree. So next time when it tries to fetch same machine from rb_tree,
it won't fail. Actually it was the case before refactoring of code in
aforementioned commit.

This patch is generated from acme perf/core branch.

Below I've mention an example that demonstrate the behaviour before and
after applying patch.

Before applying patch:
[Note: One needs to run guest before recording data in host]

ravi@ravi-bangoria:~$ ./perf kvm record -a
Warning:
5903 unprocessable samples recorded.
Do you have a KVM guest running and not using 'perf kvm'?
[ perf record: Captured and wrote 1.409 MB perf.data.guest (285 samples) ]

ravi@ravi-bangoria:~$ ./perf kvm report --stdio
Warning:
5903 unprocessable samples recorded.
Do you have a KVM guest running and not using 'perf kvm'?
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 285  of event 'cycles'
# Event count (approx.): 88715406
#
# Overhead  Command  Shared Object  Symbol
# ........  .......  .............  ......
#

#
# (For a higher level overview, try: perf report --sort comm,dso)
#

After applying patch:

ravi@ravi-bangoria:~$ ./perf kvm record -a
[ perf record: Captured and wrote 1.188 MB perf.data.guest (17 samples) ]

ravi@ravi-bangoria:~$ ./perf kvm report --stdio
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 17  of event 'cycles'
# Event count (approx.): 700746
#
# Overhead  Command  Shared Object     Symbol
# ........  .......  ................  ......................
#
    34.19%  :5758    [unknown]         [g] 0xffffffff818682ab
    22.79%  :5758    [unknown]         [g] 0xffffffff812dc7f8
    22.79%  :5758    [unknown]         [g] 0xffffffff818650d0
    14.83%  :5758    [unknown]         [g] 0xffffffff8161a1b6
     2.49%  :5758    [unknown]         [g] 0xffffffff818692bf
     0.48%  :5758    [unknown]         [g] 0xffffffff81869253
     0.05%  :5758    [unknown]         [g] 0xffffffff81869250

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
---
 tools/perf/util/session.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index c35ffdd..468de95 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -972,7 +972,7 @@ static struct machine *machines__find_for_cpumode(struct machines *machines,
 
 		machine = machines__find(machines, pid);
 		if (!machine)
-			machine = machines__find(machines, DEFAULT_GUEST_KERNEL_ID);
+			machine = machines__findnew(machines, DEFAULT_GUEST_KERNEL_ID);
 		return machine;
 	}
 
-- 
2.1.4

Re: [PATCH] perf kvm record/report: 'unprocessable sample' error while recording/reporting guest data

[Ravi Bangoria]

Hi acme,

I sent this patch few days ago. Unfortunately nobody has payed attention.

Can you please pick this up.

Regards,
Ravi

On Monday 07 December 2015 12:25 PM, Ravi Bangoria wrote:
> While recording guest samples in host using perf kvm record, it will
> populate unprocessable sample error, though samples will be recorded
> properly. While generating report using perf kvm report, no samples will
> be processed and same error will populate. We have seen this behaviour
> with upstream perf(4.4-rc3) on x86 and ppc64 hardware.
>
> Reason behind this failure is, when it tries to fetch machine from rb_tree
> of machines, it fails. As a part of tracing a bug, we figured out that this
> code was incorrectly refactored in commit 54245fdc357613633954bfd38cffb71cb9def067
> ("perf session: Remove wrappers to machines__find")
>
> This patch will change the functionality such that if it can't fetch
> machine in first trial, it will create one node of machine and add that to
> rb_tree. So next time when it tries to fetch same machine from rb_tree,
> it won't fail. Actually it was the case before refactoring of code in
> aforementioned commit.
>
> This patch is generated from acme perf/core branch.
>
> Below I've mention an example that demonstrate the behaviour before and
> after applying patch.
>
> Before applying patch:
> [Note: One needs to run guest before recording data in host]
>
> ravi@ravi-bangoria:~$ ./perf kvm record -a
> Warning:
> 5903 unprocessable samples recorded.
> Do you have a KVM guest running and not using 'perf kvm'?
> [ perf record: Captured and wrote 1.409 MB perf.data.guest (285 samples) ]
>
> ravi@ravi-bangoria:~$ ./perf kvm report --stdio
> Warning:
> 5903 unprocessable samples recorded.
> Do you have a KVM guest running and not using 'perf kvm'?
> # To display the perf.data header info, please use --header/--header-only options.
> #
> #
> # Total Lost Samples: 0
> #
> # Samples: 285  of event 'cycles'
> # Event count (approx.): 88715406
> #
> # Overhead  Command  Shared Object  Symbol
> # ........  .......  .............  ......
> #
>
> #
> # (For a higher level overview, try: perf report --sort comm,dso)
> #
>
> After applying patch:
>
> ravi@ravi-bangoria:~$ ./perf kvm record -a
> [ perf record: Captured and wrote 1.188 MB perf.data.guest (17 samples) ]
>
> ravi@ravi-bangoria:~$ ./perf kvm report --stdio
> # To display the perf.data header info, please use --header/--header-only options.
> #
> #
> # Total Lost Samples: 0
> #
> # Samples: 17  of event 'cycles'
> # Event count (approx.): 700746
> #
> # Overhead  Command  Shared Object     Symbol
> # ........  .......  ................  ......................
> #
>      34.19%  :5758    [unknown]         [g] 0xffffffff818682ab
>      22.79%  :5758    [unknown]         [g] 0xffffffff812dc7f8
>      22.79%  :5758    [unknown]         [g] 0xffffffff818650d0
>      14.83%  :5758    [unknown]         [g] 0xffffffff8161a1b6
>       2.49%  :5758    [unknown]         [g] 0xffffffff818692bf
>       0.48%  :5758    [unknown]         [g] 0xffffffff81869253
>       0.05%  :5758    [unknown]         [g] 0xffffffff81869250
>
> Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
> ---
>   tools/perf/util/session.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
> index c35ffdd..468de95 100644
> --- a/tools/perf/util/session.c
> +++ b/tools/perf/util/session.c
> @@ -972,7 +972,7 @@ static struct machine *machines__find_for_cpumode(struct machines *machines,
>
>   		machine = machines__find(machines, pid);
>   		if (!machine)
> -			machine = machines__find(machines, DEFAULT_GUEST_KERNEL_ID);
> +			machine = machines__findnew(machines, DEFAULT_GUEST_KERNEL_ID);
>   		return machine;
>   	}
>

Re: [PATCH] perf kvm record/report: 'unprocessable sample' error while recording/reporting guest data

[Arnaldo Carvalho de Melo]

Em Mon, Dec 07, 2015 at 12:25:02PM +0530, Ravi Bangoria escreveu:
> While recording guest samples in host using perf kvm record, it will
> populate unprocessable sample error, though samples will be recorded
> properly. While generating report using perf kvm report, no samples will
> be processed and same error will populate. We have seen this behaviour
> with upstream perf(4.4-rc3) on x86 and ppc64 hardware.

Thanks, good catch, applied.

- Arnaldo

[tip:perf/urgent] perf kvm record/report: 'unprocessable sample' error while recording/reporting guest data

[tip-bot for Ravi Bangoria]

Commit-ID:  3caeaa562733c4836e61086ec07666635006a787
Gitweb:     http://git.kernel.org/tip/3caeaa562733c4836e61086ec07666635006a787
Author:     Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
AuthorDate: Mon, 7 Dec 2015 12:25:02 +0530
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Fri, 15 Jan 2016 16:31:58 -0300

perf kvm record/report: 'unprocessable sample' error while recording/reporting guest data

While recording guest samples in host using perf kvm record, it will
populate unprocessable sample error, though samples will be recorded
properly. While generating report using perf kvm report, no samples will
be processed and same error will populate. We have seen this behaviour
with upstream perf(4.4-rc3) on x86 and ppc64 hardware.

Reason behind this failure is, when it tries to fetch machine from
rb_tree of machines, it fails. As a part of tracing a bug, we figured
out that this code was incorrectly refactored in commit 54245fdc3576
("perf session: Remove wrappers to machines__find").

This patch will change the functionality such that if it can't fetch
machine in first trial, it will create one node of machine and add that to
rb_tree. So next time when it tries to fetch same machine from rb_tree,
it won't fail. Actually it was the case before refactoring of code in
aforementioned commit.

This patch is generated from acme perf/core branch.

Below I've mention an example that demonstrate the behaviour before and
after applying patch.

Before applying patch:
[Note: One needs to run guest before recording data in host]

  ravi@ravi-bangoria:~$ ./perf kvm record -a
  Warning:
  5903 unprocessable samples recorded.
  Do you have a KVM guest running and not using 'perf kvm'?
  [ perf record: Captured and wrote 1.409 MB perf.data.guest (285 samples) ]

  ravi@ravi-bangoria:~$ ./perf kvm report --stdio
  Warning:
  5903 unprocessable samples recorded.
  Do you have a KVM guest running and not using 'perf kvm'?
  # To display the perf.data header info, please use --header/--header-only options.
  #
  # Total Lost Samples: 0
  #
  # Samples: 285  of event 'cycles'
  # Event count (approx.): 88715406
  #
  # Overhead  Command  Shared Object  Symbol
  # ........  .......  .............  ......
  #

  # (For a higher level overview, try: perf report --sort comm,dso)
  #

After applying patch:

  ravi@ravi-bangoria:~$ ./perf kvm record -a
  [ perf record: Captured and wrote 1.188 MB perf.data.guest (17 samples) ]

  ravi@ravi-bangoria:~$ ./perf kvm report --stdio
  # To display the perf.data header info, please use --header/--header-only options.
  #
  # Total Lost Samples: 0
  #
  # Samples: 17  of event 'cycles'
  # Event count (approx.): 700746
  #
  # Overhead  Command  Shared Object     Symbol
  # ........  .......  ................  ......................
  #
      34.19%  :5758    [unknown]         [g] 0xffffffff818682ab
      22.79%  :5758    [unknown]         [g] 0xffffffff812dc7f8
      22.79%  :5758    [unknown]         [g] 0xffffffff818650d0
      14.83%  :5758    [unknown]         [g] 0xffffffff8161a1b6
       2.49%  :5758    [unknown]         [g] 0xffffffff818692bf
       0.48%  :5758    [unknown]         [g] 0xffffffff81869253
       0.05%  :5758    [unknown]         [g] 0xffffffff81869250

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: stable@vger.kernel.org # v3.19+
Fixes: 54245fdc3576 ("perf session: Remove wrappers to machines__find")
Link: http://lkml.kernel.org/r/1449471302-11283-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/session.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index d5636ba..40b7a0d 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -1149,7 +1149,7 @@ static struct machine *machines__find_for_cpumode(struct machines *machines,
 
 		machine = machines__find(machines, pid);
 		if (!machine)
-			machine = machines__find(machines, DEFAULT_GUEST_KERNEL_ID);
+			machine = machines__findnew(machines, DEFAULT_GUEST_KERNEL_ID);
 		return machine;
 	}