[PATCH 0/2] tracing/function_graph fixes for 2.6.31

[PATCH 0/2] tracing/function_graph fixes for 2.6.31

[Frederic Weisbecker]

Ingo, Thomas,

Those are two fixes for the function graph tracer for .31

The first one fixes a possible race condition between a function graph trace
reader and ring buffer writers, leading to a mixup inside an event entry, or
worst.

The second one fixes some issues in the output of the function graph tracer
while using splice on the trace_pipe file or ftrace_dump().

Both fixes should also be backported on .30 for the -stable branch.

Thanks,
Frederic.

The following changes since commit 4733fd328f14280900435d9dbae1487d110a4d56:
  Benjamin Herrenschmidt (1):
        mm: Remove duplicate definitions in MIPS and SH

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git tracing/fixes

Lai Jiangshan (2):
      tracing: Fix invalid function_graph entry
      tracing: Fix missing function_graph events when we splice_read from trace_pipe

 kernel/trace/trace.c                 |   10 +++++++---
 kernel/trace/trace_functions_graph.c |   11 +++++++++--
 2 files changed, 16 insertions(+), 5 deletions(-)

[PATCH 1/2] tracing: Fix invalid function_graph entry

[Frederic Weisbecker]

From: Lai Jiangshan <laijs@cn.fujitsu.com>

When print_graph_entry() computes a function call entry event, it needs
to also check the next entry to guess if it matches the return event of
the current function entry.
In order to look at this next event, it needs to consume the current
entry before going ahead in the ring buffer.

However, if the current event that gets consumed is the last one in the
ring buffer head page, the ring_buffer may reuse the page for writers.
The consumed entry will then become invalid because of possible
racy overwriting.

Me must then handle this entry by making a copy of it.

The fix also applies on 2.6.30

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: stable@kernel.org
LKML-Reference: <4A6EEAEC.3050508@cn.fujitsu.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
---
 kernel/trace/trace_functions_graph.c |   11 +++++++++--
 1 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index d2249ab..420ec34 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -843,9 +843,16 @@ print_graph_function(struct trace_iterator *iter)
 
 	switch (entry->type) {
 	case TRACE_GRAPH_ENT: {
-		struct ftrace_graph_ent_entry *field;
+		/*
+		 * print_graph_entry() may consume the current event,
+		 * thus @field may become invalid, so we need to save it.
+		 * sizeof(struct ftrace_graph_ent_entry) is very small,
+		 * it can be safely saved at the stack.
+		 */
+		struct ftrace_graph_ent_entry *field, saved;
 		trace_assign_type(field, entry);
-		return print_graph_entry(field, s, iter);
+		saved = *field;
+		return print_graph_entry(&saved, s, iter);
 	}
 	case TRACE_GRAPH_RET: {
 		struct ftrace_graph_ret_entry *field;
-- 
1.6.2.3

[PATCH 2/2] tracing: Fix missing function_graph events when we splice_read from trace_pipe

[Frederic Weisbecker]

From: Lai Jiangshan <laijs@cn.fujitsu.com>

About a half events are missing when we splice_read
from trace_pipe. They are unexpectedly consumed because we ignore
the TRACE_TYPE_NO_CONSUME return value used by the function graph
tracer when it needs to consume the events by itself to walk on
the ring buffer.

The same problem appears with ftrace_dump()

Example of an output before this patch:

1)               |      ktime_get_real() {
1)   2.846 us    |          read_hpet();
1)   4.558 us    |        }
1)   6.195 us    |      }

After this patch:

0)               |      ktime_get_real() {
0)               |        getnstimeofday() {
0)   1.960 us    |          read_hpet();
0)   3.597 us    |        }
0)   5.196 us    |      }

The fix also applies on 2.6.30

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: stable@kernel.org
LKML-Reference: <4A6EEC52.90704@cn.fujitsu.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
---
 kernel/trace/trace.c |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 8bc8d8a..da984ad 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -3085,7 +3085,8 @@ tracing_fill_pipe_page(size_t rem, struct trace_iterator *iter)
 			break;
 		}
 
-		trace_consume(iter);
+		if (ret != TRACE_TYPE_NO_CONSUME)
+			trace_consume(iter);
 		rem -= count;
 		if (!find_next_entry_inc(iter))	{
 			rem = 0;
@@ -4233,8 +4234,11 @@ static void __ftrace_dump(bool disable_tracing)
 		iter.pos = -1;
 
 		if (find_next_entry_inc(&iter) != NULL) {
-			print_trace_line(&iter);
-			trace_consume(&iter);
+			int ret;
+
+			ret = print_trace_line(&iter);
+			if (ret != TRACE_TYPE_NO_CONSUME)
+				trace_consume(&iter);
 		}
 
 		trace_printk_seq(&iter.seq);
-- 
1.6.2.3

[GIT PULL] tracing/function_graph fixes for 2.6.31

[Frederic Weisbecker]

On Tue, Jul 28, 2009 at 11:55:32PM +0200, Frederic Weisbecker wrote:
> Ingo, Thomas,
> 
> Those are two fixes for the function graph tracer for .31
> 
> The first one fixes a possible race condition between a function graph trace
> reader and ring buffer writers, leading to a mixup inside an event entry, or
> worst.
> 
> The second one fixes some issues in the output of the function graph tracer
> while using splice on the trace_pipe file or ftrace_dump().
> 
> Both fixes should also be backported on .30 for the -stable branch.
> 
> Thanks,
> Frederic.



(Changing the title with the [GIT PULL] prefix, in case you miss it.)

Thanks.

 
> The following changes since commit 4733fd328f14280900435d9dbae1487d110a4d56:
>   Benjamin Herrenschmidt (1):
>         mm: Remove duplicate definitions in MIPS and SH
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git tracing/fixes
> 
> Lai Jiangshan (2):
>       tracing: Fix invalid function_graph entry
>       tracing: Fix missing function_graph events when we splice_read from trace_pipe
> 
>  kernel/trace/trace.c                 |   10 +++++++---
>  kernel/trace/trace_functions_graph.c |   11 +++++++++--
>  2 files changed, 16 insertions(+), 5 deletions(-)

Re: [PATCH 0/2] tracing/function_graph fixes for 2.6.31

[Ingo Molnar]

* Frederic Weisbecker <fweisbec@gmail.com> wrote:

> Ingo, Thomas,
> 
> Those are two fixes for the function graph tracer for .31
> 
> The first one fixes a possible race condition between a function 
> graph trace reader and ring buffer writers, leading to a mixup 
> inside an event entry, or worst.
> 
> The second one fixes some issues in the output of the function 
> graph tracer while using splice on the trace_pipe file or 
> ftrace_dump().
> 
> Both fixes should also be backported on .30 for the -stable 
> branch.
> 
> Thanks,
> Frederic.
> 
> The following changes since commit 4733fd328f14280900435d9dbae1487d110a4d56:
>   Benjamin Herrenschmidt (1):
>         mm: Remove duplicate definitions in MIPS and SH
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git tracing/fixes
> 
> Lai Jiangshan (2):
>       tracing: Fix invalid function_graph entry
>       tracing: Fix missing function_graph events when we splice_read from trace_pipe
> 
>  kernel/trace/trace.c                 |   10 +++++++---
>  kernel/trace/trace_functions_graph.c |   11 +++++++++--
>  2 files changed, 16 insertions(+), 5 deletions(-)

Pulled, thanks a lot Frederic!

	Ingo