[PATCH] tracing: wake up tasks reading trace_pipe on write to trace_marker

[PATCH] tracing: wake up tasks reading trace_pipe on write to trace_marker

[Marcin Slusarz]

Currently we rely on other code periodically waking up trace reader.
If there aren't any other data than markers, reader will never be woken up.
Fix it.

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
---
 kernel/trace/trace.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 086d363..02e04c8 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -1520,6 +1520,7 @@ int trace_array_vprintk(struct trace_array *tr,
 	if (!filter_check_discard(call, entry, buffer, event)) {
 		ring_buffer_unlock_commit(buffer, event);
 		ftrace_trace_stack(buffer, irq_flags, 6, pc);
+		trace_wake_up();
 	}
 
  out_unlock:
-- 
1.7.1.1

Re: [PATCH] tracing: wake up tasks reading trace_pipe on write to trace_marker

[Steven Rostedt]

On Wed, 2010-07-28 at 00:44 +0200, Marcin Slusarz wrote:
> Currently we rely on other code periodically waking up trace reader.
> If there aren't any other data than markers, reader will never be woken up.
> Fix it.
> 
> Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> ---
>  kernel/trace/trace.c |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index 086d363..02e04c8 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -1520,6 +1520,7 @@ int trace_array_vprintk(struct trace_array *tr,
>  	if (!filter_check_discard(call, entry, buffer, event)) {
>  		ring_buffer_unlock_commit(buffer, event);
>  		ftrace_trace_stack(buffer, irq_flags, 6, pc);
> +		trace_wake_up();
>  	}
>  

This can't work. trace_printk() and friends must be able to be used
anywhere. This can cause race conditions with the rq locks in the
scheduler.

But you do bring up a good idea. That is, perhaps we should have a way
to attach to known safe tracepoints that we can hook to to check if a
wake up should happen or not.

-- Steve

Re: [PATCH] tracing: wake up tasks reading trace_pipe on write to trace_marker

[Frederic Weisbecker]

On Fri, Aug 06, 2010 at 04:38:56PM -0400, Steven Rostedt wrote:
> On Wed, 2010-07-28 at 00:44 +0200, Marcin Slusarz wrote:
> > Currently we rely on other code periodically waking up trace reader.
> > If there aren't any other data than markers, reader will never be woken up.
> > Fix it.
> > 
> > Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> > Cc: Frederic Weisbecker <fweisbec@gmail.com>
> > Cc: Ingo Molnar <mingo@redhat.com>
> > ---
> >  kernel/trace/trace.c |    1 +
> >  1 files changed, 1 insertions(+), 0 deletions(-)
> > 
> > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > index 086d363..02e04c8 100644
> > --- a/kernel/trace/trace.c
> > +++ b/kernel/trace/trace.c
> > @@ -1520,6 +1520,7 @@ int trace_array_vprintk(struct trace_array *tr,
> >  	if (!filter_check_discard(call, entry, buffer, event)) {
> >  		ring_buffer_unlock_commit(buffer, event);
> >  		ftrace_trace_stack(buffer, irq_flags, 6, pc);
> > +		trace_wake_up();
> >  	}
> >  
> 
> This can't work. trace_printk() and friends must be able to be used
> anywhere. This can cause race conditions with the rq locks in the
> scheduler.
> 
> But you do bring up a good idea. That is, perhaps we should have a way
> to attach to known safe tracepoints that we can hook to to check if a
> wake up should happen or not.


This could be a simple macro that takes the name of the trace event:


DEFINE_EVENT(event_tpl, event_name, ...);


TRACE_EVENT_NO_WAKE(event_name);

I think trace events should be wakeable by default as it looks safe for
most of them. But probably we don't want that per event class.

In the unsafe list, I only have some sched and lock events in
mind, but I bet there are some others.

Re: [PATCH] tracing: wake up tasks reading trace_pipe on write to trace_marker

[Steven Rostedt]

On Fri, 2010-08-06 at 22:55 +0200, Frederic Weisbecker wrote:

> > This can't work. trace_printk() and friends must be able to be used
> > anywhere. This can cause race conditions with the rq locks in the
> > scheduler.
> > 
> > But you do bring up a good idea. That is, perhaps we should have a way
> > to attach to known safe tracepoints that we can hook to to check if a
> > wake up should happen or not.
> 
> 
> This could be a simple macro that takes the name of the trace event:
> 
> 
> DEFINE_EVENT(event_tpl, event_name, ...);
> 
> 
> TRACE_EVENT_NO_WAKE(event_name);
> 

Yeah, that may be worth doing for 2.6.37. Might as well also add a
trace_printk_nowake() too, when you know you are in dangerous locations
like the scheduler or NMI.


> I think trace events should be wakeable by default as it looks safe for
> most of them. But probably we don't want that per event class.
> 
> In the unsafe list, I only have some sched and lock events in
> mind, but I bet there are some others.

Yep, will put that on my todo list.

Thanks,

-- Steve

Re: [PATCH] tracing: wake up tasks reading trace_pipe on write to trace_marker

[Frederic Weisbecker]

On Fri, Aug 06, 2010 at 05:29:35PM -0400, Steven Rostedt wrote:
> On Fri, 2010-08-06 at 22:55 +0200, Frederic Weisbecker wrote:
> 
> > > This can't work. trace_printk() and friends must be able to be used
> > > anywhere. This can cause race conditions with the rq locks in the
> > > scheduler.
> > > 
> > > But you do bring up a good idea. That is, perhaps we should have a way
> > > to attach to known safe tracepoints that we can hook to to check if a
> > > wake up should happen or not.
> > 
> > 
> > This could be a simple macro that takes the name of the trace event:
> > 
> > 
> > DEFINE_EVENT(event_tpl, event_name, ...);
> > 
> > 
> > TRACE_EVENT_NO_WAKE(event_name);
> > 
> 
> Yeah, that may be worth doing for 2.6.37. Might as well also add a
> trace_printk_nowake() too, when you know you are in dangerous locations
> like the scheduler or NMI.



Yeah.


 
> 
> > I think trace events should be wakeable by default as it looks safe for
> > most of them. But probably we don't want that per event class.
> > 
> > In the unsafe list, I only have some sched and lock events in
> > mind, but I bet there are some others.
> 
> Yep, will put that on my todo list.
> 
> Thanks,


Cool. This is going to be useful in perf as well. The "nmi" argument in
perf_swevent_add tells wether we can wake up or not. If not we do a
kind of delayed wake up using a self IPI.

Currently we always consider we can't wake up when a trace event triggers.
If we know we can wake up, this is going to be less costly.