[PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

As stack tracing now requires "rcu watching", force RCU to be watching when
recording a stack trace.

Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org

Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---

Changes since v1:

   My testing discovered that the stack trace can be called with
   interrupts enabled, which is a no no to have when calling
   rcu_irq_enter(). When interrupts are enabled, as with being in an
   NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index fcc9a2d..34a98ba 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {
+		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+		return;
+	}
+
+	/*
+	 * It is possible that a function is being traced in a
+	 * location that RCU is not watching. A call to
+	 * rcu_irq_enter() will make sure that it is, but there's
+	 * a few internal rcu functions that could be traced
+	 * where that wont work either. In those cases, we just
+	 * do nothing.
+	 */
+	if (unlikely(rcu_irq_enter_disabled()))
+		return;
+
+	rcu_irq_enter();
+	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+	rcu_irq_exit();
 }
 
 /**
-- 
2.9.3

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Miroslav Benes]

On Thu, 18 May 2017, Steven Rostedt wrote:

> 
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> 
> As stack tracing now requires "rcu watching", force RCU to be watching when
> recording a stack trace.
> 
> Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> 
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> 
> Changes since v1:
> 
>    My testing discovered that the stack trace can be called with
>    interrupts enabled, which is a no no to have when calling
>    rcu_irq_enter(). When interrupts are enabled, as with being in an
>    NMI, RCU will also be watching.

Would rcu_irq_enter_irqson() help then? This is what Petr used in a live 
patching handler.

Your solution works too, of course. Just asking if I am not missing 
something.

Thanks,
Miroslav

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Steven Rostedt]

On Thu, 18 May 2017 15:48:55 +0200 (CEST)
Miroslav Benes <mbenes@suse.cz> wrote:

> On Thu, 18 May 2017, Steven Rostedt wrote:
> 
> > 
> > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > 
> > As stack tracing now requires "rcu watching", force RCU to be watching when
> > recording a stack trace.
> > 
> > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > 
> > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> > 
> > Changes since v1:
> > 
> >    My testing discovered that the stack trace can be called with
> >    interrupts enabled, which is a no no to have when calling
> >    rcu_irq_enter(). When interrupts are enabled, as with being in an
> >    NMI, RCU will also be watching.  
> 
> Would rcu_irq_enter_irqson() help then? This is what Petr used in a live 
> patching handler.
> 

Yes, that could work too, but I wanted to avoid disabling interrupts if
we didn't have to.

> Your solution works too, of course. Just asking if I am not missing 
> something.
>

Nope, I was just trying to keep the overhead down. As this can be
called by every event enabled, as well as functions being traced. I
figured that local_save_irqs() is faster than a pair of
local_irq_save()/ local_irq_restore() calls.

-- Steve

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Miroslav Benes]

On Thu, 18 May 2017, Steven Rostedt wrote:

> On Thu, 18 May 2017 15:48:55 +0200 (CEST)
> Miroslav Benes <mbenes@suse.cz> wrote:
> 
> > On Thu, 18 May 2017, Steven Rostedt wrote:
> > 
> > > 
> > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > 
> > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > recording a stack trace.
> > > 
> > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > > 
> > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > > 
> > > Changes since v1:
> > > 
> > >    My testing discovered that the stack trace can be called with
> > >    interrupts enabled, which is a no no to have when calling
> > >    rcu_irq_enter(). When interrupts are enabled, as with being in an
> > >    NMI, RCU will also be watching.  
> > 
> > Would rcu_irq_enter_irqson() help then? This is what Petr used in a live 
> > patching handler.
> > 
> 
> Yes, that could work too, but I wanted to avoid disabling interrupts if
> we didn't have to.

Ok, that makes sense.

> > Your solution works too, of course. Just asking if I am not missing 
> > something.
> >
> 
> Nope, I was just trying to keep the overhead down. As this can be
> called by every event enabled, as well as functions being traced. I
> figured that local_save_irqs() is faster than a pair of
> local_irq_save()/ local_irq_restore() calls.

(noticed Paul's reply)... yeah, it'd great. Damn, this is mindblowing.

Thanks,
Miroslav

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Steven Rostedt]

On Thu, 18 May 2017 09:38:09 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> 
> As stack tracing now requires "rcu watching", force RCU to be watching when
> recording a stack trace.
> 
> Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> 
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

Paul,

Can you give me an ack for this version too.

Thanks!

-- Steve

> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> 
> Changes since v1:
> 
>    My testing discovered that the stack trace can be called with
>    interrupts enabled, which is a no no to have when calling
>    rcu_irq_enter(). When interrupts are enabled, as with being in an
>    NMI, RCU will also be watching.
> 
> kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
>  1 file changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index fcc9a2d..34a98ba 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
>  void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
>  		   int pc)
>  {
> -	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> +	struct ring_buffer *buffer = tr->trace_buffer.buffer;
> +
> +	/*
> +	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> +	 * Also, RCU is always enabled when interrupts are.
> +	 */
> +	if (!irqs_disabled() || in_nmi()) {
> +		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> +		return;
> +	}
> +
> +	/*
> +	 * It is possible that a function is being traced in a
> +	 * location that RCU is not watching. A call to
> +	 * rcu_irq_enter() will make sure that it is, but there's
> +	 * a few internal rcu functions that could be traced
> +	 * where that wont work either. In those cases, we just
> +	 * do nothing.
> +	 */
> +	if (unlikely(rcu_irq_enter_disabled()))
> +		return;
> +
> +	rcu_irq_enter();
> +	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> +	rcu_irq_exit();
>  }
>  
>  /**

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Paul E. McKenney]

On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> 
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> 
> As stack tracing now requires "rcu watching", force RCU to be watching when
> recording a stack trace.
> 
> Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> 
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> 
> Changes since v1:
> 
>    My testing discovered that the stack trace can be called with
>    interrupts enabled, which is a no no to have when calling
>    rcu_irq_enter(). When interrupts are enabled, as with being in an
>    NMI, RCU will also be watching.
> 
> kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
>  1 file changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index fcc9a2d..34a98ba 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
>  void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
>  		   int pc)
>  {
> -	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> +	struct ring_buffer *buffer = tr->trace_buffer.buffer;
> +
> +	/*
> +	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> +	 * Also, RCU is always enabled when interrupts are.
> +	 */
> +	if (!irqs_disabled() || in_nmi()) {

You lost me on this one.  RCU might not be watching if irqs are
enabled, for example, in the idle loop.  What am I missing here?

						Thanx, Paul

> +		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> +		return;
> +	}
> +
> +	/*
> +	 * It is possible that a function is being traced in a
> +	 * location that RCU is not watching. A call to
> +	 * rcu_irq_enter() will make sure that it is, but there's
> +	 * a few internal rcu functions that could be traced
> +	 * where that wont work either. In those cases, we just
> +	 * do nothing.
> +	 */
> +	if (unlikely(rcu_irq_enter_disabled()))
> +		return;
> +
> +	rcu_irq_enter();
> +	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> +	rcu_irq_exit();
>  }
> 
>  /**
> -- 
> 2.9.3
> 

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Steven Rostedt]

On Thu, 18 May 2017 08:47:11 -0700
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:

> On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> > 
> > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > 
> > As stack tracing now requires "rcu watching", force RCU to be watching when
> > recording a stack trace.
> > 
> > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > 
> > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> > 
> > Changes since v1:
> > 
> >    My testing discovered that the stack trace can be called with
> >    interrupts enabled, which is a no no to have when calling
> >    rcu_irq_enter(). When interrupts are enabled, as with being in an
> >    NMI, RCU will also be watching.
> > 
> > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> >  1 file changed, 25 insertions(+), 1 deletion(-)
> > 
> > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > index fcc9a2d..34a98ba 100644
> > --- a/kernel/trace/trace.c
> > +++ b/kernel/trace/trace.c
> > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> >  void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> >  		   int pc)
> >  {
> > -	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > +	struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > +
> > +	/*
> > +	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > +	 * Also, RCU is always enabled when interrupts are.
> > +	 */
> > +	if (!irqs_disabled() || in_nmi()) {  
> 
> You lost me on this one.  RCU might not be watching if irqs are
> enabled, for example, in the idle loop.  What am I missing here?
> 

Hmm, no, maybe I'm missing something :-/

OK, so if we trace in the idle loop, rcu may not be watching, so I may
need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(

-- Steve


> 						Thanx, Paul
> 
> > +		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > +		return;
> > +	}
> > +
> > +	/*
> > +	 * It is possible that a function is being traced in a
> > +	 * location that RCU is not watching. A call to
> > +	 * rcu_irq_enter() will make sure that it is, but there's
> > +	 * a few internal rcu functions that could be traced
> > +	 * where that wont work either. In those cases, we just
> > +	 * do nothing.
> > +	 */
> > +	if (unlikely(rcu_irq_enter_disabled()))
> > +		return;
> > +
> > +	rcu_irq_enter();
> > +	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > +	rcu_irq_exit();
> >  }
> > 
> >  /**
> > -- 
> > 2.9.3
> >   

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Paul E. McKenney]

On Thu, May 18, 2017 at 12:39:14PM -0400, Steven Rostedt wrote:
> On Thu, 18 May 2017 08:47:11 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> 
> > On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> > > 
> > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > 
> > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > recording a stack trace.
> > > 
> > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > > 
> > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > > 
> > > Changes since v1:
> > > 
> > >    My testing discovered that the stack trace can be called with
> > >    interrupts enabled, which is a no no to have when calling
> > >    rcu_irq_enter(). When interrupts are enabled, as with being in an
> > >    NMI, RCU will also be watching.
> > > 
> > > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > >  1 file changed, 25 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > > index fcc9a2d..34a98ba 100644
> > > --- a/kernel/trace/trace.c
> > > +++ b/kernel/trace/trace.c
> > > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > >  void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > >  		   int pc)
> > >  {
> > > -	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > > +	struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > > +
> > > +	/*
> > > +	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > > +	 * Also, RCU is always enabled when interrupts are.
> > > +	 */
> > > +	if (!irqs_disabled() || in_nmi()) {  
> > 
> > You lost me on this one.  RCU might not be watching if irqs are
> > enabled, for example, in the idle loop.  What am I missing here?
> > 
> 
> Hmm, no, maybe I'm missing something :-/
> 
> OK, so if we trace in the idle loop, rcu may not be watching, so I may
> need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(

If rcu_is_watching() returns false, yes.  Of course, if rcu_is_watching()
returns true, you are golden.

							Thanx, Paul

> -- Steve
> 
> 
> > 						Thanx, Paul
> > 
> > > +		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > > +		return;
> > > +	}
> > > +
> > > +	/*
> > > +	 * It is possible that a function is being traced in a
> > > +	 * location that RCU is not watching. A call to
> > > +	 * rcu_irq_enter() will make sure that it is, but there's
> > > +	 * a few internal rcu functions that could be traced
> > > +	 * where that wont work either. In those cases, we just
> > > +	 * do nothing.
> > > +	 */
> > > +	if (unlikely(rcu_irq_enter_disabled()))
> > > +		return;
> > > +
> > > +	rcu_irq_enter();
> > > +	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > > +	rcu_irq_exit();
> > >  }
> > > 
> > >  /**
> > > -- 
> > > 2.9.3
> > >   
> 

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Steven Rostedt]

On Thu, 18 May 2017 11:45:28 -0700
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:

> On Thu, May 18, 2017 at 12:39:14PM -0400, Steven Rostedt wrote:
> > On Thu, 18 May 2017 08:47:11 -0700
> > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> >   
> > > On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:  
> > > > 
> > > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > > 
> > > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > > recording a stack trace.
> > > > 
> > > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > > > 
> > > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > > ---
> > > > 
> > > > Changes since v1:
> > > > 
> > > >    My testing discovered that the stack trace can be called with
> > > >    interrupts enabled, which is a no no to have when calling
> > > >    rcu_irq_enter(). When interrupts are enabled, as with being in an
> > > >    NMI, RCU will also be watching.
> > > > 
> > > > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > > >  1 file changed, 25 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > > > index fcc9a2d..34a98ba 100644
> > > > --- a/kernel/trace/trace.c
> > > > +++ b/kernel/trace/trace.c
> > > > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > > >  void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > > >  		   int pc)
> > > >  {
> > > > -	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > > > +	struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > > > +
> > > > +	/*
> > > > +	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > > > +	 * Also, RCU is always enabled when interrupts are.
> > > > +	 */
> > > > +	if (!irqs_disabled() || in_nmi()) {    
> > > 
> > > You lost me on this one.  RCU might not be watching if irqs are
> > > enabled, for example, in the idle loop.  What am I missing here?
> > >   
> > 
> > Hmm, no, maybe I'm missing something :-/
> > 
> > OK, so if we trace in the idle loop, rcu may not be watching, so I may
> > need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(  
> 
> If rcu_is_watching() returns false, yes.  Of course, if rcu_is_watching()
> returns true, you are golden.
> 

Hmm, so I can just replace the entire if with:

	if (rcu_is_watching())

because it should be watching in NMI context, correct?

-- Steve

Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace

[Paul E. McKenney]

On Thu, May 18, 2017 at 03:30:26PM -0400, Steven Rostedt wrote:
> On Thu, 18 May 2017 11:45:28 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> 
> > On Thu, May 18, 2017 at 12:39:14PM -0400, Steven Rostedt wrote:
> > > On Thu, 18 May 2017 08:47:11 -0700
> > > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> > >   
> > > > On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:  
> > > > > 
> > > > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > > > 
> > > > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > > > recording a stack trace.
> > > > > 
> > > > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > > > > 
> > > > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > > > ---
> > > > > 
> > > > > Changes since v1:
> > > > > 
> > > > >    My testing discovered that the stack trace can be called with
> > > > >    interrupts enabled, which is a no no to have when calling
> > > > >    rcu_irq_enter(). When interrupts are enabled, as with being in an
> > > > >    NMI, RCU will also be watching.
> > > > > 
> > > > > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > > > >  1 file changed, 25 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > > > > index fcc9a2d..34a98ba 100644
> > > > > --- a/kernel/trace/trace.c
> > > > > +++ b/kernel/trace/trace.c
> > > > > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > > > >  void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > > > >  		   int pc)
> > > > >  {
> > > > > -	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > > > > +	struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > > > > +
> > > > > +	/*
> > > > > +	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > > > > +	 * Also, RCU is always enabled when interrupts are.
> > > > > +	 */
> > > > > +	if (!irqs_disabled() || in_nmi()) {    
> > > > 
> > > > You lost me on this one.  RCU might not be watching if irqs are
> > > > enabled, for example, in the idle loop.  What am I missing here?
> > > >   
> > > 
> > > Hmm, no, maybe I'm missing something :-/
> > > 
> > > OK, so if we trace in the idle loop, rcu may not be watching, so I may
> > > need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(  
> > 
> > If rcu_is_watching() returns false, yes.  Of course, if rcu_is_watching()
> > returns true, you are golden.
> > 
> 
> Hmm, so I can just replace the entire if with:
> 
> 	if (rcu_is_watching())
> 
> because it should be watching in NMI context, correct?

Yes, and that would be even better!  ;-)

However, this assumes that there is no possibility of this being called
during the time that in_nmi() returns true but RCU is not aware of the
NMI.  (Yes, we did go through this a bit earlier, just my usual paranoia
showing...)

							Thanx, Paul