[PATCH v2] ftrace: Make ftrace_graph_is_dead() a static branch

[PATCH v2] ftrace: Make ftrace_graph_is_dead() a static branch

[Christophe Leroy]

ftrace_graph_is_dead() is used on hot paths, it just reads a variable
in memory and is not worth suffering function call constraints.

For instance, at entry of prepare_ftrace_return(), inlining it avoids
saving prepare_ftrace_return() parameters to stack and restoring them
after calling ftrace_graph_is_dead().

While at it using a static branch is even more performant and is
rather well adapted considering that the returned value will almost
never change.

Inline ftrace_graph_is_dead() and replace 'kill_ftrace_graph' bool
by a static branch.

The performance improvement is noticeable.

Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
v2: Use a static branch instead of a global bool var.
---
 include/linux/ftrace.h | 16 +++++++++++++++-
 kernel/trace/fgraph.c  | 17 +++--------------
 2 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 9999e29187de..9e639e05666d 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -18,6 +18,7 @@
 #include <linux/types.h>
 #include <linux/init.h>
 #include <linux/fs.h>
+#include <linux/jump_label.h>
 
 #include <asm/ftrace.h>
 
@@ -1006,7 +1007,20 @@ unsigned long ftrace_graph_ret_addr(struct task_struct *task, int *idx,
 extern int register_ftrace_graph(struct fgraph_ops *ops);
 extern void unregister_ftrace_graph(struct fgraph_ops *ops);
 
-extern bool ftrace_graph_is_dead(void);
+/**
+ * ftrace_graph_is_dead - returns true if ftrace_graph_stop() was called
+ *
+ * ftrace_graph_stop() is called when a severe error is detected in
+ * the function graph tracing. This function is called by the critical
+ * paths of function graph to keep those paths from doing any more harm.
+ */
+DECLARE_STATIC_KEY_FALSE(kill_ftrace_graph);
+
+static inline bool ftrace_graph_is_dead(void)
+{
+	return static_branch_unlikely(&kill_ftrace_graph);
+}
+
 extern void ftrace_graph_stop(void);
 
 /* The current handlers in use */
diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c
index 22061d38fc00..15acd48f0718 100644
--- a/kernel/trace/fgraph.c
+++ b/kernel/trace/fgraph.c
@@ -10,6 +10,7 @@
 #include <linux/suspend.h>
 #include <linux/ftrace.h>
 #include <linux/slab.h>
+#include <linux/jump_label.h>
 
 #include <trace/events/sched.h>
 
@@ -23,24 +24,12 @@
 #define ASSIGN_OPS_HASH(opsname, val)
 #endif
 
-static bool kill_ftrace_graph;
+DEFINE_STATIC_KEY_FALSE(kill_ftrace_graph);
 int ftrace_graph_active;
 
 /* Both enabled by default (can be cleared by function_graph tracer flags */
 static bool fgraph_sleep_time = true;
 
-/**
- * ftrace_graph_is_dead - returns true if ftrace_graph_stop() was called
- *
- * ftrace_graph_stop() is called when a severe error is detected in
- * the function graph tracing. This function is called by the critical
- * paths of function graph to keep those paths from doing any more harm.
- */
-bool ftrace_graph_is_dead(void)
-{
-	return kill_ftrace_graph;
-}
-
 /**
  * ftrace_graph_stop - set to permanently disable function graph tracing
  *
@@ -51,7 +40,7 @@ bool ftrace_graph_is_dead(void)
  */
 void ftrace_graph_stop(void)
 {
-	kill_ftrace_graph = true;
+	static_branch_enable(&kill_ftrace_graph);
 }
 
 /* Add a function return address to the trace stack on thread info.*/
-- 
2.35.1

Re: [PATCH v2] ftrace: Make ftrace_graph_is_dead() a static branch

[Steven Rostedt]

On Fri, 25 Mar 2022 09:03:08 +0100
Christophe Leroy <christophe.leroy@csgroup.eu> wrote:

> --- a/kernel/trace/fgraph.c
> +++ b/kernel/trace/fgraph.c
> @@ -10,6 +10,7 @@
>  #include <linux/suspend.h>
>  #include <linux/ftrace.h>
>  #include <linux/slab.h>
> +#include <linux/jump_label.h>
>  

Small nit. Please order the includes in "upside-down x-mas tree" fashion:

#include <linux/jump_label.h>
#include <linux/suspend.h>
#include <linux/ftrace.h>
#include <linux/slab.h>

Thanks,

-- Steve

Re: [PATCH v2] ftrace: Make ftrace_graph_is_dead() a static branch

[Christophe Leroy]

Le 30/03/2022 à 04:07, Steven Rostedt a écrit :
> On Fri, 25 Mar 2022 09:03:08 +0100
> Christophe Leroy <christophe.leroy@csgroup.eu> wrote:
> 
>> --- a/kernel/trace/fgraph.c
>> +++ b/kernel/trace/fgraph.c
>> @@ -10,6 +10,7 @@
>>   #include <linux/suspend.h>
>>   #include <linux/ftrace.h>
>>   #include <linux/slab.h>
>> +#include <linux/jump_label.h>
>>   
> 
> Small nit. Please order the includes in "upside-down x-mas tree" fashion:
> 
> #include <linux/jump_label.h>
> #include <linux/suspend.h>
> #include <linux/ftrace.h>
> #include <linux/slab.h>
> 

That's the first time I get such a request. Usually people request 
#includes to be in alphabetical order so when I see a file that has 
headers in alphabetical order I try to not break it, but here that was 
not the case so I put it at the end of the list.

I'll send v3

Thanks
Christophe

Re: [PATCH v2] ftrace: Make ftrace_graph_is_dead() a static branch

[Steven Rostedt]

On Wed, 30 Mar 2022 06:55:26 +0000
Christophe Leroy <christophe.leroy@csgroup.eu> wrote:

> > Small nit. Please order the includes in "upside-down x-mas tree" fashion:
> > 
> > #include <linux/jump_label.h>
> > #include <linux/suspend.h>
> > #include <linux/ftrace.h>
> > #include <linux/slab.h>
> >   
> 
> That's the first time I get such a request. Usually people request 
> #includes to be in alphabetical order so when I see a file that has 
> headers in alphabetical order I try to not break it, but here that was 
> not the case so I put it at the end of the list.

This is something that Ingo Molnar started back in 2009 or so. And I do
find it easier on the eyes ;-)  I may be the only one today trying to keep
it (albeit poorly).

It's not a hard requirement, but I find it makes the code look more like
art, which it is :-D

> 
> I'll send v3

Thanks,

-- Steve