Re: [PATCH tip/core/urgent 2/2] rcu: remove INIT_RCU_HEAD, RCU_HEAD_INIT, RCU_HEAD

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

* Paul E. McKenney (paulmck@linux.vnet.ibm.com) wrote:
> On Thu, Mar 18, 2010 at 03:35:20PM -0400, Mathieu Desnoyers wrote:
> > * Paul E. McKenney (paulmck@linux.vnet.ibm.com) wrote:
> > > From: Alexey Dobriyan <adobriyan@gmail.com>
> > > 
> > > call_rcu() will unconditionally reinitialize RCU head anyway.  New users
> > > of these macros constantly appear, so remove them.
> > 
> > Hrm. So do we have something that checks for double-use of a RCU head at
> > the moment ? (using call_rcu() twice on the same head without being
> > certain that the first callback have finished its execution).
> > 
> > I think that hiding rcu head initialization into call_rcu() is one more
> > step towards misuses that will silently corrupt rcu head lists. So I
> > think we should first add the double-use debugging option before we
> > remove the RCU head initializations.
> 
> So your thought is to have rcu_do_batch() do something like the
> following?
> 
> 	...
> 
> 	next = list->next;
> 	prefetch(next);
> 	list->next = RCU_HEAD_INIT_PTR;
> 	func = list->func;
> 	list->func = RCU_HEAD_INIT_PTR;
> 	func(list);
> 	... /* touching anything referenced by "list" is use-after-free. */
> 
> Then have __call_rcu() do something like the following before initializing
> the ->func and ->next pointers:
> 
> 	WARN_ON_ONCE(head->next != RCU_HEAD_INIT_PTR ||
> 		     head->func != RCU_HEAD_INIT_PTR);
> 
> And then require that all users of call_rcu() and friends use one of the
> RCU_INIT() macros?
> 
> Or did you have something else in mind?

More precisely poisoning an extra field of the rcu_head, as done in the
following patch.

I posted it a few months ago, but has been rejected on the ground that
it should be re-done in within the debug objects infrastructure.  But I
had to focus on other things and never found time to do these changes.
It needs a separate patch which adds missing INIT_RCU_HEAD() to a few
more kernel sites.

The reason why I add a supplementary field for the poison is to be able
to warn for detection of incoherent list_head both in call_rcu and in
rcu_do_batch(), which does not seem possible with the scheme you propose
above. The sequence is:

init ->         debug = NULL
call_rcu ->     WARN_ON_ONCE(debug != NULL)
                debug = LIST_POISON1
rcu_do_batch -> WARN_ON_ONCE(debug != LIST_POISON1)
                debug = NULL


tree rcu: Add debug RCU head option

Poisoning the rcu_head callback list. Only for rcu tree for now.

Helps finding racy users of call_rcu(), which results in hangs because list
entries are overwritten and/or skipped.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
CC: mingo@elte.hu
CC: akpm@linux-foundation.org
---
 include/linux/rcupdate.h |   11 +++++++++++
 include/net/dst.h        |    2 ++
 kernel/rcutree.c         |   10 ++++++++++
 lib/Kconfig.debug        |    9 +++++++++
 4 files changed, 32 insertions(+)

Index: linux-2.6-lttng/include/linux/rcupdate.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/rcupdate.h	2009-11-22 20:25:49.000000000 -0500
+++ linux-2.6-lttng/include/linux/rcupdate.h	2009-11-22 22:11:48.000000000 -0500
@@ -49,6 +49,9 @@
 struct rcu_head {
 	struct rcu_head *next;
 	void (*func)(struct rcu_head *head);
+#ifdef CONFIG_DEBUG_RCU_HEAD
+	struct rcu_head *debug;
+#endif
 };
 
 /* Exported common interfaces */
@@ -77,11 +80,19 @@ extern int rcu_scheduler_active;
 #error "Unknown RCU implementation specified to kernel configuration"
 #endif
 
+#ifdef CONFIG_DEBUG_RCU_HEAD
+#define RCU_HEAD_INIT 	{ .next = NULL, .func = NULL, .debug = NULL }
+#define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
+#define INIT_RCU_HEAD(ptr) do { \
+       (ptr)->next = NULL; (ptr)->func = NULL; (ptr)->debug = NULL; \
+} while (0)
+#else
 #define RCU_HEAD_INIT	{ .next = NULL, .func = NULL }
 #define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
 #define INIT_RCU_HEAD(ptr) do { \
        (ptr)->next = NULL; (ptr)->func = NULL; \
 } while (0)
+#endif
 
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
 extern struct lockdep_map rcu_lock_map;
Index: linux-2.6-lttng/kernel/rcutree.c
===================================================================
--- linux-2.6-lttng.orig/kernel/rcutree.c	2009-11-22 21:38:56.000000000 -0500
+++ linux-2.6-lttng/kernel/rcutree.c	2009-11-22 22:10:49.000000000 -0500
@@ -39,6 +39,7 @@
 #include <asm/atomic.h>
 #include <linux/bitops.h>
 #include <linux/module.h>
+#include <linux/poison.h>
 #include <linux/completion.h>
 #include <linux/moduleparam.h>
 #include <linux/percpu.h>
@@ -1010,6 +1011,10 @@ static void rcu_do_batch(struct rcu_stat
 		next = list->next;
 		prefetch(next);
 		trace_rcu_tree_callback(list);
+#ifdef DEBUG_RCU_HEAD
+		WARN_ON_ONCE(list->debug != LIST_POISON1);
+		list->debug = NULL;
+#endif
 		list->func(list);
 		list = next;
 		if (++count >= rdp->blimit)
@@ -1291,6 +1296,11 @@ __call_rcu(struct rcu_head *head, void (
 	unsigned long flags;
 	struct rcu_data *rdp;
 
+#ifdef DEBUG_RCU_HEAD
+	WARN_ON_ONCE(head->debug);
+	head->debug = LIST_POISON1;
+#endif
+
 	head->func = func;
 	head->next = NULL;
 
Index: linux-2.6-lttng/lib/Kconfig.debug
===================================================================
--- linux-2.6-lttng.orig/lib/Kconfig.debug	2009-11-22 22:01:03.000000000 -0500
+++ linux-2.6-lttng/lib/Kconfig.debug	2009-11-22 22:10:49.000000000 -0500
@@ -652,6 +652,15 @@ config DEBUG_LIST
 
 	  If unsure, say N.
 
+config DEBUG_RCU_HEAD
+	bool "Debug RCU callbacks"
+	depends on DEBUG_KERNEL
+	depends on TREE_RCU
+	help
+	  Enable this to turn on debugging of RCU list heads (call_rcu() usage).
+	  Seems to find problems more quickly with stress-tests in single-cpu
+	  mode.
+
 config DEBUG_SG
 	bool "Debug SG table operations"
 	depends on DEBUG_KERNEL
Index: linux-2.6-lttng/include/net/dst.h
===================================================================
--- linux-2.6-lttng.orig/include/net/dst.h	2009-11-22 20:25:49.000000000 -0500
+++ linux-2.6-lttng/include/net/dst.h	2009-11-22 22:10:49.000000000 -0500
@@ -154,7 +154,9 @@ static inline void dst_hold(struct dst_e
 	 * If your kernel compilation stops here, please check
 	 * __pad_to_align_refcnt declaration in struct dst_entry
 	 */
+#ifndef CONFIG_DEBUG_RCU_HEAD
 	BUILD_BUG_ON(offsetof(struct dst_entry, __refcnt) & 63);
+#endif
 	atomic_inc(&dst->__refcnt);
 }
 

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com