Re: [linux-next:master 6618/6917] kernel/sched/psi.c:1230:13: sparse: error: incompatible types in comparison expression (different address spaces)

["Paul E. McKenney" <paulmck@linux.ibm.com>]

On Mon, Feb 11, 2019 at 05:00:37PM -0800, Andrew Morton wrote:
> > > 
> > > Paul, can you please shed light?
> > 
> > First, please avoid using rcu_dereference_raw() where possible.  It is
> > intended for situations where the developer cannot easily state what
> > is to be protecting access to an RCU-protected data structure.  So...
> > 
> > 1.	If the access needs to be within an RCU read-side critical
> > 	section, use rcu_dereference().  With the new consolidated
> > 	RCU flavors, an RCU read-side critical section is entered
> > 	using rcu_read_lock(), anything that disables bottom halves,
> > 	anything that disables interrupts, or anything that disables
> > 	preemption.
> > 
> > 2.	If the access might be within an RCU read-side critical section
> > 	on the one hand, or protected by (say) my_lock on the other,
> > 	use rcu_dereference_check(), for example:
> > 	
> > 		p1 = rcu_dereference_check(p->rcu_protected_pointer,
> > 					   lockdep_is_held(&my_lock));
> > 
> > 
> > 3.	If the access might be within an RCU read-side critical section
> > 	on the one hand, or protected by either my_lock or your_lock on
> > 	the other, again use rcu_dereference_check(), for example:
> > 
> > 		p1 = rcu_dereference_check(p->rcu_protected_pointer,
> > 					   lockdep_is_held(&my_lock) ||
> > 					   lockdep_is_held(&your_lock));
> > 
> > 4.	If the access is on the update side, so that it is always protected
> > 	by my_lock, use rcu_dereference_protected():
> > 
> > 		p1 = rcu_dereference_protected(p->rcu_protected_pointer,
> > 					       lockdep_is_held(&my_lock));
> > 
> > 	This can be extended to handle multiple locks as in #3 above,
> > 	and both can be extended to check other conditions as well.
> > 
> > 5.	If the protection is supplied by the caller, and is thus unknown
> > 	to this code, that is when you use rcu_dereference_raw().  Or
> > 	I suppose you could use it when the lockdep expression would be
> > 	excessively complex, except that a better approach in that case
> > 	might be to take a long hard look at your synchronization design.
> > 	Still, there are data-locking cases where any one of a very
> > 	large number of locks or reference counters suffices to protect the
> > 	pointer, so rcu_derefernce_raw() does have its place.
> > 
> > 	However, its place is probably quite a bit smaller than one
> > 	might expect given the number of uses in the current kernel.
> > 	Ditto for its synonym, rcu_dereference_protected( ... , 1).  :-/
> 
> Is this documented anywhere (apart from here?)

In the docbook headers for these functions, apart from rcu_dereference_raw(),
whose use I am not encouraging.

But having it in one place with examples might be helpful.  Does the
patch at the end of this email seem reasonable?

> > Now on to this sparse checking and what the point of it is.  This sparse
> > checking is opt-in.  Its purpose is to catch cases where someone
> > mistakenly does something like:
> > 
> > 	p = q->rcu_protected_pointer;
> > 
> > When they should have done this instead:
> > 
> > 	p = rcu_dereference(q->rcu_protected_pointer);
> > 
> > If you wish to opt into this checking, you need to mark the pointer
> > definitions (in this case ->private) with __rcu.  It may also
> > be necessary to mark function parameters as well, as is done for
> > radix_tree_iter_resume().  If you do not wish to use this checking,
> > you should ignore these sparse warnings.
> > 
> > Unfortunately, I don't know of a way to inform 0-day test robot of
> > the various maintainers' opt-in/out choices.
> 
> Oh geeze.
> 
> Good luck, Suren ;)

Ummm...  OK...

							Thanx, Paul

------------------------------------------------------------------------

commit abf0d8830a2885af9d17c41cfb7fe32321df94cb
Author: Paul E. McKenney <paulmck@linux.ibm.com>
Date:   Tue Feb 12 07:51:24 2019 -0800

    doc: Describe choice of rcu_dereference() APIs and __rcu usage
    
    Reported-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>

diff --git a/Documentation/RCU/rcu_dereference.txt b/Documentation/RCU/rcu_dereference.txt
index ab96227bad42..bf699e8cfc75 100644
--- a/Documentation/RCU/rcu_dereference.txt
+++ b/Documentation/RCU/rcu_dereference.txt
@@ -351,3 +351,106 @@ garbage values.
 
 In short, rcu_dereference() is -not- optional when you are going to
 dereference the resulting pointer.
+
+
+WHICH MEMBER OF THE rcu_dereference() FAMILY SHOULD YOU USE?
+
+First, please avoid using rcu_dereference_raw() and also please avoid
+using rcu_dereference_check() and rcu_dereference_protected() with a
+second argument with a constant value of 1 (or true, for that matter).
+With that caution out of the way, here is some guidance for which
+member of the rcu_dereference() to use in various situations:
+
+1.	If the access needs to be within an RCU read-side critical
+	section, use rcu_dereference().  With the new consolidated
+	RCU flavors, an RCU read-side critical section is entered
+	using rcu_read_lock(), anything that disables bottom halves,
+	anything that disables interrupts, or anything that disables
+	preemption.
+
+2.	If the access might be within an RCU read-side critical section
+	on the one hand, or protected by (say) my_lock on the other,
+	use rcu_dereference_check(), for example:
+
+		p1 = rcu_dereference_check(p->rcu_protected_pointer,
+					   lockdep_is_held(&my_lock));
+
+
+3.	If the access might be within an RCU read-side critical section
+	on the one hand, or protected by either my_lock or your_lock on
+	the other, again use rcu_dereference_check(), for example:
+
+		p1 = rcu_dereference_check(p->rcu_protected_pointer,
+					   lockdep_is_held(&my_lock) ||
+					   lockdep_is_held(&your_lock));
+
+4.	If the access is on the update side, so that it is always protected
+	by my_lock, use rcu_dereference_protected():
+
+		p1 = rcu_dereference_protected(p->rcu_protected_pointer,
+					       lockdep_is_held(&my_lock));
+
+	This can be extended to handle multiple locks as in #3 above,
+	and both can be extended to check other conditions as well.
+
+5.	If the protection is supplied by the caller, and is thus unknown
+	to this code, that is the rare case when rcu_dereference_raw()
+	is appropriate.  In addition, rcu_dereference_raw() might be
+	appropriate when the lockdep expression would be excessively
+	complex, except that a better approach in that case might be to
+	take a long hard look at your synchronization design.  Still,
+	there are data-locking cases where any one of a very large number
+	of locks or reference counters suffices to protect the pointer,
+	so rcu_dereference_raw() does have its place.
+
+	However, its place is probably quite a bit smaller than one
+	might expect given the number of uses in the current kernel.
+	Ditto for its synonym, rcu_dereference_check( ... , 1), and
+	its close relative, rcu_dereference_protected(... , 1).
+
+
+SPARSE CHECKING OF RCU-PROTECTED POINTERS
+
+The sparse static-analysis tool checks for direct access to RCU-protected
+pointers, which can result in "interesting" bugs due to compiler
+optimizations involving invented loads and perhaps also load tearing.
+For example, suppose someone mistakenly does something like this:
+
+	p = q->rcu_protected_pointer;
+	do_something_with(p->a);
+	do_something_else_with(p->b);
+
+If register pressure is high, the compiler might optimize "p" out
+of existence, transforming the code to something like this:
+
+	do_something_with(q->rcu_protected_pointer->a);
+	do_something_else_with(q->rcu_protected_pointer->b);
+
+This could fatally disappoint your code if q->rcu_protected_pointer
+changed in the meantime.  Nor is this a theoretical problem:  Exactly
+this sort of bug cost Paul E. McKenney (and several of his innocent
+colleagues) a three-day weekend back in the early 1990s.
+
+Load tearing could of course result in dereferencing a mashup of a pair
+of pointers, which also might fatally disappoint your code.
+
+These problems could have been avoided simply by making the code instead
+read as follows:
+
+	p = rcu_dereference(q->rcu_protected_pointer);
+	do_something_with(p->a);
+	do_something_else_with(p->b);
+
+Unfortunately, these sorts of bugs can be extremely hard to spot during
+review.  This is where the sparse tool comes into play, along with the
+"__rcu" marker.  If you mark a pointer declaration, whether in a structure
+or as a formal parameter, with "__rcu", which tells sparse to complain if
+this pointer is accessed directly.  It will also cause sparse to complain
+if a pointer not marked with "__rcu" is accessed using rcu_dereference()
+and friends.  For example, ->rcu_protected_pointer might be declared as
+follows:
+
+	struct foo __rcu *rcu_protected_pointer;
+
+Use of "__rcu" is opt-in.  If you choose not to use it, then you should
+ignore the sparse warnings.