[PATCH -tip] rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

[PATCH -tip] rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

[Lai Jiangshan]

It is documented that local_irq_disable() also delimits
RCU_SCHED read-site critical sections.
See the document of synchronize_sched() or
Documentation/RCU/whatisRCU.txt.

So we have to test irqs_disabled() in rcu_read_lock_sched_held().
Otherwise rcu-lockdep brings incorrect complaint.

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
---
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 3024050..2ce5674 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -160,7 +160,7 @@ static inline int rcu_read_lock_sched_held(void)
 		return 1;
 	if (debug_locks)
 		lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
-	return lockdep_opinion || preempt_count() != 0;
+	return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)
@@ -191,7 +191,7 @@ static inline int rcu_read_lock_bh_held(void)
 #ifdef CONFIG_PREEMPT
 static inline int rcu_read_lock_sched_held(void)
 {
-	return !rcu_scheduler_active || preempt_count() != 0;
+	return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)

Re: [PATCH -tip] rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

[Ingo Molnar]

* Lai Jiangshan <laijs@cn.fujitsu.com> wrote:

> It is documented that local_irq_disable() also delimits
> RCU_SCHED read-site critical sections.
> See the document of synchronize_sched() or
> Documentation/RCU/whatisRCU.txt.
> 
> So we have to test irqs_disabled() in rcu_read_lock_sched_held().
> Otherwise rcu-lockdep brings incorrect complaint.

It would be useful to include the warning in question in the changelog - so 
that others who might be affected by it can see the fix and can track its 
progress. (and dont start a parallel effort debugging/reporting/fixing it)

Thanks,

	Ingo

Re: [PATCH -tip] rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

[Paul E. McKenney]

On Tue, Mar 16, 2010 at 07:09:21PM +0800, Lai Jiangshan wrote:
> It is documented that local_irq_disable() also delimits
> RCU_SCHED read-site critical sections.
> See the document of synchronize_sched() or
> Documentation/RCU/whatisRCU.txt.
> 
> So we have to test irqs_disabled() in rcu_read_lock_sched_held().
> Otherwise rcu-lockdep brings incorrect complaint.

Interesting -- I was under the impression that preempt_count() covered
this as well, due to the following in include/linux/hardirq.h:

	#define PREEMPT_MASK    (__IRQ_MASK(PREEMPT_BITS) << PREEMPT_SHIFT)
	#define SOFTIRQ_MASK    (__IRQ_MASK(SOFTIRQ_BITS) << SOFTIRQ_SHIFT)
	#define HARDIRQ_MASK    (__IRQ_MASK(HARDIRQ_BITS) << HARDIRQ_SHIFT)
	#define NMI_MASK        (__IRQ_MASK(NMI_BITS)     << NMI_SHIFT)

But irqs_disabled() does look to sample the actual interrupt hardware.

So, if there are cases where RCU is used where the interrupt hardware
is disabled, but preempt_count() has not been updated, this patch is
the right thing to do.

							Thanx, Paul

> Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> ---
> diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
> index 3024050..2ce5674 100644
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -160,7 +160,7 @@ static inline int rcu_read_lock_sched_held(void)
>  		return 1;
>  	if (debug_locks)
>  		lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
> -	return lockdep_opinion || preempt_count() != 0;
> +	return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
>  }
>  #else /* #ifdef CONFIG_PREEMPT */
>  static inline int rcu_read_lock_sched_held(void)
> @@ -191,7 +191,7 @@ static inline int rcu_read_lock_bh_held(void)
>  #ifdef CONFIG_PREEMPT
>  static inline int rcu_read_lock_sched_held(void)
>  {
> -	return !rcu_scheduler_active || preempt_count() != 0;
> +	return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
>  }
>  #else /* #ifdef CONFIG_PREEMPT */
>  static inline int rcu_read_lock_sched_held(void)
> 

Re: [PATCH -tip] rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

[Lai Jiangshan]

Paul E. McKenney wrote:
> On Tue, Mar 16, 2010 at 07:09:21PM +0800, Lai Jiangshan wrote:
>> It is documented that local_irq_disable() also delimits
>> RCU_SCHED read-site critical sections.
>> See the document of synchronize_sched() or
>> Documentation/RCU/whatisRCU.txt.
>>
>> So we have to test irqs_disabled() in rcu_read_lock_sched_held().
>> Otherwise rcu-lockdep brings incorrect complaint.
> 
> Interesting -- I was under the impression that preempt_count() covered
> this as well, due to the following in include/linux/hardirq.h:
> 
> 	#define PREEMPT_MASK    (__IRQ_MASK(PREEMPT_BITS) << PREEMPT_SHIFT)
> 	#define SOFTIRQ_MASK    (__IRQ_MASK(SOFTIRQ_BITS) << SOFTIRQ_SHIFT)
> 	#define HARDIRQ_MASK    (__IRQ_MASK(HARDIRQ_BITS) << HARDIRQ_SHIFT)
> 	#define NMI_MASK        (__IRQ_MASK(NMI_BITS)     << NMI_SHIFT)
> 
> But irqs_disabled() does look to sample the actual interrupt hardware.
> 
> So, if there are cases where RCU is used where the interrupt hardware
> is disabled, but preempt_count() has not been updated, this patch is
> the right thing to do.
> 
> 							Thanx, Paul
> 

local_irq_disable() does not touch preempt_count, it touchs
a register of current CPU.

The following quick test module is a case where RCU_SCHED is used where
the interrupt hardware is disabled, but preempt_count() has
not been updated, and it raises rcu-lockdep complaint.

#include <linux/module.h>
#include <linux/hardirq.h>
#include <linux/rcupdate.h>

void *test = &test;

int test_init(void)
{
	local_irq_disable();
	printk(KERN_INFO "%p\n", rcu_dereference_sched(test));
	local_irq_enable();

	return 0;
}

void test_exit(void) {}

module_init(test_init);
module_exit(test_exit);
MODULE_LICENSE("GPL");

Re: [PATCH -tip] rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

[Paul E. McKenney]

On Tue, Mar 16, 2010 at 06:21:56AM -0700, Paul E. McKenney wrote:
> On Tue, Mar 16, 2010 at 07:09:21PM +0800, Lai Jiangshan wrote:
> > It is documented that local_irq_disable() also delimits
> > RCU_SCHED read-site critical sections.
> > See the document of synchronize_sched() or
> > Documentation/RCU/whatisRCU.txt.
> > 
> > So we have to test irqs_disabled() in rcu_read_lock_sched_held().
> > Otherwise rcu-lockdep brings incorrect complaint.
> 
> Interesting -- I was under the impression that preempt_count() covered
> this as well, due to the following in include/linux/hardirq.h:
> 
> 	#define PREEMPT_MASK    (__IRQ_MASK(PREEMPT_BITS) << PREEMPT_SHIFT)
> 	#define SOFTIRQ_MASK    (__IRQ_MASK(SOFTIRQ_BITS) << SOFTIRQ_SHIFT)
> 	#define HARDIRQ_MASK    (__IRQ_MASK(HARDIRQ_BITS) << HARDIRQ_SHIFT)
> 	#define NMI_MASK        (__IRQ_MASK(NMI_BITS)     << NMI_SHIFT)
> 
> But irqs_disabled() does look to sample the actual interrupt hardware.
> 
> So, if there are cases where RCU is used where the interrupt hardware
> is disabled, but preempt_count() has not been updated, this patch is
> the right thing to do.

In light of later emails in this thread, first a big "thank you!!!"
to Lai, and I will pull this one in.  Good stuff!!!

							Thanx, Paul

> > Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> > ---
> > diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
> > index 3024050..2ce5674 100644
> > --- a/include/linux/rcupdate.h
> > +++ b/include/linux/rcupdate.h
> > @@ -160,7 +160,7 @@ static inline int rcu_read_lock_sched_held(void)
> >  		return 1;
> >  	if (debug_locks)
> >  		lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
> > -	return lockdep_opinion || preempt_count() != 0;
> > +	return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
> >  }
> >  #else /* #ifdef CONFIG_PREEMPT */
> >  static inline int rcu_read_lock_sched_held(void)
> > @@ -191,7 +191,7 @@ static inline int rcu_read_lock_bh_held(void)
> >  #ifdef CONFIG_PREEMPT
> >  static inline int rcu_read_lock_sched_held(void)
> >  {
> > -	return !rcu_scheduler_active || preempt_count() != 0;
> > +	return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
> >  }
> >  #else /* #ifdef CONFIG_PREEMPT */
> >  static inline int rcu_read_lock_sched_held(void)
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/