Re: Lockdep complains about commit 1331e7a1bb ("rcu: Remove _rcu_barrier() dependency on __stop_machine()")

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Oct 03, 2012 at 09:05:31AM +0530, Srivatsa S. Bhat wrote:
> On 10/03/2012 03:47 AM, Jiri Kosina wrote:
> > On Wed, 3 Oct 2012, Srivatsa S. Bhat wrote:
> > 
> >> I don't see how this circular locking dependency can occur.. If you are using SLUB,
> >> kmem_cache_destroy() releases slab_mutex before it calls rcu_barrier(). If you are
> >> using SLAB, kmem_cache_destroy() wraps its whole operation inside get/put_online_cpus(),
> >> which means, it cannot run concurrently with a hotplug operation such as cpu_up(). So, I'm
> >> rather puzzled at this lockdep splat..
> > 
> > I am using SLAB here.
> > 
> > The scenario I think is very well possible:
> > 
> > 
> > 	CPU 0				CPU 1
> > 	kmem_cache_destroy()
> 
> What about the get_online_cpus() right here at CPU0 before
> calling mutex_lock(slab_mutex)? How can the cpu_up() proceed
> on CPU1?? I still don't get it... :(
> 
> (kmem_cache_destroy() uses get/put_online_cpus() around acquiring
> and releasing slab_mutex).

The problem is that there is a CPU-hotplug notifier for slab, which
establishes hotplug->slab.  Then having kmem_cache_destroy() call
rcu_barrier() under the lock establishes slab->hotplug, which results
in deadlock.  Jiri really did explain this in an earlier email
message, but both of us managed to miss it.  ;-)

							Thanx, Paul

> Regards,
> Srivatsa S. Bhat
> 
> > 	mutex_lock(slab_mutex)
> > 	 				_cpu_up()
> > 					cpu_hotplug_begin()
> > 					mutex_lock(cpu_hotplug.lock)
> > 	rcu_barrier()
> > 	_rcu_barrier()
> > 	get_online_cpus()
> > 	mutex_lock(cpu_hotplug.lock)
> > 	 (blocks, CPU 1 has the mutex)
> > 					__cpu_notify()
> > 					mutex_lock(slab_mutex)
> > 
> > Deadlock.
> > 
> > Right?
> > 
> 
>