Re: [PATCH RFC] v7 expedited "big hammer" RCU grace periods

Lai Jiangshan
2009-05-27T05:40:34Z

Paul E. McKenney wrote:
> OK, good point! I do need to think about this.
>
> In the meantime, where do you see a need to run
> synchronize_sched_expedited() from within a hotplug CPU notifier?
>
> Thanx, Paul
>

I don't worry about synchronize_sched_expedited() called
from within a hotplug CPU notifier:

1st synchronize_sched_expedited() is newly, nobody calls it before current.
2nd get_online_cpus() will not cause DEADLOCK in CPU notifier:
get_online_cpus() finds itself owns the cpu_hotplug.lock, it will
not take it again.

I worry DEADLOCK like this:(ABBA DEADLOCK)
> get_online_cpus() is a large lock, a lot's of lock in kernel is required
> after cpu_hotplug.lock.
>
> _cpu_down()
> cpu_hotplug_begin()
> mutex_lock(&cpu_hotplug.lock)
> __raw_notifier_call_chain(CPU_DOWN_PREPARE)
> Lock a-kernel-lock.
>
> It means when we have held a-kernel-lock, we can not call
> synchronize_sched_expedited(). get_online_cpus() narrows
> synchronize_sched_expedited()'s usages.

One thread calls _cpu_down() which do "mutex_lock(&cpu_hotplug.lock)"
and then do "Lock a-kernel-lock", other thread calls
synchronize_sched_expedited() with a-kernel-lock held,
ABBA DEADLOCK would happen:

thread 1 | thread 2
_cpu_down() | Lock a-kernel-lock.
mutex_lock(&cpu_hotplug.lock) | synchronize_sched_expedited()
------------------------------------------------------------------------
Lock a-kernel-lock.(wait thread2) | mutex_lock(&cpu_hotplug.lock)
(wait thread 1)

cpuset_lock() is an example of a-kernel-lock as described before.
cpuset_lock() is required in CPU notifier.

But some work in cpuset need get_online_cpus().
(cpuset_lock() and then get_online_cpus(), we can
not release cpuset_lock() temporarily)

The fix is putting this work done in workqueue.
(get_online_cpus() and then cpuset_lock());

Thanx.
Lai