Re: [RFC][PATCH 2/3] CGroups: Use hierarchy_mutex in memory controller

[Li Zefan <lizf@cn.fujitsu.com>]

Balbir Singh wrote:
> * KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2008-12-11 10:05:01]:
> 
>> On Wed, 10 Dec 2008 16:52:57 -0800
>> Paul Menage <menage@google.com> wrote:
>>
>>> On Wed, Dec 10, 2008 at 4:49 PM, KAMEZAWA Hiroyuki
>>> <kamezawa.hiroyu@jp.fujitsu.com> wrote:
>>>>        an operation like rmdir() in somewhere.
>>>>                hierarchy_lock for A (acquired)
>>>>                hierarchy_lock for B (waiting)
>>>>
>>>>        in subsys A.
>>>>                mmap_sem (acquired)
>>>>                hierarchy_lock for A (waiting)
>>>>        in subsys B.
>>>>                hierarchy_lock for B (aquired)
>>>>                mmap_sem             (waiting)
>>>>
>>> That's a valid deadlock - you'd need to require the mmap_sem nests
>>> either inside all hierarchy_mutexes or else outside all of them.
>>>
>> This was a found dead lock between memcg and cpuset.
>>
>> another one was 
>>
>> 	an operation like rmdir() in somewhere.
>> 		hierarchy_lock for memcg (acquired)
>> 		hierarchy_lock for B (waiting)
>>
>> 	in subsys B.
>> 		hierarchy_lock for B (aquired)
> 
> But then the hierarchy_locks acquired will be different right?
> 

Yes, I'm worrying this too. The lock order by cgroup_lock_hierarchy() is:
	lock A -> lock B -> lock C
But a call chain may end up with:
	... -> lock B -> ... lock A -> ...

So though this hierarchy lock proprosal can solve specific deadlock between
cpuset and memcg by making cpuset holding cgroup_lock and memcg holding hierarchy_lock,
but we'll probably encounter other deadlocks describled above.

>> 		have to do some memory reclaim -> hierarchy_lock for memcg (waiting)
>>
>> I have no objections to hierarchy_lock itself but calling context to memcg is very
>> complicated and simple replace of these locks will be just a small help.
> 
> Could you please explain the race better?
>