[Cluster-devel] Re: [PATCH] configfs: Silence lockdep on mkdir(), rmdir() and configfs_depend_item()

[Joel Becker <Joel.Becker@oracle.com>]

On Thu, Dec 18, 2008 at 01:28:28PM +0100, Peter Zijlstra wrote:
> In fact, both (configfs) mkdir and rmdir seem to synchronize on
> su_mutex..
> 
>  mkdir B/C/bar
> 
>    C.i_mutex
>      su_mutex
> 
> vs
> 
>  rmdir foo
> 
>    parent(foo).i_mutex
>      foo.i_mutex
>        su_mutex
> 
> 
> once holding the rmdir su_mutex you can check foo's user-content, since
> any mkdir will be blocked. All you have to do is then re-validate in
> mkdir's su_mutex that !IS_DEADDIR(C).

	We explicitly do not take any i_mutex locks after taking
su_mutex.  That's an ABBA risk.  su_mutex protects the hierarchy of
config_items.  i_mutex protects the vfs view thereof.
	If you look in mkdir, we take su_mutex, get a new item from the
client subsystem, then drop su_mutex.  After that, we go about building
our filesystem structure, using i_mutex where appropriate.  More
importantly is rmdir(2), where we use i_mutex in
configfs_detach_group(), but are not holding su_sem.  Only when
configfs_detach_group() has successfully returned and we have torn down
the filesystem structure do we take su_mutex and tear down the
config_item structure.
	In fact, we're part of the way there.  Check out that
USET_DROPPING flag we set in detach_prep() while scanning for user
objects.  That flags us racing mkdir(2).  When we are done with
detach_prep(), we know that mkdir(2) calls racing behind us will do
nothing until we safely lock them out with the locking in
detach_group().  All mkdir(2) calls will have exited by the time we get
the mutex, and no new mkdir(2) call can start because we have the mutex.
	Now look in detach_groups().  We drop the groups children before
marking them DEAD.  Louis' plan, I think, is to perhaps mark a group
DEAD, disconnect it from the vfs, and then operate on its children.  In
this fashion, perhaps we can unlock the trailing lock like a normal VFS
operation.
	This will require some serious auditing, however, because now
vfs functions can get into the vfs objects behind us.  And more vfs
changes affect us.  Whereas the current locking relies on the vfs's
parent->child lock ordering only, something that isn't likely to be
changed.

Joel


-- 

"You cannot bring about prosperity by discouraging thrift. You cannot
 strengthen the weak by weakening the strong. You cannot help the wage
 earner by pulling down the wage payer. You cannot further the
 brotherhood of man by encouraging class hatred. You cannot help the
 poor by destroying the rich. You cannot build character and courage by
 taking away a man's initiative and independence. You cannot help men
 permanently by doing for them what they could and should do for
 themselves."
	- Abraham Lincoln 

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127