Re: reiserfs locking (v2)

[Al Viro <viro@ZenIV.linux.org.uk>]

On Fri, Jul 02, 2010 at 03:12:52PM +0200, Frederic Weisbecker wrote:
> Right.
> 
> 
> The problem is:
> 
> vfs_readdir() {						do_munmap() {
> 	mutex_lock(inode);					read or write(don't know)_lock(mm->mmap_sem)
> 	reiserfs_readdir() {					reiserfs_file_release() {
> 		read_lock(mm->mmap_sem)					mutex_lock(inode);
> 	}							}
> }							}
> 
> 
> 
> I don't think the deadlock can really happen, as we can't release the directory while
> we are reading it. Plus I guess we can't mmap a directory (someone correct me if
> I'm wrong).

Gyah...  For the 1001st time: readdir() is far from being the only thing that
nests mmap_sem inside i_mutex.  In particular, write() does the same thing.

So yes, it *is* a real deadlock, TYVM, with no directories involved.  Open the
same file twice, mmap one fd, close it, then have munmap() hitting i_mutex
in reiserfs_file_release() race with write() through another fd.

Incidentally, reiserfs_file_release() checks in the fastpath look completely
bogus.  Checking i_count?  What the hell is that one about?  And no, these
checks won't stop open() coming between them and grabbing i_mutex, so they
couldn't prevent the deadlock in question anyway.