Re: [PATCH 6/6] kill-the-BKL/reiserfs: release the write lock on flush_commit_list()

[Ingo Molnar <mingo@elte.hu>]

* Chris Mason <chris.mason@oracle.com> wrote:

> On Fri, 2009-05-01 at 15:13 +0200, Frederic Weisbecker wrote:
> > On Fri, May 01, 2009 at 07:42:47AM +0200, Ingo Molnar wrote:
> > > 
> > > * Frederic Weisbecker <fweisbec@gmail.com> wrote:
> > > 
> > > > flush_commit_list() uses ll_rw_block() to commit the pending log blocks.
> > > > ll_rw_block() might sleep, and the bkl was released at this point. Then
> > > > we can also relax the write lock at this point.
> > > > 
> > > > [ Impact: release the reiserfs write lock when it is not needed ]
> > > > 
> > > > Cc: Jeff Mahoney <jeffm@suse.com>
> > > > Cc: Chris Mason <chris.mason@oracle.com>
> > > > Cc: Alexander Beregalov <a.beregalov@gmail.com>
> > > > Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> > > > ---
> > > >  fs/reiserfs/journal.c |    7 +++++--
> > > >  1 files changed, 5 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/fs/reiserfs/journal.c b/fs/reiserfs/journal.c
> > > > index 373d080..b1ebd5a 100644
> > > > --- a/fs/reiserfs/journal.c
> > > > +++ b/fs/reiserfs/journal.c
> > > > @@ -1120,8 +1120,11 @@ static int flush_commit_list(struct super_block *s,
> > > >  		    SB_ONDISK_JOURNAL_SIZE(s);
> > > >  		tbh = journal_find_get_block(s, bn);
> > > >  		if (tbh) {
> > > > -			if (buffer_dirty(tbh))
> > > > -			    ll_rw_block(WRITE, 1, &tbh) ;
> > > > +			if (buffer_dirty(tbh)) {
> > > > +		            reiserfs_write_unlock(s);
> > > > +			    ll_rw_block(WRITE, 1, &tbh);
> > > > +			    reiserfs_write_lock(s);
> > > > +			}
> > > >  			put_bh(tbh) ;
> > > >  		}
> > > >  	}
> > > 
> > > there's 7 other instances of ll_rw_block():
> > > 
> > > fs/reiserfs/journal.c-			spin_unlock(lock);
> > > fs/reiserfs/journal.c:			ll_rw_block(WRITE, 1, &bh);
> > > fs/reiserfs/journal.c-			spin_lock(lock);
> > > --
> > > fs/reiserfs/journal.c-		            reiserfs_write_unlock(s);
> > > fs/reiserfs/journal.c:			    ll_rw_block(WRITE, 1, &tbh);
> > > fs/reiserfs/journal.c-			    reiserfs_write_lock(s);
> > > --
> > > fs/reiserfs/journal.c-	/* read in the log blocks, memcpy to the corresponding real block */
> > > fs/reiserfs/journal.c:	ll_rw_block(READ, get_desc_trans_len(desc), log_blocks);
> > > fs/reiserfs/journal.c-	for (i = 0; i < get_desc_trans_len(desc); i++) {
> > > --
> > > fs/reiserfs/journal.c-		set_buffer_dirty(real_blocks[i]);
> > > fs/reiserfs/journal.c:		ll_rw_block(SWRITE, 1, real_blocks + i);
> > > fs/reiserfs/journal.c-	}
> > > --
> > > fs/reiserfs/journal.c-	}
> > > fs/reiserfs/journal.c:	ll_rw_block(READ, j, bhlist);
> > > fs/reiserfs/journal.c-	for (i = 1; i < j; i++)
> > > --
> > > fs/reiserfs/stree.c-		if (!buffer_uptodate(bh[j]))
> > > fs/reiserfs/stree.c:			ll_rw_block(READA, 1, bh + j);
> > > fs/reiserfs/stree.c-		brelse(bh[j]);
> > > --
> > > fs/reiserfs/stree.c-						    reada_blocks, reada_count);
> > > fs/reiserfs/stree.c:			ll_rw_block(READ, 1, &bh);
> > > fs/reiserfs/stree.c-			reiserfs_write_unlock(sb);
> > > --
> > > fs/reiserfs/super.c-{
> > > fs/reiserfs/super.c:	ll_rw_block(READ, 1, &(SB_BUFFER_WITH_SB(s)));
> > > fs/reiserfs/super.c-	reiserfs_write_unlock(s);
> > > 
> > > in particular the second stree.c one and the super.c has a 
> > > write-unlock straight before the lock-drop.
> > > 
> > > I think the stree.c unlock could be moved to before the 
> > > ll_rw_block() call straight away.
> > 
> > 
> > Indeed.
> > 
> > 
> > > 
> > > The super.c one needs more care: first put &(SB_BUFFER_WITH_SB(s)) 
> > > into a local variable, then unlock the wite-lock, then call 
> > > ll_rw_block(). (This is important because &(SB_BUFFER_WITH_SB(s)) is 
> > > global filesystem state that has to be read with the lock held.)
> > 
> > 
> > Indeed &(SB_BUFFER_WITH_SB(s)) is a pointer to blocks that
> > reflect the state of the filesystem but it was already not
> > safe on the old code.
> > 
> 
> SB_BUFFER_WITH_SB isn't going to change.  It gets set once during 
> mount and will return the same buffer from then on.

Are we holding a permanent reference to it after the first read? If 
yes then any bread done on an uptodate bh will return immediately 
without scheduling.

	Ingo