Re: [PATCH] Avoid useless inodes and dentries reclamation

[Tim Chen <tim.c.chen@linux.intel.com>]

On Fri, 2013-09-06 at 10:55 +1000, Dave Chinner wrote:
> On Tue, Sep 03, 2013 at 11:38:27AM -0700, Tim Chen wrote:
> > On Sat, 2013-08-31 at 19:00 +1000, Dave Chinner wrote:
> > > On Fri, Aug 30, 2013 at 09:21:34AM -0700, Tim Chen wrote:
> > > >
> > > > 
> > > > Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
> > > > ---
> > > > diff --git a/fs/super.c b/fs/super.c
> > > > index 73d0952..4df1fab 100644
> > > > --- a/fs/super.c
> > > > +++ b/fs/super.c
> > > > @@ -112,9 +112,6 @@ static unsigned long super_cache_count(struct shrinker *shrink,
> > > >  
> > > >  	sb = container_of(shrink, struct super_block, s_shrink);
> > > >  
> > > > -	if (!grab_super_passive(sb))
> > > > -		return 0;
> > > > -
> > > 
> > > I think the function needs a comment explaining why we aren't
> > > grabbing the sb here, otherwise people are going to read the code
> > > and ask why it's different to the scanning callout.
> > > 
> > > >  	if (sb->s_op && sb->s_op->nr_cached_objects)
> > > >  		total_objects = sb->s_op->nr_cached_objects(sb,
> > > >  						 sc->nid);
> > > 
> > 
> > Yes, those comments are needed.
> > I also need to remove the corresponding
> > 	drop_super(sb);
> > 
> > So probably something like:
> > 
> > ---
> > diff --git a/fs/super.c b/fs/super.c
> > index 73d0952..7b5a6e5 100644
> > --- a/fs/super.c
> > +++ b/fs/super.c
> > @@ -112,9 +112,14 @@ static unsigned long super_cache_count(struct shrinker *shrink,
> >  
> >  	sb = container_of(shrink, struct super_block, s_shrink);
> >  
> > -	if (!grab_super_passive(sb))
> > -		return 0;
> > -
> > +	/*
> > +	 * Don't call grab_super_passive as it is a potential 
> > +	 * scalability bottleneck. The counts could get updated 
> > +	 * between super_cache_count and super_cache_scan anyway.
> > +	 * Call to super_cache_count with shrinker_rwsem held
> > +	 * ensures the safety of call to list_lru_count_node() and 
> > +	 * s_op->nr_cached_objects().
> > +	 */
> 
> Well, that's not true of s_op->nr_cached_objects() right now. It's
> only going to be true if the shrinker deregistration is moved before
> ->kill_sb()....
> 
> > > Let me have a bit more of a think about this - the solution may
> > > simply be unregistering the shrinker before we call ->kill_sb() so
> > > the shrinker can't get called while we are tearing down the fs.
> > > First, though, I need to go back and remind myself of why I put that
> > > after ->kill_sb() in the first place.  
> > 
> > Seems very reasonable as I haven't found a case where the shrinker 
> > is touched in ->kill_sb() yet. It looks like unregistering the
> > shrinker before ->kill_sb() should be okay.
> 
> Having looked at it some more, I have to agree. I think the original
> reason for unregistering the shrinker there was to avoid problems
> with locking - the shrinker callouts are run holding the
> shrinker_rwsem in read mode, and then we lock the sb->s_umount in
> read mount. In the unmount case, we currently take the sb->s_umount
> lock in write mode (thereby locking out the shrinker) but we drop it
> before deregistering the shrinker and so there is no inverted
> locking order.
> 
> The thing is, grab_super_passive does a try-lock on the sb->s_umount
> now, and so if we are in the unmount process, it won't ever block.
> That means what used to be a deadlock and races we were avoiding
> by using grab_super_passive() is now:
> 
> 	shrinker			umount
> 
> 	down_read(shrinker_rwsem)
> 					down_write(sb->s_umount)
> 					shrinker_unregister
> 					  down_write(shrinker_rwsem)
> 					    <blocks>
> 	grab_super_passive(sb)
> 	  down_read_trylock(sb->s_umount)
> 	    <fails>
> 	<shrinker aborts>
> 	....
> 	<shrinkers finish running>
> 	up_read(shrinker_rwsem)
> 					  <unblocks>
> 					  <removes shrinker>
> 					  up_write(shrinker_rwsem)
> 					->kill_sb()
> 					....
> 
> And so it appears to be safe to deregister the shrinker before
> ->kill_sb().
> 
> Can you do this as two patches? The first moves the shrinker
> deregistration to before ->kill_sb(), then second is the above patch
> that drops the grab-super_passive() calls from the ->count_objects
> function?

I've sent the patches on a separate mail thread.  Please add
your sign-off if the patches look okay.

Thanks.

Tim