Re: [PATCH] Prevent large file writeback starvation

[David Chinner <dgc@sgi.com>]

On Mon, Feb 06, 2006 at 05:04:11PM -0800, Andrew Morton wrote:
> David Chinner <dgc@sgi.com> wrote:
> >
> > > So to fix both these problems we need to be smarter about terminating the
> > > wb_kupdate() loop.  Something like "loop until no expired inodes have been
> > > written".
> > > 
> > > Wildly untested patch:
> > 
> > Wildly untested assertion - it won't fix my case for the same reason I'm seeing
> > the current code not working - we abort higher up in writeback_inodes()
> > on the age check.
> 
> You mean that we're in the state
> 
> a) big-dirty-expired inode is on s_dirty
> 
> b) small-dirty-not-expired inode is on s_io
> 
> sync_sb_inodes() sees the small-dirty-not-expired inode on s_io and gives up?

Yes, that's right.

> In which case, yes, perhaps leaving big-dirty-expired inode on s_io is the
> right thing to do.  But should we be checking that it has expired before
> deciding to do this?

Well, we are writing it out because it has expired in the first place,
right? And it remains expired until we actually clean it, so I
don't see any need for a check such as this....

> We don't want to get in a situation where continuous
> overwriting of a large file causes other files on that fs to never be
> written out.

Agreed. That's why i proposed the s_more_io queue - it works on those inodes
that need more work only after all the other inodes have been written out.
That prevents starvation, and makes large inode flushes background work (i.e.
occur when there is nothing else to do). it will get much better disk
utilisation than the method I originally proposed, as well, because it'll keep
the disk near congestion levels until the data is written out...

Cheers,

Dave.
-- 
Dave Chinner
R&D Software Enginner
SGI Australian Software Group