Re: Tracking actual disk write sources instead of flush thread

[Jan Kara <jack@suse.cz>]

On Wed 16-04-14 11:15:41, Phillip Susi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 4/16/2014 10:01 AM, Matthew Wilcox wrote:
> > On Tue, Apr 15, 2014 at 10:01:27PM -0400, Phillip Susi wrote:
> >> A lot of disk writes, especially when they are small individual
> >> files being written by several different processes, are hidden
> >> behind the flush thread.  Is there no way to properly track the
> >> process actually responsible for the IO, even when it is the
> >> flush thread that initiates the writeout?
> > 
> > Correct.
> 
> Wow.  If I understand things correctly, this also means that if
> process A dirties a ton of cache pages, then process B tries to write
> a relatively small amount, it can end up blocking in the synchronous
> flush path, and so it will appear that process B and flush are doing
> all of the writes, and not process A.
  There is nothing like synchronous flush path anymore (since 3.1 or
something like that). All flushing is done by the flusher works these days
(and sync(2), fsync(2) obviously). And you are right we have no way to
attribute particular dirty pages in page cache to a process. However we do
track how many pages each process has dirtied and use this information in
throttling processes in balance_dirty_pages() (the process waits in that
function for a time proportional to the number of dirtied pages since
it last entered that function).

However that dirty counter isn't very useful for administrative purposes
because it gets zeroed out in balance_dirty_pages(). But as others have
pointed out for tracking originators of dirty pages, we would like
something which survives longer than a process anyway.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR