XFS doesn't correctly account for IO-Wait for directory reading

XFS doesn't correctly account for IO-Wait for directory reading

[Matthias Schniedermeyer]

Hi


Some days ago Mr. Chinner(?, don't have the e-mail anymore) said that 
XFS fakes ( :-) ) it's way around IO-wait accounting for file-deletion 
by deferring it to the log.

Today i thought again about the initial 'rm -rf'-isn't-accounted-properly
"problem", and the bigger part of "rm -rf" is the 
directory-traversal(IOW read) and not the actual "unlink"-part.

So what better test than a simple 'find'.

Situation: Cache is cold:
find /<wherever> >/dev/null
While running (which takes some time) it shows exactly 0.0%wa in top on 
an otherwise completely idle system, where there should be a near 50%wa 
(Dual-Core system) or 100% on a UP system.

For plain old file-reading the IO-Wait appears to show correctly (AFAICT).
In a short test:
cat <somelargefiles> > /dev/null
peaked at 48%wa (said Dual-Core) with an average around 45%wa.

SO how das XFS fake around IO-wait accounting this time?

Unfortunatly i don't have any sufficiently large non-XFS-filesystems to 
do a good(tm) comparison, but a test on a small ext3-fs appeared to 
correctly(tm) account IO-Wait in directory traversal.

Tested Kernels: 2.6.23 & 2.6.24rc6







Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: XFS doesn't correctly account for IO-Wait for directory reading

[Christoph Hellwig]

Try this one-liner patch which should give you much better I/O wait
reporting.  There's some more I/O waits hidden in the log code, but to
fix this we'd need to dig into the sv_t abstraction.  Given that it only
has four users left I'm probably going to simply remove it and fix the
I/O wait accounting while I'm at it.


--- linux-2.6-xfs.orig/fs/xfs/linux-2.6/xfs_buf.c	2008-01-23 13:08:48.000000000 +0100
+++ linux-2.6-xfs/fs/xfs/linux-2.6/xfs_buf.c	2008-01-23 13:08:54.000000000 +0100
@@ -976,7 +976,7 @@ xfs_buf_wait_unpin(
 			break;
 		if (atomic_read(&bp->b_io_remaining))
 			blk_run_address_space(bp->b_target->bt_mapping);
-		schedule();
+		io_schedule();
 	}
 	remove_wait_queue(&bp->b_waiters, &wait);
 	set_current_state(TASK_RUNNING);

Re: XFS doesn't correctly account for IO-Wait for directory reading

[Matthias Schniedermeyer]

On 23.01.2008 12:17, Christoph Hellwig wrote:
> 
> Try this one-liner patch which should give you much better I/O wait
> reporting.  There's some more I/O waits hidden in the log code, but to
> fix this we'd need to dig into the sv_t abstraction.  Given that it only
> has four users left I'm probably going to simply remove it and fix the
> I/O wait accounting while I'm at it.

Either with 2.6.23.12 & 2.6.24-git-HEAD, and the shown line changed, the 
%wa reported by top is still at 0.0





Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: XFS doesn't correctly account for IO-Wait for directory reading

[David Chinner]

On Wed, Jan 23, 2008 at 12:00:27PM +0100, Matthias Schniedermeyer wrote:
> Hi
> 
> 
> Some days ago Mr. Chinner(?, don't have the e-mail anymore) said that 
> XFS fakes ( :-) ) it's way around IO-wait accounting for file-deletion 
> by deferring it to the log.
> 
> Today i thought again about the initial 'rm -rf'-isn't-accounted-properly
> "problem", and the bigger part of "rm -rf" is the 
> directory-traversal(IOW read) and not the actual "unlink"-part.
> 
> So what better test than a simple 'find'.
> 
> Situation: Cache is cold:
> find /<wherever> >/dev/null
> While running (which takes some time) it shows exactly 0.0%wa in top on 
> an otherwise completely idle system, where there should be a near 50%wa 
> (Dual-Core system) or 100% on a UP system.

XFS issues readahead on directories, so when the I/O has not completed,
it waits on semaphores in the xfs_buf layer, not on the I/O itself.
Hence we cannot account for the wait time as iowait as we cannot
tell the semaphore to call io_schedule() instead of schedule().

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group