Why is deleting (or reading) files not counted as IO-Wait in top?

Why is deleting (or reading) files not counted as IO-Wait in top?

[Matthias Schniedermeyer]

Hi


Currently i'm deleting about 500.000 files on a XFS-filesystem which 
takes a few minutes, as i had a top open i saw that 'wa' is shown as 
0.0% (Nothing else running currently) and everything except 'id' is near 
the bottom too. Kernel is 2.6.23.11.

So, as 'rm -rf' is essentially a IO (or seek, to be more correct)-bound 
task, shouldn't that count as "Waiting for IO"?

The man-page of top says:
'Amount of time the CPU has been waiting for I/O to complete.'

But AFAICT wa only seams to be (ac)counted for writing and not for 
reading. I come to that conclusion because, when i fire 'sync' i can see 
some percent wa for a few seconds.




Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: Why is deleting (or reading) files not counted as IO-Wait in top?

[Maxim Levitsky]

On Wednesday, 2 January 2008 21:35:03 Matthias Schniedermeyer wrote:
> Hi
> 
> 
> Currently i'm deleting about 500.000 files on a XFS-filesystem which 
> takes a few minutes, as i had a top open i saw that 'wa' is shown as 
> 0.0% (Nothing else running currently) and everything except 'id' is near 
> the bottom too. Kernel is 2.6.23.11.
> 
> So, as 'rm -rf' is essentially a IO (or seek, to be more correct)-bound 
> task, shouldn't that count as "Waiting for IO"?
> 
> The man-page of top says:
> 'Amount of time the CPU has been waiting for I/O to complete.'
> 
> But AFAICT wa only seams to be (ac)counted for writing and not for 
> reading. I come to that conclusion because, when i fire 'sync' i can see 
> some percent wa for a few seconds.
> 
> 
> 
> 
> Bis denn
> 


The IOWAIT time is the IDLE time that was spent waiting
for I/O. (meaning that there were no tasks running, but some were waiting on I/O)

Thus if you have another task that is not I/O bound, it can run in that time,
and ideally, you shouldn't notice any I/O slowdown, but the iowait time will decrease.

It wasn't the case before CFS introduction. I did few tests that showed almost 50% slowdown 
when running another task in that iowait time.
It is not longer a problem with CFS.


Regards,
	Maxim Levitsky

Re: Why is deleting (or reading) files not counted as IO-Wait in top?

[Matthias Schniedermeyer]

On 03.01.2008 02:16, Maxim Levitsky wrote:
> On Wednesday, 2 January 2008 21:35:03 Matthias Schniedermeyer wrote:
> > Hi
> > 
> > 
> > Currently i'm deleting about 500.000 files on a XFS-filesystem which 
> > takes a few minutes, as i had a top open i saw that 'wa' is shown as 
> > 0.0% (Nothing else running currently) and everything except 'id' is near 
> > the bottom too. Kernel is 2.6.23.11.
> > 
> > So, as 'rm -rf' is essentially a IO (or seek, to be more correct)-bound 
> > task, shouldn't that count as "Waiting for IO"?
> > 
> > The man-page of top says:
> > 'Amount of time the CPU has been waiting for I/O to complete.'
> > 
> > But AFAICT wa only seams to be (ac)counted for writing and not for 
> > reading. I come to that conclusion because, when i fire 'sync' i can see 
> > some percent wa for a few seconds.
> 
> The IOWAIT time is the IDLE time that was spent waiting
> for I/O. (meaning that there were no tasks running, but some were waiting on I/O)
> 
> Thus if you have another task that is not I/O bound, it can run in that time,
> and ideally, you shouldn't notice any I/O slowdown, but the iowait time will decrease.
> 
> It wasn't the case before CFS introduction. I did few tests that showed almost 50% slowdown 
> when running another task in that iowait time.
> It is not longer a problem with CFS.

I can understand that, but in my case nothing else was running, so i 
would expect about 46%-50% wa (Dual Core Processor).




Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: Why is deleting (or reading) files not counted as IO-Wait in top?

[Andrew Morton]

On Wed, 2 Jan 2008 20:35:03 +0100 Matthias Schniedermeyer <ms@citd.de> wrote:

> Hi
> 
> 
> Currently i'm deleting about 500.000 files on a XFS-filesystem which 
> takes a few minutes, as i had a top open i saw that 'wa' is shown as 
> 0.0% (Nothing else running currently) and everything except 'id' is near 
> the bottom too. Kernel is 2.6.23.11.
> 
> So, as 'rm -rf' is essentially a IO (or seek, to be more correct)-bound 
> task, shouldn't that count as "Waiting for IO"?
> 
> The man-page of top says:
> 'Amount of time the CPU has been waiting for I/O to complete.'
> 
> But AFAICT wa only seams to be (ac)counted for writing and not for 
> reading. I come to that conclusion because, when i fire 'sync' i can see 
> some percent wa for a few seconds.
> 

Yes, you would absolutely expect `rm' to be stuck in D state and
contributing to both load average and io-wait in this situation.

I'd think that either XFS is playing games (and it'd take some pretty
inventive games to do this) or your observations are in error.

Re: Why is deleting (or reading) files not counted as IO-Wait in top?

[Matthias Schniedermeyer]

On 05.01.2008 01:31, Andrew Morton wrote:
> On Wed, 2 Jan 2008 20:35:03 +0100 Matthias Schniedermeyer <ms@citd.de> wrote:
> 
> > Hi
> > 
> > 
> > Currently i'm deleting about 500.000 files on a XFS-filesystem which 
> > takes a few minutes, as i had a top open i saw that 'wa' is shown as 
> > 0.0% (Nothing else running currently) and everything except 'id' is near 
> > the bottom too. Kernel is 2.6.23.11.
> > 
> > So, as 'rm -rf' is essentially a IO (or seek, to be more correct)-bound 
> > task, shouldn't that count as "Waiting for IO"?
> > 
> > The man-page of top says:
> > 'Amount of time the CPU has been waiting for I/O to complete.'
> > 
> > But AFAICT wa only seams to be (ac)counted for writing and not for 
> > reading. I come to that conclusion because, when i fire 'sync' i can see 
> > some percent wa for a few seconds.
> > 
> 
> Yes, you would absolutely expect `rm' to be stuck in D state and
> contributing to both load average and io-wait in this situation.
> 
> I'd think that either XFS is playing games (and it'd take some pretty
> inventive games to do this) or your observations are in error.

I don't think that i could do much wrong(tm).

rm -rf <big dir> (Took >30 minutes to delete)
Everything else was sleeping at the time.

The sleeping processes are the usual handful of about 200 processes, you 
have laying around today. 47 kernel threads, another about 40 or so 
demons, and the rest >100 processes was my KDE-Session (mostly 
'konsole's with bashes inside) with Iceweasel (Firefox) as sugar on top. 
But as i said, there weren't doing anything at all, 'us' peaked at about 
1-1.4 when i watched top.





Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: Why is deleting (or reading) files not counted as IO-Wait in top?

[David Chinner]

On Wed, Jan 02, 2008 at 08:35:03PM +0100, Matthias Schniedermeyer wrote:
> Hi
> 
> 
> Currently i'm deleting about 500.000 files on a XFS-filesystem which 
> takes a few minutes, as i had a top open i saw that 'wa' is shown as 
> 0.0% (Nothing else running currently) and everything except 'id' is near 
> the bottom too. Kernel is 2.6.23.11.

Simply because the only I/O that XFS does during a delete is
to the log and the log does async I/O and hence the process
never blocks in I/O.

Instead, it blocks in a far more complex space reservation that
may or may not be related to I/O wait....

> So, as 'rm -rf' is essentially a IO (or seek, to be more correct)-bound 
> task, shouldn't that count as "Waiting for IO"?

rm -rf is not seek bound on XFS - it's generally determined by
the sequential write speed of the block device or how fast your
CPU is....

> The man-page of top says:
> 'Amount of time the CPU has been waiting for I/O to complete.'

Async I/O means that typically your CPU does not get held up
waiting for I/O to complete....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group