Disk IO statistics still buggy

Disk IO statistics still buggy

[Jochen Suckfuell]

Hi!

The IO statistics displayed in /proc/partitions are still buggy, because
after some time, the value for the currently running requests gets too
high or too low (see the archives, look for "/proc/partitions").

Is anyone working on a fix?

Or does someone have an idea where the error might occur?
Which parts of the kernel play a role in starting/stopping block IO
requests (that are not part of ll_rw_blk.c) and might not handle the
accounting correctly?

I'd like to fix this and would appreciate any pointers, since after a
first look there is no obvious error in ll_rw_blk.c .

Thanks in advance,

Jochen Suckfuell

Re: Disk IO statistics still buggy

[Jochen Suckfuell]

> The IO statistics displayed in /proc/partitions are still buggy, because
> after some time, the value for the currently running requests gets too
> high or too low (see the archives, look for "/proc/partitions").
I forgot: This is on all recent 2.4 kernels; I can't tell for 2.5.

Bye
Jochen

Re: Disk IO statistics still buggy

[Adrian Bunk]

On Thu, 4 Jul 2002, Jochen Suckfuell wrote:

> Hi!

Hi Jochen!

> The IO statistics displayed in /proc/partitions are still buggy, because
> after some time, the value for the currently running requests gets too
> high or too low (see the archives, look for "/proc/partitions").
>
> Is anyone working on a fix?
>...

Marcelos' BK repository (that will become 2.4.19-rc2) includes a patch to
remove these statistics completely from /proc/partitions...

> Thanks in advance,
>
> Jochen Suckfuell

cu
Adrian

-- 

You only think this is a free country. Like the US the UK spends a lot of
time explaining its a free country because its a police state.
								Alan Cox

Re: Disk IO statistics still buggy

[Bill Davidsen]

On Thu, 4 Jul 2002, Adrian Bunk wrote:

> On Thu, 4 Jul 2002, Jochen Suckfuell wrote:
> 
> > Hi!
> 
> Hi Jochen!
> 
> > The IO statistics displayed in /proc/partitions are still buggy, because
> > after some time, the value for the currently running requests gets too
> > high or too low (see the archives, look for "/proc/partitions").
> >
> > Is anyone working on a fix?
> >...
> Marcelos' BK repository (that will become 2.4.19-rc2) includes a patch to
> remove these statistics completely from /proc/partitions...

Is this the new Linux way of life? Removing modules is hard, GET RID OF
THE FEATURE! Stats in /proc/partitions are not always correct, GET RID OF
THE FEATURE!

The IDE support in 2.5 sucks rocks off the bottom of the ocean, is that
next?

To the point, the stats are there, there are used by many of the widely
used distributions, this is supposed to be stable, that's the excuse for
not adding features, how can that justify taking out a feature? That's one
of the useful things for determining which (if any) partitions have enough
traffic to be best candidates for a separate drive. Yes I know there are
other more complex ways than looking at human readable numbers. Why is
that desirable? Why not fix the feature (I don't agree it's all that far
off in most cases).

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Disk IO statistics still buggy

[Andries Brouwer]

On Sat, Jul 06, 2002 at 12:15:47AM -0400, Bill Davidsen wrote:

> > Marcelos' BK repository (that will become 2.4.19-rc2) includes a patch to
> > remove these statistics completely from /proc/partitions...
> 
> Is this the new Linux way of life? Removing modules is hard, GET RID OF
> THE FEATURE! Stats in /proc/partitions are not always correct, GET RID OF
> THE FEATURE!

Shouting and exclamation marks - a bad sign.

You are mistaken. This has never been a feature.
It is not in 2.4.18, and it looks like it will not be in 2.4.19.

It is in some vendor kernels, but it is ugly and causes various problems.
If somebody cares about having statistics she should submit a patch
adding /proc/diskstat.

Andries

Re: Disk IO statistics still buggy

[Thunder from the hill]

Hi,

On Sat, Jul 06, 2002 at 12:15:47AM -0400, Bill Davidsen wrote:
> > Marcelos' BK repository (that will become 2.4.19-rc2) includes a patch to
> > remove these statistics completely from /proc/partitions...
> 
> Is this the new Linux way of life? Removing modules is hard, GET RID OF
> THE FEATURE! Stats in /proc/partitions are not always correct, GET RID OF
> THE FEATURE!

You misunderstood. It was nothing incorrect about the stats in 
/proc/partitions, it was just moved because it just didn't belong into 
/proc/partitions.

							Regards,
							Thunder
-- 
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

[PATCH] Re: Disk IO statistics still buggy

[Jochen Suckfuell]

Hi!

On Sat, Jul 06, 2002 at 07:42:42AM -0600, Thunder from the hill wrote:
> On Sat, Jul 06, 2002 at 12:15:47AM -0400, Bill Davidsen wrote:
> > > Marcelos' BK repository (that will become 2.4.19-rc2) includes a patch to
> > > remove these statistics completely from /proc/partitions...
> > 
> > Is this the new Linux way of life? Removing modules is hard, GET RID OF
> > THE FEATURE! Stats in /proc/partitions are not always correct, GET RID OF
> > THE FEATURE!
> 
> You misunderstood. It was nothing incorrect about the stats in 
> /proc/partitions, it was just moved because it just didn't belong into 
> /proc/partitions.

And it hasn't been removed either. Only the output has been removed; the
data is still collected. Meanwhile I found the bug that leads to wrong
ios_in_flight values on SCSI systems:

The accounting was done on a copy of the request _after_ the request has
been dequeued and the irq_request_lock released. I fixed this by taking
this lock again while calling the accounting function (see the patch
below).

** This patch is relevant regardless where the statistics will finally
be printed. **

One issue is still left: on my non-SCSI machine, the ios_in_flight value
is at -1 when although requests are running. It doesn't get any more
wrong than that it seems, and I'm not sure where that happened. Maybe
something went wrong when initializing this value at boot time? I have a
workaround that corrects negative ios_in_flight values to zero each time
/proc/partitions is read. After further testing I will post it to the
list.

Bye
Jochen Suckfüll

-- 
Jochen Suckfuell  ---  http://www.suckfuell.net/jochen/  ---

--- linux/drivers/scsi/scsi_lib.c Mon Jul  8 16:15:27 2002
+++ linux_work/drivers/scsi/scsi_lib.c Tue Jul  9 17:56:39 2002
@@ -426,7 +426,9 @@
   if (req->waiting != NULL) {
    complete(req->waiting);
   }
+  spin_lock_irq(&io_request_lock);
   req_finished_io(req);
+  spin_unlock_irq(&io_request_lock);
   add_blkdev_randomness(MAJOR(req->rq_dev));
 
         SDpnt = SCpnt->device;

Re: [PATCH] Re: Disk IO statistics still buggy

[Marcelo Tosatti]

Christoph,

Can you take a look at this one ?

On Tue, 9 Jul 2002, Jochen Suckfuell wrote:

> And it hasn't been removed either. Only the output has been removed; the
> data is still collected. Meanwhile I found the bug that leads to wrong
> ios_in_flight values on SCSI systems:
>
> The accounting was done on a copy of the request _after_ the request has
> been dequeued and the irq_request_lock released. I fixed this by taking
> this lock again while calling the accounting function (see the patch
> below).
>
> ** This patch is relevant regardless where the statistics will finally
> be printed. **
>
> One issue is still left: on my non-SCSI machine, the ios_in_flight value
> is at -1 when although requests are running. It doesn't get any more
> wrong than that it seems, and I'm not sure where that happened. Maybe
> something went wrong when initializing this value at boot time? I have a
> workaround that corrects negative ios_in_flight values to zero each time
> /proc/partitions is read. After further testing I will post it to the
> list.
>
> Bye
> Jochen Suckfüll
>
> --
> Jochen Suckfuell  ---  http://www.suckfuell.net/jochen/  ---
>
> --- linux/drivers/scsi/scsi_lib.c Mon Jul  8 16:15:27 2002
> +++ linux_work/drivers/scsi/scsi_lib.c Tue Jul  9 17:56:39 2002
> @@ -426,7 +426,9 @@
>    if (req->waiting != NULL) {
>     complete(req->waiting);
>    }
> +  spin_lock_irq(&io_request_lock);
>    req_finished_io(req);
> +  spin_unlock_irq(&io_request_lock);
>    add_blkdev_randomness(MAJOR(req->rq_dev));
>
>          SDpnt = SCpnt->device;

Re: [PATCH] Re: Disk IO statistics still buggy

[Jens Axboe]

On Tue, Jul 09 2002, Jochen Suckfuell wrote:
> --- linux/drivers/scsi/scsi_lib.c Mon Jul  8 16:15:27 2002
> +++ linux_work/drivers/scsi/scsi_lib.c Tue Jul  9 17:56:39 2002
> @@ -426,7 +426,9 @@
>    if (req->waiting != NULL) {
>     complete(req->waiting);
>    }
> +  spin_lock_irq(&io_request_lock);
>    req_finished_io(req);
> +  spin_unlock_irq(&io_request_lock);
>    add_blkdev_randomness(MAJOR(req->rq_dev));
>  
>          SDpnt = SCpnt->device;

good spotting, that's definitely a bug

-- 
Jens Axboe

no msg when linux-kernel boot.

[louie miranda]

Hi,

I saw this server twice. When she boots, it does not show extra kernel
"echo, requests" (Initializations)

ex:


LILO: linux...
Booting the kernel

and after that it pauses for a little while

and just shows the login prompt


Linux

Login:


---

Any ideas on this is much appreciated..





Thanks,
Louie...

Re: no msg when linux-kernel boot.

[Tigran Aivazian]

check /proc/cmdline to see if there is a "quiet" option listed there. If
so, edit /etc/lilo.conf and get rid of it from "append=" line and rerun
lilo and reboot.

Regards
Tigran

On Fri, 12 Jul 2002, louie miranda wrote:

> Hi,
>
> I saw this server twice. When she boots, it does not show extra kernel
> "echo, requests" (Initializations)
>
> ex:
>
>
> LILO: linux...
> Booting the kernel
>
> and after that it pauses for a little while
>
> and just shows the login prompt
>
>
> Linux
>
> Login:
>
>
> ---
>
> Any ideas on this is much appreciated..
>
>
>
>
>
> Thanks,
> Louie...
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: no msg when linux-kernel boot.

[szonyi calin]

 --- louie miranda <louie@chikka.com> a écrit : > Hi,
> 
> I saw this server twice. When she boots, it does not
> show extra kernel
> "echo, requests" (Initializations)
> 
> ex:
> 
> 
> LILO: linux...
> Booting the kernel
> 
> and after that it pauses for a little while
> 
> and just shows the login prompt
> 
> 
> Linux
> 
> Login:
> 

He could be ashamed so he hides himself :-)
You can play with linux and make him really quiet :-)
It's actually a feature :-)



=====
--
The UNIX Hierarchy - Beginner
- insecure with the concept of a terminal 
- has yet to learn the basics of "vi" 
- has not figured out how to get a directory 
- still has trouble with typing <RETURN> after each line of input

___________________________________________________________
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Yahoo! Mail : http://fr.mail.yahoo.com

Re: [PATCH] Re: Disk IO statistics still buggy

[Jochen Suckfuell]

Hi!

I'm just curious about the status: will this patch be included in 2.4.19
final?

On Thu, Jul 11, 2002 at 06:27:25PM -0300, Marcelo Tosatti wrote:
> 
> Christoph,
> Can you take a look at this one ?
> 
> On Tue, 9 Jul 2002, Jochen Suckfuell wrote:
> > The accounting was done on a copy of the request _after_ the request has
> > been dequeued and the irq_request_lock released. I fixed this by taking
> > this lock again while calling the accounting function (see the patch
> > below).
> >
> > [...]
> >
> > --- linux/drivers/scsi/scsi_lib.c Mon Jul  8 16:15:27 2002
> > +++ linux_work/drivers/scsi/scsi_lib.c Tue Jul  9 17:56:39 2002
> > @@ -426,7 +426,9 @@
> >    if (req->waiting != NULL) {
> >     complete(req->waiting);
> >    }
> > +  spin_lock_irq(&io_request_lock);
> >    req_finished_io(req);
> > +  spin_unlock_irq(&io_request_lock);
> >    add_blkdev_randomness(MAJOR(req->rq_dev));
> >
> >          SDpnt = SCpnt->device;

Bye
Jochen