Re: [PATCH] cfq + io priorities

Re: [PATCH] cfq + io priorities

[Albert Cahalan]

Jens Axboe writes:
> On Sun, Nov 09 2003, Guillaume Chazarain wrote:

>> A process has an assigned io nice level, anywhere
>> from 0 to 20. Both of
>>
>> OK, I ask THE question : why not using the normal
>> nice level, via current->static_prio ? This way,
>> cdrecord would be RT even in IO, and nice -19
>> updatedb would have a minimal impact on the system.
>
> I don't want to tie io prioritites to cpu
> priorities, that's a design decision.

Sure, but do it in a way that's friendly to
all the apps and admins that only know "nice".

nice_cpu   sets CPU niceness
nice_net   sets net niceness
nice_disk  sets disk niceness
...
nice       sets all niceness values at once

>>> these end values are "special" - 0 means the process
>>> is only allowed to do io if the disk is idle, and 20
>>> means the process io is considered
>>
>> So a process with ioprio == 0 can be forever
>> starved. As it's not
>
> Yes
>
>> done this way for nice -19 tasks (unlike FreeBSD),
>> wouldn't it be safer to give a very long deadline
>> to ioprio == 0 requests ?
>
> ioprio == 0 means idle IO. It follows from that that
> you can risk infinite starvation if other io is happening.
> Otherwise it would not be idle io :-)
>
> CFQ doesn't assign request deadlines. That would
> be another way of handling starvation.

Keeping IO niceness as similar to CPU niceness as
you can would be very good for admins.

Re: [PATCH] cfq + io priorities

[Herbert Xu]

Albert Cahalan <albert@users.sf.net> wrote:
> 
> Sure, but do it in a way that's friendly to
> all the apps and admins that only know "nice".
> 
> nice_cpu   sets CPU niceness
> nice_net   sets net niceness
> nice_disk  sets disk niceness
> ...
> nice       sets all niceness values at once

That's a user space problem.  No matter what Jens does, you can always
make nice(1) do what you said.
-- 
Debian GNU/Linux 3.0 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Ali Magik

[Rama]

hello,


i have read the past post in archive, but i still cannot figure how in 
2.4.22 i can use ali magik ide with dma.
when i enable it via "hdparm" the system freeze after short.

in lilo, i have add in append the ide1=autotune hda=autotune, i have 
read somewhere that this can help in detection of hd.

there is somewhere a patch for 2.4.22 that i can use for this problem?

tnx for help!

best regards

Re: [PATCH] cfq + io priorities

[Albert Cahalan]

On Mon, 2003-11-10 at 05:19, Herbert Xu wrote:
> Albert Cahalan <albert@users.sf.net> wrote:
> > 
> > Sure, but do it in a way that's friendly to
> > all the apps and admins that only know "nice".
> > 
> > nice_cpu   sets CPU niceness
> > nice_net   sets net niceness
> > nice_disk  sets disk niceness
> > ...
> > nice       sets all niceness values at once
> 
> That's a user space problem.  No matter what Jens
> does, you can always make nice(1) do what you said.

It's not just the nice command. There's a syscall
interface you know, and lots of apps use it.

#include <unistd.h>
int nice(int inc);

You planning to hack ALL those apps? You'll
convince BSD-centric developers to include
this Linux-specific change?

Besides, the kernel load average was changed to
include processes waiting for IO. It just plain
makes sense to mix CPU usage with IO usage by
default. Wanting different niceness for CPU
and IO is a really unusual thing.

Re: [PATCH] cfq + io priorities

[P]

Albert Cahalan wrote:
> Besides, the kernel load average was changed to
> include processes waiting for IO. It just plain
> makes sense to mix CPU usage with IO usage by
> default. Wanting different niceness for CPU
> and IO is a really unusual thing.

I strongly agree. Of course it would be
nice/necessary to have seperate nice values,
but setting the global one should set the
underlying ones (cpu, disk, ...) also.

Pádraig.

Re: [PATCH] cfq + io priorities

[Jens Axboe]

On Mon, Nov 10 2003, P@draigBrady.com wrote:
> Albert Cahalan wrote:
> >Besides, the kernel load average was changed to
> >include processes waiting for IO. It just plain
> >makes sense to mix CPU usage with IO usage by
> >default. Wanting different niceness for CPU
> >and IO is a really unusual thing.
> 
> I strongly agree. Of course it would be
> nice/necessary to have seperate nice values,
> but setting the global one should set the
> underlying ones (cpu, disk, ...) also.

Global one? nice is CPU in Linux, period. ionice is io priority. I'm not
going to change this. So Albert and you can agree as much as you want,
unless you have some heavier arguments it's not going to help one bit.

-- 
Jens Axboe

Re: [PATCH] cfq + io priorities

[P]

Jens Axboe wrote:
> On Mon, Nov 10 2003, P@draigBrady.com wrote:
> 
>>Albert Cahalan wrote:
>>
>>>Besides, the kernel load average was changed to
>>>include processes waiting for IO. It just plain
>>>makes sense to mix CPU usage with IO usage by
>>>default. Wanting different niceness for CPU
>>>and IO is a really unusual thing.
>>
>>I strongly agree. Of course it would be
>>nice/necessary to have seperate nice values,
>>but setting the global one should set the
>>underlying ones (cpu, disk, ...) also.
> 
> Global one? nice is CPU in Linux, period.

Currently this is what it actually does.
But functionally, high nice value (to me at least)
means this process is low priority on the system =>
other processes get more system resources, and
the fact that this doesn't apply to IO until
now is just a defect. Now I can see some advantage
to splitting the tunables but not requiring
a new interface to turn this functionalit on.
I would like the new interface to turn it off.
I.E. I would like:

nice
     cpu
     IO

whereas you would like:

really_nice
     nice
     ionice

> ionice is io priority. I'm not
> going to change this. So Albert and you can agree as much as you want,
> unless you have some heavier arguments it's not going to help one bit.

fair enough.

Pádraig.

Re: [PATCH] cfq + io priorities

[Albert Cahalan]

On Mon, 2003-11-10 at 08:37, Jens Axboe wrote:
> On Mon, Nov 10 2003, P@draigBrady.com wrote:
> > Albert Cahalan wrote:
> > >Besides, the kernel load average was changed to
> > >include processes waiting for IO. It just plain
> > >makes sense to mix CPU usage with IO usage by
> > >default. Wanting different niceness for CPU
> > >and IO is a really unusual thing.
> > 
> > I strongly agree. Of course it would be
> > nice/necessary to have seperate nice values,
> > but setting the global one should set the
> > underlying ones (cpu, disk, ...) also.
> 
> Global one? nice is CPU in Linux, period.
> ionice is io priority. I'm not going to
> change this. So Albert and you can agree
> as much as you want, unless you have some
> heavier arguments it's not going to help
> one bit.

In other words, you're ignoring at least 3
people because you can. Coder's right, etc.

FWIW, here's a heavier argument.

I studied the UNIX spec today. I find nothing
that says the "nice" value is only for %CPU.

Your objection has been raised before, when
Linus changed the load average computation
to include processes waiting for IO. Linus
argued that users were really interested in
the feel of the system, no matter what the
source of load. That applies here as well.

In other words, it's a bug that some sources
of load aren't handled (viewed, limited, etc.)
by the normal mechanisms.

I have at least 27 binaries on my system that
use nice() in an attempt to reduce system load
(as in xlock) or increase their ability to
get things done. I can't think of a case in
which a matching IO priority wouldn't be desired.
It sure sounds painful to modify all those
binaries to do what is really wanted, especially
for the ones that aren't Linux-specific and
managed with autoconf.

I suppose the nice() wrapper in glibc could
be modified... but that's kind of silly when
the kernel is getting modified anyway.

Re: [PATCH] cfq + io priorities

[Pavel Machek]

Hi!

> I suppose the nice() wrapper in glibc could
> be modified... but that's kind of silly when
> the kernel is getting modified anyway.

Perhaps modifying glibc is the right way after all. We do not want to
have nice(), ionice() and then cpunice() as a kernel interface.

OTOH it would be good to keep ionice() on the same scale as other
"nice" values so that "do-it-all" interface is easier.

								Pavel
-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Re: [PATCH] cfq + io priorities

[Toon van der Pas]

On Mon, Nov 10, 2003 at 08:07:54AM -0500, Albert Cahalan wrote:
> On Mon, 2003-11-10 at 05:19, Herbert Xu wrote:
> > Albert Cahalan <albert@users.sf.net> wrote:
> > > 
> > > Sure, but do it in a way that's friendly to
> > > all the apps and admins that only know "nice".
> > > 
> > > nice_cpu   sets CPU niceness
> > > nice_net   sets net niceness
> > > nice_disk  sets disk niceness
> > > ...
> > > nice       sets all niceness values at once
> > 
> > That's a user space problem.  No matter what Jens
> > does, you can always make nice(1) do what you said.
> 
> It's not just the nice command. There's a syscall
> interface you know, and lots of apps use it.
> 
> #include <unistd.h>
> int nice(int inc);
> 
> You planning to hack ALL those apps? You'll
> convince BSD-centric developers to include
> this Linux-specific change?

Yes, I really would like all those apps to be hacked.
Because I want to REMOVE those nice() calls, not because I want to
add another piece of functionality that doesn't belong in apps in the
first place.

My opinion is that it's the system administrator's job to assign
nice-values to processes. Apps that do that themselves make the task
of the admin harder, if not impossible.

Regards,
Toon.