Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Jens-Uwe Mager]

On Wed, 16 Jan 2002 03:00:45 GMT, Robert Love <rml@tech9.net> wrote:
>On Tue, 2002-01-15 at 21:04, Davide Libenzi wrote:
>
>> On 15 Jan 2002, Robert Love wrote:
>> > This isn't a bad idea, as long as we don't use it as a crutch or
>> > excuse.  That is, answer scheduling problems with "properly nice your
>> > tasks" -- the scheduler should be smart enough, to some degree.
>> >
>> > FWIW, Solaris actually implements a completely different scheduling
>> > policy, SCHED_INTERACT or something.  It is for windowed tasks in X --
>> > they get a large interactivity bonus.
>
>> Now ( with 2.5.3-pre1 ) intractivity is *very good* but SCHED_INTERACT
>> would help *a lot* to get things even more right.
>
>I looked it up; its called class IA.  I don't know if it grows from a
>limitation of their scheduler (i.e. they can't calculate priority and be
>as fair to interactive tasks as us) or if it offers a fundamental
>advantage.  I suspect their are a myriad of things things we can do with
>an interactive/GUI scheduling policy.
>
>One thing this is, since their kernel is preemptible, it marks processes
>that very much always deserve a scheduling boost based on interactivity,
>and thus their interactivity is quite nice.

I would believe the IA class is a hack to maintain good interactive
performance for the X window apps if the scheduler does not maintain
this itself in the TS (time sharing class). The standard Solaris
scheduler is not favoring I/O bound programs properly, especially if
these are driven by network connections. I have several programs that
are of the type poll() - recv() - work a little - send() type loops and
these get slow to a crawl if run in the TS class if the foreground
process (also in TS) is compute bound.

-- 
Jens-Uwe Mager	<pgp-mailto:62CFDB25>

Re: [patch] O(1) scheduler-H6/H7 and nice +19

[Davide Libenzi]

On Mon, 14 Jan 2002, Ed Tomlinson wrote:

> On January 14, 2002 09:33 pm, Davide Libenzi wrote:
> > try to replace :
> >
> > PRIO_TO_TIMESLICE() and RT_PRIO_TO_TIMESLICE() with :
> >
> > #define NICE_TO_TIMESLICE(n)    (MIN_TIMESLICE + ((MAX_TIMESLICE - \
> > 	MIN_TIMESLICE) * ((n) + 20)) / 39)
> >
> >
> > NICE_TO_TIMESLICE(p->__nice)
>
> Not sure about this change.  gkrellm shows the compile getting about 40%
> cpu.  Best result here seems to be with a larger range of timeslices.  ie
> 1-15 ((10*HZ)/1000...) instead lets the compile get 80% of the cpu.  wonder
> if this might be the way to go?

What's the MIN/MAX_TIMESLICE range that you used to get 80% of cpu ?




- Davide

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Ed Tomlinson]

The 2.4.17-I0 patch makes things much better here.  Does this one
suffer from the same bugs that the 2.5.2 version has?  

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
  790 ed        44  19 14320  13M   640 R N  69.4  2.7 166:18 setiathome
 7676 ed         0   0 14908  14M 11036 R    16.7  2.8   0:13 kmail
 5703 root       0 -10 82596  23M  1808 R <  11.2  4.6   2:23 XFree86
 7725 ed         0   0  1016 1016   776 R     1.3  0.1   0:00 top
 5803 ed         0   0  3764 3764  2904 R     0.5  0.7   0:15 gkrellm
 7720 ed         0   0  9752 9752  7856 R     0.3  1.8   0:04 kdeinit
 5725 ed         0   0  7524 7520  6888 S     0.1  1.4   0:01 kdeinit
    1 root       0   0   520  472   452 S     0.0  0.0   0:07 init
    2 root       0   0     0    0     0 SW    0.0  0.0   0:00 keventd
    3 root      17  19     0    0     0 SWN   0.0  0.0   0:00 ksoftirqd_CPU0
    4 root       0   0     0    0     0 SW    0.0  0.0   0:00 kswapd
    5 root      25   0     0    0     0 SW    0.0  0.0   0:00 bdflush
    6 root       0   0     0    0     0 SW    0.0  0.0   0:02 kupdated
    7 root      12   0     0    0     0 SW    0.0  0.0   0:00 khubd
   18 root       0   0     0    0     0 SW    0.0  0.0   0:00 kreiserfsd
   60 root       0   0     0    0     0 SW    0.0  0.0   0:00 mdrecoveryd
  219 root       0   0     0    0     0 SW    0.0  0.0   0:00 usb-storage-0
  220 root       0   0     0    0     0 SW    0.0  0.0   0:00 scsi_eh_0
  234 root       0   0   648  644   528 S     0.0  0.1   0:00 syslogd
  238 root      -2   0  1344 1344  1264 S     0.0  0.2   0:00 watchdog
  243 root       0   0  1184 1176   456 S     0.0  0.2   0:00 klogd
  249 daemon     0   0   472  460   380 S     0.0  0.0   0:00 portmap

Major difference from older version of the patch is that top shows many 
processes with PRI 0.   I am not sure this is intended?

Thanks
Ed Tomlinson

On January 14, 2002 10:27 pm, Davide Libenzi wrote:
> On Mon, 14 Jan 2002, Ed Tomlinson wrote:
> > On January 14, 2002 09:33 pm, Davide Libenzi wrote:
> > > try to replace :
> > >
> > > PRIO_TO_TIMESLICE() and RT_PRIO_TO_TIMESLICE() with :
> > >
> > > #define NICE_TO_TIMESLICE(n)    (MIN_TIMESLICE + ((MAX_TIMESLICE - \
> > > 	MIN_TIMESLICE) * ((n) + 20)) / 39)
> > >
> > >
> > > NICE_TO_TIMESLICE(p->__nice)
> >
> > Not sure about this change.  gkrellm shows the compile getting about 40%
> > cpu.  Best result here seems to be with a larger range of timeslices.  ie
> > 1-15 ((10*HZ)/1000...) instead lets the compile get 80% of the cpu. 
> > wonder if this might be the way to go?

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Davide Libenzi]

On Tue, 15 Jan 2002, Ed Tomlinson wrote:

> The 2.4.17-I0 patch makes things much better here.  Does this one
> suffer from the same bugs that the 2.5.2 version has?
>
>   PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
>   790 ed        44  19 14320  13M   640 R N  69.4  2.7 166:18 setiathome
>  7676 ed         0   0 14908  14M 11036 R    16.7  2.8   0:13 kmail
>  5703 root       0 -10 82596  23M  1808 R <  11.2  4.6   2:23 XFree86
>  7725 ed         0   0  1016 1016   776 R     1.3  0.1   0:00 top
>  5803 ed         0   0  3764 3764  2904 R     0.5  0.7   0:15 gkrellm
>  7720 ed         0   0  9752 9752  7856 R     0.3  1.8   0:04 kdeinit
>  5725 ed         0   0  7524 7520  6888 S     0.1  1.4   0:01 kdeinit
>     1 root       0   0   520  472   452 S     0.0  0.0   0:07 init
>     2 root       0   0     0    0     0 SW    0.0  0.0   0:00 keventd
>     3 root      17  19     0    0     0 SWN   0.0  0.0   0:00 ksoftirqd_CPU0
>     4 root       0   0     0    0     0 SW    0.0  0.0   0:00 kswapd
>     5 root      25   0     0    0     0 SW    0.0  0.0   0:00 bdflush
>     6 root       0   0     0    0     0 SW    0.0  0.0   0:02 kupdated
>     7 root      12   0     0    0     0 SW    0.0  0.0   0:00 khubd
>    18 root       0   0     0    0     0 SW    0.0  0.0   0:00 kreiserfsd
>    60 root       0   0     0    0     0 SW    0.0  0.0   0:00 mdrecoveryd
>   219 root       0   0     0    0     0 SW    0.0  0.0   0:00 usb-storage-0
>   220 root       0   0     0    0     0 SW    0.0  0.0   0:00 scsi_eh_0
>   234 root       0   0   648  644   528 S     0.0  0.1   0:00 syslogd
>   238 root      -2   0  1344 1344  1264 S     0.0  0.2   0:00 watchdog
>   243 root       0   0  1184 1176   456 S     0.0  0.2   0:00 klogd
>   249 daemon     0   0   472  460   380 S     0.0  0.0   0:00 portmap
>
> Major difference from older version of the patch is that top shows many
> processes with PRI 0.   I am not sure this is intended?
>
> Thanks
> Ed Tomlinson
>
> On January 14, 2002 10:27 pm, Davide Libenzi wrote:
> > On Mon, 14 Jan 2002, Ed Tomlinson wrote:
> > > On January 14, 2002 09:33 pm, Davide Libenzi wrote:
> > > > try to replace :
> > > >
> > > > PRIO_TO_TIMESLICE() and RT_PRIO_TO_TIMESLICE() with :
> > > >
> > > > #define NICE_TO_TIMESLICE(n)    (MIN_TIMESLICE + ((MAX_TIMESLICE - \
> > > > 	MIN_TIMESLICE) * ((n) + 20)) / 39)
> > > >
> > > >
> > > > NICE_TO_TIMESLICE(p->__nice)
> > >
> > > Not sure about this change.  gkrellm shows the compile getting about 40%
> > > cpu.  Best result here seems to be with a larger range of timeslices.  ie
> > > 1-15 ((10*HZ)/1000...) instead lets the compile get 80% of the cpu.
> > > wonder if this might be the way to go?

The above macro is wrong, this is right :

#define NICE_TO_TIMESLICE(n)    (MIN_TIMESLICE + ((MAX_TIMESLICE - \
	MIN_TIMESLICE) * (19 - (n))) / 39)




- Davide

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Ingo Molnar]

On Tue, 15 Jan 2002, Ed Tomlinson wrote:

> The 2.4.17-I0 patch makes things much better here.  Does this one
> suffer from the same bugs that the 2.5.2 version has?

i'll do a -I3 patch in a minute.

> Major difference from older version of the patch is that top shows
> many processes with PRI 0.  I am not sure this is intended?

yes, it's intended. Lots of interactive (idle) tasks. Right now the time
under which we detect a task as interactive is pretty short, but if you
run 'top' with 's 0.3' then you can see how tasks grow/shrink their
priorities, depending on the load they generate.

	Ingo

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Ed Tomlinson]

On January 15, 2002 08:49 pm, Ingo Molnar wrote:
> On Tue, 15 Jan 2002, Ed Tomlinson wrote:
> > The 2.4.17-I0 patch makes things much better here.  Does this one
> > suffer from the same bugs that the 2.5.2 version has?
>
> i'll do a -I3 patch in a minute.
>
> > Major difference from older version of the patch is that top shows
> > many processes with PRI 0.  I am not sure this is intended?
>
> yes, it's intended. Lots of interactive (idle) tasks. Right now the time
> under which we detect a task as interactive is pretty short, but if you
> run 'top' with 's 0.3' then you can see how tasks grow/shrink their
> priorities, depending on the load they generate.

OK I3 also works fine with respect to my nice test.  One thing I do note
and I am not too sure how it might be fixed, is what happens when starting 
what will be interactive programs.  

Watching with top 's 0.3' I can see them lose priority in the 3-10 seconds it
takes them to setup.  This is not that critical if they are the only thing trying
to run.  If you have another (not niced) task eating cpu (like a kernel compile) 
then intactive startup time suffers.  Startup time is wait time that _is_ noticed
by users.

Is there some way we could tell the scheduler or the scheduler could learn that 
a given _program_ is usually interactive so it should wait at bit (10 seconds on my 
box would work) before starting to increase its priority numbers?

TIA
Ed Tomlinson

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Rene Rebe]

Hi.

I3 still shows exactly the same behavior. For a test I simply compiled
ALSA and executed xstart. The screen went black (for a minute?) and
continued to start when ALSA finished.

Also dragging a xterm arround (during a compilation) results in 1-2
frames/per second refresh.

The lst one I tried was sched-patch was G1, it worked fine.

Athlon XP 1700+, SiS735, 512MB RAM, Matrox-G450 ...

From: Ed Tomlinson <tomlins@cam.org>
Subject: Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19
Date: Tue, 15 Jan 2002 19:44:51 -0500

> On January 15, 2002 08:49 pm, Ingo Molnar wrote:
> > On Tue, 15 Jan 2002, Ed Tomlinson wrote:
> > > The 2.4.17-I0 patch makes things much better here.  Does this one
> > > suffer from the same bugs that the 2.5.2 version has?
> >
> > i'll do a -I3 patch in a minute.
> >
> > > Major difference from older version of the patch is that top shows
> > > many processes with PRI 0.  I am not sure this is intended?
> >
> > yes, it's intended. Lots of interactive (idle) tasks. Right now the time
> > under which we detect a task as interactive is pretty short, but if you
> > run 'top' with 's 0.3' then you can see how tasks grow/shrink their
> > priorities, depending on the load they generate.
> 
> OK I3 also works fine with respect to my nice test.  One thing I do note
> and I am not too sure how it might be fixed, is what happens when starting 
> what will be interactive programs.  
> 
> Watching with top 's 0.3' I can see them lose priority in the 3-10 seconds it
> takes them to setup.  This is not that critical if they are the only thing trying
> to run.  If you have another (not niced) task eating cpu (like a kernel compile) 
> then intactive startup time suffers.  Startup time is wait time that _is_ noticed
> by users.
> 
> Is there some way we could tell the scheduler or the scheduler could learn that 
> a given _program_ is usually interactive so it should wait at bit (10 seconds on my 
> box would work) before starting to increase its priority numbers?
> 
> TIA
> Ed Tomlinson
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Ingo Molnar]

On Tue, 15 Jan 2002, Ed Tomlinson wrote:

> OK I3 also works fine with respect to my nice test. [...]

good!

> Watching with top 's 0.3' I can see them lose priority in the 3-10
> seconds it takes them to setup.  This is not that critical if they are
> the only thing trying to run.  If you have another (not niced) task
> eating cpu (like a kernel compile)  then intactive startup time
> suffers.  Startup time is wait time that _is_ noticed by users.

well, the kernel needs some 'proof' that a task is interactive, before it
gives it special attention.

the scheduler will give newly started up tasks some credit (if the parent
is interactive), but if they take too long to start up then there is
nothing it can do but to penalize them.

> Is there some way we could tell the scheduler or the scheduler could
> learn that a given _program_ is usually interactive so it should wait
> at bit (10 seconds on my box would work) before starting to increase
> its priority numbers?

there is a way: renicing. Either use nice +19 on the compilation job or
use nice -5 on the 'known good' tasks. Perhaps we should allow a nice
decrease of up to -5 from the default level - and things like KDE or Gnome
could renice interactive tasks, while things like compilation jobs would
run on the default priority.

	Ingo

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Robert Love]

On Tue, 2002-01-15 at 21:48, Ingo Molnar wrote:

> there is a way: renicing. Either use nice +19 on the compilation job or
> use nice -5 on the 'known good' tasks. Perhaps we should allow a nice
> decrease of up to -5 from the default level - and things like KDE or Gnome
> could renice interactive tasks, while things like compilation jobs would
> run on the default priority.

This isn't a bad idea, as long as we don't use it as a crutch or
excuse.  That is, answer scheduling problems with "properly nice your
tasks" -- the scheduler should be smart enough, to some degree.

FWIW, Solaris actually implements a completely different scheduling
policy, SCHED_INTERACT or something.  It is for windowed tasks in X --
they get a large interactivity bonus.

	Robert Love

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Davide Libenzi]

On 15 Jan 2002, Robert Love wrote:

> On Tue, 2002-01-15 at 21:48, Ingo Molnar wrote:
>
> > there is a way: renicing. Either use nice +19 on the compilation job or
> > use nice -5 on the 'known good' tasks. Perhaps we should allow a nice
> > decrease of up to -5 from the default level - and things like KDE or Gnome
> > could renice interactive tasks, while things like compilation jobs would
> > run on the default priority.
>
> This isn't a bad idea, as long as we don't use it as a crutch or
> excuse.  That is, answer scheduling problems with "properly nice your
> tasks" -- the scheduler should be smart enough, to some degree.
>
> FWIW, Solaris actually implements a completely different scheduling
> policy, SCHED_INTERACT or something.  It is for windowed tasks in X --
> they get a large interactivity bonus.

Now ( with 2.5.3-pre1 ) intractivity is *very good* but SCHED_INTERACT
would help *a lot* to get things even more right.




- Davide

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Robert Love]

On Tue, 2002-01-15 at 21:04, Davide Libenzi wrote:

> On 15 Jan 2002, Robert Love wrote:
> > This isn't a bad idea, as long as we don't use it as a crutch or
> > excuse.  That is, answer scheduling problems with "properly nice your
> > tasks" -- the scheduler should be smart enough, to some degree.
> >
> > FWIW, Solaris actually implements a completely different scheduling
> > policy, SCHED_INTERACT or something.  It is for windowed tasks in X --
> > they get a large interactivity bonus.

> Now ( with 2.5.3-pre1 ) intractivity is *very good* but SCHED_INTERACT
> would help *a lot* to get things even more right.

I looked it up; its called class IA.  I don't know if it grows from a
limitation of their scheduler (i.e. they can't calculate priority and be
as fair to interactive tasks as us) or if it offers a fundamental
advantage.  I suspect their are a myriad of things things we can do with
an interactive/GUI scheduling policy.

One thing this is, since their kernel is preemptible, it marks processes
that very much always deserve a scheduling boost based on interactivity,
and thus their interactivity is quite nice.

	Robert Love

Re: [patch] O(1) scheduler-H6/H7/I0 and nice +19

[Linus Torvalds]

On 15 Jan 2002, Robert Love wrote:
>
> I looked it up; its called class IA.  I don't know if it grows from a
> limitation of their scheduler (i.e. they can't calculate priority and be
> as fair to interactive tasks as us) or if it offers a fundamental
> advantage.  I suspect their are a myriad of things things we can do with
> an interactive/GUI scheduling policy.

I really doubt there is a _single_ interesting case apart from X itself.

X is kind of special, in that the X server can use a lot of CPU time if
you have people scrolling data on the screen etc, yet you want it to be
interactive because it does its own scheduling of actual user input etc.

So you can have the X server showing all the signs of a CPU hog, while
still being really important.

The current scheduler is pretty good at handling it, and since I don't
believe it is a generic problem, and since there _is_ a specific answer
for the specific case of X (ie "renice -10 X") already, I see no real
reason to have a new scheduling class.

(That said, I personally had running X _without_ the renice as my test of
scheduler interactivity goodness. It breaks down in the cases where X
really becomes CPU-bound, but if you ignore the really pathological case
where X is a CPU-hog it's a really good interactivity tester. The current
scheduler passes at least my personal criteria with flying colors in this
sense).

In short: I don't want to need to renice X to get good interactive
behaviour under any normal load, but I want even _less_ to start making up
scheduler classes for it.

		Linus