2.5.66-(bk) == horrible response times for non X11 applications

2.5.66-(bk) == horrible response times for non X11 applications

[Zwane Mwaikambo]

The X server is not reniced nor are any of the called tasks. This is very 
easy to reproduce by doing the following;

open two xterms, run glxgears from one and then 5 or so seconds after that 
run an application which you can easily determine completion, such as 
ifconfig. You will notice that the console app doesn't make much progress 
and essentially get's starved.

ifconfig      R 00000000 5738320  1213   1032                     (NOTLB)       
Call Trace:                                                                     
 [<c011ee20>] autoremove_wake_function+0x0/0x40                                 
 [<c02b6462>] generic_make_request+0x112/0x1b0                                  
 [<c011dc4d>] io_schedule+0x2d/0x40                                             
 [<c015851a>] __wait_on_buffer+0xba/0xc0                                        
 [<c011ee20>] autoremove_wake_function+0x0/0x40     

glxgears      S C010A1E8 4265977488  1209   1019                   (NOTLB)    
Call Trace:                                                                     
 [<c010a1e8>] common_interrupt+0x18/0x20                                        
 [<c01298d5>] schedule_timeout+0xb5/0xc0                                        
 [<c03e8490>] unix_poll+0x20/0x90                                               
 [<c0391e69>] sock_poll+0x19/0x20                                               
 [<c0169c22>] do_select+0x132/0x270                                             
 [<c0169960>] __pollwait+0x0/0xa0         

Afflicted system was;

K6 500MHz laptop
128M RAM w/ 8M shared (Trident X11 Server)

This was not reproduceable with 2.4.18-3 nor with 2.5.66 and the 
following patch applied;

http://www.kernel.org/pub/linux/kernel/people/akpm/patches/2.5/2.5.66/2.5.66-mm3/broken-out/sched-interactivity-backboost-revert.patch

-- 
function.linuxpower.ca

Re: 2.5.66-(bk) == horrible response times for non X11 applications

[Andrew Morton]

Zwane Mwaikambo <zwane@linuxpower.ca> wrote:
>
> This was not reproduceable with 2.4.18-3 nor with 2.5.66 and the 
> following patch applied;
> 
> http://www.kernel.org/pub/linux/kernel/people/akpm/patches/2.5/2.5.66/2.5.66-mm3/broken-out/sched-interactivity-backboost-revert.patch
> 

There have been multiple reports of this patch fixing starvation problems.

We need to either apply it or fix up the problem by other means.



From: Ingo Molnar <mingo@elte.hu>

the patch below fixes George's setiathome problems (as expected).  It
essentially turns off Linus' improvement, but i dont think it can be fixed
sanely.

the problem with setiathome is that it displays something every now and
then - so it gets a backboost from X, and hovers at a relatively high
priority.



 kernel/sched.c |   13 +------------
 1 files changed, 1 insertion(+), 12 deletions(-)

diff -puN kernel/sched.c~sched-interactivity-backboost-revert kernel/sched.c
--- 25/kernel/sched.c~sched-interactivity-backboost-revert	2003-03-28 22:30:08.000000000 -0800
+++ 25-akpm/kernel/sched.c	2003-03-28 22:30:08.000000000 -0800
@@ -379,19 +379,8 @@ static inline int activate_task(task_t *
 		 * boosting tasks that are related to maximum-interactive
 		 * tasks.
 		 */
-		if (sleep_avg > MAX_SLEEP_AVG) {
-			if (!in_interrupt()) {
-				sleep_avg += current->sleep_avg - MAX_SLEEP_AVG;
-				if (sleep_avg > MAX_SLEEP_AVG)
-					sleep_avg = MAX_SLEEP_AVG;
-
-				if (current->sleep_avg != sleep_avg) {
-					current->sleep_avg = sleep_avg;
-					requeue_waker = 1;
-				}
-			}
+		if (sleep_avg > MAX_SLEEP_AVG)
 			sleep_avg = MAX_SLEEP_AVG;
-		}
 		if (p->sleep_avg != sleep_avg) {
 			p->sleep_avg = sleep_avg;
 			p->prio = effective_prio(p);

_

Re: 2.5.66-(bk) == horrible response times for non X11 applications

[Jakob Oestergaard]

On Sun, Apr 06, 2003 at 06:24:35PM -0700, Andrew Morton wrote:
...
> the patch below fixes George's setiathome problems (as expected).  It
> essentially turns off Linus' improvement, but i dont think it can be fixed
> sanely.
> 
> the problem with setiathome is that it displays something every now and
> then - so it gets a backboost from X, and hovers at a relatively high
> priority.

Why are niced processes boosted?   Does that make sense?

If only non-niced processes were boosted, wouldn't that pretty much fix
the problem?

Cheers,

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: 2.5.66-(bk) == horrible response times for non X11 applications

[Helge Hafting]

Jakob Oestergaard wrote:
> On Sun, Apr 06, 2003 at 06:24:35PM -0700, Andrew Morton wrote:
[...]
>>the problem with setiathome is that it displays something every now and
>>then - so it gets a backboost from X, and hovers at a relatively high
>>priority.
> 
> 
> Why are niced processes boosted?   Does that make sense?
> 
> If only non-niced processes were boosted, wouldn't that pretty much fix
> the problem?
> 
You'd have exactly the same problem for any non-niced cpu hog
that displays something now and then. A non-niced cpu hog is supposed
to be ok because the interactive processes are boosted above them.

Gui is becoming popular, so many compute tasks get some sort of
display, even if it only is a progress bar.

Helge Hafting

Re: 2.5.66-(bk) == horrible response times for non X11 applications

[Jakob Oestergaard]

On Wed, Apr 09, 2003 at 10:01:09AM +0200, Helge Hafting wrote:
> Jakob Oestergaard wrote:
> >On Sun, Apr 06, 2003 at 06:24:35PM -0700, Andrew Morton wrote:
> [...]
> >>the problem with setiathome is that it displays something every now and
> >>then - so it gets a backboost from X, and hovers at a relatively high
> >>priority.
> >
> >
> >Why are niced processes boosted?   Does that make sense?
> >
> >If only non-niced processes were boosted, wouldn't that pretty much fix
> >the problem?
> >
> You'd have exactly the same problem for any non-niced cpu hog
> that displays something now and then. A non-niced cpu hog is supposed
> to be ok because the interactive processes are boosted above them.

A non-niced CPU hog that has interactivity, like eh, Descent or
Half-Life (under Wine) - well, I want those to get one heck of a lot
more boost than my seti...  or my emacs...

glxgears - if I run it (I don't know why I would, but let's say I wanted
to) - I don't want it to drop frames to favour my emacs or kword or
whatever (in the case of glxgears maybe I would - but I wouldn't run it
in the first place - let's just assume that glxgears is really some
important real-time visualization application).  If emacs behaves "less
interactively" than glxgears, then this is because glxgears is doing
something intensive, eg. it really gets hurt if it is not heavily
boosted.   Yes, you would be insane to run glxgears and some other
interactive application simultaneously, and expect that both can get 90%
of the resources - but this is the problem of limited resources and no
amount of cleverness is going to solve that.

> 
> Gui is becoming popular, so many compute tasks get some sort of
> display, even if it only is a progress bar.

And if they update the progress bar 100 times per second, then either it
is crucially important that this is not delayed, or the app needs
fixing.

You can break Linux in a million ways from user space by writing poor
user space code.

It wouldn't update 100 times per second unless it was important would
it?   And if it is important, then it's a lot better to make sure it
happens, than to ensure that the bash in the xterm next door gets it's
timeslices exactly when it wants them.   After all, you can type in a
laggy terminal (and if you type enough, it will get similar
interactiveness boost and we're back to the problem of limited
resources).

Look, I'm not trying to defend the boosting mechanism at all costs - I'm
just trying to argue that it's not fundamentally insane.   :)

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: 2.5.66-(bk) == horrible response times for non X11 applications

[Helge Hafting]

Jakob Oestergaard wrote:
> On Wed, Apr 09, 2003 at 10:01:09AM +0200, Helge Hafting wrote:
> 
>>Jakob Oestergaard wrote:
>>
>>>On Sun, Apr 06, 2003 at 06:24:35PM -0700, Andrew Morton wrote:
>>
>>[...]
>>
>>>>the problem with setiathome is that it displays something every now and
>>>>then - so it gets a backboost from X, and hovers at a relatively high
>>>>priority.
>>>
>>>
>>>Why are niced processes boosted?   Does that make sense?
>>>
>>>If only non-niced processes were boosted, wouldn't that pretty much fix
>>>the problem?
>>>
>>
>>You'd have exactly the same problem for any non-niced cpu hog
>>that displays something now and then. A non-niced cpu hog is supposed
>>to be ok because the interactive processes are boosted above them.
> 
> 
> A non-niced CPU hog that has interactivity, like eh, Descent or
> Half-Life (under Wine) - well, I want those to get one heck of a lot
> more boost than my seti...  or my emacs...
> 
Sure.  I don't think your emacs will be a problem - you aren't typing
in it while playing half-life? :-)   And if you're multi-user - the other
user with his emacs has as much right to the cpu as your half-life game.

> glxgears - if I run it (I don't know why I would, but let's say I wanted
> to) - I don't want it to drop frames to favour my emacs or kword or
> whatever (in the case of glxgears maybe I would - but I wouldn't run it
> in the first place - let's just assume that glxgears is really some
> important real-time visualization application).  If emacs behaves "less
> interactively" than glxgears, then this is because glxgears is doing
> something intensive, eg. it really gets hurt if it is not heavily
> boosted.   Yes, you would be insane to run glxgears and some other
> interactive application simultaneously, and expect that both can get 90%
> of the resources - but this is the problem of limited resources and no
> amount of cleverness is going to solve that.
> 
> 
>>Gui is becoming popular, so many compute tasks get some sort of
>>display, even if it only is a progress bar.
> 
> 
> And if they update the progress bar 100 times per second, then either it
> is crucially important that this is not delayed, or the app needs
> fixing.

I'm not sure I agree on that - see below.

> You can break Linux in a million ways from user space by writing poor
> user space code.
> 
Sure.

> It wouldn't update 100 times per second unless it was important would
> it?   And if it is important, then it's a lot better to make sure it
> happens, than to ensure that the bash in the xterm next door gets it's
> timeslices exactly when it wants them.   After all, you can type in a
> laggy terminal (and if you type enough, it will get similar
> interactiveness boost and we're back to the problem of limited
> resources).
> 
> Look, I'm not trying to defend the boosting mechanism at all costs - I'm
> just trying to argue that it's not fundamentally insane.   :)

I agree that it isn't fundamentally insane, but using the 2.5.66
mechanism might force you to
concider some apps "broken" that were fine before.  This forces the
question of how hard it should be to write a user program.

100 screen updates per second doesn't mean it is important if it only
is twiddling of a baton or some progress bar.  People simply stick such
things in an outer loop - and that worked out to a update or two
per second in 1989.  Same code on todays machines results in hundreds
of updates per second.  Are we going to fix apps as our pcs becomes faster?

Cpu speed independent animations/screen updates are indeed possible, 
something
every game writer has to deal with.  But do we want to force people to do
that only so their progress bar won't go over some update treshold and
become an "interactive cpu hog" on some future fast machine?

Helge Hafting

Re: 2.5.66-(bk) == horrible response times for non X11 applications

[Jakob Oestergaard]

On Wed, Apr 09, 2003 at 11:10:02AM +0200, Helge Hafting wrote:
...
> >Look, I'm not trying to defend the boosting mechanism at all costs - I'm
> >just trying to argue that it's not fundamentally insane.   :)
> 
> I agree that it isn't fundamentally insane, but using the 2.5.66
> mechanism might force you to
> concider some apps "broken" that were fine before.  This forces the
> question of how hard it should be to write a user program.

That is a good and valid point

> 
> 100 screen updates per second doesn't mean it is important if it only
> is twiddling of a baton or some progress bar.  People simply stick such
> things in an outer loop - and that worked out to a update or two
> per second in 1989.  Same code on todays machines results in hundreds
> of updates per second.  Are we going to fix apps as our pcs becomes faster?

nice my_old_compute_intensive_app

We agree that only CPU intensive apps will cause the problem right?

So if we keep the CPU boosting mechanism, but only boost non-niced
processes (makes good sense in my twisted mind at least), we provide the
interactiveness benefits for the general case, and provide an easy way
to run unmaintained (or "semi broken" or whatever we will call them)
applications.

We end up "forcing" the user to nice some (hopefully few) CPU intensive
applications that didn't need nicing before.   I can see that that will
be a problem.

But will it be a bigger problem than not having the interactiveness
boost at all?

Cheers,

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............: