2.4.10 much better than previous 2.4.x :-)

2.4.10 much better than previous 2.4.x :-)

[Michael Rothwell]

This is mainly a thank you for 2.4.10. It performs much better than
2.4.7 (RedHat version), from which I upgraded. Interactive performance
for applications (Gnome, Evolution, Mozilla) is much improved, and my
swap load is at zero, which is probably where it should be (2.4.7 would
regularly be using 256MB of swap with the same applications running).

Thanks!

--Michael

Re: 2.4.10 much better than previous 2.4.x :-)

[Rik van Riel]

On 24 Sep 2001, Michael Rothwell wrote:

> This is mainly a thank you for 2.4.10. It performs much better than
> 2.4.7 (RedHat version), from which I upgraded. Interactive performance
> for applications (Gnome, Evolution, Mozilla) is much improved,

If you have the time, could you also test 2.4.9-ac15 ?

(The -ac VM has basically branched off at 2.4.7 and has
evolved quite a bit since ... last week I fixed a stupid
page aging bug and things should be a lot better than
before now)

regards,

Rik
-- 
IA64: a worthy successor to i860.

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: 2.4.10 much better than previous 2.4.x :-)

[Paul]

Rik van Riel <riel@conectiva.com.br>, on Mon Sep 24, 2001 [10:35:53 PM] said:
> 
> If you have the time, could you also test 2.4.9-ac15 ?
> 
> (The -ac VM has basically branched off at 2.4.7 and has
> evolved quite a bit since ... last week I fixed a stupid
> page aging bug and things should be a lot better than
> before now)
> 
> regards,
> 
> Rik

	K6-333 128M ram
	2.4.9-ac15 my impression: Well, running mutt with 80M
folder open, desktop with several aterms and netscape with a few
windows open, I started building a kernel in one term, and a
'find / -type f -exec md5sum {}' in another, then started reading
the mail, and occasionally jumping around the virtual desktop,
exposing netscapes... (this is pretty much my normal working
load, except for the find.)
	I thought it worked very well; exposed netscapes were
either just there, or drew almost instantly. (other kernels I
have used, under the same load would usually take quite a bit
longer for exposed netscape to draw itself.) 'interactiveness'
seemed good.

	Then, I read a post in this thread about swap being
funny. I noticed that no swap was being reported as used all
during my test. So, I forced the issue with an endless malloc.
Very quickly, the system seems to freeze, and the disk is
yammering away. I was waiting for the OOM killer to kick in,
but it never did. <alt><sysrq> works well, and I used it to print
out the Mem stats after several minutes. Eventually, I used
sysrq to sync and kill (couldnt get it to reBoot, though).
	Im not complaining-- Im just curious why no OOM killing, 
and the Mem stats report 337148k swap free (I have 337168k).
Does this memmory report look  proper for a machine thrashing
itself to death from endless mallocs?

Paul
set@pobox.com

SysRq : Show Memory
Mem-info:
Free pages:        1512kB (     0kB HighMem)
( Active: 63, inactive_dirty: 172, inactive_clean: 0, free: 378 (351 702 1053) )
1*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB = 508kB)
25*4kB 3*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB = 1004kB)
= 0kB)
Swap cache: add 5, delete 5, find 0/0
Page cache size: 79
Buffer mem: 156
Ramdisk pages: 0
Free swap:       337148kB
32764 pages of RAM
0 pages of HIGHMEM
1038 reserved pages
115 pages shared
0 pages swap cached
0 pages in page table cache
Buffer memory:      624kB

Re: 2.4.10 much better than previous 2.4.x :-)

[José Luis Domingo López]

On Tuesday, 25 September 2001, at 20:35:15 -0400,
Paul wrote:

> Rik van Riel <riel@conectiva.com.br>, on Mon Sep 24, 2001 [10:35:53 PM] said:
> > 
> > If you have the time, could you also test 2.4.9-ac15 ?
> > [...]
> 	Im not complaining-- Im just curious why no OOM killing, 
> and the Mem stats report 337148k swap free (I have 337168k).
> Does this memmory report look  proper for a machine thrashing
> itself to death from endless mallocs?
> 
I've done several test with various versions of 2.4.x kernels, just to
make sure OOM worked right or not. I've used setups with both swap and no
swap, with swap double the RAM and equal to it, from a single user mode
and full multiuser with tons of applications running.

To reach OOM I try one of two methods: first, the well-know glob() DoS 
(ls ../*/../*/../*/ etc), second, starting as many applications as I can,
loading and creating huge images with gimp, etc.

In my test, OOM seems to work well most of the time, but not always. When
in works, it works fine, that is, it doesn't kill applications too early,
and (in recent kernel), multithreaded applications (like mozilla and
staroffice) and fully wiped from memory ("old" 2.4.x kernels didn't kill
all the threads, just the selected process ID).

When OOM doesn't work, the disk starts spinning like crazy, responsiveness
in null, mouse doesn't move, consoles don't update, unability to switch to
text consoles, etc. Giving time to the machine to recover itself is not
helpful: after more than 15 minutes the disk continue to spin and sound
like they were to inmediately crash :)

But in this situation, SysRq+K work fine most of the times: in a couple of
seconds the disk stops its crazyness, and the machine recovers. The text
console is unusable (can't display a thing), but issuing a "startx"
blindly works as expected, as if nothing had happened.

I've tried playing with "freepages" tunnable (where it exists), to raise
limits and (hopefully) keep more RAM free for the kernel for the hard
times where it tries to recover from OOM. OOM still fails sometimes, but
maybe I don't understand what freepages.[min|low|high] mean (having read
documentation under linux/Documentation :)

-- 
José Luis Domingo López
Linux Registered User #189436     Debian Linux Woody (P166 64 MB RAM)
 
jdomingo EN internautas PUNTO org  => ¿ Spam ? Atente a las consecuencias
jdomingo AT internautas DOT   org  => Spam at your own risk

Re: 2.4.10 much better than previous 2.4.x :-)

[Helge Hafting]

José Luis Domingo López wrote:

> In my test, OOM seems to work well most of the time, but not always. When
> in works, it works fine, that is, it doesn't kill applications too early,
> and (in recent kernel), multithreaded applications (like mozilla and
> staroffice) and fully wiped from memory ("old" 2.4.x kernels didn't kill
> all the threads, just the selected process ID).
> 
> When OOM doesn't work, the disk starts spinning like crazy, responsiveness
> in null, mouse doesn't move, consoles don't update, unability to switch to
> text consoles, etc. Giving time to the machine to recover itself is not
> helpful: after more than 15 minutes the disk continue to spin and sound
> like they were to inmediately crash :)

Seems to me you aren't necessarily OOM, that's why the killer don't
kick in.  You simply have a working set larger than RAM, and is
trashing into a hopeless slowness.  This slowness may even postpone
further allocations so you need more time to go OOM.  

If you _want_ to get OOM killed quickly - allocate way too much memory
but keep the working set small.

Helge Hafting

Re: 2.4.10 much better than previous 2.4.x :-)

[Paul]

	Hi.

	Got 2.4.9-ac16, and ran the 'test' described in my previous
post in this thread, and some benchmarks. Interactive performance 
was fine, but not quite as snappy as 2.4.9-ac15. I think this is
because 2.4.9-ac15 just didnt want to swap, due no doubt to the
mispatch. I guess everything I do 'normally' tends to fit well
within my available memory.
	Aiming the gun squarely at my toe again, I modified my
malloc program to write to the mem it allocs, and sleep 50us,
so I can watch it progress. Quite different results this time:
	(I have 128M ram and 330M swap)
	Mal's RSS climbed to around 60-90M, eating up free mem
until free mem was at 2.8M. Then some of the buffers and cache
drained. (mot much though-- I kept moving around the virtual
desktop and doing stuff, and interactivity was fine, and disk
was extremely quiet during whole test). After Mal drained as much
from cache and buff as it could, it would go to swap periodicly,
until eventually all of swap was used up. Then it started on the
buff and cache again-- it didnt seem to completely deplete them,
but they got down pretty low. Then it finally started getting
some of that 2.8M free memory that had been reserved this whole
time. Swap was completely full, cache and buff were very low, and
free mem went to zip. Then the OOM killer took it. At no point was the
machine thrashing.
	I dont know if this really says anything, but it was
comforting to see such a seemingly rational smooth progression
and termination:)
	Then I modified Mal to write to every chunk it had malloc'd
each time it malloc'd more. This is punishing:) After a while it
is a fight with Mal and everyone to be in memory. Disk activity
becomes contant, and things keep getting swapped out, however, Im
writing this with some degree of interactive degradation as it
goes on. Load is only 2, and Mal has slowed to a crawl, but the
machine is running as well as can be expected.

Thanks;
Paul
set@pobox.com

Something Broken in 2.4.9-ac15

[Kent Borg]

On Mon, Sep 24, 2001 at 08:29:44PM -0400, Michael Rothwell wrote:
> my swap load is at zero, which is probably where it should be (2.4.7
> would regularly be using 256MB of swap with the same applications
> running).

That got my attention.  I think something is broken here and you
are into swap but the reporting is quite wrong.

I have 192 MB in my laptop running 2.4.9-ac15 w/preemption patch
turned on.  I fired up a couple Mozilla windows, a couple Netscape
windows, Staroffice, a couple acroread windows, a couple emacs
windows, Abiword, and I forget what all.  Still no swap usage
reported.  I fired up a second X session and my computer came to a
crawl as it tried to run all those apps again.  The disk was grinding
away--it felt just like a swapping death and yet xosview and top both
reported 0 swap usage.  I don't believe it was so.

I killed most everything, and now the computer feels like it is back
to normal, but xosview thinks the CPU is pinned at 100% SYS when
nothing is going on.  I drag a window around and I get plenty of USR
and even some FREE, but let go and it goes back to 100%.

Something is broken here, me thinks.


-kb, the Kent who might go back to 2.4.9-ac12 later today and ry it
again.

Re: Something Broken in 2.4.9-ac15

[Kent Borg]

On Tue, Sep 25, 2001 at 11:20:12AM -0400, I, Kent Borg wrote:
> xosview thinks the CPU is pinned at 100% SYS when
> nothing is going on.  I drag a window around and I get plenty of USR
> and even some FREE, but let go and it goes back to 100%.

OK, so the above is nonsense.  I reverted through each of the kernels
I have built of late--and kapm.idled is what is taking up all the
time.  Which I guess is the right thing to do, I had heard about this
once, but then I got confused by it when I suddenly started paying
lots of attention.  It is unfortunate that an idle daemon doesn't get
counted as free, but I think I understand why.  Sorry for the
confusion.

Still, I am convinced the constant report of zero swap usage is wrong
in 2.4.9-ac15.


-kb

Re: 2.4.10 much better than previous 2.4.x :-)

[Andrea Arcangeli]

On Mon, Sep 24, 2001 at 08:29:44PM -0400, Michael Rothwell wrote:
> 
> This is mainly a thank you for 2.4.10. It performs much better than
> 2.4.7 (RedHat version), from which I upgraded. Interactive performance
> for applications (Gnome, Evolution, Mozilla) is much improved, and my
> swap load is at zero, which is probably where it should be (2.4.7 would
> regularly be using 256MB of swap with the same applications running).

if you apply vm-tweaks-1 it should get even better ;).

Andrea