vm in 2.4.5

vm in 2.4.5

[J . A . Magallon]

Hi.

This is a little experiment to smash 2.4 vm, and there is something I do not
understand.

Experiment: compile a C file with, say, 100k lines of puts("test"), auto
generated. Box is running vanilla 2.4.5, on 256Mb of ram.

State before gcc tst.c (just logged in a Gnome session with a couple rxvt's
open):

werewolf:~> free
             total       used       free     shared    buffers     cached
Mem:        255688      99572     156116          0       6788      42876
-/+ buffers/cache:      49908     205780
Swap:       152576          0     152576

Start gcc tst.c, and used memory begins to grow linearly in time, until:

werewolf:~> free
             total       used       free     shared    buffers     cached
Mem:        255688     213856      41832          0       6788      42876
-/+ buffers/cache:     164192      91496
Swap:       152576          0     152576

Stays some moments this way, and suddenly both memory and swap are full:

werewolf:~> free
             total       used       free     shared    buffers     cached
Mem:        255688     251268       4420          0       3752     179948
-/+ buffers/cache:      67568     188120
Swap:       152576     152576          0

It does not begin to use swap in a growing fashion, it just appears full in
a moment.

When compiler (cc1) ends, and assembler starts, some mem is freed:

werewolf:~> free
             total       used       free     shared    buffers     cached
Mem:        255688     195784      59904          0       1708     178512
-/+ buffers/cache:      15564     240124
Swap:       152576     152576          0

And when all the gcc process ends, my mem ends up like:

werewolf:~> free
             total       used       free     shared    buffers     cached
Mem:        255688     191232      64456          0       1760     174628
-/+ buffers/cache:      14844     240844
Swap:       152576     152576          0

What process do belong the 150Mb of swap ???!!!!
Shouldn't that pages have been freed when gcc ends ?

In short, I begin in one state, gcc something, and I do not return to the
same state (I know it could not be the same, something has been paged out,
and still has not get in core again, but should not totals be roughly the
same ???). It looks like the files gcc has used are still cached and also
swapped out and reserved, or the like...

Perhaps this helps someone.

BTW, I do not understand what the hell for needs gcc 256 Mb to compile 100k
sequential sentences...

-- 
J.A. Magallon                           #  Let the source be with you...        
mailto:jamagallon@able.es
Linux Mandrake release 8.1 (Cooker) for i586
Linux werewolf 2.4.4-ac15 #1 SMP Wed May 23 21:55:23 CEST 2001 i686

Re: vm in 2.4.5

[Rik van Riel]

On Sat, 26 May 2001, J . A . Magallon wrote:

> It does not begin to use swap in a growing fashion, it just appears
> full in a moment.

It gets _allocated_ in a moment, but things don't actually get
swapped out. This isn't a problem.

The real problem is that we don't actively reclaim swap space
when it gets full. We just assign swap to parts of processes,
but we never reclaim it when we need swap space for something
else; at least, not until the process exit()s or exec()s.

> And when all the gcc process ends, my mem ends up like:

>              total       used       free     shared    buffers     cached
> Swap:       152576     152576          0
> 
> What process do belong the 150Mb of swap ???!!!!
> Shouldn't that pages have been freed when gcc ends ?

Linux reclaims swap cache (and swap space) when it encounters
them in its scan of memory. It doesn't take the trouble of
freeing the swap on exit() but the swap space will be freed
later.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: vm in 2.4.5

[Andrzej Krzysztofowicz]

> On Sat, 26 May 2001, J . A . Magallon wrote:
> > It does not begin to use swap in a growing fashion, it just appears
> > full in a moment.
> 
> It gets _allocated_ in a moment, but things don't actually get
> swapped out. This isn't a problem.
> 
> The real problem is that we don't actively reclaim swap space
> when it gets full. We just assign swap to parts of processes,
> but we never reclaim it when we need swap space for something
> else; at least, not until the process exit()s or exec()s.

As I understand the original message, before the compilation and after the
system is in a "similar" state ("the same" processes should be running; even
less of them if some were killed by OOM).

:              total       used       free     shared    buffers     cached
: -/+ buffers/cache:      49908     205780
: Swap:       152576          0     152576

~49 MB in use befere the test,

: -/+ buffers/cache:      14844     240844
: Swap:       152576     152576          0

~149 MB allocated swap. By processes of total size of 49 MB ? Strange...

Either the test shows some leak in running userspace processes (they
allocate a lot of memory) as an effect of memory shortage (strange) or there
is some leak in the kernel. Or the test is bad (something else is running
when it finishes).

Am I right ?

Andrzej

Re: vm in 2.4.5

[J . A . Magallon]

On 05.26 Rik van Riel wrote:
> On Sat, 26 May 2001, J . A . Magallon wrote:
> 
> > It does not begin to use swap in a growing fashion, it just appears
> > full in a moment.
> 
> It gets _allocated_ in a moment, but things don't actually get
> swapped out. This isn't a problem.
> 
> The real problem is that we don't actively reclaim swap space
> when it gets full. We just assign swap to parts of processes,
> but we never reclaim it when we need swap space for something
> else; at least, not until the process exit()s or exec()s.
> 
> > And when all the gcc process ends, my mem ends up like:
> 
> >              total       used       free     shared    buffers     cached
> > Swap:       152576     152576          0
> > 
> > What process do belong the 150Mb of swap ???!!!!
> > Shouldn't that pages have been freed when gcc ends ?
> 
> Linux reclaims swap cache (and swap space) when it encounters
> them in its scan of memory. It doesn't take the trouble of
> freeing the swap on exit() but the swap space will be freed
> later.
> 

That seems to be partially true, if I start just the same comilation when
the first finishes, it behaves like preivous, in the moment the new gcc
needs swap the old swap gets freed, but not the in-core cache, so if I choose
carefully the number of 'puts' lines to stress my system just to the
border, I can't do two times the same 'gcc tst.c'.

-- 
J.A. Magallon                           #  Let the source be with you...        
mailto:jamagallon@able.es
Linux Mandrake release 8.1 (Cooker) for i586
Linux werewolf 2.4.4-ac15 #1 SMP Wed May 23 21:55:23 CEST 2001 i686