Memory, where's it going?

Memory, where's it going?

[Shaun]

I have a server with 8GB of RAM thats running 2.6.14.3 with the UML Ska's 3 
Patch.   It seams the system is eating more and more memory everyday but i 
cant seam to figure out where it's going.

I free shows the following...
[root@hostXXX ~]# free
             total       used       free     shared    buffers     cached
Mem:       8312484    8040500     271984          0     120664    7734420
-/+ buffers/cache:     185416    8127068
Swap:      4192924       2500    4190424

I see that free shows that 7.7GB is cached and i'm not sure why so much is 
cached.

When running through a ps auxf grabbing VSZ and RSS i get the following 
results...

For VSZ...
[root@hostXXX ~]# TOTAL=0;for i in `ps auxf|awk '{print $5}'`;do TOTAL=$(($i 
+ $TOTAL));done;echo $TOTAL
2459016

For RSS...
[root@hostXXX ~]# TOTAL=0;for i in `ps auxf|awk '{print $6}'`;do TOTAL=$(($i 
+ $TOTAL));done;echo $TOTAL
2239188

Anybody have any idea what is going on here?

-- 

~Shaun 

Re: Memory, where's it going?

[DervishD]

    Hi Shaun :)

 * Shaun <mailinglists@unix-scripts.com> dixit:
> I see that free shows that 7.7GB is cached and i'm not sure why so
> much is cached.

    Because free memory is a *waste* of memory. Why leaving unused
memory when it can be used for caching? The kernel will (up to some
extent, I suppose) try to use all free memory for caching if no app
needs it.

    If you have 8GB of memory, it's a bit difficult to fill it just
with the running apps, so the kernel cleverly uses the rest for
caching things so the system runs faster.

    For example, I have 1GB of RAM, and even when I use X (seldom...)
I never eat up more than, let's say, 500MB. So, I have another 500MB
of memory unused: the kernel uses it as cache, and that makes my
system run much faster (I noticed a speed increase when I switch from
512 to 1G).

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Memory, where's it going?

[Shaun]

I understand the concept and why things are cached, i've just never seen it 
cache this much before.. usually on a machine with bearly anything running 
on it (like this one) it uses bearly any memory at all.  My concern is that 
with bearly anything running on it i already have dug into swap.

Thanks for the responce, i just wanted to make sure their wasnt some type of 
a memory bug.

~Shaun
----- Original Message ----- 
From: "DervishD" <lkml@dervishd.net>
To: "Shaun" <mailinglists@unix-scripts.com>
Cc: <linux-kernel@vger.kernel.org>
Sent: Wednesday, December 28, 2005 12:53 AM
Subject: Re: Memory, where's it going?


>    Hi Shaun :)
>
> * Shaun <mailinglists@unix-scripts.com> dixit:
>> I see that free shows that 7.7GB is cached and i'm not sure why so
>> much is cached.
>
>    Because free memory is a *waste* of memory. Why leaving unused
> memory when it can be used for caching? The kernel will (up to some
> extent, I suppose) try to use all free memory for caching if no app
> needs it.
>
>    If you have 8GB of memory, it's a bit difficult to fill it just
> with the running apps, so the kernel cleverly uses the rest for
> caching things so the system runs faster.
>
>    For example, I have 1GB of RAM, and even when I use X (seldom...)
> I never eat up more than, let's say, 500MB. So, I have another 500MB
> of memory unused: the kernel uses it as cache, and that makes my
> system run much faster (I noticed a speed increase when I switch from
> 512 to 1G).
>
>    Raúl Núñez de Arenas Coronado
>
> -- 
> Linux Registered User 88736 | http://www.dervishd.net
> http://www.pleyades.net & http://www.gotesdelluna.net
> It's my PC and I'll cry if I want to...
> 

Re: Memory, where's it going?

[DervishD]

    Hi Shaun :)

 * Shaun <mailinglists@unix-scripts.com> dixit:
> I understand the concept and why things are cached, i've just never
> seen it cache this much before..

    That's probably because most of the work the machine did was CPU
bound and not IO bound. If you run many IO bound apps, which read a
lot from disk, you fill cache pretty fast. That's normal.

    For example, if your machine is an FTP server which most of the
time serves the same 10 files, only those files will end up filling
the cache, and the machine will not eat much memory caching. This
way, you probably end up with lots of unused memory, because it is
not needed even for cache.

    On the other hand, if your machine is an FTP server mostly
inactive but you serve a file whose size is 1GB, you will use most of
the memory with cache.

> My concern is that 
> with bearly anything running on it i already have dug into swap.

    Swap memory is not used just when the machine has no free memory.
Although this is a rough explanation and doesn't describe exactly the
swap mechanism, some apps will remain into swap space even if there's
plenty of free RAM available, as long as they are not used. And
that's good, because an unused app should not eat memory even if
there's free memory.

    Think about this situation: you have a running app whose memory
comsumption is cyclic, it eats most of the memory for a few minutes
and after that it frees the memory. The other apps eat a whole bunch
of memory, but they're sleeping, and only run once a day. In that
scenario, if apps were put out of swap and into main memory as soon
as free memory is available, the system will be very busy moving
sleeping apps from swap into core and back. And it will be wasted
time, because the moved apps are sleeping.

    What the kernel does is that it sents unused pages of memory to
swap when another app needs the memory (it is a bit more complex than
that, but it will do for an explanation), and after that it doesn't
put back the pages into main memory when it's free again. Instead, it
puts back the pages when they're accessed again, not before.

    Main memory must be used for things like running apps and
caching, not for storing sleeping apps. The system runs faster that
way, because it uses the expensive RAM to actually do work, not as a
drawer to put there apps "just in case they're needed" ;)

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Memory, where's it going?

[Tommy Reynolds]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

Uttered DervishD <lkml@dervishd.net>, spake thus:

> > I understand the concept and why things are cached, i've just never
> > seen it cache this much before..
> Swap memory is not used just when the machine has no free memory.
> Although this is a rough explanation and doesn't describe exactly the
> swap mechanism, some apps will remain into swap space even if there's
> plenty of free RAM available, as long as they are not used. 

Yes.  Let me amplify that..

It is not apps that are being evicted from main memory, only some
of their pages.  The kernel will do "anticipatory swapping", moving
pages that have not be used for some time out to the paging store, to
make room just in case it might be needed for a burst of activity in
the future.  This is a very neat feature.

As you pointed out, do not get excited about having no free memory:
the kernel is just doing its job especially well.

However, DO get excited if swap space gets short or you see frequent
page-in activity.

Cheers

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Memory, where's it going?

[DervishD]

    Hi Tommy :)

 * Tommy Reynolds <Tommy.Reynolds@MegaCoder.com> dixit:
> Uttered DervishD <lkml@dervishd.net>, spake thus:
> > > I understand the concept and why things are cached, i've just never
> > > seen it cache this much before..
> > Swap memory is not used just when the machine has no free memory.
> > Although this is a rough explanation and doesn't describe exactly the
> > swap mechanism, some apps will remain into swap space even if there's
> > plenty of free RAM available, as long as they are not used. 
[...] 
> It is not apps that are being evicted from main memory, only some
> of their pages.

    That's why I told "rough explanation" ;) I didn't want to go into
details, although I did anyway...

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...