alpha kernel memory leak

alpha kernel memory leak

[Bob McElrath]

[-- Attachment #1: Type: text/plain, Size: 1813 bytes --]

I have noticed that my alphas seems to have a gigantic memory leak.  I
have two machines, one with 2GB memory and one with 512MB memory.  When
I first boot up the machines run fine, but over time the amount of
memory available decreases steadily to zero.

At first I thought it was just poor VM code in linux that caused it, but
I ran for a couple of days with the swap turned off, and the results are
the same.  

'ps' does not correctly report memory information anymore (latest debian
unstable -- why?), but using top and summing the VIRT, I get 202MB on
the 512MB machine, and top reports 505MB used, 5MB free, and 25MB
cached. Clearly something is horribly wrong.  The 2GB machine is
reporting that almost all memory is full (free + buffers + cached is
only different from the sum of VIRT by 150 MB).  But it is not used as
heavily so presumably it is leaking slower.  uptime is 4 days on the
512MB machine, 9 days on the 2GB machine.

The kernels on these two machines are 2.4.21-pre7 (2GB) and
2.4.21-rc1-ac1 (512MB), but this is something that I have noticed for
more than a year accross many kernels.

I have plotted the memory usage every 60 seconds over the last 3 days on
the 512MB machine (swap was turned off).  On first bootup I can run many
xterms, galeon, and xmms with no problem.  After 4 days both were killed
with OOM, but notice in the plots that the amount of memory available
hasn't changed, despite the fact that these (and X) are the largest
memory consuming processes running.  It is here:
    http://mcelrath.org/mem.log.ps.gz

Cheers,
Bob McElrath [Univ. of Wisconsin at Madison, Department of Physics]

    "You measure democracy by the freedom it gives its dissidents, not the
    freedom it gives its assimilated conformists." -- Abbie Hoffman


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: alpha kernel memory leak

[mcompengr]

(Please correct me where I'm wrong.)

Generally, a memory leak is where an often called piece of code
dynamically allocates itself some memory for temporary usage, and
then fails to release that memory before being called again.

This situation might be indicated by running out of swap space, at
which point the machine should grind to a halt (all processes), but
the memory usage reflected by the "top" or "free" commands won't show
it.  Swap space should be twice the size of physical memory.

> ...top reports 505MB used, 5MB free, ...
>
Such memory usage is normal and desirable, and is not an Alpha or
a Debian or even a Linux thing.  That's what the memory is for: to
be used to the max for minimum paging/swaping.  Since no process can
run until it is loaded into memory, processes are unloaded only when
it's time for some other process to run, and if that new process does
not already reside in memory.  Only a portion of a process's code is
loaded as needed, so even if no new processes are started after booting,
the memory usage will increase to approach the max, as needed portions
are loaded and un-needed blocks are left as is until there is no more
memory available (then swaping commences apace).

-Martin


Bob McElrath wrote:
> 
> I have noticed that my alphas seems to have a gigantic memory leak.  I
> have two machines, one with 2GB memory and one with 512MB memory.  When
> I first boot up the machines run fine, but over time the amount of
> memory available decreases steadily to zero.
> 
> At first I thought it was just poor VM code in linux that caused it, but
> I ran for a couple of days with the swap turned off, and the results are
> the same.
> 
> 'ps' does not correctly report memory information anymore (latest debian
> unstable -- why?), but using top and summing the VIRT, I get 202MB on
> the 512MB machine, and top reports 505MB used, 5MB free, and 25MB
> cached. Clearly something is horribly wrong.  The 2GB machine is
> reporting that almost all memory is full (free + buffers + cached is
> only different from the sum of VIRT by 150 MB).  But it is not used as
> heavily so presumably it is leaking slower.  uptime is 4 days on the
> 512MB machine, 9 days on the 2GB machine.
> 
> The kernels on these two machines are 2.4.21-pre7 (2GB) and
> 2.4.21-rc1-ac1 (512MB), but this is something that I have noticed for
> more than a year accross many kernels.
> 
> I have plotted the memory usage every 60 seconds over the last 3 days on
> the 512MB machine (swap was turned off).  On first bootup I can run many
> xterms, galeon, and xmms with no problem.  After 4 days both were killed
> with OOM, but notice in the plots that the amount of memory available
> hasn't changed, despite the fact that these (and X) are the largest
> memory consuming processes running.  It is here:
>     http://mcelrath.org/mem.log.ps.gz
> 
> Cheers,
> Bob McElrath [Univ. of Wisconsin at Madison, Department of Physics]
> 
>     "You measure democracy by the freedom it gives its dissidents, not the
>     freedom it gives its assimilated conformists." -- Abbie Hoffman
>

Re: alpha kernel memory leak

[Bob McElrath]

[-- Attachment #1: Type: text/plain, Size: 1985 bytes --]

mcompengr@earthlink.net [mcompengr@earthlink.net] wrote:
> 
> (Please correct me where I'm wrong.)
> 
> Generally, a memory leak is where an often called piece of code
> dynamically allocates itself some memory for temporary usage, and
> then fails to release that memory before being called again.
> 
> This situation might be indicated by running out of swap space, at
> which point the machine should grind to a halt (all processes), but
> the memory usage reflected by the "top" or "free" commands won't show
> it.  Swap space should be twice the size of physical memory.

The machine does, in fact grind to a halt.  When I first boot the memory
usage is ~250MB and I have no problems using or starting any program.

After a few days the memory usage is ~500MB and I cannot start programs
(they are killed immediatly with OOM).  Note that in both cases the SAME
programs are running.  Just sitting here and watching 'top' shortly
after bootup, the memory usage goes up by ~2k/s.  This is with the
network down, so the machine should be quiescent and hopefully no memory
allocations taking place.  I just checked again with all services
stopped and the network down, at the console (no X), only init, tcsh and
my little perl memory logging script running.  It still leaks by 1.8k/s,
according to my calculations.

If I turn swap back on, most operations cause a massive amount of
swapping (switching desktops in X).

Are there any tools to examine how memory is being used, that report
sensible numbers?  As I mentioned 'ps' no longer reports any memory
numbers (procps 3.1.8, debian unstable).  And using top, the sum of VIRT
is never equal to 'used'.  If there is a memory leak, how do I determine
that definitively?

Cheers,
Bob McElrath [Univ. of Wisconsin at Madison, Department of Physics]

    "You measure democracy by the freedom it gives its dissidents, not the
    freedom it gives its assimilated conformists." -- Abbie Hoffman


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]