Re: out of memory problem

[Andreas Ericsson <ae@op5.se>]

guo tang wrote:
> On Mon, Sep 22, 2008 at 12:32 AM, Andreas Ericsson <ae@op5.se> wrote:
> 
>> Guo Tang wrote:
>>
>>> Gentlemen,
>>>
>>> I try to run "git gc" on linux kernel tree. The virtual memory keeps going
>>> up until over 3GB, then crash. Tried twice with the v1.6.0.2, same result.
>>> Then I used the git coming with FC9 (v1.5.5.1), the peak virutal memory
>>> usage is about 1.5GB. "git gc" finished without any trouble.
>>> Could there be a memory leak in v1.6.0.2?
>>>
>>>
>> There could be, but most likely it's commit
>> 38bd64979a2a3ffa178af801c6a62e6fcd658274 (Enable threaded delta
>> search on BSD and Linux). Do you have multiple cpu's in the
>> computer where 'git gc' was running? If so, and if you've set
>> pack.threads = 0, or --threads=0 it will autodetect the number
>> of CPU's you have and then saturate all of them with work. Each
>> thread will however consume memory close to that of a single
>> process running the repack, so for large repositories you might
>> want to set pack.threads = 1 in such large repositories.
> 
> 
> It is a Pentium M single core machine. But I am not sure whether it is using
> just a single thread or
> multiple threads. I will  try setting pack.threads parameter next I run into
> trouble.
> 

Unless you explicitly told it to run multiple threads (which
would be a bit silly on a single-core machine), it just ran
one thread.

>> It's a shame you didn't save the unpacked repository, or this could
>> have been properly debugged. While it's possible there is a memory
>> leak, it's a dismal project trying to locate it by staring at the
>> code, and the time it takes to repack huge repositories with memory
>> intensive parameters is sort of prohibitive for finding the possible
>> leak by bisection.
> 
> 
> Yes, the repository is already packed now. One question, beside the
> bisecting method, do we have
> this ability built into kernel:
> 1. Turn a flag on for a process.
> 2. OS will keep track off process malloc(), free() calls and the call stack.
> 
> 3. For the malloc() calls without the the free() call (a memory leak), OS
> will keep it count based on malloc() call stack.
> 4. After some time, be able to dump this information out based on biggest
> leak spot.
> 

No, there's not. The kernel isn't the one handing out the memory when you
call malloc(). That's handled by the C library, which can (and usually does)
allocate a larger area of memory than the application needs, so that it
doesn't have to run a system call for every malloc() call you do.

You can pre-load a different memory allocator though, which can do whatever
it wants with calls to malloc(), including ofcourse logging which function
called them and how much memory was requested.

Google for "memory leak check linux" and you'll get something like 750000
results.


> The complain when I ran out of memory if from mmap failure. Is it the same
> as malloc() failure?
> 

Sort of. Read 'man 2 mmap' for a more exhaustive description.

> This kind of tool is available in Windows with its umdh (user mode heap
> dump) tool.
> 

There are a number of tools to detect leaks under Linux/Unix as well.
valgrind is probably the most frequently used of all such leak checkers.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231