Re: pack operation is thrashing my server

["Shawn O. Pearce" <spearce@spearce.org>]

Andi Kleen <andi@firstfloor.org> wrote:
> "Ken Pratt" <ken@kenpratt.net> writes:
> >
> > I'm starting to think repacking is just not feasible on a 64-bit
> > server with 256MB of RAM (which is a very popular configuration in the
> > VPS market).
> 
> I think the right fix would be to make git throttle itself (not 
> use mmap, use very small defaults etc.) on low memory systems.
> It could take a look a /proc/meminfo for this.

Well, we had thought it was already able to throttle itself, as
we did put code in to respond to mmap() and malloc() failures by
trying to release memory and retrying the failed operation again.

However what we don't do is try to limit our heap usage to some
limit that is smaller than physical memory.  We just assume that
whatever we need is available from the OS.  This fails when what
we need exceeds physical memory and the OS tries to use swap.
We can get better performance by reducing what we mmap instead.

:-|

Looking at /proc/meminfo only works on Linux, and maybe some other
OSes which support a /proc like design.  But even then we don't
really know how much we are competing with other active processes
and how much memory we can use.

-- 
Shawn.