Reducing the memory footprint

Reducing the memory footprint

[Jon Smirl]

I'm using this config file:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
        packedGitWindowSize = 1M
        packedGitLimit = 1M
        deltaBaseCacheLimit = 1M
[pack]
        windowMemory = 1M
        deltaCacheSize = 1M

And I have NO_MMAP compiled in.

git is still using over 200MB of memory or address space, my process
gets killed either way.

Ideas on how to reduce the foot print more?

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Reducing the memory footprint

[Brian Downing]

On Fri, Nov 09, 2007 at 06:38:00PM -0500, Jon Smirl wrote:
> I'm using this config file:
> 
> [pack]
>         windowMemory = 1M
>         deltaCacheSize = 1M
> 
> And I have NO_MMAP compiled in.
> 
> git is still using over 200MB of memory or address space, my process
> gets killed either way.

I'm assuming it's dying on repacking since you included the pack
parameters.

How big is your biggest object?  Even with pack.windowMemory, it still
keeps the last object around to try and delta against (in other words,
the window only shrinks to size 1), which means you have to have room
for it and its delta index.

-bcd

Re: Reducing the memory footprint

[Jon Smirl]

On 11/9/07, Brian Downing <bdowning@lavos.net> wrote:
> On Fri, Nov 09, 2007 at 06:38:00PM -0500, Jon Smirl wrote:
> > I'm using this config file:
> >
> > [pack]
> >         windowMemory = 1M
> >         deltaCacheSize = 1M
> >
> > And I have NO_MMAP compiled in.
> >
> > git is still using over 200MB of memory or address space, my process
> > gets killed either way.
>
> I'm assuming it's dying on repacking since you included the pack
> parameters.
>
> How big is your biggest object?  Even with pack.windowMemory, it still
> keeps the last object around to try and delta against (in other words,
> the window only shrinks to size 1), which means you have to have room
> for it and its delta index.

It's a Linux kernel repository. Git receive-pack is going over 200MB
and getting zapped.  I don't understand why the process is so large. I
am compiled with -DNO_MMAP.

I think I have a achieved a work around. I rsync'd in my last several
weeks of changes. Now I can 'git push' small amounts of changes
without getting killed.

I'm begging dreamhost to simply install git. Installed commands don't
get zapped.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Reducing the memory footprint

[Jon Smirl]

On 11/9/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> On 11/9/07, Brian Downing <bdowning@lavos.net> wrote:
> > On Fri, Nov 09, 2007 at 06:38:00PM -0500, Jon Smirl wrote:
> > > I'm using this config file:
> > >
> > > [pack]
> > >         windowMemory = 1M
> > >         deltaCacheSize = 1M
> > >
> > > And I have NO_MMAP compiled in.
> > >
> > > git is still using over 200MB of memory or address space, my process
> > > gets killed either way.
> >
> > I'm assuming it's dying on repacking since you included the pack
> > parameters.
> >
> > How big is your biggest object?  Even with pack.windowMemory, it still
> > keeps the last object around to try and delta against (in other words,
> > the window only shrinks to size 1), which means you have to have room
> > for it and its delta index.
>
> It's a Linux kernel repository. Git receive-pack is going over 200MB
> and getting zapped.  I don't understand why the process is so large. I
> am compiled with -DNO_MMAP.

I believe I must not have installed everything correctly with my
NO_MMAP build. After debugging for a while and fixing things I'm able
to do a push now in about 80MB of memory.

> I think I have a achieved a work around. I rsync'd in my last several
> weeks of changes. Now I can 'git push' small amounts of changes
> without getting killed.
>
> I'm begging dreamhost to simply install git. Installed commands don't
> get zapped.
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>


-- 
Jon Smirl
jonsmirl@gmail.com