Out of memory error during git push

Out of memory error during git push

[Thomas Jampen]

Hi all

I'm experiencing the following error while pushing a git repo (home
directory) to my QNAP TS-210:

user@mypc:~$ git push origin master
Counting objects: 12532, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (8974/8974), done.
fatal: Out of memory, malloc failed, 986.06 MiB | 1.65 MiB/s
error: pack-objects died of signal 13
error: pack-objects died with strange error
error: failed to push some refs to
'ssh://nas//share/MD0_DATA/user/repositories/backup.mypc'

I've searched the internet and found suggestions to try to repack with
lower values for 'depth', 'window' and 'windowMemory'. I used 3, 3, 50
respectively, but a push after the repack command resulted in the same
error.

I tried git fsck which reported a few dangling blobs only and I got the
same error again during the next push.

I watched /proc/meminfo and /proc/swaps on the NAS while pushing and saw
that there are always 20-50MB of free RAM and that from 512MB swap space
only about 180MB are used.

What else can I try to successfully push my repo to my NAS?

Thanks for your help,
Tom

Re: Out of memory error during git push

[Jonathan Nieder]

(+cc: Nicolas, who knows the packing code pretty well)

Hi,

Thomas Jampen wrote:

> I'm experiencing the following error while pushing a git repo (home
> directory) to my QNAP TS-210:
> 
> user@mypc:~$ git push origin master
> Counting objects: 12532, done.
> Delta compression using up to 2 threads.
> Compressing objects: 100% (8974/8974), done.
> fatal: Out of memory, malloc failed, 986.06 MiB | 1.65 MiB/s
> error: pack-objects died of signal 13

What version of git are you using?  Do you have overcommit accounting
enabled?

I am not sure this is the problem you are running into, but pack-objects
like many other parts of git uses mmap() to read packfiles, which can
make the address space usage high.

> I've searched the internet and found suggestions to try to repack with
> lower values for 'depth', 'window' and 'windowMemory'. I used 3, 3, 50
> respectively, but a push after the repack command resulted in the same
> error.
> 
> I tried git fsck which reported a few dangling blobs only and I got the
> same error again during the next push.
> 
> I watched /proc/meminfo and /proc/swaps on the NAS while pushing and saw
> that there are always 20-50MB of free RAM and that from 512MB swap space
> only about 180MB are used.

I don't know --- this is out of my depth. :)

Good luck,
Jonathan

Re: Out of memory error during git push

[Thomas Jampen]

Hi

>> I'm experiencing the following error while pushing a git repo (home
>> directory) to my QNAP TS-210:
>>
>> user@mypc:~$ git push origin master
>> Counting objects: 12532, done.
>> Delta compression using up to 2 threads.
>> Compressing objects: 100% (8974/8974), done.
>> fatal: Out of memory, malloc failed, 986.06 MiB | 1.65 MiB/s
>> error: pack-objects died of signal 13
> 
> What version of git are you using?  Do you have overcommit accounting
> enabled?

I'm using git version 1.7.0.4 on my Kubuntu 10.04 Laptop and git version
1.7.1.1 on my QNAP NAS.

On my Laptop and on the NAS, when I execute: sudo sysctl vm.overcommit_memory
I get:
vm.overcommit_memory = 0

Do you propose to set this to '2' in order to disable overcommitting? If yes, where (on the Laptop and/or on the NAS)?

Thanks,
Tom

Re: Out of memory error during git push

[Jonathan Nieder]

Thomas Jampen wrote:

> vm.overcommit_memory = 0
> 
> Do you propose to set this to '2' in order to disable overcommitting?

It might be interesting to try

 echo 1 >/proc/sys/vm/overcommit_memory

but 0 is generally the only sane setting.

Re: Out of memory error during git push

[Thomas Jampen]

> > vm.overcommit_memory = 0
> >
> > Do you propose to set this to '2' in order to disable
> > overcommitting?
> 
> It might be interesting to try
> 
> echo 1 >/proc/sys/vm/overcommit_memory
> 
> but 0 is generally the only sane setting.

I set the value on both (Laptop and NAS) and got the following error:
Counting objects: 13051, done.
Delta compression using up to 2 threads.
warning: suboptimal pack - out of memory
Compressing objects: 100% (9848/9848), done.
error: index-pack died of signal 9), 1.27 GiB | 308 KiB/s
error: pack-objects died of signal 13
error: pack-objects died with strange error
error: failed to push some refs to 'ssh://...'

Any ideas?
Tom

Re: Out of memory error during git push

[Nicolas Pitre]

On Fri, 20 Aug 2010, Jonathan Nieder wrote:

> Thomas Jampen wrote:
> 
> > vm.overcommit_memory = 0
> > 
> > Do you propose to set this to '2' in order to disable overcommitting?
> 
> It might be interesting to try
> 
>  echo 1 >/proc/sys/vm/overcommit_memory
> 
> but 0 is generally the only sane setting.

Don't know the meaning for specific values, but you definitively don't 
want to turn off memory overcommit.


Nicolas

Re: Out of memory error during git push

[Nicolas Pitre]

On Fri, 20 Aug 2010, Jonathan Nieder wrote:

> (+cc: Nicolas, who knows the packing code pretty well)
> 
> Hi,
> 
> Thomas Jampen wrote:
> 
> > I'm experiencing the following error while pushing a git repo (home
> > directory) to my QNAP TS-210:
> > 
> > user@mypc:~$ git push origin master
> > Counting objects: 12532, done.
> > Delta compression using up to 2 threads.
> > Compressing objects: 100% (8974/8974), done.
> > fatal: Out of memory, malloc failed, 986.06 MiB | 1.65 MiB/s
> > error: pack-objects died of signal 13

How much memory is there on the machine where the pushed data is going?


Nicolas

Re: Out of memory error during git push

[Thomas Jampen]

On 20.08.2010 22:59, Nicolas Pitre wrote:
> On Fri, 20 Aug 2010, Jonathan Nieder wrote:
> 
>> (+cc: Nicolas, who knows the packing code pretty well)
>>
>> Hi,
>>
>> Thomas Jampen wrote:
>>
>>> I'm experiencing the following error while pushing a git repo (home
>>> directory) to my QNAP TS-210:
>>>
>>> user@mypc:~$ git push origin master
>>> Counting objects: 12532, done.
>>> Delta compression using up to 2 threads.
>>> Compressing objects: 100% (8974/8974), done.
>>> fatal: Out of memory, malloc failed, 986.06 MiB | 1.65 MiB/s
>>> error: pack-objects died of signal 13
> 
> How much memory is there on the machine where the pushed data is going?

My NAS has 256MB RAM. While the push is running, there is between
20-50MB RAM available according to MemFree reported by 'head
/proc/meminfo'. The harddrive space should not be an issue, its about 2TB.